What is this?

I noticed a few days ago that my lab has brownish dirt looking something in his ears (pic for reference). He likes getting ear scratches, so while I was scratching it, I noticed it. His other ear doesn’t have it and the one that does, smells a little bad. Should I take him to the vet and should I be worried about it?