I'm actually very skillful at having conversations without names, for a variety of reasons beyond memory problems (my life has been very complicated at various points). It boggles my mind when I suggest that someone who isn't sure how to address someone just not use a name (in casual/passing interactions) and they give me a blank stare like, "What do you mean?"

But then if you watch them, nine times out of ten they end up not saying the person's name, because you rarely do when you're speaking directly to someone face to face. It just feels weird to them to not have a handle to hang on someone.