Had it happen to me on a few occasions.
Most of the time, I pretend that I don't know them, for privacy reasons.
If they acknowledge me and alone, it is ok. If not, I keep walking.
But, this can go the other way too. Sometimes you are out in public with friends, you don't want to be...