Some people are going to look down on you, and most people will some of the time. It happens, right? There are moments when we seem less than we usually are.
IMO the best scenario would be to know that someone's looking down on you (you might occasionally need such information), but not really be bothered by it.
Looking down, looking up. They're just emotions. Curious to note. Not nearly as significant as our old monkey brains want us to believe.