Allow me to simplify:
Let's say you and I are friends and you invite me over to your house. I visit 5 times and you live with your parents but when I come over, I immediately go to your bedroom. Each time, I avoid your parents in the living room, not out of spite, but simply because I didn't use the front door and came through the garage. Is it rude that I didn't bother to say hi not 1 of those 5 times?