Continue to live with him I guess but cut him off from using your car, and anything else that belongs to you. He's a grown ass man. If he wants to whore around......let him. Just don't enable it.
I guess he'll have to find some other method of getting men to the house