Why is Joe Pitt confused by Louis' invitation to come home with him?

Joe follows Louis into Central Park in Act 3, Scene 7, unsure of what he intends to do. Louis invites him home, but Joe is confused. They barely know each other, and Louis seems to hate everything that Joe believes.