In the poem, To a Louse, the speaker does not feel that the louse belongs in the place where he sees it. Why?

The speaker sees the louse on a fine lady's hat, and he believes that a louse is most at home and should be found in lower places, like on cattle, on low born people, in fields, on small, dirty children. In the end, though, he says it makes equals of us all.