Why does Kirsha claim is the reason people gossip about him in the novel, Midaq Alley?

Kirsha claims that people gossip about him not because they disapprove of his actions, but rather, because they like to belittle him. He also notes that when substance is lacking, they invent things.