Why do the townspeople turn against George after he begins employing Prentiss and Landry?

The townspeople turn on George after he begins employing Prentiss and Landry because they do not think Black people deserve to get paid for work; they are generally very racist and in favor of slavery, even though the South has all but lost the Civil War.