why did some settlers not want to join the militia?

Most of the settlers only fight because they're forced to fight. They fight because it's their duty. Given a choice, they'd stay home and look to their own interests..... the safety of their families, their farms, and putting food on the table. The settlers' homes are isolated, thus, when the men leave, their families are left to protect themselves.