Why does Frank leave the Christmas mass service early during his first Christmas in New York as noted in the nonfiction book, ‘Tis: A Memoir?

Frank attends Christmas mass at Saint Vincent's on his first Christmas in New York, only to wind up feeling uncomfortable. His discomfort led him to leave before the service was over.