Why do Marie and Louise not tell anyone that they think Byron and Pearce are sleeping together?

Marie and Louise go to clean the garconniere. When they change the bed linens, they discover evidence that Byron and Pearce are sleeping together. They decide not to tell anyone because if they do, they will be sold. Then they will lose their husbands and their children.