Why are Amanda and Elyot reluctant to discuss another marriage?

Amanda is somewhat nervous about what will happen to them socially if they divorce their current spouses right away and remarry, though it does not seem to bother Elyot too much. Also, they are both fearful about whether they can handle being together again, even though their current run at their relationship has been going smoothly.