Why does Kaoru begin to regret his refusal of Nakanokimi?

Nakanokimi's husband, Niou is arranged to marry Rokunokimi. Nakanokimi is not happy about being left behind by her husband. Kaoru goes to console Nakanokimi and is astonished at her beauty. She greatly reminds him of Oigimi and he begins to regret refusing her.