In A Feast for Crows Jaime has taken a vow never again to raise a sword to a Tully or a Stark. Since a Tully is holding the castle, he is loathe to swarm it. Secondly, the new owner of the castle doesn't want it bombarded with cannon and other implements of war because it will destroy his new castle.
