The President considered the dispute one of form rather than substance, and having a deep regard, not only for the Postmaster-General, but for his brother, General Frank Blair, and for his distinguished father, was most reluctant to take action against him.  Even in the bosom of the government, however, a strong hostility to Mr. Blair manifested itself.  As long as Chase remained in the cabinet there was smoldering hostility between them, and his attitude toward Seward and Stanton was one of increasing enmity.  General Halleck, incensed at some caustic remarks Blair was reported to have made about the defenders of the capital after Early’s raid, during which the family estate near Washington had suffered, sent an angry note to the War Department, wishing to know if such “wholesale denouncement” had the President’s sanction; adding that either the names of the officers accused should be stricken from the rolls, or the “slanderer dismissed from the cabinet.”  Mr. Stanton sent the letter to the President without comment.  This was too much; and the Secretary received an answer on the very same day, written in Mr. Lincoln’s most masterful manner: 

“Whether the remarks were really made I do not know, nor do I suppose such knowledge is necessary to a correct response.  If they were made, I do not approve them; and yet, under the circumstances, I would not dismiss a member of the cabinet therefore.  I do not consider what may have been hastily said in a moment of vexation at so severe a loss is sufficient ground for so grave a step....  I propose continuing to be myself the judge as to when a member of the cabinet shall be dismissed.”

Not content with this, the President, when the cabinet came together, read them this impressive little lecture: 

“I must myself be the judge how long to retain in and when to remove any of you from his position.  It would greatly pain me to discover any of you endeavoring to procure another’s removal, or in any way to prejudice him before the public.  Such endeavor would be a wrong to me, and, much worse, a wrong to the country.  My wish is that on this subject no remark be made nor question asked by any of you, here or elsewhere, now or hereafter.”

This is one of the most remarkable speeches ever made by a President.  The tone of authority is unmistakable.  Washington was never more dignified; Jackson was never more peremptory.

The feeling against Mr. Blair and the pressure upon the President for his removal increased throughout the summer.  All through the period of gloom and discouragement he refused to act, even when he believed the verdict of the country likely to go against him, and was assured on every side that such a concession to the radical spirit might be greatly to his advantage.  But after the turn had come, and the prospective triumph of the Union cause became evident, he felt that he ought no longer to retain in his cabinet a member who, whatever his personal merits, had lost the confidence of the great body of Republicans; and on September 9 wrote him a kindly note, requesting his resignation.