It is evidently the intention of the Constitution that the first magistrate should be responsible for the executive department.  So far, therefore, as we do not make the officers who are to aid him in the duties of that department responsible to him, he is not responsible to the country.  Again:  Is there no danger that an officer, when he is appointed by the concurrence of the Senate and has friends in that body, may choose rather to risk his establishment on the favor of that branch than rest it upon the discharge of his duties to the satisfaction of the executive branch, which is constitutionally authorized to inspect and control his conduct?  And if it should happen that the officers connect themselves with the Senate, they may mutually support each other, and for want of efficacy reduce the power of the President to a mere vapor, in which case his responsibility would be annihilated, and the expectation of it is unjust.  The high executive officers, joined in cabal with the Senate, would lay the foundation of discord, and end in an assumption of the executive power only to be removed by a revolution in the Government.

Mr. Sedgwick, in the same debate, referring to the proposition that a head of Department should only be removed or suspended by the concurrence of the Senate, used this language: 

But if proof be necessary, what is then the consequence?  Why, in nine cases out of ten, where the case is very clear to the mind of the President that the man ought to be removed, the effect can not be produced, because it is absolutely impossible to produce the necessary evidence.  Are the Senate to proceed without evidence?  Some gentlemen contend not.  Then the object will be lost.  Shall a man under these circumstances be saddled upon the President who has been appointed for no other purpose but to aid the President in performing certain duties?  Shall he be continued, I ask again, against the will of the President?  If he is, where is the responsibility?  Are you to look for it in the President, who has no control over the officer, no power to remove him if he acts unfeelingly or unfaithfully?  Without you make him responsible you weaken and destroy the strength and beauty of your system.  What is to be done in cases which can only be known from a long acquaintance with the conduct of an officer?

I had indulged the hope that upon the assembling of Congress Mr. Stanton would have ended this unpleasant complication according to his intimation given in his note of August 12.  The duty which I have felt myself called upon to perform was by no means agreeable, but I feel that I am not responsible for the controversy or for the consequences.

Unpleasant as this necessary change in my Cabinet has been to me upon personal considerations, I have the consolation to be assured that so far as the public interests are involved there is no cause for regret.

Salutary reforms have been introduced by the Secretary ad interim, and great reductions of expenses have been effected under his administration of the War Department, to the saving of millions to the Treasury.