The question came before the Supreme Court of the United States in January, 1839, ex parte Hennen.  It was declared by the court on that occasion that the power of removal from office was a subject much disputed, and upon which a great diversity of opinion was entertained in the early history of the Government.  This related, however, to the power of the President to remove officers appointed with the concurrence of the Senate, and the great question was whether the removal was to be by the President alone or with the concurrence of the Senate, both constituting the appointing power.  No one denied the power of the President and Senate jointly to remove where the tenure of the office was not fixed by the Constitution, which was a full recognition of the principle that the power of removal was incident to the power of appointment; but it was very early adopted as a practical construction of the Constitution that this power was vested in the President alone, and such would appear to have been the legislative construction of the Constitution, for in the organization of the three great Departments of State, War, and Treasury, in the year 1789, provision was made for the appointment of a subordinate officer by the head of the Department, who should have charge of the records, books, and papers appertaining to the office when the head of the Department should be removed from office by the President of the United States.  When the Navy Department was established, in the year 1798, provision was made for the charge and custody of the books, records, and documents of the Department in case of vacancy in the office of Secretary by removal or otherwise.  It is not here said “by removal of the President,” as is done with respect to the heads of the other Departments, yet there can be no doubt that he holds his office with the same tenure as the other Secretaries and is removable by the President.  The change of phraseology arose, probably, from its having become the settled and well-understood construction of the Constitution that the power of removal was vested in the President alone in such cases, although the appointment of the officer is by the President and Senate. (13 Peters, p. 139.)

Our most distinguished and accepted commentators upon the Constitution concur in the construction thus early given by Congress, and thus sanctioned by the Supreme Court.  After a full analysis of the Congressional debate to which I have referred, Mr. Justice Story comes to this conclusion: 

After a most animated discussion, the vote finally taken in the House of Representatives was affirmative of the power of removal in the President, without any cooperation of the Senate, by the vote of 34 members against 20.  In the Senate the clause in the bill affirming the power was carried by the casting vote of the Vice-President.  That the final decision of this question so made was greatly influenced by the exalted character of the President then in office was asserted at the time