Background
Impeachment may be conducted by congress for the following reasons outlined in the U.S. Constitution:
Article I, Section 2, Clause 5 of the United States Constitution provides:
The House of Representatives shall choose their Speaker and other Officers; and shall have the sole Power of Impeachment.
Article I, Section 3, Clauses 6 and 7 provide:
The Senate shall have the sole Power to try all Impeachments. When sitting for that Purpose, they shall be on Oath or Affirmation. When the President of the United States is tried, the Chief Justice shall preside: And no Person shall be convicted without the Concurrence of two-thirds of the Members present. Judgment in Cases of Impeachment shall not extend further than to removal from Office, and disqualification to hold and enjoy any Office of honor, Trust or Profit under the United States; but the Party convicted shall nevertheless be liable and subject to Indictment, Trial, Judgment and Punishment, according to Law.
Article II, Section 2 provides:
[The President] ... shall have power to grant reprieves and pardons for offenses against the United States, except in cases of impeachment.
Article II, Section 4 provides:
The President, Vice President and all civil Officers of the United States, shall be removed from Office on Impeachment for, and Conviction of, Treason, Bribery, or other high Crimes and Misdemeanors.
The first two, Treason and Bribery parallel crimes in the legal system (I say parallel because an impeachment charge by the House of Representatives is different from a Grand Jury indictment) and could easily draw upon the legal system for specific details as to what qualifies treason and bribery. However, "high crimes and misdemeanors" is morphic term from what I understand, and thus the potential is wide open for a federal officer to be removed from office for something that isn't necessarily a crime just because both Houses of Congress voted enough to say it is. If that was the case, then, given enough votes, both houses of congress could use impeachment as a political cudgel to oust anyone they so chose based off the loose definition of high crimes and misdemeanors.
Now, as a matter of practicality, the occurrences of impeachment in the United States are few and far between and in recent memory only the impeachments of the President have proven to be controversial (contrived or not). However, it is this potential toxic side effect - that if someone acting in their official duty happens to go against the politics of the day - lead me to ask the following question...
Question
Given that what qualifies as a high crime or misdemeanor is left to the legislature to define, why allow both houses of congress to remove someone from office with what may potentially not be a crime?