Lies before Congress, including in an impeachment trial, can be penalized.
In a trial of an impeachment in the US Senate, any witnesses give their testimony under oath, and are subject to the same potential penalties for perjury as any witness in any ordinary trial or other court proceeding.
However, the case managers from the house (who are in effect the prosecutors) and the defense counsel, are not under oath. Neither are the individual senators who may comment during debate over an intermediate or final vote. If either could be shown to have knowingly introduced false statements of fact (not of opinion) a charge of Contempt of Congress could in theory be brought. This is similar to Contempt of Court, with a similar range of penalties. Such a charge in such a situation is very unusual. And if such falsity were shown during the trial, it might tend to sway senators to the opposite side of the issue. The House managers could be censured or expelled by a vote of the House for any improper behavior.
Congress has an "inherent power" to imprison for Contempt, which could include perjury before Congress (or a committee) or other improper behavior before Congress or a committee, but it hasn't used this power for about 100 years. (Partly because the courts ruled that it can detain a person only until the end of its session.) It can vote to cite a person for contempt, and does so when it thinks fit. Normally, such citations are transmitted to the DOJ for prosecution. Congress could, however, under current law ask a Federal judge to appoint a special prosecutor to present a perjury or contempt case before a court, if the DOJ did not. It could also pass a law creating a special court to try such accusations.
The May 2019 article "The House’s contempt powers explained" from Constitution Daily says:
Based on precedent, statutes, and court rulings, the House and the Senate each have the power to invoke three types of contempt proceedings ....
The first type of contempt power is a citation of criminal contempt of Congress. This power comes from a statute passed by Congress in 1857. ... the Speaker of the House or Senate President refers the matter to the appropriate U.S. attorney’s office, “whose duty it shall be to bring the matter before the grand jury for its action.” However, the Executive Branch in prior situations has claimed that it has the discretion to decide if a grand jury should be convened to hear the charges. But if the case goes to a grand jury, fines and a jail term could result from the ensuing criminal prosecution.
The second type of contempt power comes in the form of a civil lawsuit brought by the House or Senate, asking a court to enforce a subpoena. The Senate and its committees are authorized to bring such a lawsuit under a federal statute. ...
The third type of contempt power—Congress’s dormant inherent contempt power—is rarely used in modern times. Inherent contempt was the mode employed by Congress to directly enforce contempt rulings under its own constitutional authority until criminal and civil contempt statutes were passed, and it remained in use into the twentieth century. Under inherent contempt proceedings, the House or Senate has its Sergeant-At-Arms, or deputy, take a person into custody for proceedings to be held in Congress.
Although these powers are not directly stated in the Constitution, the Supreme Court has ruled on multiple occasions that they are implicit as an essential legislative power held by Congress.
The CRS Report "Congress’s Contempt Power and the Enforcement of Congressional Subpoenas: Law, History, Practice, and Procedure" (Updated May 12, 2017) says (on page 1 of the report):
First, the long dormant inherent contempt power permits Congress to rely on its own constitutional authority to detain and imprison a contemnor until the individual complies with congressional demands. Second, the criminal contempt statute permits Congress to certify a contempt citation to the executive branch for the criminal prosecution of the contemnor. Finally, Congress may rely on the judicial branch to enforce a congressional subpoena. Under this procedure, Congress may seek a civil judgment from a federal court declaring that the individual in question is legally obligated to comply with the congressional subpoena.
In the case of Anderson v. Dunn, 19 U.S. (6 Wheat.) 204 (1821) The US Supreme Court upheld the inherent power of either house of congress to deal directly with contempt. (Dunn was the House Sargent at Arms, and arrested Anderson by direction of the Speaker. Anderson sued Dunn claiming false arrest and other things.) The CRS report linked above says of this case (on page 7 of the report):
The Supreme Court heard the case in February of 1821 and concluded that the Congress possessed the inherent authority to punish for contempt and dismissed the charges against Mr. Dunn. The Court noted that while the Constitution does not explicitly grant either House of Congress the authority to punish for contempt, except in situations involving its own Members, such a power is necessary for Congress to protect itself. The Court asserted that if the House of Representatives did not possess the power of contempt it would “be exposed to every indignity and interruption, that rudeness, caprice, or even conspiracy, may meditate against it.” (citations omitted)
There is no appellate review of Senate trials, and no mechanism to overturn a result for improper procedure. The presiding officer, or the senate as a whole, can stop anyone who is deemed out of line, and prevent a given course of conduct from continuing. But once the trial has concluded, there is no way to change its result based on the conduct of the people involved, however improper it may appear.
Moreover the senators are protected by the "Speech and Debate clause" (from Article I section 6 of the constitution) which says: "and for any Speech or Debate in either House, they [the Senators and Representatives] shall not be questioned in any other Place." This would prevent any legal action other than a resolution of censure or expulsion in the Senate itself.
