Suppose that a meteor strikes the Capitol building in the middle of the night. No one is close enough to be hurt, but the building is destroyed and totally unusable. Where would Congress hold its votes as a result of the Capitol being unavailable?
Anywhere Congress can agree to.
From Article I, Section 5 of the Constitution:
Neither House, during the session of Congress, shall, without the consent of the other, adjourn for more than three days, nor to any other place than that in which the two Houses shall be sitting.
In United States v. Ballin, Article I, Section 5 was further interpreted to meaning that:
[Congress] may not by its rules ignore constitutional restraints or violate fundamental rights, and there should be a reasonable relation between the mode or method of proceeding established by the rule and the result which is sought to be attained. But within these limitations all matters of method are open to the determination of the house [...] The power to make rules is not one which once exercised is exhausted. It is a continuous power, always subject to be exercised by the house, and within the limitations suggested, absolute and beyond the challenge of any other body or tribunal.
This sets forth that Congress can meet wherever it pleases and there's nothing anyone else can do about it. By tradition it has always happened at The Capitol and no one in their right mind would suggest meeting anywhere else, sans meteor.
The U.S. Capitol was burned when the British captured Washington, D.C. for a few hours during the War of 1812. It was ultimately Washington, D.C. businessmen that promoted and financed the construction of a temporary brick structure where Congress could meet while the U.S. Capitol was rebuilt called the Old Brick Capitol (at the same site where the current-day U.S. Supreme Court now stands). They were afraid that they would lose business and influence if the capital would move to the interior of the country and were keen on keeping Congress in the district.
That worked then, I imagine in your scenario there could be widespread destruction throughout the district so staying there to conduct the business of the country probably wouldn't be feasible. In reality what I believe would happen in a catastrophic scenario (or even non-catastrophic scenario where only the U.S. Capitol building is affected) is the Speaker, the Vice President as leader of the Senate, and the majority and minority leaders from both houses would come to some consensus on what makes the most sense, and all other (or at least a quorum of) members would fall in line. They may go so far as to have a quick session with enough people on the steps of the remains of the building (or where the steps used to be) to simply adjourn to "somewhere else" to do business, but I don't think it would be legally necessary.