Both the House of Representatives and the Senate rule themselves according to the rules they adopt - typically at the beginning of each session. Article I, Section 5 of the Constitution lays this out, stating:
Each House shall be the Judge of the Elections, Returns and Qualifications of its own Members, and a Majority of each shall constitute a Quorum to do Business; but a smaller Number may adjourn from day to day, and may be authorized to compel the Attendance of absent Members, in such Manner, and under such Penalties as each House may provide.
Each House may determine the Rules of its Proceedings, punish its Members for disorderly Behaviour, and, with the Concurrence of two thirds, expel a Member.
Each House shall keep a Journal of its Proceedings, and from time to time publish the same, excepting such Parts as may in their Judgment require Secrecy; and the Yeas and Nays of the Members of either House on any question shall, at the Desire of one fifth of those Present, be entered on the Journal.
Neither House, during the Session of Congress, shall, without the Consent of the other, adjourn for more than three days, nor to any other Place than that in which the two Houses shall be sitting
In a nutshell, this says that each chamber needs to adopt its own rules. And, at the beginning of each session, they do. The rules of the House for example, are here. The first order of business is to elect a speaker, the second is to adopt the rules. The rules are usually written by the majority (although with the knowledge that one day, the tables will be turned), and agreed to by the house. A simple majority is all that is required.
Again, the real check on these rules is mostly the axiom that "turnabout is fair play." In the Senate, for example, Filibusters are controversial, in that they allow the minority party to stop the will of the majority. And, when overplayed, you will often hear about the nuclear option - the proposal to overturn the filibuster. But there is one thing that keeps it from being approved - the knowledge that eventually the majority party will be in the minority some day. And when that happens, they don't want to lose that power.
Specifically in regards to who can call for a vote, however - that's long been the perogative of the Speaker. Indeed, by controlling the calendar, the Speaker has the ability to influence legislation far more than if he actually voted on it. Sam Rayburn (a Democrat Speaker in the 50s and 60s), in particular, held tight control over the calendar in order to ensure his priorities were the priorities of the House.