You didn't clarify as to whether you meant "break" in the sense of "This rule expressly prohibits this, but we did it anyway" or in the sense of "This rule is still on the books, but it no longer has any meaning". For clarity's sake, I will refer to the first as violating
the rule and the second is invalidating
the rule.
The senate didn't violate its own rules, so much as it bypassed them by invoking other rules to invalidate the rule in question. The ability of a simple majority to change the need for a supermajority is why it's called the "nuclear option": a procedural maneuver with potentially serious consequences, to be used as a last resort to overcome political opposition.
1
The rule in question is part of Senate Rule XXII:
And if [the cloture question] shall be decided in the affirmative by three-fifths of the Senators duly chosen and sworn -- except on a measure or motion to amend the Senate rules, in which case the necessary affirmative vote shall be two-thirds of the Senators present and voting -- then said measure, motion, or other matter pending before the Senate, or the unfinished business, shall be the unfinished business to the exclusion of all other business until disposed of.
Since rule changes can be filibustered the same as any other vote, cloture can be required on them. The bolded section means that cloture on rule changes needs the full 2/3 supermajority, rather than the lesser 3/5 one. This is an attempt to protect the rules from constant editing. However, it's important to realize that this only affects cloture votes on rule changes. It does not affect votes on rule changes themselves (if no one filibusters and thus requires cloture), nor does it explicitly prevent alternative means of altering the rules which don't require cloture.
That last is what the "nuclear option" entails. The mechanics of the process are discussed on the Wikipedia page:
A point of order is a parliamentary motion used to remind the body of its written rules and established precedents, usually when a particular rule or precedent is not being followed. When a senator raises a point of order, the presiding officer of the Senate immediately rules on the validity of the point of order, but this ruling may be appealed and reversed by the whole Senate. Ordinarily, a point of order compels the Senate to follow its rules and precedents; however, the Senate may choose to vote down the point of order. When this occurs, a new precedent is established, and the old rule or precedent no longer governs Senate procedure. Similarly, it is possible to raise a point of order and state that the standard procedure of the Senate is actually different from what the current rules and precedents suggest. If this point of order is sustained, a new precedent is established, and it controls Senate procedure thenceforth.
This is how it played out. Specifically:
- Senator Reid raised a point of order. ref 1
- The point of order was ruled invalid by the chairman, because it wasn't the rule. ref 1
- Reid appealed the ruling to the full senate. ref 1
- The senate as a whole voted 48-53 to fail to uphold the chairman's ruling. ref 2
- The chairman (as required) declared that this introduced new precedent such that that Reid's point of order was now the rule to use going forward. ref 2
- Senator McConnell raised a point of order that the existing rules prevented exactly this. ref 2
- The senate as a whole voted 52-48 to sustain the chair's ruling that new precedent was set.ref 2
Thus, rule XXII was invalidated by introducing a precedent for changing it which doesn't violate it. Whether this counts as "breaking" the rule or not is up to you. However, if the rule were actually violated then the change would be invalid, the same way that a senator casting a voice vote of "Cookies" wouldn't count as a valid vote. If the Senate continued operating under violated rules, it would be theoretically permissible for the Executive branch to ignore them (due to the constitutional violation of Each House may determine the Rules of its Proceedings
), and would potentially spark a constitutional crisis.