If the Supreme Court decided that same-sex marriage was a right on the federal level, what would happen if a specific state decided not to follow their ruling?
What power does the federal government have over states?
I don't know if this is the worst case confrontation, but I suppose it ranks pretty high; following Brown v. Board of Education there were numerous attempts to resist, some solved with troops and some after legal battles forced the states' own courts to intervene:
In September 1957, Arkansas governor Orval Faubus called out the Arkansas Army National Guard to block the entry of nine black students, later known as the "Little Rock Nine", after the desegregation of Little Rock Central High School. President Dwight D. Eisenhower responded by asserting federal control over the Arkansas National Guard and deploying troops from the U.S. Army's 101st Airborne Division stationed at Fort Campbell to ensure the black students could safely register for and attend classes. [...]
In June 1963, Alabama governor George Wallace personally blocked the door to the University of Alabama's Foster Auditorium to prevent the enrollment of two black students in what became known as the "Stand in the Schoolhouse Door" incident. Wallace sought to uphold his "segregation now, segregation tomorrow, segregation forever" promise he had given in his 1963 inaugural address. Wallace moved aside only when confronted by General Henry V. Graham of the Alabama National Guard, whom President John F. Kennedy had ordered to intervene. [...]
Virginia had one of the companion cases in Brown, involving the Prince Edward County schools. Significant opposition to the Brown verdict included U.S. Senator Harry F. Byrd, who led the Byrd Organization and promised a strategy of Massive Resistance. [...] in 1956, a special session of the Virginia legislature adopted a legislative package which allowed the governor to simply close all schools under desegregation orders from federal courts. In early 1958, newly elected Governor J. Lindsay Almond closed public schools in Charlottesville, Norfolk, and Warren County rather than comply with desegregation orders, leaving 10,000 children without schools despite efforts of various parent groups. However, he reconsidered when on the Lee-Jackson state holiday, the Virginia Supreme Court ruled the closures violated the state constitution, and a panel of federal judges ruled they violated the U.S. Constitution.
And yeah, one holdout county (Prince Edward) kept all their public schools closed until 1964, basically 10 years after Brown. (Private) "segregation academies" were ruled unconstitutional in 1976, but in the meantime (1954-1976) they provided a legal way to circumvent Brown.
Basically, if one were to generalize from the above, given a determined federal executive, it seems the only way to ultimately avoid federal troops enforcing a ruling is to make it non-applicable by making the locus of the dispute vanish/nil. (Both Eisenhower and Kennedy invoked the Insurrection Act of 1807 for their use of troops in the events described above.) The resistance was far less massive following Obergefell, but it involved a number of counties following the latter strategy, i.e. not issuing any marriage licenses, essentially being "closed for business" in that regard.
The current wording of the Insurrection Act provision (which has been amended a few times since initial adoption), according to Wikipedia:
Whenever the President considers that unlawful obstructions, combinations, or assemblages, or rebellion against the authority of the United States, make it impracticable to enforce the laws of the United States in any State by the ordinary course of judicial proceedings, he may call into Federal service such of the militia of any State, and use such of the armed forces, as he considers necessary to enforce those laws or to suppress the rebellion.
It provides a statutory exception to the otherwise broad prohibition against using the US armed forces "as a posse comitatus, or otherwise, for the purpose of executing the laws".
The President can take control of a state's national guard and enforce martial law in extreme cases. This happened a lot with desegregation of schools.
A less drastic option is finding the governor in contempt of court, but this is largely a symbolic gesture for a head of state. Fines could be applied, though making them apply to the governor and not the state would likely be difficult. Trying to actually arrest a governor supported by their state police would probably be ugly.
The President could try to use existing laws to make not following the decision extremely inconvenient. Congress passing additional laws could make this more effective, but carries a lot of the challenges as contempt of court.
Whether this is about Politics or Law, doesn't the Answer remain the same: something between nothing and a new civil war, depending on whether enough people with enough power care enough?
Well inside those extremes are the possibilities that indeed the National Guard - or the regular Military - could be sent in, that new laws might be enacted or both.
Again, it depends on which people have power, or inclination or both.
Broadly the law and the constitution are what they are, but they can be enforced only to the extent people have both the will and the ability to show them as they "should" be seen…
We don't have to guess. The remedy applied was the usual one, of having a court hold the individual in charge of obeying the U.S. Supreme Court ruling, usually the lowest official with the authority to act, in contempt of court for not doing so until compliance is achieved.
The U.S. Supreme Court on Monday rejected an appeal from Kim Davis, the former Kentucky county clerk who gained national attention five years ago when she cited her religious beliefs in refusing to issue marriage licenses to same-sex couples.
Although the court was apparently unanimous in refusing to hear her appeal, two of the conservative justices said the 2015 ruling making same-sex marriage the law of the land amounted to a "cavalier treatment of religion." Davis "may have been one of the first victims" of the decision, "but she will not be the last," wrote Clarence Thomas for himself and Samuel Alito.
After the marriage ruling, gay couples sued Davis for refusing to issue them marriage licenses, and she was briefly jailed for contempt when she continued to refuse after a court ordered her to grant them. Kentucky later changed state law so that marriage licenses were no longer issued in the name of the county clerk.