Question 1: Would the law pass in this scenario?
No. Passing requires a majority of the chamber, not of the quorum. In other words, it requires a majority of all possible votes, not just of the votes cast (or the valid votes cast).
You actually already mentioned this in your question without realizing it: this is exactly the same reasoning as for abstention.
Question 2: Are there historical precedents where some states voted invalid in the Bundesrat and this allowed the passage of a law against the wishes of one political party?
As per the answer to your question 1, it is not possible to pass a law this way. So, I will focus on the first half of the question.
There are, as of now, only two cases of a "split vote" of a Bundesland in the Bundesrat. The first happened in 1949 and was a simple mistake that got resolved immediately.
The second one is the (in)famous case of the vote concerning the Zuwanderungsgesetz (immigration law) from March, 22nd, 2002. It is infamous because it is widely regarded as an extremely undignified moment in the history of (what is supposed to be) the more polite, deliberative, collaborative of the two Houses, where both of the leading parties (ab)used the floor for political theater complete with choreographed fake outrage and a staged walkout.
The bill was introduced by the Red-Green Coalition government of SDP and Greens in the Bundestag and passed with the government's majority against fierce opposition especially by the CDU. Because the bill affected administrative procedure, it needed confirmation in the Bundesrat.
The Bundesrat was almost evenly split between SDP-led and CDU-led States with a very slim majority for the SPD. It was clear that all CDU-led States would vote No and all SPD-led States would vote Yes. The deciding vote came down to the 4 votes of the State of Brandenburg, which was governed by a Red-Black Coalition of SDP as the senior partner and CDU, under SPD prime minister Manfred Stolpe. The coalition agreement between the two parties states that in case the parties do not agree on a vote in the Bundesrat, Brandenburg will Abstain.
However, Manfred Stolpe announced internally that he intends to vote Yes and that he would make use of his privilege as the Prime Minister to fire any Minister who opposes him more than once.
When it came to the vote of Brandenburg, President of the Bundesrat Klaus Wowereit (SPD) polled the State of Brandenburg for their vote and one of the other SPD Ministers answered "Yes" after which CDU Minister Jörg Schönbohm immediately interjected "No".
The CDU's intent with Jörg Schönbohms interjection was to create an invalid, split vote. The SPD's intent with not having the leader of the State answer first, was that now President of the Bundesrat Wowereit could ask the State's leader, SDP Prime Minister Manfred Stolpe, for clarification (which he did after citing the relevant section of the Grundgesetz and instructing Brandenburg that the votes of one State can only be delivered unanimously), who stated "The State of Brandenburg votes Yes", to which Jörg Schönbohm amended "Mr. President, you know my opinion".
Schönbohm couldn't answer "No" a second time if he wanted to keep his job. In the eyes of the CDU, his vague statement was enough to consider the vote invalid, whereas in the eyes of the SPD, since he did not clearly object (which would have gotten him fired), he agreed and thus the vote was not split, since Stolpe had clarified the answer.
President of the Bundesrat Wowereit concluded "I hereby record that the State of Brandenburg has voted Yes". This resulted in an immediate reaction from most CDU ministers in the session, most notably Roland Koch, CDU Prime Minister of Hessia and one of the leaders of the CDU, which was widely criticized as an over-the-top theatrical performance, complete with banging his fist on the desk a whopping 25 times.
Wowereit then offered "I can ask the State of Brandenburg if they need to clarify some matters" and then addressed again Prime Minister Stolpe, who re-affirmed his "Yes", this time without any interjection from Schönbohm (who would risk his job if he interjected a third time).
At that time, Wowereit announced that he considered Brandenburg having declared its vote and continued polling the other States, which resulted in the bill passing with the narrowest of margins.
The "theater" was not over, though: the CDU Prime Ministers and Ministers walked out of the session in protest, and outside of the Bundesrat, the CDU's youth organization organized a "spontaneous protest", where protesters "spontaneously" showed up with thousands of flyers commenting on what had happened only a few minutes earlier.
Both parties were heavily criticized by the media, other parties, as well as distinguished senior members of their own parties for essentially turning the highest, most somber, legislative body of Germany into a stage for an amateur theater production. In addition, both parties accused each other of the same.
Here is a TV clip from about a week after the session, explaining what was going on and showing excerpts of the session.
In the end, the law was struck down by the Bundesverfassungsgericht (Federal Constitutional Court, Germany's Supreme Court), based on the majority opinion of the court that:
- There was no affirmative vote by the State of Brandenburg because the 4 votes were not unanimous.
- While the President of the Bundesrat is allowed to seek clarification on unclear votes, he can only do so by asking the State as a whole again for its vote, but not by polling individual representatives of the State, effectively giving them the opportunity to override other representatives.
- Even if one were to allow polling of individual representatives, he would at least need to also ask Jörg Schönbohm to clarify his vote instead of taking his silence as an affirmation. There is no requirement for Schönbohm to re-state his earlier answer unprompted – as long as he does not actively change the answer, it stands.
The official website you linked to actually mentions the Supreme Court ruling:
Aus dieser Konzeption des Grundgesetzes für den Bundesrat folgt – so hat das Bundesverfassungsgericht 2002 entschieden – dass der Abgabe der Stimmen durch einen Stimmführer jederzeit durch ein anderes Bundesratsmitglied desselben Landes widersprochen werden kann und damit die Voraussetzungen der Stimmführerschaft insgesamt entfallen.
After some additional rounds through the Bundestag, the Bundesrat (where in the meantime there was a clear CDU majority and thus the bill was rejected without drama), the Reconciliation Committee, a subcommittee, and finally the Bundestag again, the bill was signed into Law by Bundespräsident Horst Köhler, who again criticized both parties, over two years later.