May 2021 – when this vote was held – was before the most recent general election which took place in September 2021. At this time, the SPD formed a coalition government with the CDU/CSU. You can download the coalition agreement of these parties in German from the Federal Government’s website (unfortunately, I doubt an official English version exists).
Important for the context of the question is section XIV Arbeitsweise der Regierung und Fraktionen (my translation: Mode of Operation of the Government and Parliamentary Groups). As is standard for coalition agreements in Germany, they contain the clause (lines 8261 and following):
Im Bundestag und in allen von ihm beschickten Gremien stimmen die Koalitionsfraktionen einheitlich ab. Das gilt auch für Fragen, die nicht Gegenstand der vereinbarten Politik sind. Wechselnde Mehrheiten sind ausgeschlossen.
The parliamentary groups of the coalition parties will vote uniformly in the Bundestag and all committees it forms. This includes questions which do not form part of the agreed goals. Varying majorities are not permitted.
Running a ctrl-f search for transs (the beginning of transsexuell and Transsexuellengesetz which would definitely be mentioned if this were an agreed-upon question) did not yield any results. User Max, however, pointed out in a comment that: lines 793 and following read:
Wir respektieren geschlechtliche Vielfalt. Alle Menschen sollen unabhängig von ihrer sexuellen Identität frei und sicher leben können – mit gleichen Rechten und Pflichten. Homosexuellen- und Transfeindlichkeit verurteilen wir und wirken jeder Diskriminierung entgegen. Wir werden die Vorgaben des Bundesverfassungsgerichts hierzu umsetzen.
We respect diversity in society. All people should live freely and safely regardless of their sexual identity – with the same rights and duties. We condemn homophobia and transphobia and work against discrimination of all kinds. We will implement the guidelines set out by the Federal Constitutional Court.
The guidelines as mentioned here refer to court decisions that ruled parts of the law unconstitutional; however, this usually permits the remainder of the law to remain on the books. Thus, we can assume that there was no consensus on abolishing the law, only for tweaking it to keep it in line with the Constitutional Court's rulings. As there is no clear wording, the coalition parties would likely have discussed how to vote on this bill separately, when it was introduced.
The specific bill you have linked to was introduced by the group of the Green Party. It was a bill that, if you look at typical party manifestos, the Green Party, Left Party, SPD and FDP would generally find favourable but the CDU/CSU and AfD would generally reject. While to the best of my knowledge nobody on this site was privy to the discussions (and if they were, they would probably not be permitted to disclose the information), we can generally assume that the following arguments played a role in the decision of the coalition parties to reject the bill:
It is very much not a position that the CDU/CSU wishes to support, so they would have had strong objections. As the chancellor was CDU, their opinion tends to have a greater weight in coalition discussions especially if they feel strongly. They probably assumed that it would alienate parts of their voters while not really allowing them to win new voters in the process.
The CDU but more so the CSU has rather strong objections to the Green Party, often decrying everything the Green Party introduces as terrible ideas. This may be in part because the Greens have overtaken the SPD as the major opposition force in the Bavarian state parliament which is the CSU's home turf.
The bill was introduced very shortly before the election (four months). All parties were already more or less in campaigning mode. It would be highly unusual for the ruling coalition to agree to an opposition bill at this time. In fact, at least in the Bavarian state parliament it is common for the CSU to vote against an opposition bill and then a couple of months later introduce their own not-so-very-differently-worded bill that then gets passed with their votes. Had they voted with the Green Party on a Green Party bill, that would have been seen as a very strong attempt to win the Green Party into a post-election coalition which they most certainly wanted to avoid.
Taking a brief look at poll data, the German-language site Wahlrecht.de has compiled an archive of published polls by polling institute including publication date. Choosing the archive data of the Forsa institute, we can see that on the 12th May they published a poll showing the CDU/CSU at 24 %, SPD at 15 % and the Greens at 27 %. At the time of polling, the was a left-wing majority of Greens, SPD and Left Party which would have allowed the candidate of the Green Party (Baerbock) to become chancellor. Simply from an election strategy perspective, permitting the Green bill to pass might have been disastrous.
After the election, there was a majority for the SPD, Greens and FDP – all three of which are socially liberal or even progressive. In their coalition agreement (also available on the Federal Government's website, albeit only in German), they included the sentence on page 119:
Wir werden das Transsexuellengesetz abschaffen und durch ein Selbstbestimmungsgesetz ersetzen.
We will abolish the Transsexuellengesetz (Transsexuals Law) and replace it with a Selbstbestimmungsgesetz (Self-Determination Law).
This corresponds to:
a phrase in the SPD election manifesto (page 44):
Wir wollen, dass trans-, inter- und nicht binäre Menschen im Recht gleich behandelt werden, deshalb werden wir das Transsexuellengesetz reformieren.
We want trans, inter and non-binary people to be treated equally in the law; therefore we will reform the Transsexuellengesetz.
one in the FDP's manifesto (page 34):
Wir Freie Demokraten wollen das Transsexuellengesetz abschaffen und durch ein Selbstbestimmungsgesetz ersetzen.
We Free Democrats want to abolish the Transsexuellengesetz and replace it with a Selbstbestimmungsgesetz.
And the Green Party's (page 193):
Mit einem Selbstbestimmungsgesetz werden wir dafür sorgen, dass das überholte Transsexuellengesetz endlich aufgehoben wird.
We will ensure that the dated Transsexuellengesetz will finally be abolished with the help of a Selbstbestimmungsgesetz.
Basically, all parties forming the current coalition had pledged to do exactly that which was entered into the agreement. The reason why the SPD rejected the proposal in May 2021 was not because they were against it – they could not vote in favour due to the coalition agreement with the CDU/CSU who was actually against it.