The original AfD formed back in what was known as the debt crisis when conservative economists criticised the ruling coalition in general but Merkel in particular for 'saving Greece' and others; their criticism essentially implied that German money was being used to pay the Greek national debt which they strongly objected to. On this platform, they were elected to the European Parliament in 2014. Starting around this time with the rise of Pegida, anti-immigration voiced in the party became stronger which culminated in the original party splitting in 2015. With the arrival of larger numbers of refugees in late 2015, the AfD became more and more associated with far-right, nationalist positions such as exiting the EU, closing the borders, sending back refugees and not accepting any more, etc. Importantly, they once again put the blame for what they saw as a terrible situation for Germany squarely on Merkel.
Thus, until at least this election the AfD would under no circumstances have accepted any cooperation with the party of Merkel, although they hinted numerous times that they would welcome cooperation with a more conservative, de-Merkelised CDU. This is especially reflected in the hunting comment that Johannes_K cites in his answer.
In addition, the other parties saw the direction the AfD was taking and drew up a 'firewall' against it as correctly highlighted by Fizz. When the parties after the election are saying that they will 'discuss with all democratic parties to determine whether we can form a government', the word democratic serves to explicitly exclude the AfD from the list of options. What happens if they aren't careful can be seen in Thuringia as outlined by AuronTLG.
An episode from one of the recent pre-election TV debates of the chancellor candidates highlights this. It is necessary to mention, that the CDU has also excluded cooperating or forming a coalition with the Left Party (die Linke). This tidbit should surprise nobody, as they are on opposite ends of the spectrum of parties in parliament. Nevertheless, they found it important to explicitly rule out cooperation with said party during a party conference in 2018 (source in German) – the same decision also ruled out cooperations with the AfD. During the debate, Laschet, the CDU's chancellor candidate, was asked what his stance on Left Party and AfD was and whether he considered them equivalent. His answer was as follows:
Ich sage Ihnen. Wir werden mit [der Linken] nicht koalieren. [...] Ich sage der AfD. Mit ihnen kooperieren wir nicht, verhandeln wir nicht und werden nie koalieren. Wir tun alles, dass Sie nicht mehr in deutschen Parlamenten vertreten sind.
My translation:
I'm telling you. We will not form a coalition with [the Left Party]. [...] I'm telling the AfD. We will not cooperate, we will not negotiate and we will never form a coalition with you. We will do everything to ensure you will no longer be present in German parliaments.
And this is really the core of the issue: at least for the forseeable future the CDU sees the AfD as a party that should not even be in parliament, let alone anywhere near the government benches. Therefore, any coalition that would include both CDU and AfD is practically impossible.
Finally, and sort of as an addendum: Franz Josef Strauß, former Minister President of Bavaria and long-time leader of the CSU, the CDU's sister party which is only active in Bavaria, is often quoted as having said: 'Only the wall can be to the right of the CSU!' Whether he uttered that phrase exactly like that I don't know, but he uttered something in a very similar vein in 1987:
... Ich habe erklärt – im Übrigen in vollem Einvernehmen mit Helmut Kohl, der sich ja genau zu dieser Formulierung bekannt hat, die hundertmal mit mir besprochen hat –, dass es rechts von der CDU/CSU keine demokratisch legitimierte Partei geben darf. Wir denken hier nicht, natürlich, an rechtsradikale Narren, mit denen wir gar nichts zu tun haben wollen, aber an normale, demokratische, konservative Kräfte, die bei uns ihre politische Heimat behalten müssen.
My translation:
I have declared – by the way, in full agreement with Helmut Kohl, who affirmed exactly this phrasing which he discussed with me hundreds of times – that there must not be a democratically legitimised party to the right of the CDU/CSU. Of course, we are not talking about extreme-right fools with whom we want nothing to do, but about normal, democratic, conservative forces that should continue finding their political home with us.
One may debate how accurate a statement from the late 80's may be (at the time, the extreme-right Republikaner (Republicans) were one the rise, made it into a couple of state parliaments but never into the Bundestag), but to the best of my knowledge it has never formally been retracted and so the view of the CDU/CSU should be that the AfD falls under these 'extreme-right fools'.
