As other answers have pointed out, the host country can just declare the ambassador a persona non grata and they would legally be forced to leave the country.
"Storming the embassy" by Nigerien troops to expel them would be a violation of the embassy and a (further) diplomatic incident. Technically, cutting off the embassy from supplies will also cause an incident.
The ambassador staying in the embassy also is a violation of the treaty, however. So we reach an impasse.
In the case of Niger, the issue is that the current "government" is considered illegitimate by France. Hence, they lack the authority to expel the French ambassador. If France were to withdraw their ambassador, they could be seen as legitimizing the coup. So, for the time being, this is not an option (until the costs become too high, and another excuse can be used).
Addendum: usually, withdrawing the ambassador would be seen as delegitimizing the host country and a sign of cooling relations. But that only counts, if the sending country does it of their own volition. So, France stays put and waits.