Because such treaties aren't quite as binding as they may seem at first glance, and helpfully include clauses intended to allow a little wiggle room.
First an analogy:
Civil law contracts and agreements drawn up unilaterally often throw in a pile of "gimme" clauses that ask or imply requirements exist for many more things than the signers must actually legally provide. A slight exaggeration:
Rumpelstilzchen Inc. also requires the firstborn child of signer within 10 days of birth via FedEx courier for processing to be sold as unguents... this is a binding contract ... bla bla bla ... These Terms are void where prohibited by law
Of course the firstborn child requirement is prohibited by law and therefore not legally binding, and therefore is void, but every now and then some signer who didn't read far enough innocently provides Rumpelstilzchen Inc. with free feedstock anyway, so it's sometimes profitable for the drafters of contracts to leave all sorts of junk clauses in, followed by an overarching void clause.
End of analogy.
For treaties, there's are similar catch-all clauses roughly equivalent to "void where prohibited by law". For example, the Single Convention on Narcotic Drugs, has the catch-all clause "Subject to its constitutional limitations":
- (a) Subject to its constitutional limitations, each Party shall adopt such measures as will ensure that ... bla bla bla ...
...and anything in that list of provisions conflicts Canada's Constitution is not binding, or is at least subordinate to its Constitution.
And so on for all treaties with similar void clauses...