Lots of good answers already but one basic fact needs to be highlighted: EU fisheries (and not only French fisheries) depend on access to British waters. It's that simple and it's the problem with “agree[ing] to only fish in their territorial waters”.
You can make a lot of arguments based on sovereignty, morality, historical precedents, quota/sales purchases, ressource management, fish migration, tie fishing to market access for fish, the small size of the sector (for both sides), compare fisheries with other areas of negotiation, haggle over the duration of the transition period, argue that 25% is a large gain compared to the statu quo or that 60% is well short of the 100% the UK is entitled to, etc. etc. etc. but at the end of the day retreating to the EU's territorial waters (or its EEZ) will create a huge disruption and devastate an entire industry.
From the EU perspective, losing access is a problem (not an obvious solution) and any (sovereign) country would use whatever leverage they have to prevent that or expect significant benefits in return from abandoning that leverage (and that's withouttrue even before getting into the symbolic and political weight fishermen can have in some countries).