How would the people in US, especially those who most like to pillory Iran for not being docile and compliant, react if a bunch of foreigners, led by Iranians, wanted to come into the US and, completely at their own whims, go to any facility, unannounced, to poke around and inspect them? The howls of outrage would be heard half way across the globe, which means they'd be heard all around the globe, I guess.
Issues of sovereignty, security and prestige/image would make any country at least partially resistant to that kind of intrusion. That's why so many found it notable that Iran agreed to a level of inspections and deference to an inspection regime that has never been achieved before.
Debunking the Myths... - Center for Arms Control and Non-proliferation
Downvoters - please read what I actually say in my answer. I'm not even stating that Iran DOES oppose inspections. I'm pointing out that some wrangling or desire to mitigate access is a normal reaction from any nation.