A suspect is read their Miranda rights to protect any immediate statements they might make so those statements are admissible in criminal proceeding — In the US, citizens are protected under the Fifth Amendment against being compelled to incriminate themselves of a crime.
The "public safety exemption" says that, in very specific circumstances, some statements given prior to Miranda warnings can be admissible into evidence if the situation posed an immediate and great danger to public safety.
This is seen often in murder investigations where a suspect is found at the scene of a shooting. If the weapon is not immediately located, it is generally admissible for the police to ask "where's the gun!?" without fear of acquittal due to self-incrimination. The need for spontaneity overrides the desire to protect those who might make "legally unadvised statements."
Of course, the risk of asking such questions (without advising them of their right to remain silent) is after the fact — the courts will have to determine if the public safety exemptions were indeed warranted before admitting the evidence. There's also a risk that the admission was "compelled" beyond someone's reasonable will to withhold such information.
If a suspect's rights are sufficiently violated, the authorities might not be able to hold them at all. And if the courts decide that there wasn't an imminent and substantial public safety issue, the courts will be forced to throw out anything said during his arrest. The problem becomes that any "fruits from the poisonous tree" (i.e. any secondary evidence obtained directly from what he said) can lead to evidence being ruled inadmissible, making it more likely the defendant being found not guilty or even summarily acquitted of all charges.
