The United States (US) only has two parties because of the way that presidential elections work. In general, plurality voting systems lead to two parties. You can read more about why at Duverger's law. The basic idea is that plurality voting forces voters to choose which governing coalition should win the district before voting and the presidency is basically a national district.
In other systems, voters have more ability to choose parties that represent them better. In the US, they have to vote for atypical members of their parties, e.g. Blue Dog Democrats or Main Street Republicans. In many other systems, those would be different parties. The Congressional Black Caucus and Tea Party might also have been separate parties in other countries.
The United Kingdom (UK) can have more than two parties because it is a parliamentary system (although still with plurality voting). So while each district only has one or two serious parties, different districts can have different pairs. Because the country as a whole has multiple parties, it is easier for the pairs to change in a district. This helps avoid the stabilization into a two party system. Note that it is still dominated by two parties. Either Labour or the Tories are always in the governing coalition and have been since the demise of the Liberal party.
In the last general election, the Tories used the promise of a referendum to keep the party from splitting into pro-Brexit and anti-Brexit factions. One can view the pro-Brexit faction as having some similarity to the Tea Party movement in the US. The Scottish National Party has some similarities to the Congressional Black Caucus. It votes almost exclusively with Labour (vs. the Tories), but in the UK, it is a separate party rather than a Labour faction.
Since 1996, New Zealand has had a form of proportional representation. Proportional representation encourages multiple parties as it is easier for small parties to get enough votes for representation. People can choose a party that represents almost all of their beliefs. The parties then choose a governing coalition that makes reasonable compromises. Parties that don't support enough of the governing coalition's agenda join the opposition instead.
New Zealand is not the only country with proportional representation. It's actually rather common. And in general, those countries have multiple parties in both the governing coalition and the opposition.