To appear on the ballot paper, the political party must be registered with the Electoral Commission. They are an independent statutory body, tasked with running elections.
You have to submit a lot of details, like your party constitution, aims, and "identity marks" (ie names and logos). You cannot register a name that is likely to cause confusion. "Libel Democrats" would not be permitted.
The rules for identity marks (emphasis mine):
As a guide, we cannot register an identity mark that:
- is likely to mislead voters as to the effect of their vote
- is the same as another registered identity mark which is on
the same register
- is the same as an identity mark from a deregistered party
which was on the same register and is protected until the
end of that party’s financial year in the year they were
- is likely to result in voters confusing it with another party
identity mark that is already registered or protected
- is likely to contradict or hinder instructions or guidance given
- is obscene or offensive
- contains certain prohibited words
- is, or contains an acronym or abbreviation that is not well
known and widely used and not spelt out
- links in any way to online material or contains reference to
- contains a reference to a person’s name unless it is a
person directly associated with your party
- an emblem that contains text which cannot be read at the
size emblems appear on ballot papers (2cm square)
- is longer than six words
- is not in Roman script
- is likely to amount to an offence if published.
No doubt this has been tried, which is why there are rules to prevent it.
The other rules prevent, for example, a pro-EU party registering "UK Out" (misleading as to the effect). You couldn't register "Tick this box ->" (hinders guidance for voting). Nor could you register "Kick Theresa May out" (refers to a person not associated with your party).
In other words, your name should be a clear reflection of the positive aims of your party, and is likely to be rejected if it is not.