The only instance I found in an admittedly quick internet search of a such an event was Edward VIII of England, but he was given the Dukedom of Windsor and became a Duke quickly after, and I couldn't find what his title in between was.
In the Netherlands the monarch may abdicate by signing an instrument of abdication. That transfers some titles held by the previous monarch to their successor. According to Wikipedia, when then Queen Beatrix abdicated in favor of her son who is now King Willem-Alexander:
a number of titles previously held by Queen Beatrix (excluding those of Princess of Lippe-Biesterfeld and Princess of Orange-Nassau) were bestowed upon Willem-Alexander.
The two titles which she did not pass on to her son were titles she obtained from birth. According to her Wikipedia page:
From birth till her inauguration as queen she had the following name and titles, to which she reverted after her abdication: Her Royal Highness Princess Beatrix of the Netherlands, Princess of Orange-Nassau, Princess of Lippe-Biesterfeld.
If the current King Willem-Alexander abdicates then he will revert to the titles he obtained at birth, specifically according to his Wikipedia page:
From birth, Willem-Alexander has held the titles Prince of the Netherlands (Dutch: Prins der Nederlanden), Prince of Orange-Nassau (Dutch: Prins van Oranje-Nassau), and Jonkheer of Amsberg (Dutch: Jonkheer van Amsberg).
The title of Prince or Princess of Orange-Nassau for the departing monarch (as well as the ruling monarch and the presumed successor) is also regulated by law in article 9(1) of the Law Membership Royal House, which states (in Dutch):
De Koning, diens vermoedelijke opvolger en de Koning die afstand van het koningschap heeft gedaan dragen de titel «Prins (Prinses) van Oranje-Nassau».
The King, their presumed successor and the King who has abdicated bears the title «Prince (Princess) of Orange-Nassau».
The Dutch Wikipedia page on the King of the Netherlands says some more about the title (should one wonder why the law article above doesn't mention the female version Queen):
De Koning kan zowel een man als een vrouw zijn. Als de "Koning" een vrouw is krijgt deze de aanspreektitel Koningin (met grondwettelijke taak).
Trying to translate it literally, that becomes:
The King can be a man or a woman. If the "King" is a woman they receive the title Queen (with constitutional task).
Nitpick: there are no "medieval style monarchies" in Europe. Some monarchies may claim a lineage to medieval institutions, but that is not the same. And AFAIK, none is directly descending from those (except perhaps some Scandinavian ones), with most having had republican governments lapses.
I think the style will depend wildly of the country, and even personal preferences.
Juan Carlos of Spain abdicated a few years ago, but the protocol was that he kept the treatment of "King". So the actual king - with all of its rights and duties - is Felipe, but you should address both as king.
The media routinely refers to him as "king emeritus" (rey emérito).
When Queen Beatrix of the Netherlands abdicated, her new title became
Her Royal Highness Princess Beatrix of the Netherlands, Princess of Orange-Nassau, Princess of Lippe-Biesterfeld
which is the same title she held before her coronation.
In the case of Belgium where King Albert abdicated in 2013, he kept his title of King, but no longer King of the Belgians. Just a minor semantic change:
After his abdication on 21 July 2013 it was decided that he would be styled as His Majesty King Albert II,the same form of address granted to his father, Leopold III, after his abdication.
Source As the quote indicates, it was a decision made at that time because there was nothing forseen. They could have decided otherwise had they wished so.
A monarch who wants to go into retirement and let their heir take over usually does not resign officially. They retain their titles and simply delegate all their decisions and duties to their heir.
There is actually an European monarchy where this is the case right now: Liechtenstein. Which also happens to be the only remaining European monarchy where the monarch is still making decisions in everyday politics. The official head of state is Fürst Hans-Adam II, Prince of Liechtenstein. However, he retired and leaves all of his duties to his son Alois, Hereditary Prince and Regent of Liechtenstein, Count of Rietberg. Hans-Adam II is still "officially" the monarch of Liechtenstein and retains his title as Fürst, even though "de facto" the country is ruled by his son.
By the way: There is a similar arrangement in another still existing monarchy outside of Europe: Saudi-Arabia. The official king is Salman bin Abdulaziz, but the de-facto ruler is his son Mohammed bin Salman. The king retains all his titles even though the son already wields the authority those titles bestow.
There are no consistently established rules regarding this
The European constitutional monarchies have their roots in the hereditary dictatorships of the medieval era, and to the European monarchs of that era the idea of voluntarily surrendering their power prematurely would have been almost unimaginable. With real power on the line, there would have been no truce to be had between the kings. In cases such as the abdication of Edward II where he was forced to abdicate in favour of his son, Edward II died eight months later and is widely believed to have been murdered on the orders of the new regime.
Because an aged monarch stepping down in favour of their offspring (not necessarily a son any more) is a relatively modern invention, there are no well established rules and traditions to follow and different occurrences are being handled differently.
From British Titles (1951) by Valentine Heywood:
I was at that time responsible for the news columns of The Sunday Times, and we had to decide in what way we were to refer to the ex-King, a point on which there was then no official guidance. The view I then expressed was that as all the pereage titles he had held formerly had been extinguished by his succession to the Throne, the KIng reverted to the state in which he had been born, i.e. His Royal Highness and a Prince of Great Britain, styles which had not been created for him personally but which were his right by reason both of his being the son of one Sovereign or the grandson in the male line of another.
That, in fact, was the style accorded him when, on the occasion of his farewell broadcast, Sir John Reith introduced him to the world as “His Royal Highness Prince Edward”. I think it can be taken for granted that the then Director-General of the British Broadcasting Corporation was not speaking without having sought advice and received authority.
Later, of course, he was made Duke, and also reappointed to the orders of knights that he no longer headed.