It's mainly because there will always be people who do not have credit cards and still uses cash, for example, 96% in Sweden in the article that you cited. 4% is still a relatively huge number.
This article by BBC addresses Sweden, noting that not everyone welcomes the move:
Like the Netherlands and its Scandinavian neighbours, Sweden is among the front-runners in the race to eradicate cash. But not everyone is welcoming.
One of the reasons the article mentioned is that many people are reluctant to give up using cash, including Germany, as they believe that "cash gives them better control over their spending".
The Guardian mentioned that some are not quite ready for this drastic change, so it will still take sometime to convince those who still prefer cash. And it's also a political decision and thus it will need to be popular with Swedes before the government can abandon cash.
Old people’s organisations also fear that those who prefer cash, out of a reluctance to use new technology or simply because they find it easier to keep track of their spending, will be disadvantaged, while educators worry that young people will be tempted to spend money they do not have.
For these and other social reasons, Arvidsson said, cash is not dead quite yet. “Even if, in the next few years, Swedes use almost no cash at all, going 100% cashless needs a political decision,” he said.“The idea of cash, even in Sweden, remains very strong.”
(emphasis mine)
Totally abandoning cash will also result in problems with interest rates, since one could no longer keep cash at home and have to always be subjected to the bank's interest rate, unless governments introduce reforms regulating this but it will take time.
A central bank by all means can adopt a negative rate, charging retail banks to hold their surplus funds on deposit ... But in their hordes, customers would up and leave, withdrawing their cash and locking it up at home. Unless they no longer could.
Unless there were no longer notes and coins to withdraw, and they were confined to a digital ecosystem, free to move their money between banks but not away from them.
(emphasis mine)
Some other challenges include privacy concerns, since electronic payments leave a trace and can be possibly tracked back by companies. So, without cash, anonymous payments will not be possible.
This is something most of us have taken for granted as a huge advantage of the old Greenback. It is the most private, and direct way to transmit value. If you use a debit card, Bitcoin, or a wire transfer, you are creating a massive paper trail that can compromise your identity or security, both now and in the future. Some are just now realizing how important these benefits are today.
(emphasis mine)