Because it's hard to do, prone to failure and very costly. And because the newer alternatives aren't always a big improvement.
Very costly in the sense that failure costs in the multiple 100M$ are frequent and could easily become an election issue (which makes it a political issue).
Take a look, for example, at Canada's revamped payroll system. $1B+ and counting, IBM was the lead on that. Or look at Capita's activities in the UK and its litany of project failures. Or, 70M$ for a failed asset management system in Australia, for a population base of 250 000 people .
Even with the best of intentions, we in the IT industry do not yet have a proven, repeatable, low-risk solution to migrating old government IT systems. Nor is it all that clear what the next good-for-60-years technology solution is. Until that is the case, there will be a strong case for keeping seemingly obsolete, but working, systems running as long as possible.
Bad project management in public sector
It is very difficult to reduce risks when doing this kind of work.
One option is to do a strict migration from the old system to the new system, with it doing the exact same thing as before. This is very rarely done, usually all sorts of new bells and whistles are attached which makes it very hard to judge if the new system does indeed provide the same results as the old. Often, under the guise of the rewrite, the existing business processes and organization are revamped, on the basis of the promises made by the promoters of the new system.
Governments IT is often driven by consensus and stakeholders. Give too many people a say in what a complex system should do and what should be fairly straightforward degenerates into endless meetings and shifting requirements. Worse, rather than picking a representative sample of senior front-line users of the existing system, the stakeholders are usually their managers. Often, retraining of, and engagement with, front-line users takes a backseat to more glamorous techy and PM stuff.
Many government departments also consider themselves special and do not like off-the-shelf existing solutions. That ranges from not buying a commercial product but writing it from scratch to acquiring an off-the-shelf product but then customizing it all out of recognition.
Even when intentionally trying to keep things simple there are inherent complexities to government IT. A payroll system will have to keep track of multiple union agreements and seniority scales, for example. Not all payroll vendors are able to do this without custom development.
Unclear and shifting requirements, and to a lesser extent, incompetence by the outsourcing contractors is what kills most government IT projects.
Big IT projects require a Mr./Ms. No. The successful projects I have participated in all usually circled back to the project director having the authority and willingness to tell the different stakeholders to buzz off, but only when necessary. Their driving principle is to keep things simple, a single-minded focus on reducing risk and to get a solid working basic system out as quickly as possible. That person needs to be backed at the highest level. It's rare to see that recognized, and doubly so in a public sector setting.
Limited choice of, not always good, contractors
Public sector IT relies heavily on outsourcing to big, high-profile, companies. IBM, Capita, Accenture, etc... (in the distant past, when governments were a major driver in early IT a lot of this work was done in-house).
The government consulting companies are often more skilled in navigating the complexities of government procurement and contracts than they are at actually producing working products. However, when a project fails, the immediate government reaction is to be more careful the next time and select a trustworthy, credible, vendor.
Small companies, even assuming they somehow knew better than the big consulting giants, don't have that credibility, or the commercial skills to win a bid, and the next big project tends to be awarded to yet again the IBMs and Capitas of this world, even while these companies outsource more and more to lower cost countries like India.
The next line of defense is to write longer and longer specifications documents, with every contingency covered. That 1.2B$ Canadian payroll project had a 6000 page spec at some point. Specs cost, but are not working code at the end of the day. No one is going to read 6000 pages of tech specs, making it very hard for any one person to fully understand the project.
A primary point of effort for a consultancy (their core competency to cite a comment) then becomes managing the spec and limiting their contractual risks, rather than getting things done. At the opposite end, the much vaunted spec-less methods like Agile are woefully inadequate in these contexts.
COBOL is 60 yrs old, about as unhip as you can get and is suffering from a shrinking pool of talent. On the other hand, it is very stable, you can take 30 yr old code, run it in a modern environment and expect it to work and it also quite clear to read. It's there and it works.
The primary alternative would be Java. 25 yrs old, no shortage of talent. But... there is much less version-to-version stability, the codebases are rarely very clear and there is a strong culture of overcomplexity in its practitioners. Java is also quite allergic to non-Java stuff, its perfect world is one in which everything is also written in Java. Unfortunately, that is not the way the real world works.
Java's shortcomings are being recognized and it's start to have the whiff of "legacy" as well. In the last 5-10 years not many would have picked Java as a technology stack for complex new websites for example, unless they are already invested in that technology.
However, there are no other real contenders for massive-scale, government-level, IT development languages.
Such a language would need to be:
- predictably available 30-50 years from now
- secure by design, plays well with others, not overly verbose
- suitable for line-of-business apps (i.e. processing health claims, rather than doing high-performance 3D graphics)
- clear and with low initial complexity for junior programmers (John Carmack ain't gonna be working at NJ's DMV)
- appropriate to projects in which tens or hundreds of developers work together on a tightly integrated system
COBOL comes close to this ideal - its main weakness are scarcity of skilled labor and verbosity. Java less so IMHO and I can't think of another widely adopted language that combines these attributes. Simple modern compiled languages like Swift, Go (C#?) might be adapted/adopted for this, but that's not their current aim and they lack vendor neutrality.
Why does this matter to politics?
Because software matters. Banks have sometimes been described as IT companies specialized in customer service and money management. A modern government relies on IT in order to deliver all sorts of services and to collect taxes. Large scale IT failures have a budgetary impact, crowding out spending in other areas and can directly affect how citizens interact with their government. And they become talking points during election cycles.
On a related subject, the inability of local government IT, for schools, hospitals or cities, to manage and secure their systems have made them prey to ransomware attacks, which can very directly affect thousands of citizens.
In order to function well, governments are going to have to improve considerably in this field.