The government is in fact concerned about it, the problem in large part is that those elected to government positions to make fiscal policy have diverging interests from one another, and their collective inability in an age of hyper-partisanship to find compromise leads to a whip lash effect that only serves to exacerbate the problem.
One of the two main political parties is not willing to budge when it comes to spending on entitlements, and the other isn't willing to budge on increasing tax revenue. It is indeed a strategy of many American Conservatives to "Starve the Beast," that is to attempt to force capitulation of some sort on entitlements through denying funding. At the same time, it is also an absolute imperative for Progressives to ensure that government programs that people have paid in to for the majority of their working lives are still around for them in a meaningful way when it's their turn to take advantage of them.
Given that most politicans would rather enjoy the fruits of their political labor by keeping the Federal Government around for a while, the only give and take it seems possible between the two parties when negotiating spending bills seems to be leaving both entitlements and tax rates largely untouched. Even when negotiations aren't strictly necessary during single party rule, the party in charge can't seem to find a feasible path towards backtracking on the other party's successes during their tenure, looking instead to enact successes of their own (see: Obamacare, Trump's Tax Cuts). This is even when the backlash to the other party's success could arguably be the major contributing factor to such single party rule in the first place. Because of this, what winds up happening is that spending is generally expected to increase during a Democratic majority, followed by decreasing tax rates during a Republican majority, neither of which ever get reversed.
Where does the government get this money from?
Generally by selling US Treasuries, aka "Treasury Bills", "T-Bills", and US Savings Bonds. These are government-backed securities that are basically IOUs from the government, you give them your money and they promise to pay you back at a later date with a set amount of interest. These are also bought and sold between private parties on the open market after initially being purchased directly from the government. "Treasuries" are generally only referred to when bought from the Federal Government, but states and municipalities also sell things referred to as "bonds" ("municipal bonds" aka "muni bonds" are ways local governments sometimes raise money for large infrastructure projects). If you wish to loan the U.S. Government some of your money (or just learn more about it), you can do so here.
Won't such a tremendous rise in debt lead to future interest payments skyrocketing?
"Skyrocketing" is a bit of a loaded term, but see this quote from the CBO's Budget and Economic Outlook report from May 2022.
Beyond 2032, if current laws remained generally unchanged, deficits would continue to grow relative to the size of the economy over the following 20 years, keeping debt measured as a percentage of GDP on an upward trajectory throughout that period. Those large budget deficits would arise because outlays—particularly for interest on federal debt and for Medicare—would grow steadily under current law, and revenues would not keep pace with those outlays.
The CBO always assumes that "current laws remain generally unchanged" in their estimates. Because of some of the reasons given in @Trilarion's answer, it is entirely impossible to tell the future. There's also always the chance for Black Swan events to occur, such as a worldwide pandemic or some form of easy-to-use fusion technology discovery that could dramatically alter anyone's forecast in either direction.
