Who owns the debt?
People and organizations who want to own what has historically been a rather stable investment for most countries. This also differs somewhat from country to country.
In Japan, the second largest owners of debt are citizens in the form of postal savings. The largest is the central bank.
In the United States, debt owed to the public (as per the balance):
- Foreign - $6.281 trillion
- Federal Reserve - $2.463 trillion
- Mutual funds - $1.379 trillion
- State and local government, including their pension funds - $874 billion
- Private pension funds - $544 billion
- Banks - $570 billion
- Insurance companies - $304 billion
- U.S. savings bonds - $169 billion
Foreign debt may be owed to countries. For example, China and Japan own large stakes so as to be able to keep their currencies lower than they otherwise would be. In some cases, it may also represent money held by foreign agents, rich individuals or countries.
Another significant holding is intragovernmental savings. For example, Social Security owns about $2 trillion of US government debt.
Money after taxes
Where does the government get the money they spend after they've spent all the taxes they collected? Does it just magically appear because the government prints another billion dollars?
It could, but it usually doesn't. A typical government obtains money beyond taxes by selling government debt. It creates a bond and sells it. The buyer could be the central bank, an individual, an organization, or even another government. If no one wants to buy, then it has to change the price until someone does.
A bond is basically a promise to pay an amount of money at some future time. The cheaper the current price, the better the return (called the yield). So the government might sell a bond for $95 with the intent of paying $100 a year from now.
Most countries manage their money supply through a central bank. The central bank is usually tasked at expanding the money supply at a rate that keeps inflation low without starving transactions due to a lack of money to process them. If they miss on the low end, this creates what is called deflation. Along with a fall in prices, production falls as there simply isn't enough money to go around. If they miss on the high end, they get inflation. High enough inflation also causes production to fall. Falls in production are associated with job loss and increasing unemployment.
Because central banks are usually independent and arcane, they aren't as driven by political needs as governments. Most people don't know what they would want the central bank to do, so there is less pressure on them to take short term actions.