tl;dr It depends upon what you mean by "didn't pay I to the system." The non-partisan Government Accountibility Office (GAO) has released several reports, and there seems to be a wide range ($0 to as few as 18 months) of how much you have to pay into FICA to begin collecting Social Security checks.
As a government employee, you might pay as little as $0. In the GAO 2010 report, SOCIAL SECURITY ADMINISTRATION
Management Oversight Needed to Ensure Accurate Treatment of State and Local Government Employees noted:
At present, SSA and IRS managers do not know the extent to which wages are reported accurately or to which Social Security taxes are paid in accordance with program rules
State and federal officials found that, over decades, [...] in hundreds of districts, [...] some teachers were incorrectly paying Social Security taxes when they were not eligible to receive Social Security coverage. Others were not paying Social Security taxes although they were covered by Social Security.
Since IRS did not collect any back taxes from audited school districts, SSA and IRS officials told us that the U.S. Treasury and Social Security Trust Funds would effectively bear the cost of any long-standing coverage errors and FICA taxes that school districts and employees did not pay.
You might argue that those we're clerical errors, and shouldn't count, just because the mistakes occurred to long ago for the IRS to correct the error. There are other cases of government employees working only a single day, paying as little as $3 in FICA and collecting thousands in annual social security checks, fully following the law.
In 2004, Congress closed the loophole with legislation requiring public employees be employed for five years in a job covered by Social Security in order to be eligible for full spousal benefits.
The GAO report, SOCIAL SECURITY
Congress Should Consider Revising the Government Pension Offset "Loophole" notes:
Social Security’s Government Pension Offset (GPO) exemption [...] prevents workers from receiving a full Social Security spousal benefit on top of a pension earned from government employment not covered by Social Security. However, the law provides an exemption from the GPO if an individual’s last day of state/local government employment is in a position that is covered by both Social Security and their state/local pension system. In these cases, the GPO will not apply, and Social Security spousal benefits will not be reduced> In 2002, one-fourth (or 3,521) of all Texas public education retirees took advantage of this exemption.
In most schools, teachers typically worked a single day [...] the Social Security contributions deducted from their pay would total about $3 for the day. We estimate that the average annual spousal benefit resulting from these last-day transfers would be about $5,200.
So, before 2004 you could work as little a one day, and your spouse would be entitled to full spousal benefits. After 2004, as a government employee you only need to work five years instead of the typical ten years.
Illegal aliens are also entiltled to Social Security benefits, even if they weren't working in the U.S. legally at the time. In fact, the Totalizaion agreement with Mexico encourages them to do so, because they are entilted to benefits from working as few as 18 months, to as much as 9.75 years, even under a stolen identity. The 2003 GAO report SOCIAL SECURITY -
Proposed Totalization Agreement with Mexico Presents Unique Challenges notes:
SSA provided no information showing that it assessed the reliability of Mexican earnings data and the internal controls used to ensure the integrity of information that SSA will rely on to pay social security benefits.
Mexican workers [...] could qualify for partial social security benefits with as few as 6 coverage credits. In addition, under the proposed agreement, more family members of covered Mexican workers would become newly entitled
Under the Social Security Act, all earnings from covered employment in the United States count towards earning social security benefits, regardless of the lawful presence of the worker, his or her citizenship status, or country of residence.