This article details many of the 1,138 documented rights and responsibilities associated with marriage in the United States according to the Government Accountability Office (GAO). They run the spectrum from tax incentives, medical decision making and benefits, entitlement program (Social Security, Medicaid, Medicare) eligibility, to family services like adoption and survivor benefits.
As for the rationale, as @DVK notes, there is a wide range that have evolved over time and not all of them are all that rational. However, there is a very good discussion of the various reasons for championing marriage in the law found here.