As outlined in Why did CJ Roberts apply the Fourteenth Amendment to Harvard, a private school?, the affirmative action only applies to schools that receive federal funding. But the federal funds seem like a drop in the bucket for Harvard:
With a $40.9 Billion Endowment, Harvard Gets $22 Million in Federal Work Study Funding. Harvard received $22.2 million from 2016 until present from the U.S. Department of Education's Federal Work Study Program.
So... why not just refuse future federal funding and continue to run your admissions program as you please? Is there a political reason for not doing so?