Controversy cannot be avoided within the realm of politics, so-called "affirmative action" is no exception.
Examining the actual language at the question it is necessary to state the fact that the United States is a racist organization. Why would someone label the U.S. as a racist organization the reader might ask. The answer is that the U.S. government officially promulgates the political construct of "race". Though as the AAA thoroughly vetted above, "race" is entirely a legal fiction, with only a single party benefiting from the fraud: the so-called "white race", the ruling political class in the U.S.
Quoting the language used at the linked news article
US Attorney General Jeff Sessions revoked 24 guidance documents on
Tuesday, many involving race in schools and affirmative action
The construct of "race" was invented specifically by white supremacists for the political advantage of the invented "white race".
AAA (American Anthropological Association) Statement on Race
How people have been accepted and treated within the context of a
given society or culture has a direct impact on how they perform in
that society. The "racial" worldview was invented to assign some
groups to perpetual low status, while others were permitted access to
privilege, power, and wealth. The tragedy in the United States has
been that the policies and practices stemming from this worldview
succeeded all too well in constructing unequal populations among
Europeans, Native Americans, and peoples of African descent. Given
what we know about the capacity of normal humans to achieve and
function within any culture, we conclude that present-day inequalities
between so-called "racial" groups are not consequences of their
biological inheritance but products of historical and contemporary
social, economic, educational, and political circumstances.
Thus, any policy decision based on "race", whether that fictitious "race" be Negro, Black, Asian, Some Other Race (an official "race" in the United States), the ethnicity Hispanic, or White, is a racist policy by its very definition.
From one Negro/Black/African in America perspective, there is nothing that the United States can "Order" or "scrap" as to a policy regarding "race" that can or will ever benefit Negros/Blacks/Africans in America other than reparations, whether in the form of land or direct payment for at least two hundred years of hard labor as prisoners of war within the concentration camps (states) of the British Colonies, United States, and Confederate States of America. This perspective holds true that the U.S. is inherently a white supremacist organization, incapable of change to an organization which does not make each and every decision with the primary interest being the political advantage of the so-called "white race". Negro/Black/African in America were never intended to be and are not currently U.S. citizens, notwithstanding the 14th Amendment to the Constitution of the United States.
From another Negro/Black/African in America perspective, the United States is a neutral organization, capable of being influenced by Negro/Black/African individuals from within and without. The Federal Government is capable of protecting the rights of individuals and groups, given adequate pressure and an appeal to the conscience, religious sympathies and liberal and conservative members of the ruling political class (the "white race").
Thus, even within a single affected class, so-called Negro/Black/African in America, there is sharp division as to ideas of the capacity of the U.S. to actually either not be a white supremacist organization after being founded as such; or whether continued lawsuits, protests, and "integration" into as members or lobbying the political system for redress of grievances of inequality can be a productive path to "justice" and "equality", two of the supposed pursued hallmarks of the U.S. society via political negotiation.
For the former view, so-called "affirmative action" in college admissions is utterly worthless. Negro/Black/African in America educated themselves even while being formal prisoners of war forced to a life of hard labor in concentration camps (states) prior to the conclusion of the American Civil War (1865). Thus, Negro/Black/African in America do not want or need any handouts from the U.S. Government. If the U.S. is to do anything as to redress the descendants of prisoners of war, the U.S. should forthwith set aside land for said descendants of Negro/Black/African in America who were forced under arms to hard labor for life in the Several States; reparations via dollars, provided appropriate accrued interest, would amount to trillions. A politically pessimistic view which does not even address the atrocities performed by the U.S. upon Chinese and Native Americans in the U.S.
For the latter view, the concepts of the Founding Fathers are what must be considered, as the term "race", "white", "negro", "asian", and "some other race" do not appear in the U.S. Constitution.
For Sigma Pi Phi Fraternity also known as the Boulé, who adopt a Greek identity in colleges rather than an African one, neither so-called "affirmative action" nor the former nor latter worldview of Negro/Black/African in America is important, as they had no issue being invited to colleges in the first instance, with guarantees of careers in politics, industry, education and business after completing college, and fundamentally view themselves as Greeks, suckling at the teat of western academia and civilization as "council of nobles advising a king", where the same principles can be found historically among a group of Melkite Coptic Egyptians who, for political power over their own people within the context of the Greek invaders of Ancient Egypt (that is Alexander of Macedonia (332 B.C.E.) where that date is the origin of western civilization though western academia generally recognizes circa 600 C.E. as the beginning of so-called "western civilization") agreed to create the amalgamated creature Serapis for the express purpose of "integration" into the Ancient Egyptian Temple Societies (school systems), whom had rejected the notion of accepting an invading Greek as a deity within their Temple Societies. Serapis later became "Jesus the Christ" after many more years of controversy, see The Historical Origins of Christianity by Walter Williams.. Thus, some Blacks suckled at the teats of western civilization 2,000 years ago, some do so today, some reject the very idea of wanting or needing any "admission" into western civilizations' institutions of higher learning, where such an admission would be to enter the house of the invader. Africans have been educating themselves thousands of years before 332 B.C.E. and do so today, without any reliance on favors from governments, ruling political classes, or asking anyone for anything.
From the view of the Negro/Black/African in America, depending on their worldview, so-called "affirmative action" could be a positive policy which assists in "integrating" into the ruling class society, or "the white race"; an irrelevant scheme which has absolutely no effect on whether Negro/Black/African families in American to even allow their children to attend western academic based schools, or if so, to not rely on them being anything but racist in nature; or, a means to recruit members into the secret societies of Blacks who adopt Greek culture, and who, if their handlers tell them to, shall suppress any non-Greek Negro/Black/Africans in America, for the same reasons the single group of Melkite Coptic Egyptians did 2,000 years ago: individual political kickbacks.
The question asks why there is controversy surrounding so-called affirmative action. The answer is that there is at least two-thousand years of controversy as to the education of a conquered people. In Ancient Egypt there was controversy over the closing of Temples by Ptolemy (western civilization), later a single Temple sought political power by creating the fiction Serapis which became Jesus the Christ. In the current political reality, the policy to "scrap" so-called "affirmative" action is controversial because institutional education is one element of political power, that is, institutions survive generations. The ruling class, for thousands of years, has sought to control the information consumed by the non-ruling classes, and decide when to and not to accept some limited few into their lower ranks.
Brown v. Board of Education, 347 U.S. 483 (1954)
Segregation of white and colored children in public schools has a
detrimental effect upon the colored children. The impact is greater
when it has the sanction of the law, for the policy of separating the
races is usually interpreted as denoting the inferiority of the negro
group. A sense of inferiority affects the motivation of a child to
learn. Segregation with the sanction of law, therefore, has a tendency
to [retard] the educational and mental development of negro children
and to deprive them of some of the benefits they would receive in a
racial[ly] integrated school system.
There was no harmful impact on Negro children being educated by Negro teachers, as old Negro/Black/African people have relayed that that is the best education that they ever received. Black history from a black perspective; not beginning in 332 B.C.E., where Ancient Egypt has a Temple School System that was at least 5,000 years old before the invasion of Alexander of Macedonia. The harmful impact was Negro/Black/African were to be taught western history by non-Negro/Black/African teachers, thus co-opting and controlling their intellect. That is fine with the Boule, who seek to increase their ranks of so-called Black Greeks at colleges, perhaps fine with Negro/Black/Africans in American who believe that the U.S. Constitution in and of itself is a noble document, not just a piece of paper; not fine with Negro/Black/Africans in America who take issue with the fraudulent means by which so-called "science" came to that conclusion; e.g., by excluding and presenting misleading evidence at Court. Negro/Black/Africans in America who were taught by the same did fine without "admission" into "white" colleges, as they do today; for the latter, the U.S. can keep its political trick of "affirmative action".
If the goal of the U.S. Government is to truly bring about Justice and Equality, a start would be to eliminate "race" altogether from any and all federal laws and regulations; including "Negro", "Black", "White", "Some Other Race", "Asian" the ethnicity "Hispanic" or "Latino", etc. Those terms are purely political constructs invented for the express purpose of division, oppression and inequality. There is no political benefit or advantage for any individual to self-identify with a "race" other than to self-identify with the so-called "white race", which is currently the ruling political class in the United States.