This answer focuses on the why aspect of the question, rather than the what aspect focused on in other answers.
Why Ask About Race And Ethnicity?
The U.S. Census, first and foremost is a constitutionally mandated part of the process of deciding how many representatives in the U.S. House and how many electoral votes each U.S. state gets, which is called "reapportionment."
Originally, there were three relevant categories: free people, slaves, and Indians not taxed. Each of these categories was almost perfectly correlated with race in the pre-civil war era.
With the adoption of the 13th, 14th and 15th Amendments to the U.S. Constitution after the U.S. Civil War, the distinction between free people and slaves was no longer relevant. In 1924, the category of people counting as "Indians not taxed" was abolished.
But since the census was used for redistricting within states, as well as reapportionment among states, it was helpful to have a better sense of who lived in particular places for the purpose of having information for drawing of Congressional and other political districts, and for other policy making purposes.
Historically, race and ethnicity were desirable things to ask about for those purposes because the political and cultural behavior of people has been closely associated with their historic ancestry, represented at a more general level by the concept of race, and at a more specific level by the concept of ethnicity. Race was conceived of as a biological concept, but the reason people cared about it was because it had a tight correlation with culture and political attitudes, not because census takers really cared about how people looked or people's genetics.
Also, race was a legally meaningful concept in the era of segregation from 1865-1954 when, after the Civil War, and prior to Brown v. Board of Education, a legal regime of racial segregation was allowed and was widely practiced in the American South, and before 1924 because some Indians had different political rights than non-Indians.
After this regime of legally enforced racial segregation call "Jim Crow" was struck down, data on race and ethnicity was still relevant to gauging the extent to which civil rights laws and court rulings enacted to full implement the dismantlement of the Jim Crow regime were working.
This is where the nation was in 1970, when the landmark Civil Rights Act and Voting Rights Act were both less than a decade old.
In that context, the Hispanic category became a matter of interest, first of all, because "national origin" was a protected class under the Civil Rights Act and Voting Rights Act, and also because failure to consider Hispanic individuals as a distinct group injected "noise" into data about discrimination against African-American former slaves in Census data, that officials wanted to minimize.
General Considerations In Classifying People By Race And Ethnicity
The challenge for someone designing a census is to gain as much information about the people studied as possible with a minimum of simple, clear questions. With regard to race and ethnicity, this means one has to define categories that have political and cultural relevance but are easy to evaluate in individual cases with few gray area edge cases, although some edge cases will always be inevitable.
To be easily used, the categories adopted have to be understandable by people filling out census forms, which means that they have to correspond to categories that make sense and are widely understood to the people using them in their cultural context.
There is always an arbitrary line that has to be drawn between being too specific and being too general.
The 1970 Addition Of The Hispanic Ethnicity Category
In this case, the concern was that the racial designations "white" and "black" were each including some people who had little in common culturally with people historically classified by the census of white (descendants of European immigrants and colonists) and black (descendants of former slaves), and had more in common with each other (a shared Latin American culture derived mostly from Spanish culture). So, the census sought to be more specific somehow to address this issue.
It also didn't help that people in this emerging group of people didn't assign the same meaning to terms like "white" and "black" that people in the U.S. usually gave to those terms, and that they also had other categories that the census had not previously considered because it hadn't needed to do so since the number of people involved had been modest in places where it was politically relevant to make the distinction.
The census invented the Hispanic designation as an ethnicity in an effort, primarily, to distinguish a culturally cohesive Latin American community which was an increasingly large subset of the total U.S. population due to growing migration from Latin America (the U.S. population had an all time low 4.7% percentage of its population that was foreign born in 1970, a third of the current percentage).
Historic Racial Classification In Latin America
Doing so was difficult because Latin Americans themselves in their home countries categorized themselves different in terms of "race" (a term which historically and at the time was intended to refer to one's predominant broad genetic ancestry although in 1997, it was redefined based upon self-identification).
Historically, the main racial categories in Latin America have been indigenous, "mestizo" (who were admixed individuals with indigenous South American and Iberian ancestry, with a larger share of indigenous than Iberian ancestry in most cases), black (meaning of sub-Saharan African origin), mulatto (meaning someone with a black parent and an ancestrally European parent), various finer gradations for people with one black grandparent, one black great-grandparent, and so on (such as "quadroons" and "octaroons"), and white (people of European origin, who were predominantly Iberian in origins, in practice, until the 20th century in Latin America).
Other, more obscure categories were sometimes recognized for someone with African and indigenous ancestry, or with African, indigenous and European ancestry.
If you go to the Denver Art Museum you will see many sets of multiple oil paintings in each set, with each set having one painting depicting each of these racial categories in a setting representing the assigned social standing of the people in that category, basically as an educational effort by which Latin American Colonial regimes defined their racial categories.
The many mixes of peoples of different ancestries in Latin America arose because the Iberian colonists of Latin America, in a process that started earlier than North American Anglophone colonization, were mostly male dominated "Conquistadors" who took local wives out of necessity, rather than bringing wives with them from Iberia.
Historic Racial Classification In Anglophone North America
In contrast, in Anglophone North America, the "one drop rule" formally considered anyone with any African ancestry to be black, and the colonists largely replaced rather than admixed with existing indigenous populations.
The reality that there were some mixed race people with African origins (since essentially all blacks were also distinguished as slaves, and interracial sexual relationships were eventually made illegal as "miscegenation") was not acknowledged.
Likewise, there was not a sufficient number of people with European-Native American ancestry, or black-Native American ancestry, that was publicly acknowledged to necessitate a socially recognized racial category for them. If they looked more Native American, or they were in a Native American tribal community, they were considered Native American. If they could pass for white and were integrated into white society, they were considered white.
Children of slaves who had a Native American parent were still considered "black" under the "one drop rule" and were still slaves. Often, black-Native American admixture arose when Native Americans were enslaved by someone who also had black slaves. But, for reasons beyond the scope of this answer, early efforts to enslave significant numbers of Native Americans largely failed (not for want of trying) and had largely been forgotten by the time the first U.S. Census was conducted. While many U.S. African-Americans have low levels of Native American ancestry, the pairings that led to that admixture were already so remote in 1790 and beyond in most cases, that they no longer had any legal, cultural, or political relevance to their descendants.
Asians were so vanishingly rare in early North America and South America alike that there were almost no people with significant recent Asian admixture in either Anglophone North America or in Latin America at the time that the foundations of the racial classifications that would prevail in the respective cultures were emerging in these places. So, neither Latin America nor Anglophone North America has terms to describe mixed race people with Asian origins.
Francophone racial categories were more similar to those of the Spanish colonies and Brazil, than to Anglophone North America (particularly in Louisiana prior to it joining the United States). But in what would become Quebec, replacement of existing populations by gender balanced, endogamous colonists was the norm, and slavery wasn't significantly practiced, so the issue of new racial categories other than whites and indigenous people didn't really come up. In the Southern part of the Louisiana territory, Francophone attitudes towards race were rather promptly replaced by those of the United States to which it was annexed (there are a some good literary accounts of that transition, for what it is worth), to the great detriment of free people of color in what is now Louisiana.
Racial Classification In Modern Latin America And The U.S. Compared
But, in modern Latin America, racial categories for people with 1/4 or less African ancestry, and with mixed indigenous and African ancestry (with or without European ancestry) have largely been abandoned. People who have low levels of African ancestry and little indigenous ancestry, even if they would not quite "pass" as white in the U.S. often self-identify and are considered "white" by Latin American standards. People with more African ancestry, who are not fully African, tend to identify or be categorized either as mulatto or black, even if they have some significant indigenous ancestry.
The stereotypical Hispanic phenotype, which is predominant in Mexico, the largest source of U.S. Hispanic immigration, is mestizo, a racial category that didn't exist in the U.S. in 1970 and is the main source of people who identify as an "other race". But the racial mix in different countries varies considerably even among Latin American countries. Some, like Argentina and Cuba have many people who are overwhelmingly European in ancestry. Other countries have fewer, and/or have more people with some African ancestry relative to mestizo individuals, as a share of the population, than Mexico does.
In modern Latin America, many people who identify as "white" have a small amount of sub-Saharan African admixture to a degree that would be quite unusual among U.S. people who identify as "white". The threshold of "passing" for white is less strict in Latin America than in the U.S.
But, an average person who identifies as "black" in Latin America would have less sub-Saharan African ancestry than an average person who identifies as "black" in the U.S.
Why Ask About A Hispanic Ethnicity?
The census adopted the Hispanic categorization to distinguish this demographically and sociologically coherent group of people from other people who identify as black, white or Asian or Native American or Native Alaskan or Native Pacific Islander in the U.S., thinking that most people in that category would secondarily have a white or black racial identity, and ignoring the lack of a mestizo category (perhaps thinking that mestizo people would identify as Native American, which they usually do not).
Hispanic people of all races share a language and a culture distinct from U.S. African-Americans (almost all having African slave ancestors with modest amounts of white admixture), from people considered "white" in the U.S. with almost pure European ancestry (although low levels of African ancestry and Native American ancestry, often unacknowledged, are common in Southern whites in the U.S.), from people considered "Asian" in the U.S., and from people considered to be Native American, considered to be Alaska Natives, or considered to be Pacific Natives, in the U.S.
Treating Hispanic as an ethnicity rather than a race was also a convenient choice because the census already had an ethnicity question, primarily used to identify the place of geographic origin identity of European-Americans (e.g. German, Italian, Irish, English). So, it did not require a major restructuring of census forms to implement.
The Formal Definition
As noted in the answer by @reirab:
According to the Census Bureau:
The U.S. Office of Management and Budget (OMB) requires federal
agencies to use a minimum of two ethnicities in collecting and
reporting data: Hispanic or Latino and Not Hispanic or Latino. OMB
defines "Hispanic or Latino" as a person of Cuban, Mexican, Puerto
Rican, South or Central American, or other Spanish culture or origin
regardless of race.
The Census Bureau also reiterates this definition as well as
explicitly listing out countries on this page, which is essentially a
list of all Spanish-speaking countries in Central and South America.
Note that Brazil is absent, as their primary language is Portugese,
not Spanish. Thus, they are Latino, but not Hispanic.
As formally defined, Hispanic includes people from Spanish speaking Latin America and from Spain, but not people from Portugal and not people from Brazil (which are linguistically Portuguese) although many Brazilians, in practice, are classified as Hispanic by people not familiar with this nuance of the definition, and many people from Spain identify as non-Hispanic contrary to the census definition.
It Hasn't Been A Perfect Solution
The Hispanic ethnicity question in racial and ethnic classification in the U.S. has been mixed success. But it has persisted from 1970 to the present out of inertia and out of a lack of a better solution.
It has grown particular problematic as the U.S. national conversation on race and ethnicity has grown less simplistic and the demographic make up of the U.S. has grown more varied and complex.
Lack of clarity in definitions for race and ethnicity in the U.S. census and for other purposes based upon it, among the people who have to apply them to Hispanic individuals in the U.S. have caused statistics involving the terms to have irregularities due to a failure of people to internalized the census definitions. This has also led to an "other race" category that is confusing, particularly now that the census has adopted a "more than one race" option which many Hispanic individuals utilize. Another answer from @reirab spells out the state of the confusion as of the 2000 census:
As of the 2000 Census . . . almost half of the people who
identified as Hispanic identified as "white only" for their race, with
the next largest group marking "some other" single race. Approximately
2% marked black only, approximately 1% native American only, and
around 6.3% marked two or more races, with the vast majority of those
marking "some other" + one of white, black, or native American.
This diversity of answers doesn't meaningfully reflect actually differences in ancestry or culture between people answering those questions in 2000.
The share of Hispanic Americans who identified themselves as having more than one race soared in the 2010 census to make up a large share of the total.
The Hispanic classification has probably outlived its usefulness in its current form. The Census bureau is aware of the problem and has considered alternatives. But there isn't a clear consensus around an alternative, and continuing to use the same formal definition makes current data back compatible with old data collected using the same definition.
A footnote on the U.S. Census "Asian" racial classification v. Hispanic ethnicity
If they consider "Asian" a race which has a similar but even broader
geographic meaning to it, then why not?
As I noted earlier, there is always an arbitrary line that has to be drawn between being too specific and being too general.
This line regarding generality has more to do with culture and practical percentages of the overall population than it does with any genetically meaningful measure of ancestry. This is generally speaking, a good thing. If we used genetic similarity as a threshold for drawing categories, there would be more subcategories within Finland alone, which has a lot of genetic substructure, than within all of Northern Europe, which is genetically quite homogeneous.
Asians were lumped into one category because at the time that the definition was drawn up, Asians made up a very small part of the U.S. population (in part due to the Chinese Exclusion Acts), and almost all Asians who did live in the U.S. were East Asians who lived in the Western U.S.
Also, white basically refers to the European continent, black basically refers to the African continent, Native American basically refers to the North American continent (even though it could arguably apply to both North and South America), and so referring to people from the Asian continent as Asian isn't superficially all that much more broad, except for the fact that Asia is a much bigger and more diverse continent than the other examples. (Presumably Australian aboriginal people, one of the excluded continents, would be members of an "other race" but don't get their own box to check because so few people in the U.S. would use it.)
The Census actually did have a category called "Hindu" at one time. When it existed, it referred to South Asians regardless of religion. But it was later dropped from the census, presumably because so few people on a percentage basis in the U.S. belonged in that category at the time it was dropped.
In the meantime, the census has continued the extraordinarily broad category of Asian because a second ethnicity question clarifies where within the broader Asian category someone falls, because the percentage of people who are Asian is still quite small in most U.S. states, and because unlike the Hispanic category, the Asian category doesn't actually overlap with other categories.
Also, despite their incredible cultural and linguistic diversity, in the U.S., our racial and ethnic realities have had the practical effect of having these diverse peoples choose to unite under a Pan-Asian banner for political purposes and for purposes of voluntary civic organizations (e.g. on college campuses and as interest groups within professional organizations) despite their diversity, particularly because many of them share similar and similarly timed immigration stories. So, there is not been much political pressure from people classified as Asian for census purposes to be more specific.
Even more recently, the "check more than one race" option and very high rates of exogamy among second generation (and later generation) Asian-American immigrants (often close to 50% or more), has made a focus on a specific sub-Asian ethnic identity less emotionally charged for many of the people affected who often see themselves as multi-racial or post-racial, in any case.
Equally important, while South Asians, Southeast Asians, East Asians and North Asians may be very culturally and politically different from each other, none of those classifications overlap meaningfully with people of European and North African ancestry who are called "white", people of African ancestry who are called "black", Native Americans, Alaska Natives, Pacific Islanders, or Latin Americans.
In contrast, the Hispanic category encompasses people who can also legitimately be classified in the "black", "white", "Native American" and even in rare cases "Asian" racial categories under the definitions of those categories.
So, while the Asian category is indeed an arbitrary line that is drawn very broadly, it is not an arbitrary line that overlaps with the other arbitrary lines drawn to make up the categories on the form, and thus, poses a different in kind issue from the Hispanic ethnicity issue.
Footnote on sources
I could and if I have time, I will, provide links to sources and authorities for this answer which I have mostly answered from memory. But, that will have to wait for another day when I have enough time to do so, if that day ever arrives.
