The Anti-Masonic political movement (based upon my knowledge of its history while working with primary sources related to the history of my undergraduate college was founded by people who belonged to it), was afraid of a secret oligarchy by a self-selected elite whose religious proclivities (cryto-Egyptian rituals and ecumenical religious tolerance that allowed deists to paticipate) were heterodox at best. This was something that raised a specter of amoral, pagan, or even Satanic influence in the minds of their fearful and often religiously affiliated opponents. The Masons were also seen as globalists who did not put American interests first.
They saw Masons as a nefarious threat to democracy and to virtuous ordinary people in general, who put the interests of people we would describe today as "one percenters" and foreign interests ahead of the interests of ordinary people.
Basically, the anti-Masons were conspiracy theorists, before conspiracy theories were cliche or objects of derision. And, they weren't entirely wrong. At the time, the Masons did have members who made up a remarkably large share of highly influential elites, and they were more religiously tolerant, more secular and more globalist in orientation than their political opponents.
Generally speaking, the Anti-Masons were deeply suspicious of all secret societies, which had become a feature of elite universities already, and not just the Masons.
For example, Anti-Masons at Oberlin College, early in its history, expelled all of the members of its baseball and football teams, which had previously been very competitive, because they had formed a secret fraternity, even though it wasn't particularly a Masonic one.
