Richardson v. Ramirez, a 1974 Supreme Court case, held that:
California, in disenfranchising convicted felons who have completed
their sentences and paroles, does not violate the Equal Protection
Clause.
One reason cited was the Fourteenth Amendment, which says in part (emphasis mine):
But when the right to vote at any election for the choice of electors for President and Vice President of the United States, Representatives in Congress, the Executive and Judicial officers of a State, or the members of the Legislature thereof, is denied to any of the male inhabitants of such State, being twenty-one years of age, and citizens of the United States, or in any way abridged, except for participation in rebellion, or other crime, the basis of representation therein shall be reduced in the proportion which the number of such male citizens shall bear to the whole number of male citizens twenty-one years of age in such State.
The amendment clearly contemplates (without outright saying) that the right to vote can be taken away from someone who commits a crime.
A 2016 case from the Third Circuit (which confusingly has been titled Lynch v. Binderup, Sessions v. Binderup, and Binderup v. Attorney General, due to changes in which side was appealing and who the current Attorney General was) found that a person could challenge the as-applied constitutionality of the federal law mandating the loss of their gun rights (and also held that it was indeed unconstitutional as applied to this person.) In 2017 the Supreme Court declined to hear the appeal, leaving the ruling as precedent in that circuit but apparently leaving a circuit split.