The original Constitution made no reference to self evident "rights". That was the Declaration of Independence:
We hold these truths to be self-evident, that all men are created equal, that they are endowed by their Creator with certain unalienable Rights, that among these are Life, Liberty and the pursuit of Happiness. — That to secure these rights, Governments are instituted among Men, deriving their just powers from the consent of the governed,...
The constitution, unamended, makes little reference to "rights", in fact the only reference to rights is the power of Congress to pass bills
To promote the Progress of Science and useful Arts, by securing for limited Times to Authors and Inventors the exclusive Right to their respective Writings and Discoveries;
The original constitution did not spell out other "rights". It was assumed that these were "endowed by the creator" and so a document written by man couldn't limit what the rights were.
It was felt later that an enumeration of some of the rights would be useful. But still the notion was that "rights" are from God, and that is a higher authority even than the constitution.
The rights retained are "everything else". But the writers of the constitution thought that these were defined by God, and not by man and deliberately chose not to list them. You have the right do anything, except that which is forbidden.