Mostly the (Decennial) Census
Generally, the electoral formula is Electoral Votes = Representatives + Senators. Each state has two Senators, so that's 100 of the 538 total Electoral Votes.
Washington DC gets three electors; while they technically have shadow congresspeople, those don't get to vote in the House or Senate. In keeping with historical values of "No Taxation Without Representation", they receive electors so they have some measure of representation. 100 Senators + 3 DC Electors = 103 so far
Every ten years, the US conducts its census to determine, among other things, the population of each state. Each state receives Representatives proportional to its fraction of the total US population, with a minimum of one representative per state and a total of 435.
For example, California had a population of 37.37 million in 2010 out of a total US population of 309.3 million.
37.37e6 (CA pop) / 309.3e6 (US pop) = x (CA Reps) / 435 (US Reps). Solving for x rounds to 53 seats, which is the current number of California Representatives in the House.
435 Representatives + 100 Senators + 3 DC Electors = 538 Electoral Votes
Though this is slightly outside the scope of your question, this is part of the reason the census has recently been politicized: an undercount in (often specific) urban areas would lead to lesser representation in traditionally blue areas, giving Republicans greater proportional representation.