One of the main features of authoritarianism is that the government's strength is only dependent on a small amount of people. The smaller the amount of people the government relies on to keep its power, the better, because they are more susceptible to influence such as bribery.
Take, for example, the case of Bell, California, which voted to become a charter city in a 2005 election. The turnout was fewer than 400 votes (of which half are suspected to be dubiously obtained), even though the city's population numbered in the tens of thousands. Becoming a charter city removed the limit on city officials' salaries; in 2010, the corruption was exposed, with officials being paid hundreds of thousands every year.
By voting, you help make the democratic system more representative and less susceptible to unfair influence.