What I am asking is if a country in the EU, following the rules of the EU could mandate vaccinations amongst their citizens? (the age of citizens might make a difference as some vaccines have not been widely tested on younger people, the specific vaccine might also make a difference in some cases).
The EU is primarily concerned with regulating trade between EU members and between the EU and other countries. It doesn't have laws about vaccine mandates. It does have regulations on vaccines in veterinary use (since trade in animals is an important trade) and on the labelling of medical products.
There is a EU established European centre for disease prevention and control, which coordinates the response of member nations to communicable diseases. But it doesn't bar any specific measures, such as mandatory vaccinations.
So the EU is not about vaccine mandates. It is outside of the EU's competence.
There is the European Court of Human Rights, which is not an EU body. The resource that JJJ links in a comment states that:
Mandatory vaccination interferes with a person’s right to integrity protected under Article 8 of the European Convention on Human Rights (ECHR). Nevertheless, the Court concluded such interference may be justified if considered a ‘necessity to control the spreading of infectious diseases.
This means that you can't be forced to vaccinate (citing article 8) but a country is entitled to treat you differently if you don't. You could have to pay a charge, or you could find that you are not permitted to enter certain places.
The court noted that any such measures would have to be "proportionate". This means that the country must justify that repressive legislation is necessary to achieve the aim of protecting the health of its people. But it noted that scientific matters, such as "is a vaccine safe and effective" is not a matter that the court is qualified to decide, but a question for scientists.
So following EU law the answer is "Yes". Following the decisions of the ECHR the answer is a qualified "...yes, but".