Wikipedia briefly justifies it as:
Union Territories of India have special rights and status due to their
constitutional formation and development. They may be given this
constitutional status for reasons such as safeguarding the rights of
indigenous cultures of that region, averting political turmoil related
to matters of administration and so on. These union territories could
be changed to states in the coming times to exercise better control in
their administration.
Yet that text, only partly answers the question. Here is a reference which is a first hand description of the reasons and proceedings.
In the above citation SHRI SRIBALLAV PANIGRAHI explains:
There have been references to the provisions of the
Constitution dealing with the Union Territory, statehood
etc. When we achieved Independence, we had nine regular States. At
that time, the States were coming under four categories. These were
Part 'A', Part 'B' and Part 'C' States. They were also called `A'
class, 'B' class and 'C' class States.
In Part 'A', the States were nine in number. In Part 'B', we
had Hyderabad, Jammu and Kashmir, Madhya Bharat, Mysore, Patiala and
East Punjab. In Part 'C', we had five States, namely, Ajmer, Bhopal,
Bilaspur, Kooch Bihar and Coorg. In Part 'D', only one
Territory, namely, Andaman and Nicobar Islands found its place. There
was a provision kept for future and if need be and if we acquire
a new territory in course of time, then that would join Andaman and
Nicobar Islands in Part `D' category.
There was a dramatic change in this arrangement in course of time.
As you know, there were demands from different areas for separate
States. There was also a Boundary Commission, the States
Reorganisation Commission. I think the Commission was constituted
under the Chairmanship of Fazal Ali. P.M. Panicker was one of the
three Members of the Committee. They were brilliant and seasoned
people with experience in different areas. Their recommendations were
accepted and States were reorganised. And the old arrangement
disappeared and under the new arrangement, States and Union
Territories were formed.