As a complement to armatita's good answer.
Although I cannot locate those interventions online, France's position was exposed in front of 21st and 22nd UN General Assembly, by Mr. Jeannel (during the 910th seance) and Mr. Bresson (during the 969th seance). Maybe this can help finding them on one of the UN's websites.
France's motivations have been extensively explained a few months after the Vienna conference in this article (in French, traduction mine after each paragraph).
Dans le scrutin du 22 mai 1969 par lequel la Conférence de Vienne a adopté la Convention sur le Droit des traités, la délégation française a voté contre ce texte.(...)
In the ballot of 22 May 1969 by which the Vienna Conference adopted the Convention on the Law of Treaties, the French delegation voted against this text.(...)
Pour [le gouvernement français], les effets bénéfiques du travail de la Commission du Droit international étaient d'ores et déjà atteints dans la mesure où cette dernière avait su trouver pour de nombreuses règles traditionnelles des formules qui traduisent heureusement la pratique internationale. Mais en revanche dans les domaines où les articles projetés n'étaient pas l'affirmation de règles reconnues, le moment lui semblait mal venu de tenter de forcer l'adoption par la voie conventionnelle de notions dont le sens exact et le bien-fondé appellent encore une maturation que seul le temps peut apporter.(...)
For [the French government], the beneficial effects of the work of the International Law Commission had already been achieved insofar as the latter had been able to find formulas for numerous traditional rules which happily reflect international practice. But on the other hand, in areas where the proposed articles were not the affirmation of recognized rules, it considers the moment unsuitable to try and force the adoption by conventional means of notions whose exact meaning and merit call for further maturation that only time can bring.(...)
les positions françaises se sont trouvées très éloignées, voire même opposées, sur les plus importantes des dispositions de la partie V (nullité) de la Convention, à celles défendues par une majorité(...)
French positions found themselves very distant, even opposed, on the most important of the provisions of part V (nullity) of the Convention, to those defended by a majority (...)
Les réserves françaises portaient sur les articles relatifs à certains cas de nullité, à la manière ou à la forme de l'application de la nullité, mais surtout son opposition a été fondamentale en ce qui concerne les dispositions qui ont été adoptées concernant d'une part « le jus cogens » dans le droit des traités et d'autre part les procédures de règlement des différends.(...)
The French reservations related to the articles relating to certain cases of nullity, to the manner or form of the application of nullity, but above all its opposition was fundamental with regard to the provisions which were adopted concerning, on the one hand "jus cogens" in the law of treaties and on the other hand the procedures for the settlement of disputes.(...)
la délégation française (...) s'est bornée à annoncer son vote contre l'article 50, comme contre les articles 61 et 67 qui lui sont liés, dans une intervention qui reprend les raisons d'une opposition à un texte qui « d'avance et péremptoirement, déclare nulle toute une catégorie de traités sans que l'on sache exactement ni quels sont ces traités, ni quelles sont les normes en vertu desquelles leur nullité sera prononcée, ni comment seront édictées ces normes », et qui « ferait pénétrer, à titre permanent, la contestation dans le droit des traités ».(...)
The French delegation (...) limited itself to announcing its vote against article 50, as against articles 61 and 67 which are linked to it, in an intervention which takes up the reasons for an opposition to a text which " in advance and peremptorily, declares void an entire category of treaties without it being known exactly neither what these treaties are, nor what are the norms by virtue of which their nullity will be pronounced, nor how these norms will be enacted", and which " would permanently penetrate the dispute into the law of treaties”.(…)
EDIT (by Fizz): the thrust of the complaint is against part V of the convention which contains articles regarding the nullity of some kinds of treaties. The French were concerned that the types of treaties thus prohibited isn't well defined. They voted (?) against articles 50, 61, 67 as having this unclear character. Reservations about "jus cogens" are also mentioned, but without a clear/separate article reference for that (above), but they do call them out in another portion of the paper:
les dangers inacceptables des articles 53 et 64 relatifs au jus cogens
the unacceptable dangers of articles 53 and 64 relating to jus cogens
They explain that article 53 taken together with article 44, which mandates the indivisibility of treaties, makes the convention dangerous because any obscure clause that can be found unacceptable under the convention in a given treaty would render the entire treaty void. Actually, there is some further confusion, because portions of that paper refer to draft(s) of the convention, in which article 50 then became 53 later... (that's said on p. 18 of the paper; added by Fizz)
In a recent (2016) Oxford Book International Law and Empire: Historical Explorations, an article by Umut Ozsu comes back on the debate about jus cogens. Luckily, the relevant pages 299-303 are on free access and also contain some direct quotes:
France, the most rigorous and trenchant critic of international jus cogens, stressed that the question - an 'extremely important one' that may on the 'ill-defined borderline between morality and law' - needed to be considered with great care. It was necessary to examine the issue in light of the present equality and not the past inequality of states, the French declared, this being the only way to avoid 'confrontation between the upholders of different political, social or economical systems'. It was also necessary to avoid imprecision as to jus cogens' creation, scope, and effects, especially by taking steps to control its application, as the contrary would force many states, particularly those subscribing to a monist conception of the relation between domestic and international law, to evaluate the validity of treaties in accordance with a 'supreme, undefined law', eviscerating the 'climate of security and confidence' requisite for smooth interstate relations.
I'll not pretend to understand all the law subtelties here, but generally, it seems that the French delegation was very reluctant to the introduction of new juridic concepts (mostly jus cogens) without initially backing them with strong scholar works. Disagreement on those technicalities, rather than any political motivation, justified France's negative vote on the text of the Convention, and its refusal to sign it.