Pakistan has large group of citizens (still like more than half) who harbor anti-American and surely even stronger anti-Israeli sentiments. Among these are non-trivial amount of hardcore jihadists. Any Pakistani leadership wishing to normalize ties (mutual recognition etc.) with Israel has to be weary of eroding their domestic position as a result, and not just at the polls...
There's a non-trivial precedent in the region with Sadat getting assassinated by a jihadist group after making peace with Israel, which included mutual recognition. So the leadership of any Muslim country with significant jihadist activity inside its territory has to keep this in mind when making ouvertures to Israel.
Completely suppressing jihadists in Pakistan is a tricky proposition as long as they simultaneously depend on them to keep some pressure on India/Kashmir. And if that were done, there's still the larger issue of ideological sensitivities of a larger group of Pakistani.
Wikipedia mentions that nevertheless Musharaf did try to create more cooperative relationship with Israel. What Wikipedia doesn't say, but the source (Haaretz) it cites for that does detail how that went down:
In fact, Musharraf might not recall, or be misrepresenting the reactions to his testing of the waters, because not all of it was as positive as he suggests. Secular parties in Pakistan accused him of playing up to the Americans, and religious parties there threatened street protests to oust the government if it took even a small step toward the recognition of Israel. Others in the government then backpedaled on their leader's behalf, with both Prime Minister Zafarullah Khan Jamali and Information Minister Sheikh Rashid Ahmed reiterating the country's traditional policy toward Israel, and the Foreign Office jumping in and joining the chorus.
But Musharraf insists: "There was no negative fallout. I don't remember anyone speaking out negatively against me or what I did."
In any case, it was a risk, he allows. "There is always risk in any new initiative. You can never be sure [what the reaction will be]. But a leader who is not prepared to take risks is not a leader. I believe that leaders should generally flow with public opinion. But there are times and issues where the public opinion goes astray, or is anchored in wrong premises - and to change that is the leader's job. That's where real leadership emerges. Changing the public's opinion is part of leadership. Leadership is not standing at the head of a herd and carrying out things you feel are wrong.
"Pakistan, like Israel, is an ideological state. That is the foundation of our creation. We are an Islamic republic," he says. "Which goes toward explaining why Pakistani Muslims are much more sensitive about Islam than most other Muslim countries. We are extremely sensitive about desecration of the Koran. So we are wholly sensitive to the Palestinian plight and any new initiative regarding Israel has to be proposed very delicately."
To put it in a nutshell, normalizing relations with Israel while not changing much in Pakistan is a case wanting to have the cake and eat it too. In other words it's a matter of conflicting objectives that can't all be achieved, and Pakistan's leadership[s] insofar have chosen to prioritize their more traditional objectives when push came to shove.