The hard Brexiteers don't really care about immigration. Immigration was just a tool they used to get what they really want: economic freedom to exploit the UK. Things like employment rights and financial regulations that derive from the EU limit get in the way of them making money.
The immigration angle was just something they used to get enough plebspeople to vote for brexit. Now it's a useful excuse for not honouring the pre-referendum proposals to stay in the Single Market and Customs Union.