tl;dr: The wall is only an idea with lots of blanks. People on different sides of the divide fill in the blanks differently, so they end up with different conclusions.
There is still nothing more to The Wall than the idea. A concrete project to build the wall, a plan for how it would look, an estimate what it would cost, or even a clear objective that it would achieve, is not part of the discussion. Such a plan to have a discussion about simply doesn't exist.
And it gets worse. Nobody has yet clearly and fully defined the problem The Wall is supposed to solve. Right now, any attempt to dive into the facts of The Wall discussion will fail before it starts, because none of the involved politicians can enumerate the actual real life problems The Wall is supposed to solve, let alone explain how The Wall does so better than alternative options.
So how come people on different sides fill in the blanks differently? The Wall is a simple symbol, which convinces some voters because it's easy to make strong assertions, which carry some false implied statements. Let's give an example: OP's words "Many countries around the world have a hard border with a physical barrier in place to stop illegal crossing/smuggling" imply
- That continuous walls between countries are perfectly normal
- That Trump's wall can stop or significantly reduce smuggling
- That Trump's wall can stop or significantly reduce illegal immigration
OP's words are true, but the 3 implied statements are not.* While there are untrue or unproven implicit statements in circulation on both sides, I chose the claims implied by OP because of simplicity, and because they are particularly popular falsehoods.
The divide over the idea of the wall is magnified due to the unfortunate fact that in the US there is a very peculiar situation where one of the major news networks specializes in the intentional spread of such false and misleading information through implied statements.
All in all, a wall is a simple solution that doesn't address a complex problem, except in the minds of some of the voters. And both sides - one much more so than the other - play around the issue by avoiding any actually relevant hard facts and data, to avoid being called out by the other side.
*Disclaimer: The below shows, on request, the 3 implied falsehoods are such. It is not strictly relevant to, or part of, this answer, but it will satisfy some people's curiosity. Some people in comments assumed that the below is the answer. It isn't.
1) Only a tiny fraction of worldwide borders have walls, while the overwhelming majority or borders do not have man made barriers. You'll also notice that among the small fraction of borders that sport man made barriers, a majority seems to be borders between hostile nations such as India-Pakistan, Ukraine-Russia, and Korea-Korea.
2) Drugs already cross the US-Mexican border in places where man-made barriers exist, over, under and through the barriers. Smugglers use plenty of routes and methods that will be entirely unaffected by any new wall (catapults, smuggling through ports of entry, planes/drones, tunnels).
3) Visa overstays are completely unaffected by a physical barrier of any kind. Immigrants who want to surrender themselves to request asylum don't even need to get to the other side of the wall, they just need to find a border patrol on US soil. The remaining immigrants already need tools/vehicles and often hire experienced guides to cross the border - requiring guides to spend money on a set of very cheap and lightweight tools (ladders, ropes, blankets) is unlikely to deter them.
In addition to the above, the symbolism of the Wall plays strongly into a pre-existing political divide, as outlined in Michael Kay's answer.