tl;dr: The wall is only an idea with lots of blanks. People on different sides of the divide fill in the blanks differently, so they end up with different conclusions.
The Wall is just an abstract concept. A concrete project to build The Wall, a plan for how it would look, an estimate what it would cost, where exactly it would be located, or a clear objective that it would achieve, is not part of the discussion.
And it gets worse: The problem The Wall is supposed to solve has not been clearly and fully defined. At this point in time, none of the involved politicians have enumerated the actual real life problems The Wall is supposed to solve, let alone explained how The Wall does so better than alternative options.
So how come people on different sides fill in the blanks differently? There are plenty of strong assertions around the weakly defined The Wall project. These assertions usually carry along plenty of untrue implied statements. Let's give an example: OP's words "Many countries around the world have a hard border with a physical barrier in place to stop illegal crossing/smuggling" imply
- Continuous walls between countries are perfectly normal
- The Wall can and will stop or significantly reduce smuggling
- The Wall can and will stop or significantly reduce illegal immigration
OP's words are true - if we assume "many" to mean "more than 2" - but all 3 implied statements are clearly false.* The comments provide anecdotal evidence that despite being obviously false, people do fall for these implied claims. While there are untrue or unproven implicit statements in circulation on both sides, I chose these particular claims because they are part of the original question I'm answering, and because they seem to be popular enough that I subjectively classify them as notable.
The divide over the idea of The Wall is magnified due to the unfortunate fact that in the US there is a very peculiar situation where one of the major news networks specializes in the deliberate spread of false and misleading information.
All in all, The Wall is an abstract concept that promises a simple solution to an oversimplified problem, and the question of how 'The Wall' would solve real world problems hasn't been answered yet.
*Disclaimer: The below shows, on request, why the 3 implied falsehoods are such. It is not strictly relevant to, or part of, this answer, but it will satisfy some people's curiosity.
1) Continuous walls between countries are perfectly normal - false Only a tiny fraction of worldwide borders have walls, while the overwhelming majority or borders do not have man made barriers. If you follow the link, you'll notice that even among the small fraction of borders that have man made barriers, the majority of these are related to past, present, and/or probable future armed conflict, such as Ukraine-Russia, Korea-Korea, Saudi Arabia-Jemen, India-Pakistan, Syria-Turkey, etc.
2) Trump's wall can and will stop or significantly reduce smuggling - false A majority of drugs cross the US-Mexican border through ports of entry, therefore a new wall is entirely unable to affect a majority of smuggling. In addition, smugglers already use methods that do physically defeat current walls and will also defeat any new wall (catapults, planes/drones, tunnels).
3) Trump's wall can and will stop or significantly reduce illegal immigration - false In the public discourse "illegal immigrants" (incorrectly) refers to 3 kinds of people**: Visa overstays (who make up 40%-60% of undocumented immigrants entering the US), asylum seekers, and people who try to cross the border undetected. Visa overstays are entirely unaffected by a wall. Asylum seekers can easily surrender themselves in any place where the wall has to move a few feet away from the border for physical reasons, without having to cross a wall. Asylum seekers can alternatively also surrender themselves at a border checkpoint. The last category - people who try to cross the border undetected - will be affected by a wall. These people already need tools/vehicles and often hire experienced guides to cross the border - a wall will force these guides to find ways to circumvent it (ladders, ropes, blankets).
In addition to the above, the symbolism of The Wall plays strongly into a pre-existing political divide, as outlined in Michael Kay's answer.