The land on the borderline would technically belong to both neighboring countries as each one's territory ends there. So in the case of one country deciding to build a wall, would that country build the wall just inside their border to not require consent from the other country?
Each side is free to build walls on their side of the border. See, for example, the border between the two German states during the Cold War. Since such a wall must not encroach on the foreign territory, it cannot be literally on the border -- there will be a strip of territory between the wall and the border. There are even cases where both sides build walls on their side, e.g. the Korean DMZ.
Countries might voluntarily enter treaties which effectively prohibit such a wall. Or regulate where there must be gates, and who can cross. But that is a special case, not the general one.
The country wishing to build a wall would do it inside its borders. Using even a small part of the neighbor's territory would be sure to lead to serious political issues. This is unless the neighbor agrees, which would be unlikely, as in most cases only one side sees an interest in building a wall.