Note that I am simplifying the Northern Irish conflict for the sake of both clarity and brevity.
Is there evidence that irregular spot-checking of a small minority of private traffic by customs officers would provoke violence?
The question is flawed because it assumes the wrong sequence of events. No one is going to get violent just because someone else's vehicle is being checked.
However, hearing about vehicles being checked at the border makes you think about government control, specifically the UK. This reminds you that the UK currently occipies a region that you may consider to be yours. This anti-UK sentiment, especially when experienced by groups of sufficient size (and means/outrage), is likely going to lead to anti-UK action being taken. This could be peaceful action, but it also could be violent. You just don't know.
All you know is what happened last time.
What you're dealing with here is that a part of the population adamantly rejects the notion that Northern Ireland is the property of the UK. This has lead to violence and acts of terrorism in trying to remove the UK from what they consider to be their Irish property.
Regardless of the correctness of the debate on who should own Northern Ireland; the violence became disproportionate and eventually became the main talking point, distracting from the much less in-your-face discussion on administrative ownership.
The decrease of border control has indirectly improved things in the Northern Irish region by decreasing border checks and therefore decreasing UK governmental presence. This isn't the only thing that improved things (of course the Good Friday agreement is the main key), but decreasing border control is obviously the opposite of increasing it (due to Brexit) in terms of border control, which is why I'm focusing on it.
Think of it this way: when standing in Ireland, trying to look into Northern Ireland, the first thing you see/encounter is the border, enforced by border agents who represent the UK government. That becomes a shining beacon of (perceived) oppression, which puts fuel to the flame of considering Northern Ireland to be "occupied" by the UK and requiring liberation.
Because the borders are no longer actively enforced, there is less of a UK presence visible to the Irish near Northern Ireland, which has lead to a decrease in wanting to remove the UK government. For a completely different reason (open borders), the same result (less UK involvement) was already achieved. Enough so to halt the ongoing violence and make the anti-UK sentiment go dormant.
When Brexit happens, and the UK guards its border again, the same anti-UK sentiment, which still remains in the Irish population but is currently dormant, is liable to flare up again. The increased border control will act as a reminder of the past perception of the UK's "occupation" of Northern Ireland and therefore is liable to spark more outrage again, which could lead to an increase in violence.
Nothing of this is a guarantee. But if something saved you in the past, and now you're going to lose that thing that saved you, it's more than normal to consider the possibility that the thing that threatened you in the past is going to threaten you again.
Think of it this way:
You are happily married. However, you discover that your wife has cheated on you with Ryan Reynolds. After a long process of being hurt and reevaluating the relationship, you decide to continue the marriage since she's not cheating on you anymore and has officially stated her intent to no longer cheat on you. Everything is good again.
A few years later, your wife is going to the premiere of a movie starring Ryan Reynolds, has won a meet and greet with him, and tells you how excited she is that she's going to meet him.
Fearing being cheated on again, you discuss marital fidelity with your wife.
Your neighbor has observed this, and he wonders to himself:
Is there evidence that your spouse meeting Ryan Reynolds leads to them cheating on you? According to my experience, my wife was excited to win a meet and greet with him, and she did not cheat on me.
Why would Michau assume that his wife meeting Ryan Reynolds means that she will be unfaithful to him? Where is the proof?
To a degree, your neighbor is right. You cannot know for a fact that your wife is going to cheat on you this time. But what your neighbor did not know is that there is a dormant dispute that is liable to flare up when Ryan Reynolds is discussed.
To you (and your wife), Ryan Reynolds is a painful memory. When he is brought up in a discussion, that inherently also reminds you of your wife's infidelity. You can never just talk about Ryan Reynolds without thinking about your wife cheating on you.
To your neighbor (and his wife), Ryan Reynolds is an actor. When he is brought up in a discussion, that means you're talking about a movie he starred in. When your neighbor is talking about Ryan Reynolds, he's just talking about Ryan Reynolds, nothing else.
Similarly, when raising the issue of a UK-controlled border, this acts as a reminder of the general feeling of Northern Ireland being occupied.
Last time, this lead to violence (just like how last time, your wife cheated on you with RR). It's not guaranteed to lead to violence this time (just like how your wife is not guaranteed to cheat again just because she meets RR again), but it's a valid concern that this could happen.