By looking at statistics on primary voters in 2018, as recorded by the Cooperative Congressional Election Study, we can investigate this. Rather than a tendency for mail-in voters to be more conservative ideologically, I suggest that the observed tendency to support incumbents may be due to mail-in voters identifying as stronger supporters of their party.
Intuitively, we know that older voters are generally more conservative than younger voters, and than older voters are more likely to vote by mail. Indeed, this is backed up by data from the study - the mean age of in-person primary voters was about 55.2 years, while the mean age of absentee/mail-in voters was around 60.4. At the party level, the figures were 52.4/57.4 for the Democrats, and 57.7/63.5 for the Republicans.
In addition, the study data backs up the claim that older voters are generally more conservative. The mean ideology moves ~1.7 points towards conservative on a seven-point scale from 'Somewhat Liberal' for 18-29 year olds to just below 'Somewhat Conservative' for voters over the age of 65.
However, the combination of these two factors doesn't appear to hold. Contrary to the expectation that in-person primary voters would tend to be more conservative than mail-in voters, I only observed this in voters in the Republican primary, whose average ideology moved ~0.15 points towards the conservative side of the seven-point scale. Amongst voters in the Democratic primary, I actually observed a ~0.12 shift towards the liberal end of the scale, shown in the plot below.
I would therefore suggest that rather than a red-shift, the comparatively older demographic represented in party primaries by mail-in ballots is more conservative in a rather different way - they are more likely to back the incumbent due to a sense of party loyalty. I've tested this by looking at the strength of voters' allegiance to their party. Both Democrats and Republicans using mail-in/absentee ballots are more likely to identify as stronger supporters of their respective party than those voting in-person. This factor could be what causes the tendency to support incumbents observed in the question.