This is such a good question, and here's my historical reasoning as to why it should be the first factor (degree of difficulty in opening), and this is the higher factor by a large margin.
Here's the different averages by each position, for reference:
Full Test History (1877-Present):
Op) 35.71
01) 36.85
02) 34.56
03) 39.48
04) 40.84
05) 38.08
06) 32.61
07) 27.73
08) 21.31
09) 15.53
10) 11.57
11) 8.55
3-5) 39.49
Op/3-5) 0.904
Modern Era (1970-Pres):
Op) 35.92
01) 36.96
02) 34.88
03) 39.46
04) 41.66
05) 39.41
06) 34.11
07) 29.24
08) 21.91
09) 15.53
10) 11.40
11) 8.29
3-5) 40.18
Op/3-5) 0.894
Inter/Post Wars (1914-1969):
Op) 36.74
01) 38.49
02) 35.01
03) 41.86
04) 41.50
05) 36.21
06) 30.23
07) 24.89
08) 20.05
09) 15.89
10) 11.87
11) 8.70
3-5) 39.93
Op/3-5) 0.920
Pre World Wars (1877-1913):
Op) 28.69
01) 28.85
02) 28.53
03) 30.53
04) 27.54
05) 27.98
06) 22.55
07) 19.56
08) 18.60
09) 14.27
10) 12.62
11) 10.72
3-5) 28.70
Op/3-5) 1.000
If you look at the breakdown by eras, you'll notice that the gap between production of true middle order (I define as 3-5, but you can go with 3-4) is greatest in the modern era, but decreases in size progressively until you get to the earliest Pre World Wars eras where it's basically a negligible difference. It makes sense as well, as the absolute highest producing run-scorers of earlier eras were openers (think Hobbs, Sutcliffe). Thinking about it conceptually, if all else is equal and it's not really harder to open and bat up the order than further down then of course you'd want to put your best run scorers as high up the order as possible because it gives them the maximum ability to score the most runs possible, and minimizes the chance that they run out of batting partners.
So what changed from those early days of cricket to the modern era, with the highest run scorers of the side generally needing to be shielded from the new ball? Fast bowling technique with the new ball, developed and then exploded to become the defining feature of the game. This was really in it's infancy in the earlier eras of Test cricket for mine, and now that it's a given for any aspiring great side to be able to attack with their most skilled fast bowling resources with the first new ball to effect, it's clear to me that the most difficult specialist batting role in the side by far is that of the openers.
I mean, you don't need a ****ing in depth stats analysis for that, when we can all just ****ing watch the game and it should be clear, but it's nice to see the evolution broken down in numbers as well.
PS:
The original reason that I wanted to get this breakdown is to try and determine, in the modern game, the best spot for your best middle order bat, between the 3 or 4 positions. And assuming that that position should be where that best batsman can maximize his run-scoring potential (because why the **** wouldn't you want him to), then I think that is clearly the number 4 position in the modern game. I think it ends up being in that Goldilocks zone where you're both the vast majority of the time avoiding the first new ball ( something the number 3 will see often enough to impact his scoring ), and early enough to make their mark on the game before you've lost the majority of your best batting resources (like a bat coming in at 5 would deal with) and are not in much danger of running out of partners either. Of course, individual comfort and preferences of this best bat in the side will also effect the batting spot decision, but if in doubt I think 4 is the best spot*.
*Unless you're a Pakistani bat circa the 2000s, and senior players would want to get as far away from the possibility of facing the new ball as much as possible, (i.e. Inzamam playing 5) because the openers were utterly incompetent and would put you in that position constantly.