Is Duplicate Bridge (standard Matchpoints) a game of pure skill?
Even with the same experts playing North-South, there will be luck involved.

For instance


When you have a pure guess in a two way finesse for a Q or distribution etc. 
When you have a guess during bidding (sacrifice or not etc)
System wins/losses.
When you overbid/underbid/play incorrectly and hit a lucky lie of the cards.
Same hand could be played differently on different tables, because of different actions taken (during bidding or play) by the players. One might play like a bozo, but resulting in the expert to take the wrong inferences, handing you a win (it happens!) etc.


What you say will reduce the randomization of the field (some people call it field protection), but will not eliminate luck completely.

For instance, a world class level Matchpoint contest will have very little randomization (similar to your proposed situation), but to win you need to play well (skill) and need some luck.

Of course, most of the time it is your skill that matters, but luck does factor in, especially against equally matched opponents.