Ultegra 6800 shifter with R8000 rear derailleur?
Should be OK. Both shifter and derailleur are 11 speed, only 1 generation apart. AFAIK Shimano did not change the cable pull or derailleur movement ratio.

All the other derailleur specs check out:

As you say, the R8000-GS goes up to a 34 tooth max sprocket.

Your front ring tooth difference (10) is within R8000-GS max of 16

Your total capacity required (34-11 + 44-34 =  33) is well within max of 39.