Cannot successfully setup LM741 op amp
The only thing you are doing wrong is your choice of a pitiful and ancient op-amp. Even if you bring R3 and R4 down to 100K, this op-amp is so poor quality that you will still have minor but visible offset errors.

Make R3 and R4 10K and the offset errors are less, but still there. As long as you use the LM741 you will be upset with less-than-perfect results.

Suggest you dump the POS 741 and advance way up to a TL081 or TL071 (for low noise). These are low cost JFET op-amps that can tolerate feedback and gain resistors in the low meg-ohms with little offset.

They offer offset null pins if you want offset to be near 0.000. Their bias current is thousands of times less than the 741, so the "true value" of a gain or voltage divider is much closer to the math equations.

ADVICE:
If you buy the TL071 or TL081 series do yourself a favor and buy 1% metal film low drift resistors so your readings are stable over time and temperature. Per each given 1/4 W 1% value you can get 100 pcs on cut-tape for maybe $2.50 USD at Digi-key, Mouser, Allied, etc.