Arduino weight sensor value increasing/decreasing for no reason
After many trials I found out that the issue came from 2 places: 

1 - the scale.get_units() function provided in the hx711 library allows to specify the number of samples to average in order to get a more precise reading. When I changed the number of samples from 20 to 1, the values stopped jumping around and I got a more flat line. (+- 0.02 grams between values). There is probably a bug in the library since the result should have been the reverse.

2 - I had "noise" in my circuit and resoldering parts of it solved another intermittent variation in values I experienced.

I ran tests over a period of 8 hours and could notice the readings going slightly up, then down and probably back up if I waited longer. Variations were of 0.8 grams over 8 hours like shown here:



I now account for these variations using software to calculate a drift value.

Thanks for all the help!