Overcharge Lithium-Ion when using charge controller + Arduino Load
The TP4056 reports that the charge-point accuracy is 4.2V within 1.5%. Your battery is essentially acting as a nice, large pseudo-capacitor in feeding the input of your 5V boost regulator. In an ideal world, if your solar input is sourcing enough power to keep the battery fully-charged, and run the Arduino, I would expect the TP4056 to sit in constant-voltage mode keeping the battery topped off while the boost regulator operates nominally. This would keep the voltage around 4.2V (assuming some hysteresis) and you're set.

The major 'concern' here in my head is that while the battery is a beefy enough device to feed your boost regulator, I don't know how the TP4056, which appears to be a mass-market ASIC out of Shenzen for consumer devices, deals with a potential switching load on its output. You don't want a scenario where the boost regulator causes such ripple that the TP4056 oscillates wildly; luckily, you have a large reservoir known as a battery on the line, so it should bear the brunt of the load. I assume it was at least considered, thinking about the types of product the chip ends up in, but the boost regulator will apply some ripple to the battery as it continuously switches current into its inductor. I've seen enough projects and products do this that damage is not going to happen, but I'd personally get scope waveforms of what's going on on the BAT+ rail.