Lithium Ion charging schematic
When I understand you right, you want to know whether you can connect your project directly to the battery, even when the battery is charged?
The answer is somewhat hidden in the MCP73831 data sheet, on page 13: 


  during constant-current mode, the programmed charge current is supplied to the battery or the load


So yes, your project can be connected to the battery all the time. When you connect the power supply to the charging circuit, your project will be powered by it. 

The current your project needs gets subtracted from the charging current (that means the battery will be charged at a lower rate)

Likewise, when your project needs more current than you have programmed as charging current, the battery will need to supply the additional current.

During charging, your project will be powered by the charging voltage, which will be at most 4.2 volts (OK, maybe 4.5 volts when your battery needs this), so you will be on the safe side with the Arduino Mini.