Solar panel charging issue
As described, This is a normal operation. But I think the charging profile is not suitable for Lithium batteries.
In fact, there are different charging profiles.
When you first apply the solar power the battery will be charged to a threshold value which is 13.4V in your case. Then the controller will wait till the voltage drops below another threshold (which is below 13.1V in your case).
Charging profiles change depending on the type of the battery. The following is for instance the profile used by Deepsea chargers (not designed for lithium batteries):

the float voltage definition fits with your 13.1 V.
To find out if it is really working as needed, apply a continuous load and wait to check below which threshold it will restart charging.
the following graph is an example of a charging profile for a lithium battery that is not voltage float charged and where the current is kept constant instead of the voltage (here the voltages are different than in your case) :

Some battery chargers are profile configurable either by software or simply by a button like the following:

Choosing the right profile increase the lifetime of the battery.
Since you edited your post to add the battery type. Here is one of the suitable profiles included also in a Deepsea charger for Lithium Phosphate:

In fact in this case the float charge is terminated when the charging current is too low and then after either the charge termination timer elapses or the battery is discharged the cycle will restart.
Concerning your controller, it is clearly mentioned in the manual that the battery type is &quot;any 12V DC battery that is used for cars, boats, motor cycles, etc...&quot; so it is mainly a lead acid battery. The charging profile suits these kind of batteries. But this is not to say that it wont work for your case. However it is not the optimal profile for the lifetime of your battery.