LTC3862-2 Step Up problem, output voltage drop
Your layout may have a bit to do with it (you can share it if you want more info).  As well as needing bulk capacitance on the solenoid.

The boost converter has a finite source impedance, as does the layout.  By adding a large capacitance right next to the solenoid, you effectively get an additional very low impedance source (for a limited time).
I would suggest adding several caps in parallel to create around 1000uF as a start. That will keep the total ESR lower.
To get better values, you'll need to measure the current waveform through the inductor at turn on.  If you don't have a current probe, you can place a non-inductive 0.1Ω (or smaller) resistor in series from the M1 source to ground connection, then use ohms law to convert to amps.







simulate this circuit &ndash; Schematic created using CircuitLab