Baseline holder (BLH) circuits are used widely to stabilize the analog output of application-specific integrated circuits (ASICs) for high-count-rate applications. The careful design of BLH circuits is vital to the overall stability of the analog-signal-processing chain in ASICs. Recently, we observed self-triggered fluctuations in an ASIC in which the shaping circuits have a BLH circuit in the feedback loop. In fact, further investigations showed that methods of enhancing small-signal stabilities cause an even worse situation. To resolve this problem, we used large-signal analyses to study the circuit's stability. We found that a relatively small gain for the error amplifier and a small current in the non-linear stage of the BLH are required to enhance stability in large-signal analysis, which will compromise the properties of the BLH. These findings were verified by SPICE simulations. In this paper, we present our detailed analysis of the BLH circuits, and propose an improved version of them that have only minimal self-triggered fluctuations. We summarize the design considerations both for the stability and the properties of the BLH circuits.