The safety and stability of hydraulic turbine regulating system (HTRS) in hydropower plants become increasingly important since the rapid development and the broad application of hydro energy technology. In this paper, a novel mathematical model of Francis hydraulic turbine regulating system with a straight-tube surge tank based on a few state-space equations is introduced to study the dynamic behaviors of the HTRS system, where the existence of possible unstable oscillations of this model is studied extensively and presented in the forms of the bifurcation diagram, time waveform plot, phase trajectories, and power spectrum. To eliminate these undesirable behaviors, a specified fuzzy sliding mode controller is designed. In this hybrid controller, the sliding mode control law makes full use of the proposed model to guarantee the robust control in the presence of system uncertainties, while the fuzzy system is applied to approximate the proper gains of the switching control in sliding mode technique to reduce the chattering effect, and particle swarm optimization is developed to search the optimal gains of the controller. Numerical simulations are presented to verify the effectiveness of the designed controller, and the results show that the performances of the nonlinear HTRS system assisted with the proposed controller is much better than that with the commonly used optimal PID controller.