Comparing Pros and Cons of SWAT and HSPF
This is a broad question, the best way to compare the models is to go through the manuals and compare the process algorithms to see how the various hydrologic processes are handled. There are other considerations as well, including:


what environment/watershed are you modelling? How large, how urbanized, what type of soils/environment/precipitation vs evapotranspiration, how important are snow processes, etc., and compare that to what the models were made for
what is the purpose of your modelling? Your choice of model will be different if you are looking at water balance vs flood forecasting vs water quality estimates
what data do you have available? Some models are more intensive with data requirements than others, and if you only have daily precipitation and temperature (as is typical) than that will restrict some model types
what level of accuracy do you need? The timestep options in the models will make a difference


That being said, generally my understanding from a broad perspective is that:
- HSPF is a more detailed hydrologic model with larger data requirements for input forcings, and has also a more comprehensive process library with respect to water quality
- SWAT as you said was developed for agricultural regions, has less strict data requirements and also some ability to model water quality parameters (though less detailed than HSPF)

Depending on the purpose of your modelling, you may want to consider:


starting with a simpler model, such as GR4J, just to get the ball rolling before investing more time into complex models, particularly if just need simple answers from the model
using a flexible model where the processes can be modified on the fly and you are not committed to a particular set of algorithms when you choose the modelling framework, e.g. the Raven hydrologic modelling framework


Hope that helps!