James Fitzjames entered the Royal Navy at the age of 12 in July 1825 as a Volunteer of the Second Class on HMS Pyramus, a frigate under the command of Captain Robert Gambier.
Although after this the Coninghams and the Gambiers wanted to send him to Cambridge, James Fitzjames was determined to resume his Royal Naval career.