known as Mud Island, renamed in 1976), Milson Passage the passage in the Hawkesbury River between Milson Island and the southern bank (previously known as South(ern) Channel or The Gutter, renamed in 1976), and Milson Passage the suburb on the southern bank of the Hawkesbury River opposite Milson Island (previously known as Prickly Point, renamed in 1995) are all named after Milson's son Robert Milson (1824–1886). Surname In many records James Milson's surname is shown as "Milsom". "Milsom" was the surname by which James signed letters, in which legal proceedings were transacted, and which appears on early maps such as the 1840s map shown below. By the time that James died in 1872 he was known as Milson, the name by which he and his family are known today. Family tradition In the words of the North Sydney Historical society: James Milson is a controversial figure in local history, having made many claims during his lifetime (mostly in attempts to claim land) which either cannot be substantiated, or in some cases have been positively disproved There is a Milson family tradition which includes that: (1) James Milson was "promised" all the land on the North Shore of Sydney Harbour between Hulk (Lavender) Bay and Rainbow (Neutral) Bay by Governor Philip Gidley King. Despite Milson's wife's continuous urging, he did not press the Governor for a written-out grant. However, James Milson did not arrive in Port Jackson until after Philip Gidley King had ceased to be governor on 12 August 1806. James Milson arrived on Albion on 19 August 1806 during the period that Captain William Bligh was governor from 13 August 1806 to 26 January 1808 (until the military coup by the NSW Corps). James Milson's arrival and his marriage on 8 January 1810 were both events that occurred after Philip Gidley King ceased to be Governor of New South Wales. Therefore there is no way that Philip Gidley King could have made Milson a land grant, or that Milson's wife could have urged Milson to press Philip Gidley King for a written-out grant. Milson marriage occurred during the time that Lachlan Macquarie was Governor of New South Wales. Lachlan Macquarie was governor from 1 January 1810 to 1 December 1821. James Milson was instead given land by Governor Macqaurie. In James Milson's memorial to Governor Macquarie dated 5 June 1820 it was stated that James Milson was living with