Walter Gray died in 1862 and, by 1865, John and George Harris, merchants and agents from Brisbane, had taken over the wharf site.
Initially, the ASN wharf was a small timber wharf or an "earth bank" similar to Gray's wharf.