 He rejoined the cabinet again in 1900 under the 4th Itō Hirobumi administration as Minister of Agriculture and Commerce and assisted in the formation of the Rikken Seiyūkai political party.Hayashi retired from public life in 1908.Returning to his native Kōchi Prefecture, Hayashi became an entrepreneur, forming a company to make cultured pearls.