The appointment of the chairman of energy supplier SSE and engineering firm Weir Group came as the bank was formed as a public company.
His deputy will be Sir Adrian Montague, who has led the bank's advisory group.
The green bank is being set up with Â£3bn of public money to help firms finance early-stage renewable energy schemes.
Its headquarters will be based in Edinburgh after the city beat off competition from 31 other bids.
The newly formed board will begin recruiting other directors and senior executives with a view to the bank becoming fully operational this autumn, subject to state aid approval from the European Commission.
The advisory group, led by 3i and Anglian Water Group chairman Sir Adrian, will be disbanded after the appointments.
In addition to his roles at the Weir Group and SSE, Lord Smith of Kelvin is chairman of the Glasgow 2014 Commonwealth Games organising committee and director of Standard Bank Group.
Sir Adrian was previously chairman of Friends Provident Group and British Energy Group.
Mr Cable said the  
          Green Investment Bank
 was a "major new innovation", vital to securing investment in the decarbonisation of the UK's energy supply.
"It has found two candidates of outstanding calibre well suited to leading the bank through its important early phase," he added.
Scottish Secretary Michael Moore said: "This is a UK institution headquartered in Edinburgh supported by a strong team in London and the news that Lord Smith and Sir Adrian Montague have been appointed is another step towards the bank opening for business.
"They bring a wealth of experience to the roles and will bring that to bear as we look to encourage private sector investment in key energy projects."
Although the headquarters of the bank will be in Edinburgh, its main transaction team will be based in London.
Lord Smith of Kelvin will chair the UK Green Investment Bank (GIB), Business Secretary Vince Cable has announced.