If I want to buy bitcoins, I can buy them from an online exchange, from an ATM or from another person who has bitcoins. Online exchanges and ATMs charge high premiums because they are intermediaries. If I buy bitcoins from other people, they also charge high premiums. The whole idea behind the blockchain was to get rid of third-parties. Is is possible to connect to the public ledger somehow and acquire bitcoins without a third-party?

I wonder how the first bitcoin transactions were completed when there wasn't any online exchanges, mobile wallets or ATMs. Can programmers somehow connect to the public ledger and do transactions without bitcoin wallets or fees?