How to protect the early stages of a proof-of-work application? For a given cryptocurrency or other blockchain application, proof-of-work provides intrinsic security in the algorithm itself that makes it unlikely that someone can attack the network unless they have more than 50% of the computational power of the network under their control. 

This makes sense and works once the network has a considerable size, but every network has to start somewhere, and when a new application is born and the number of miners is still very small, it would seem very easy to outpower the network to insert fraudulent transactions and still be the longest chain of blocks.

Is there a way to mitigate this risk?
... insert fraudulent transactions ...


No, fraudulent transactions and blocks containing them are rejected when nodes follow the consensus rules.

There is a greater risk of chain reorganisation, meaning miners who successfully mined a block may not eventually have the reward for that when their block is no longer part of the longest chain.

The great risk of a 51% attack is that the attacker could outpace the rest of the network to assert that the attacker's version of the blockchain is longer, re-writing history with valid transactions and claiming block rewards, or undoing their own transactions when re-writing history by spending the utxo's back to themselves first, making their previously valid transaction seem like a double spend, for example, and stealing from a merchant.

To protect a new proof-of-work application it is only necessary that the work is reasonably distributed. For example, in the case of a Bitcoin full clone the same ASIC miners would be valid, however, the cost-for-benefit would be very low initially and so an attacker must ask, what benefit do I derive?