Why wouldn't miners change the protocol? (e.g. change difficulty)
The rest of the network, which uses the existing difficulty parameters, will reject the new blocks generated by your modified miners. There will be a longer blockchain, but there will not be a longer valid blockchain according to the established rules.