Third party library added via composer.json but how can I load it in a module now?
You don't need to use require_once for autoload. It handles by the composer automatically. You should create a use statement in your php file.

As an example if you want to use a class s3 module of aws;

use Aws\S3\SomeClassName;