Use Service Worker in Plugin
Solved it by just serving the file within a controller action like this:

public function actionCacheCraftServiceWorker () {
    $serviceWorkerFile = Craft::getAlias('@skayo/cachecraft/assetbundles/cachecraft/dist/js/CacheCraftServiceWorker.min.js');

    return Craft::$app-&gt;getResponse()-&gt;sendFile(
        $serviceWorkerFile,
        'CacheCraftServiceWorker.min.js',
        [
            'mimeType' =&gt; 'application/javascript',
            'inline' =&gt; true
        ]
    );
}


and using this event to register the front-end url:

Event::on(
    UrlManager::class,
    UrlManager::EVENT_REGISTER_SITE_URL_RULES,
    function (RegisterUrlRulesEvent $event) {
        Craft::debug(
            'UrlManager::EVENT_REGISTER_SITE_URL_RULES',
            __METHOD__
        );
        $event-&gt;rules['cache-craft-service-worker'] = [
            'route' =&gt; 'cachecraft/frontend/cache-craft-service-worker',
            'suffix' =&gt; '.js'
        ];
    }
);


Now I can just access it with https://example.com/cache-craft-service-worker.js which works perfectly fine.