OOM when running Bitbucket pipelines
I suspect that Bitbucket pipelines is enforcing this memory limit, not docker-compose.  This is nothing to do with the Linux kernel's OOM killer.

The page you linked is about memory limits in additional containers (e.g. for a database) used by the pipeline, not the pipeline itself.  

Some suggestions:


Try editing bitbucket-pipelines.yml to include a size option for memory size - I would expect this is more useful than editing Docker-compose options which may not be respected.  See this doc page.
Try running the build locally under Docker Compose to get an idea of memory usage.


It would help to include the details of your bitbucket-pipelines.yml, and the Bitbucket error message about being out of memory.