Llama – A CLI for outsourcing computation to AWS Lambda
GitHub - nelhage/llama
llama -- A CLI for outsourcing computation to AWS Lambda
Llama is a tool for running UNIX commands inside of AWS Lambda. Its
goal is to make it easy to outsource compute-heavy tasks to Lambda,
with its enormous available parallelism, from your shell.
Most notably, llama includes llamacc, a drop-in replacement for
gcc or clang which executes the compilation in the cloud, allowing
for considerable speedups building large C or C++ software projects.
Lambda offers nearly-arbitrary parallelism and burst capacity for
compute, making it, in principle, well-suited as a backend for
interactive tasks that briefly require large amounts of compute. This
idea has been explored in the ExCamera and gg
papers, but is not widely accessible at present.
Performance numbers
Here are a few performance results from my testing demonstrating the
current speedups achievable from llamacc:



project
hardware
local build
local time
llamacc build
llamacc time
Approx llamacc cost




Linux v5.10 defconfig
Desktop (24-thread Ryzen 9 3900)
make -j30
1:06
make -j100
0:42
$0.15


Linux v5.10 defconfig
Simulated laptop (limited to 4 threads)
make -j8
4:56
make -j100
1:26
$0.15


clang+LLVM, -O0
Desktop (24-thread Ryzen 9 3900)
ninja -j30
5:33
ninja -j400
1:24
$0.49



As you can see, Llama is capable of speedups for large builds even on
my large, powerful desktop system, and the advantage is more
pronounced on smaller workstations.
Getting started
Dependencies

A Linux x86_64 machine. Llama only supports that platform for
now. Cross-compilation should in theory be possible but is not
implemented.
The Go compiler. Llama is tested on v1.16
but older versions may work.
An AWS account

Install llama
You'll need to install Llama from source. You can run
go install github.com/nelhage/llama/cmd/...@latest

or clone this repository and run

If you want to build C++, you'll want to symlink llamac++ to point
at llamacc:
ln -nsf llamacc "$(dirname $(which llamacc))/llamac++"

Set up your AWS credentials
Llama needs access to your AWS credentials. You can provide them in
the environment via AWS_ACCESS_KEY_ID/AWS_SECRET_ACCESS_KEY, but
the recommended approach is to use ~/.aws/credentials,
as used by. Llama will read keys out of either.
The account whose credentials you use must have sufficient permissions.  The
following should suffice:

AmazonEC2ContainerRegistryFullAccess
AmazonS3FullAccess
AWSCloudFormationFullAccess
AWSLambda_FullAccess
IAMFullAccess

Configure llama's AWS resources
Llama includes a CloudFormation template and a command which
uses it to bootstrap all required resources. You can read the
template to see what it's going to do.
Once your AWS credentials are ready, run

to create
the required AWS resources. By default, it will prompt you for an AWS
region to use; you can avoid the prompt using (e.g.) llama -region us-west-2 bootstrap.
If you get an error like
Creating cloudformation stack...
Stack created. Polling until completion...
Stack is in rollback: ROLLBACK_IN_PROGRESS. Something went wrong.
Stack status reason: The following resource(s) failed to create: [Repository, Bucket]. Rollback requested by user.

then you can go to the AWS web console, and find the relevant CloudFormation
stack.  The event log should have more useful errors explaining what went
wrong.  You will then need to delete the stack before retrying the bootstrap.
Set up a GCC image
You'll need to build a container with an appropriate version of GCC for llamacc to use.
If you are running Debian or Ubuntu, you can use
scripts/build-gcc-image to automatically build a Debian image and
Lambda function matching your local system:
$ scripts/build-gcc-image
If you want more control or are running another distribution, you can
look at images/gcc-focal for an example Dockerfile to build a
compiler package. You can build that or a similar image into a Lambda
function using llama update-function like so:
$ llama update-function --create --build=images/gcc-focal gcc
Using llamacc
To use llamacc, run a build using make or a similar build system
with a much higher -j concurrency than you normally would -- try
5-10x the number of local cores,, and using llamacc or llamac++ as
your compiler. For example, you might invoke
$ make -j100 CC=llamacc CXX=llamac++
llamacc configuration
llamacc takes a number of configuration options from the
environment, so that they're easy to pass through your build
system. The currently supported options include.



Variable
Meaning




LLAMACC_VERBOSE
Print commands executed by llamacc


LLAMACC_LOCAL
Run the compilation locally. Useful for e.g. CC=llamacc ./configure


LLAMACC_REMOTE_ASSEMBLE
Assemble .S or .s files remotely, as well as C/C++.


LLAMACC_FUNCTION
Override the name of the lambda function for the compiler


LLAMACC_LOCAL_CC
Specifies the C compiler to delegate to locally, instead of using 'cc'


LLAMACC_LOCAL_CXX
Specifies the C++ compiler to delegate to locally, instead of using 'c++'


LLAMACC_LOCAL_PREPROCESS
Run the preprocessor locally and send preprocessed source text to the cloud, instead of individual headers. Uses less total compute but much more bandwidth; this can easily saturate your uplink on large builds.


LLAMACC_FULL_PREPROCESS
Run the full preprocessor locally, not just #include processing. Disables use of GCC-specific -fdirectives-only


LLAMACC_BUILD_ID
Assigns an ID to the build. Used for Llama's internal tracing support.


LLAMACC_FILTER_WARNINGS
Filters the given comma-separated list of warnings out of all the compilations, e.g.  LLAMACC_FILTER_WARNINGS=missing-include-dirs,packed-not-aligned.



It is strongly recommended that you use absolute paths if you set
LLAMACC_LOCAL_CC and LLAMACC_LOCAL_CXX.  Not all build systems will
preserve $PATH all the way down to llamacc, so if you don't use
absolute paths, you can get build failures that are difficult to diagnose.
Other features
llama invoke
You can use llama invoke to execute individual commands inside of
Lambda. The syntax is llama invoke <function> <command> args.... <function> must be the name of a Lambda function using the
Llama runtime. So, for instance, we can inspect the OS running inside
our Lambda image:
$ llama invoke gcc uname -a
Linux 169.254.248.253 4.14.225-175.364.amzn2.x86_64 #1 SMP Mon Mar 22 22:06:01 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
If your function consumes files as input or output, you can use the
-f and -o options to specify that files should be passed between
the local and remote nodes. For instance:
$ llama invoke -f README.md:INPUT -o OUTPUT gcc sh -c 'sha256sum INPUT > OUTPUT'; cat OUTPUT
16c399c108bb783fc5c4529df4fecd0decb81bc0707096ebd981ab2b669fae20  INPUT
Note the use of LOCAL:REMOTE syntax to optionally specify different
paths between the local and remote ends.
llama xargs
llama xargs provides an xargs-like interface for running commands in
parallel in Lambda. Here's an example:
The optipng command compresses
PNG files and otherwise optimizes them to be as small as possible,
typically used in order to save bandwidth and speed load times on
image assets. optipng is somewhat computationally expensive and
compressing a large number of PNG files can be a slow operation. With
llama, we can optimize a large of images by outsourcing the
computation to lambda.
I prepared a directory full of 151 PNG images of the original Pokémon,
and benchmarked how long it took to optimize them using 8 concurrent
processes on my desktop:
$ time ls -1 *.png | parallel -j 8 optipng {} -out optimized/{/}
[...]
real    0m45.090s
user    5m33.745s
sys     0m0.924s
Once we've prepared and optipng lambda function (we'll talk about
setup in a later section), we can use llama to run the same
computation in AWS Lambda:
$ time ls -1 *.png | llama xargs -logs -j 151 optipng optipng '{{.I .Line}}' -out '{{.O (printf "optimized/%s" .Line)}}'
real    0m16.024s
user    0m2.013s
sys     0m0.569s
We use llama xargs, which works a bit like xargs(1), but runs each
input line as a separate command in Lambda. It also uses the Go
template language to provide flexibility in substitutions, and offers
the special .Input and .Output methods (.I and .O for short)
to mark files to be passed back and forth between the local
environment and Lambda.
Lambda's CPUs are slower than my desktop and the network operations
have overhead, and so we don't see anywhere near a full 151/8
speedup. However, the additional parallelism still nets us a 3x
improvement in real-time latency. Note also the vastly decreased
user time, demonstrating that the CPU-intensive work has been
offloaded, freeing up local compute resources for interactive
applications or other use cases.
This operation consumed about 700 CPU-seconds in Lambda. I configured
optipng to have 1792MB of memory, which is the point at which lambda
allocates a full vCPU to the process. That comes out to about 1254400
MB-seconds of usage, or about $0.017 assuming I'm already out of the
Lambda free tier.
Managing Llama functions
The llama runtime is designed to make it easy to bridge arbitrary
images into Lambda. You can look at images/optipng/Dockerfile in
this repository for a well-commented example explaining how you can
wrap an arbitrary image inside of Lambda for use by Llama.
Once you have a Dockerfile or a Docker image, you can use llama update-function to upload it to ECR and manage the associated Lambda
function. For instance, we could build optipng for the above example
like so:
$ llama update-function --create --build=images/optipng optipng
When specifying the memory size for your functions, note that Lambda
assigns CPU resources to functions based on their memory
allocation. At
1,769 MB, your function will have the equivalent of one full core.
Other notes
Inspiration
Llama is in large part inspired by gg, a tool for outsourcing
builds to Lambda. Llama is a much simpler tool but shares some of the
same ideas and is inspired by a very similar vision of using Lambda as
high-concurrency burst computation for interactive uses.
