AWS Lambda Layers with Serverless

AWS Lambda Layers was announced during 2018’s re:Invent as a new way bundle dependencies into functions. It allows us to include libraries, binaries and even runtimes into Lambdas.

Without Layers, dependencies need to be bundled together with the function code, inside the .zip file that is pushed to Amazon servers. This results on large deployment packages and a slow development pipeline. Some dependencies can’t even be bundled with ease, since they exceed Lambda’s current 50 MB package size limit.

Layers allows us to package and deploy dependencies independently from our functions. Every Layer also gets an unique identifier that can be shared and re-used between other Lambdas.

This tutorial shows how to publish a function and a Layer using the Serverless Framework.

Creating a Screenshot service with Chromium and Puppeteer

Let’s see how we can deploy a Lambda Function that receives an URL, takes a screenshot of the page and returns it. Puppeteer will be the library of choice to handle page loading and screenshots. Since Puppeteer relies on a compiled version of Chromium to deal with pages, we also need to provide it as a dependency by using Layers.

Tip: The process for including other prebuilt binaries - such as FFmpeg or Imagemagick - should be very similar.

Setting up the project

First, let’s create an empty repository and add the project files:

# Create the project repository
mkdir screenshot-page
 
# Enter the repository
cd screenshot-page
 
# Create a minimal package.json
yarn init -y
 
# Create a file for the function code
touch handler.js
 
# Create a Serverless configuration file
touch serverless.yml
 
# Create a folder where we'll keep the dependencies (in this case, the Chromium binary)
mkdir layer

Run the commands below to install the dependencies needed:

# Install "puppeteer-core" for taking screenshots
yarn add puppeteer-core
 
# Install "serverless" to manage and deploy our function
yarn add -D serverless

Tip: The pupeteer npm package downloads Chromium by default when installed. This would result in a huge deployment package. Since we’ll be providing the Chromium binary via Layer we can only install puppeteer-core, which doesn’t include Chromium. See more on puppeteer vs. puppeteer-core.

Tip: If you are using a global installation of serverless, make sure the version is higher than v1.34.0 (when support for Layers was added)

Downloading Chromium

Now let’s add Chromium to the project. For convenience, let’s grab one of the prebuilt binaries from the serverless-chrome repository here.

Download the file, extract the binary and add it to the layer folder:

# Still inside the project repository...
 
# Download the zip file
curl -L0 https://github.com/adieuadieu/serverless-chrome/releases/download/v1.0.0-55/stable-headless-chromium-amazonlinux-2017-03.zip -o chromium.zip
 
# Unzip it
unzip chromium.zip
 
# Move the binary into the "layer" folder
mv headless-chromium ./layer/headless-chromium
 
# Delete the zip file
rm chromium.zip

Tip: You can add the layer folder (or whatever you call it) to your .gitignore and avoid pushing large files to your Git server.

After following these steps you should end up with this structure on your project:

.
├── handler.js
├── layer
│   └── headless-chromium
├── package.json
├── serverless.yml
└── yarn.lock

Adding code and configuration

Now let’s add code to our files. Below is an example of how your serverless.yml configuration should look like.

Note that we have a new top-level property called layers. It contains the path to the folder where we store our dependencies - in this case, Chromium’s prebuilt binary.

To attach the Layer to our function, we can simply reference it on the function declaration and it will be available during the function execution.

service: screenshot-page
 
provider:
  name: aws
  runtime: nodejs8.10
  memorySize: 512
  timeout: 30
 
# Here we define the name of the layer and its path.
# Everything that's inside of /layer will be published
# in a separate package.
layers:
  chromium:
    path: layer
 
functions:
  screenshot:
    handler: handler.screenshot
    events:
      - http:
          method: get
          path: /screenshot
    # Here we reference the layer used by this function.
    # The Ref name is generated by TitleCasing the layer name & appending "LambdaLayer".
    layers:
      - { Ref: ChromiumLambdaLayer }

On handler.js we add the Puppeteer code to screenshot a page:

const puppeteer = require('puppeteer-core')
 
module.exports.screenshot = async function (event) {
  // Get the url from parameters
  const { url } = event.queryStringParameters
 
  // Launch a puppeteer instance
  const browser = await puppeteer.launch({
    headless: true,
    args: ['--no-sandbox', '--disable-gpu', '--single-process'],
    // Layer dependencies will be available on "/opt"
    // during the Lambda execution.
    executablePath: '/opt/headless-chromium',
  })
 
  const page = await browser.newPage()
 
  await page.goto(url)
 
  // Screenshots the page
  const image = await page.screenshot({ encoding: 'base64' })
 
  return {
    body: `<img src="data:image/png;base64,${image}" />`,
    headers: { 'Content-Type': 'text/html' },
    statusCode: 200,
  }
}

Note that, because we are using puppeteer-core, we need to provide the path where the Chromium binary is hosted using the executablePath option.

Another important thing about Layers is that they extract the dependencies on the /opt folder during the function execution.

Deploying the function

At this point we are ready to deploy our function and the Layer containing the Chromium binary. Just run the command below in your terminal:

yarn sls deploy

The output will look like this:

Serverless: Packaging service...
Serverless: Excluding development dependencies...
Serverless: Excluding development dependencies...
Serverless: Uploading CloudFormation file to S3...
Serverless: Uploading artifacts...
Serverless: Uploading service .zip file to S3 (790.45 KB)...
Serverless: Uploading service .zip file to S3 (47.74 MB)...
Serverless: Validating template...
Serverless: Updating Stack...
Serverless: Checking Stack update progress...
...................
Serverless: Stack update finished...
Service Information
service: screenshot-page
stage: dev
region: us-east-1
stack: screenshot-page-dev
api keys:
  None
endpoints:
  GET - https://km8mqcwfvk.execute-api.us-east-1.amazonaws.com/dev/screenshot
functions:
  screenshot: screenshot-page-dev-screenshot
layers:
  chromium: arn:aws:lambda:us-east-1:372768542448:layer:chromium:8
Serverless: Removing old service artifacts from S3...
Done in 295.39s.

Here we can see that the Layer and the function code got split into two separate .zip files and were uploaded independently.

The Layer ARN is also present on the output:

layers:
  chromium: arn:aws:lambda:us-east-1:372768542448:layer:chromium:8

If we ever need to use Chromium on any other function we can simply reference this Layer’s ARN on the AWS Dashboard.

Tip: You can also publish the Layer without any function with Serverless. Just remove the functions section on your serverless.yml and run the deployment command normally.

To test the function just visit the url:

https://km8mqcwfvk.execute-api.us-east-1.amazonaws.com/dev/screenshot?url=https://news.ycombinator.com

And you will see the screenshot:

HN Screenshot

Conclusion

Layers is a very welcome addition to the AWS Lambda toolset. Bundling dependencies on Lambdas used to be a difficult task, specially for large binaries. Now with Layers it became easier to manage, share and keep dependencies updated.

Serverless also provides a very simple and straightforward way to use and deploy Lambda Layers to AWS.