This content originally appeared on Level Up Coding - Medium and was authored by Lee James Gilmore
A practical example of using AWS Lambda Layers to generate screenshots using a headless browser in Lambda.
This blog post goes through an introduction of AWS Lambda Layers, and then builds out a serverless screenshot application using the Serverless Framework, TypeScript and Puppeteer. You can access the code repo example here. I found that most developers using Serverless forget about this feature, and it is very useful to have this in your mental toolkit when building out applications?
Puppeteer is a Node library which provides a high-level API to control Chrome or Chromium over the DevTools Protocol, which can be ran inside a lambda to perform tasks such as:
- Generate screenshots and PDFs of pages.
- Crawl a SPA (Single-Page Application) and generate pre-rendered content (i.e. “SSR” (Server-Side Rendering)).
- Automate form submission, UI testing, keyboard input, etc.
- Create an up-to-date, automated testing environment. Run your tests directly in the latest version of Chrome using the latest JavaScript and browser features.
- Capture a timeline trace of your site to help diagnose performance issues.
Another great example of using the AWS provided Lambda Layers is discussed in this earlier blog post around Feature Flags with AWS AppConfig:
What are Lambda Layers?
So what are Lambda Layers and why would I need them? Danilo Poccia (Chief Evangelist (EMEA) at Amazon Web Services) describes them as:
“When building serverless applications, it is quite common to have code that is shared across Lambda functions. It can be your custom code, that is used by more than one function, or a standard library, that you add to simplify the implementation of your business logic.
Previously, you would have to package and deploy this shared code together with all the functions using it. Now, you can put common components in a ZIP file and upload it as a Lambda Layer. Your function code doesn’t need to be changed and can reference the libraries in the layer as it would normally do.
Layers can be versioned to manage updates, each version is immutable. When a version is deleted or permissions to use it are revoked, functions that used it previously will continue to work, but you won’t be able to create new ones.”
A visual example of this reuse is shown in the following animation:
This is fantastic for code reuse and keeping functions DRY, but one additional benefit is your lambda package size for deployments when working with large dependencies such as ‘Chromium’ and ‘Puppeteer’. ? Couple this with the fact that many lambda layers are available open source to utilise in your own code directly using the ARN of the layer.
This won’t reduce your cold start times as such, but will increase the speed of deployments, ensure your lambdas are just your ‘code’, enables shared code and asset reuse, and ensures you don’t hit the dreaded ‘CodeStorageExceeded’ error (especially if you have multiple lambdas all using these large dependencies!).
Using layers in this particular code example removes 41MB from the overall function size which makes deployments quick as it is then only 61KB in size!
There is a fantastic curated list of open source Lambda Layers here which you can use directly in your serverless config by linking to the layer ARN in the same region as your function rather than a local zip file (this does however make local development a challenge with this method)
GitHub - mthenw/awesome-layers: λ A curated list of awesome AWS Lambda Layers.
How do Lambda Layers work?
Lambda functions can have up to five different lambda layers, with the total size of the Lambda deployment package with all layers unzipped not exceeding 250 MB.
A Lambda layer is a .zip file archive that can contain additional code or data. A layer can contain libraries, a custom runtime, data, or configuration files.
Lambda extracts the layer contents into the /opt directory when setting up the execution environment for the function.
Remember that all five layers will be extracted to the same /opt folder so ensure that you don’t have file names that clash or the files will be overwritten by a subsequent layer.
You can then use the assets, dependencies etc by accessing them from the /opt folder in your lambda code.
What are we building?
We are going to build an online screenshot tool using layers which:
- allows users to post a payload to the /screenshots/ endpoint, which includes the webpage we want to screenshot.
- A lambda function will be invoked to generate the screenshot of the webpage based on the payload from API Gateway (a headless browser in Lambda).
- The screenshot is then saved in a public S3 bucket for viewing, and the asset path is returned in the result to the consumer for viewing.
Let’s get building!
The following extracts from the code repo shows the main configuration in building this solution.
Building/Adding the layer
The chrome_aws_lambda layer is a zip file which is stored in a layers folder as shown here.
You can build the chrome-aws-lambda layer using the following commands which are detailed here to generate the zip file:
git clone --depth=1 https://github.com/alixaxel/chrome-aws-lambda.git && \
cd chrome-aws-lambda && \
make chrome_aws_lambda.zip
Deploying the layer
Using the serverless framework we can then ensure that this layer is deployed to AWS (which will also be versioned):
Adding the layer to the function
The layer can then be linked to the function using the following code:
Once deployed, when you log into the AWS console you will see the layer attached to the function:
Local vs AWS
We use a combination of Webpack and the Serverless Framework to ensure that if we are running locally with serverless-offline we can use the dev dependencies of chrome-aws-lambda and puppeteer, but when deployed to AWS we don’t bundle them as we have the lambda layer to use, and we just use pupeteer-core (by default this version does not download the browser) to perform the interaction with the layer (and only deploy the code).
This is one of the disadvantages of lambda layers, as many local build tools won’t work when pointing to the shared lambda layer by ARN, which can make local development difficult. Hence the workaround above.
The function code
The basic code (including verbose comments for the example) for generating the screenshot and returning the path from S3 for the user to view can be shown below (this is not production code! Its a basic example to demonstrate layers):
Generating the screenshot
Once deployed to the cloud (or ran offline) and using the Postman collection in the repo, you can invoke the endpoint with a POST call and the following payload as an example:
{ "webpage": "https://stackoverflow.com/" }
When clicking on the return URL i.e. the message property, you will be able to download the screenshot PNG of Stackoverflow as show below!
One idea I have for playing around with would be a visual diff CI/CD tool for frontend development which I am going to take a look at next.
Wrapping up
I would love to connect with you on any of the following:
https://www.linkedin.com/in/lee-james-gilmore/
https://twitter.com/LeeJamesGilmore
If you found the articles inspiring or useful please feel free to support me with a virtual coffee https://www.buymeacoffee.com/leegilmore and either way lets connect and chat! ☕️
If you enjoyed the posts please follow my profile Lee James Gilmore for further posts/series, and don’t forget to connect and say Hi ?
Please also use the ‘clap’ feature at the bottom of the post if you enjoyed it! (You can clap more than once!!)
About me
“Hi, I’m Lee, an AWS certified technical architect and Lead Software Engineer based in the UK, currently working as a Technical Cloud Architect and Principal Serverless Developer, having worked primarily in full-stack JavaScript on AWS for the past 5 years.
I consider myself a serverless evangelist with a love of all things AWS, innovation, software architecture and technology.”
** The information provided are my own personal views and I accept no responsibility on the use of the information. ***
Serverless Lambda Layers ? was originally published in Level Up Coding on Medium, where people are continuing the conversation by highlighting and responding to this story.
This content originally appeared on Level Up Coding - Medium and was authored by Lee James Gilmore
Lee James Gilmore | Sciencx (2021-07-21T17:16:10+00:00) Serverless Lambda Layers. Retrieved from https://www.scien.cx/2021/07/21/serverless-lambda-layers/
Please log in to upload a file.
There are no updates yet.
Click the Upload button above to add an update.