This content originally appeared on DEV Community and was authored by Hugh
Here is how I reduced the package size of my lambda functions 35MB per function to ~1MB and cold-start init times from 606ms to 291ms.
Introduction
We have recently been using the serverless-esbuild plugin instead of serverless-plugin-typescript package when making new Typescript serverless projects because it transpiles much faster (saving development time) and can create more optimized package sizes.
However, without proper configuration, your transpiled code can end up surprisingly large. This can impact the speed of deployment pipelines and cold start times of your lambda functions.
Related article: Size is (almost) all that matters for optimizing AWS Lambda cold starts | by Adrian Tanasa | Medium.
Furthermore, if the package size of a single function gets too large (50MB zipped), it will actually fail to deploy due to AWS limits on lambda package size (this happened to me and was the motivation for me to optimise file size and make this guide).
So, here are the steps to optimise serverless-esbuild settings:
Analyzing your bundle size
To package your serverless project locally, run:
serverless package -s offline
Note: the stage option (-s offline) is optional, depends on if you use different stages or not.
This generates a .serverless folder in the root of your project, which contains a zip file of your package. Open the zip file to check the index.js and index.js.map file sizes.
By setting the metafile option in serverless.yml, you can analyze the size of dependencies.
# serverless.yml
custom:
esbuild:
metafile: true # for local analysis only!
Don’t forget to comment out this option or revert to false before deploying, as the metafile is unnecessary in production and will increase the package size.
Once again packaging your project, you should see the meta files in the zipped package (index-meta.json).
You can use the esbuild - Bundle Size Analyzer tool to easily analyze the size of your dependencies:
In this example, the moment-timezone , sequelize, and iconv-lite shouldn’t have been in some of the functions, as those dependencies were not required by those functions, and accidentally imported into the source code through a shared common module.
This resulted in every lambda function having an extra unnecessary ~2.5MB of scripts to load.
Optimize esbuild settings
#serverless.yml
plugins:
- serverless-esbuild
package:
individually: true
custom:
# Here you can define your esbuild options
esbuild:
bundle: true
sourcemap: true
minify: true
exclude: ['@aws-sdk']
# custom plugin to remove vendor sourcemaps
plugins: esbuild-plugins.js
Explanation of settings:
package:
individually: true
The option will create a separate zip file for each lambda function, speeding up deployment and lowering size per function. If you don’t set this to true, each of your lambda functions will contain the entire source code of the whole project and all dependencies, rather than only the files it needs.
This is one of the most important features for reducing filesize.
sourcemap: true
Sourcemap option should be true because we want to be able to locate stack traces thrown in production environments easily. However, there is a major downside: sourcemaps will include all node_modules packages resulting in an increase in package size. I will explain how to remedy this in the next section.
minify: true
Minifies code, resulting in further file size reduction
exclude: ['@aws-sdk']
Exclude aws-sdk from your package (if you use it), since Node.js lambda functions automatically have access to the AWS libraries when deployed!
Removing vendor source maps
The source map files generated by esbuild can be extremely large if you have many heavy dependencies (larger than the source code). This is because esbuild generates sourcemaps not just for your code, but for all imported node_modules too.
While it can sometimes be useful to have vendor sourcemaps when debugging code, the use case for debugging 3rd party package’s source code from production stack traces is quite uncommon.
To exclude vendor scripts from your source files, we need to write a custom plugin for esbuild.
First add this option to your serverless.yml:
# serverless.yml
custom:
esbuild:
plugins: esbuild-plugins.js
Then create a esbuild-plugins.js file in the same directory as the serverless.yml file:
// esbuild-plugins.js
const { readFileSync } = require('fs');
const excludeVendorFromSourceMapPlugin = {
name: 'excludeVendorFromSourceMap',
setup(build) {
// eslint-disable-next-line consistent-return
build.onLoad({ filter: /node_modules/ }, (args) => {
if (args.path.endsWith('.js') || args.path.endsWith('.mjs') || args.path.endsWith('.cjs')) {
return {
contents:
// eslint-disable-next-line prefer-template
readFileSync(args.path, 'utf8') +
'\n//# sourceMappingURL=data:application/json;base64,eyJ2ZXJzaW9uIjozLCJzb3VyY2VzIjpbIiJdLCJtYXBwaW5ncyI6IkEifQ==',
loader: 'default',
};
}
});
},
};
module.exports = [excludeVendorFromSourceMapPlugin];
What this script does is replace sourcemaps of files from the node_modules folder with an empty sourcemap template.
Applying this plugin reduced sourcemap file in each lambda function of my project from 10MB to about 2.5MB.
Avoid importing dependencies that aren’t needed
ESBuild has tree-shaking built in, meaning that only modules your lambda function imports (from the module containing lambda handler's entry point) will be included in the bundle. However, it is easy to unintentionally import modules that your lambda function doesn’t need indirectly through commonly shared modules.
Use the bundle analyzer and find out if there are any large dependencies that look like they shouldn’t be there for thespecific lambda function you’re analyzing.
In one project, we used the sequelize package to connect to a database. However, only some of these functions actually required a database connection.
The code was unintentionally importing the ‘sequelize’ package in every lambda function indirectly through a commonly imported module, adding around 1MB of script size per lambda function.
A simple refactor to remove the sequelize import when a DB connection wasn’t needed reduced the file size significantly.
Conclusion
Before optimising esbuild, the project package was 60MB, and this was higher than the limit allowed by AWS.
Furthermore, as I hadn’t packaged functions individually, every single lambda function would have been 60MB, making the total storage space used by 18 functions over 1GB.
These optimization changes allowed me to reduce package size to around 0.5MB to 1.5MB per function, totalling only 16MB uploaded to AWS, a dramatic reduction.
Furthermore, the deployment pipeline became quicker and the init duration of a simple ‘Hello World’ function, added to cold starts, from 606ms to 291ms.
Summary
- Use esbuild - Bundle Size Analyzer (after setting metafile esbuild option to true) to observe dependencies size
- Add
aws-sdkto the exclude array in esbuild options (reducing script slightly) - the aws library should exist in the deployed environment. - Set the esbuild
minifyoption to true inserverless.yml(another slight reduction) - Package functions individually using serverless.yml setting (large reduction of total size uploaded to AWS for multiple functions)
- Added a plugin to exclude vendor sourcemaps from the scripts (big reduction) exclude node_modules from source map Issue #1685 · evanw/esbuild · GitHub
- Use the
serverless packagecommand locally to check the file size output in .serverless folder.
This content originally appeared on DEV Community and was authored by Hugh
Hugh | Sciencx (2024-07-18T11:05:37+00:00) Optimising package size for Typescript AWS Lambda functions using serverless-esbuild. Retrieved from https://www.scien.cx/2024/07/18/optimising-package-size-for-typescript-aws-lambda-functions-using-serverless-esbuild/
Please log in to upload a file.
There are no updates yet.
Click the Upload button above to add an update.