This content originally appeared on Bits and Pieces - Medium and was authored by Ashan Fernando
Can we break the monolithic Infrastructure as Code into Composable Components?
With the wide adoption of the cloud, modern infrastructure is becoming complex and rapidly evolving with continuous improvements. And a tiny change you make may have adverse effects unless you understand its impact. That’s where the infrastructure in code comes into the picture.
When writing the entire infrastructure using a programming language, we source control and version it. As a result, we can review each change we make and incrementally reflect it into the existing infrastructure.
History of IaC
In the early days, Cloud infrastructure was created manually through web consoles or by running CLI commands. This approach was time-consuming, error-prone, and lacked scalability.
Then, people started scripting the Cloud infrastructure by creating Shell scripts. These scripts contain code that creates infrastructure resources sequentially and lays the foundation for tracking infrastructure in source control systems like Git. The main advantage was that we could see our changes, making it possible to re-run the scripts to create new infrastructure whenever required. However, after running the infrastructure, the scripts lacked visibility into what was being created in the Cloud. In short, there isn’t a way to see the current state of the cloud without simply running the scripts.
Moving into the next generation, domain-specific language (DSL) emerged to define infrastructure such as Terraform, AWS CloudFormation, and Azure Bicep. These could keep track of the state in the Cloud, allowing us to make intelligent decisions when re-running scripts. Terraform took that even further, allowing Dry-Run to evaluate the effects of IaC changes and show them to the user before executing them. The main problem here was that developers needed to learn these DSLs. Not many IDEs and tools supported them.
Today, the infrastructure is written using commonly used programming languages like TypeScript. Pulumi, AWS CDK, and Azure CDK support it out of the box. Most of these tools provide ways to break the infrastructure into subgroups (e.g. Pulumi Microstacks). Yet, when it comes to the stage where we want to break things, it becomes difficult.
Composable Infrastrucuture
That’s where Composable Infrastructure comes into the picture. We can start designing the infrastructure bottom-up by using these modern tools that support general programming languages and a toolchain for componentization.
This article will use Pulumi (IaC Framework) and Bit (Toolchain for Composability) to demonstrate how composable infrastructure can be implemented.
One of the major benefits of creating Composable Infrastructure is that you can also connect how your code is deployed in a unified way similar to how you create the infrastructure.
For example, you can see a vite-react app (frontend) and hello-service (lambda function code) found as components in the same dependency graph.
The Pulumi App Component
To support Pulumi command execution on the development machine and in CI, you need to create a Pulumi App component. This component acts as the entry point for executing Pulumi commands.
You can create a new app component by using the example application.
bit fork bitpulumi.apps/pulumi-entry
You can see that it depends on the Pulumi App Type to handle Pulumi-specific functionality.
After forking the app component, you can modify its YAML files to point to your Pulumi Stack. In addition to that, you can update the entry points for use resources by editing the pulumi-entry.ts file.
import { Output } from "@pulumi/pulumi";
import * as pulumi from "@pulumi/pulumi";
import { apiRoutes } from "@bitpulumi/awsdemo.api.api-routes";
import bitpulumi from "@bitpulumi/awsx.web-s3";
const API_NAME = "api";
const apiRouteInstance = apiRoutes(API_NAME);
const WEB_BUCKET_NAME = "web-bucket";
const webBucketInstance = new bitpulumi.awsx.WebS3(
WEB_BUCKET_NAME,
require.resolve("@bitpulumi/apps.vite-app")
);
/* Pulumi stack exports */
export const bucketName = webBucketInstance.bucket.id.apply(name => name) as pulumi.Output<string>;
export const cloudfrontUrl = webBucketInstance.cloudfrontDistribution.domainName.apply(
(domainName) => `https://${domainName}`
) as pulumi.Output<string>;
export const apiUrl = apiRouteInstance.url.apply(
(url) => `${url}api/hello`
) as pulumi.Output<string>;In this example, we are creating an AWS S3 bucket to store the Vite-React app’s build artefacts, which will be served through AWS CloudFront distribution. We are also creating an API using AWS Api Gateway and Lambda functions.
To execute Pulumi commands locally, you can go into the directory and execute Pulumi commands like pulumi up as required. Another way to deploy the infrastructure is that after modifying any component, you can run;
bit tag -m "modified app component"
bit export
This will build the app in Ripple CI and execute pulumi up command to deploy the resources into AWS cloud. You need to configure the AWS credentials into Ripple Environment variables for it to work.
export PULUMI_STACK_NAME=dev
export PULUMI_ACCESS_TOKEN=<my-token>
export AWS_ACCESS_KEY_ID=<my-key>
export AWS_SECRET_ACCESS_KEY=<my-secret>
Frontend Component Unifies with IaC
In Bit, you can create a Vite-React app using the prebuilt app type for Vite. However, any of the following app types may also work. This part doesn't directly link with Pulumi since the focus is only on creating an app that can be bundled into an application.
You can create an app by running the following command or fork the one from the example.
bit create bit-app my-app
Then, you need to create the AWS S3 bucket to store the Vite app's build artefacts and AWS Cloudfront to serve via its CDN URL. Since this is a common functionality many will use, we have created the class bitpulumi.awsx.WebS3 that handles the resource creation.
const WEB_BUCKET_NAME = "web-bucket";
/* Create S3 Bucket */
const webBucketInstance = new bitpulumi.awsx.WebS3(
WEB_BUCKET_NAME,
require.resolve("@bitpulumi/apps.vite-app")
);
/* Pulumi Stack Exports */
export const bucketName = webBucketInstance.bucket.id.apply(name => name) as pulumi.Output<string>;
/* Create CloudFront distribution */
export const cloudfrontUrl = webBucketInstance.cloudfrontDistribution.domainName.apply(
(domainName) => `https://${domainName}`
) as pulumi.Output<string>;
This creates the AWS resources to host the Vite app and then deploy the Vite app’s artefacts deploying the application.
Backend Components for API
Looking at the backend components, we find three main components contributing to the API.
- API Gateway Route and Lambda Function Configuration
- Lambda Code
- Lambda Env (Bit Specific)
The following shows how to define an API endpoint and a Lambda configuration.
// @file: api-routes.ts, DefineAPI Gateway Route
import * as apigateway from "@pulumi/aws-apigateway";
import bitpulumi from "@bitpulumi/awsx.lambda";
export function apiRoutes(endpointName: string) {
const api = new apigateway.RestAPI(endpointName, {
routes: [
{
path: "/api/hello",
method: "GET",
eventHandler: new bitpulumi.awsx.Lambda(
"hello-lambda",
require.resolve("@bitpulumi/awsdemo.services.hello-service"),
{
environment: {
variables: { DATE_TYPE: "Today" }, // Optional environment variables
},
}
),
},
],
});
return api;
Here, we can also find a helper class named bitpulumi.awsx.Lambda that handles the default policies and other configurations used by many by convention.
The actual Lambda code resides in the resolved component @bitpulumi/awsdemo.services.hello-service .
import { APIGatewayProxyHandler } from 'aws-lambda';
import { format } from 'date-fns';
export const handler: APIGatewayProxyHandler = async () => {
const dateType = process.env.DATE_TYPE;
const message = `Hey!, ${dateType} is ${format(new Date(), "yyyy-MM-dd")}`;
return {
statusCode: 200,
body: JSON.stringify({ message }),
};
};You can fork the component into your workspace and write your own function code.
bit fork bitpulumi.awsdemo/services/hello-service
One major improvement when using Pulumi with Bit is that it now supports bundling the Lambda function before deploying it to AWS. This improves upon Pulumi’s highly inefficient approach of bundling the node_modules directory with all the functions.
Here, the Lambda Env handles the bundling of Lambda function code.
In addition to these, you can decide to create database tables, queues, email services, DNS mapping, etc., using the same unified approach.
Deploying Changes
Once you set up the components and their compositions, you can deploy the applications by tagging them and exporting them to the bit.cloud.
bit tag -m "Updated components"
bit export
Soon after running the commands, Ripple CI builds the components and their dependencies in parallel and deploys them into appropriate services.
Conclusion
Breaking down IaC components opens up a new way of creating composable and robust infrastructure components.
To maximize its potential, you can start breaking down Lambda functions into sub-components and sharing components with each other.
Thanks for reading !! Cheers !!
Learn More
- Modern Infrastructure as Code (IaC) with Independent Bit Components
- Creating Infrastructure Components with Pulumi and Bit
- Managing Micro-Stacks using Pulumi
Moving Beyond IaC: Infrastructure into Components was originally published in Bits and Pieces on Medium, where people are continuing the conversation by highlighting and responding to this story.
This content originally appeared on Bits and Pieces - Medium and was authored by Ashan Fernando
Ashan Fernando | Sciencx (2024-05-31T20:47:28+00:00) Moving Beyond IaC: Infrastructure into Components. Retrieved from https://www.scien.cx/2024/05/31/moving-beyond-iac-infrastructure-into-components/
Please log in to upload a file.
There are no updates yet.
Click the Upload button above to add an update.