This content originally appeared on DEV Community and was authored by Christy Jacob
Intro
Appwrite is an open-source, self-hosted Backend-as-a-Service that makes app development easier with a suite of SDKs and APIs to accelerate app development. #30DaysOfAppwrite is a month-long event focussed on giving developers a walkthrough of all of Appwrite's features, starting from the basics to more advanced features like cloud functions! Alongside we will also be building a full-featured Medium Clone to demonstrate how these concepts can be applied when building a real-world app. We also have some exciting prizes for developers who follow along with us!
Microservices of Appwrite
Welcome to Day 3 ?
Today, we take a deep dive into Appwrite's technology stack and learn what goes on under the hood. We will cover Appwrite's docker-compose file and go through all the containers that Appwrite uses. Appwrite was designed to work in a Cloud Native environment, and in keeping this spirit alive, Appwrite is conveniently packaged as a set of docker containers (17 to be precise!).
This is an overview of Appwrite's architecture, and we will talk about most of these components in the upcoming section. Each container in Appwrite handles a single microservice on its own. Since they've been containerized, each service can scale independently of any of the other microservices.
Currently, all of the Appwrite microservices communicate over the TCP protocol over a private network. You should be aware not to expose any of the services to the public-facing network, besides the public port 80 and 443, which, by default, are used to expose the Appwrite HTTP API.
Appwrite
This is the main Appwrite container and this is where all the fancy things happen! This container is built off a Dockerfile hosted here. The main Appwrite container implements the Appwrite API protocols, handles authentication, authorization, and rate-limiting. This microservice is completely stateless and can be easily replicated for scalability.
Trafeik
Traefik is a modern reverse proxy and load balancer written in Go that makes deploying microservices easy. Traefik integrates with your existing infrastructure components and configures itself automatically and dynamically. We use Traefik as the main entry point for the different Appwrite APIs. Traefik is also responsible for serving Appwrite's auto-generated SSL certificates. This microservice is completely stateless.
Redis
Appwrite uses Redis to serve three main functions.
- Caching: Appwrite uses Redis in-memory cache to fetch database queries faster.
- Pub/Sub: Appwrite uses Redis with Resque as a pub/sub mechanism to transmit messages between the Appwrite API and the different workers.
- Scheduled tasks: Appwrite uses Redis to store and trigger future tasks using the schedule container.
Appwrite's Workers
There are a lot of asynchronous tasks that need to happen in Appwrite - a good example is recording usage stats for the Appwrite API.
We use an internal pub/sub system - Resque - to accumulate all these tasks. The respective workers consume these tasks and perform the executions independently. We have eight message queues and eight workers paired with them.
-
Audits worker
The Audits worker consumes messages from the
v1-auditsqueue. Appwrite has a defined set of system events that can be found here. When these events occur, the Audits worker logs them intomariadb. The Audits worker makes use of the utopia-php/audit library. -
Certificates Worker
The Certificates worker consumes messages from the
v1-certificatesqueue. The certificate worker usescertbotfrom Let's Encrypt to create and periodically renew SSL certificates. We will be covering SSL certificates and more about the certificates worker in our upcoming article. So stay tuned to learn more. -
Deletes Worker
The Deletes worker consumes messages from the
v1-deletesqueue. As the name suggests, it performs deletions in the Appwrite Database. Delete requests for Documents, Users, Projects, Functions, etc. are handled by the Deletes Worker. In the present state, the deletes worker is triggered on certain API requests, as well as by the Maintenance worker. -
Functions Worker
The Functions worker consumes messages from the
v1-functionsqueue and handles all of the tasks related to Appwrite's Cloud Functions.
Cloud functions in Appwrite can be triggered in 3 ways:
The Functions worker does all the heavy lifting required to get Cloud Functions up and running. From pulling Docker images for the respective environments on startup, to managing and running containers, and responding to errors, the functions worker takes care of it all!
-
Mails Worker
The Mails worker consumes messages from the
v1-mailsqueue and is responsible for only one function: sending emails! It simply gathers information and uses PHPMailer to send them. -
Tasks Worker
The Tasks worker consumes messages from the
v1-tasksqueue.
Appwrite's Tasks API allows you to schedule any repeating tasks your app might need to run in the background. Each task is created by defining a CRON schedule and a target HTTP endpoint.
Each task can define any HTTP endpoint with any HTTP method, headers, or basic HTTP authentication. Inside your Appwrite console, you can view all your tasks, their current statuses, previous and next runtime, and a response log to view the result of previous executions.
-
Usage Worker
The Usage worker consumes messages from the
v1-usagequeue and makes use ofstatsdto send messages totelegrafover a UDP connection. The usage stats are then logged ininfluxDB, including function execution stats, the total number of requests, storage stats, etc. -
Webhooks Worker
The Webhooks worker consumes messages from the
v1-webhooksqueue and triggers the webhooks that were registered in the Appwrite console. The worker checks for the event that occurred and fires the corresponding webhook by making a CURL request.
Webhooks allow you to build or set up integrations that subscribe to certain events on Appwrite. When one of those events is triggered, we send an HTTP POST payload to the webhook's configured URL. Webhooks can be used to purge the cache from CDN, calculate data or send a Slack notification. You're only limited by your imagination.
Additionally, we have two more workers that delegate tasks to other workers.
-
Maintenance Worker
The maintenance worker corresponds to the
appwrite-maintenanceservice in the docker-compose file. The Maintenance worker lies here and performs some housekeeping tasks so your Appwrite server does not blow up over time! In its current state, the maintenance worker delegates deletion tasks toappwrite-worker-deleteswhich then performs the actual deletion. We use the Maintenance worker to schedule three kinds of deletions: Cleaning up Abuse logs
Cleaning up Audit Logs
Cleaning up Execution Logs
-
Schedules Worker
The Schedules worker corresponds to the
appwrite-scheduleservice in the docker-compose file. The Schedules worker uses a Resque Scheduler under the hood and handles the scheduling of CRON jobs across Appwrite. This includes CRON jobs from the Tasks API, Webhooks API, and the functions API.
Mariadb
Appwrite uses MariaDB as the default database for project collections, documents, and all other metadata. Appwrite is agnostic to the database you use under the hood and support for more databases like Postgres, CockroachDB, MySQL and MongoDB is currently under active development! ?
ClamAV
ClamAV is a TCP Anti-virus server responsible for scanning all user uploads to the Appwrite storage. The ClamAV microservice is optional and can be disabled using Appwrite environment variables. Starting with Appwrite version 0.8, this functionality is disabled by default to save memory on smaller setups. If you are having issues with excessive memory utilization, you can learn to disable it here
Influxdb
Appwrite uses InfluxDB for storing your projects' API usage metrics and stats. This is the engine used for generating your API usage graphs and handling time-series data.
Telegraf
Telegraf is a plugin-driven server agent for collecting and sending metrics and events from multiple sources to multiple destinations. Telegraf kind of protects InfluxDB by aggregating the data before sending it to the database. Telegraf operates on the UDP protocol which makes data transfer blazing fast!
Credits
We hope you liked this write-up. You can follow #30DaysOfAppwrite on Social Media to keep up with all of our posts. The complete event timeline can be found here
Feel free to reach out to us on Discord if you would like to learn more about Appwrite, Aliens, or Unicorns ?. Stay tuned for tomorrow's article! Until then ?
This content originally appeared on DEV Community and was authored by Christy Jacob
Christy Jacob | Sciencx (2021-05-03T13:44:32+00:00) #30DaysofAppwrite : Appwrite’s building blocks. Retrieved from https://www.scien.cx/2021/05/03/30daysofappwrite-appwrites-building-blocks/
Please log in to upload a file.
There are no updates yet.
Click the Upload button above to add an update.