This content originally appeared on DEV Community and was authored by koyopro
I am developing a TypeScript ORM library called Accel Record. Unlike other TypeScript/JavaScript ORM libraries, Accel Record has adopted a synchronous API instead of an asynchronous one.
In this article, I will explain the background and reasons for adopting a synchronous API in Accel Record.
The ORM We Wanted to Create
In the article "Seeking a Type-Safe Ruby on Rails in TypeScript, I Started Developing an ORM," I introduced the start of my work on a TypeScript ORM library.
My goal was to have a framework for TypeScript that is as efficient as Ruby on Rails. To achieve this, I decided to try creating an ORM in TypeScript with functionalities similar to Rails' Active Record. Hence, the first step in creating this new ORM was to imitate the API of Active Record.
Problems with Asynchronous APIs
In JavaScript/TypeScript, database access is typically done using asynchronous APIs with Promises or callbacks. Since libraries for database access also return Promises for each operation, the new ORM naturally implemented each API asynchronously.
When executing asynchronous APIs, it is common to use await to handle them in sequence. For example, when performing CRUD operations on the User model, it looks like this:
await User.create({ name: "Foo" }); // Create
const user = await User.find(1); // Read
await user.update({ name: "Bar" }); // Update
await user.delete(); // Delete
Although it’s somewhat tedious to write await each time, it wasn't a major issue initially. However, the problem became more significant when handling associations. Consider a case where the User model has a hasOne association with the Setting model.
Problem Example 1: Updating Associations
I wanted to write the update process like this, following the Active Record interface:
const setting = Setting.build({ theme: "dark" });
await user.setting = setting; // This is not possible
The reason for adding await here is that this operation might involve database access. In Rails' Active Record, if this setter causes a change in either the user or setting, a database access (save operation) occurs.
However, since TypeScript setters cannot be asynchronous, this kind of syntax is not possible. We needed to consider an alternative interface.
Problem Example 2: Loading Associations
When fetching associations, await needs to be written each time because there is a possibility of database access.
const theme = (await user.setting).theme;
In Active Record, associations are lazy-loaded, so database access may occur when fetching associations. If the user instance already has the setting cached, no database access occurs; otherwise, it does. This also necessitated reconsidering the interface due to usability issues.
Each of these issues could be somewhat resolved by designing the interface carefully. However, I felt that repeatedly making such adjustments was gradually leading the library’s usability away from the ideal. The asynchronous API was restricting the library's interface.
The Conceptual Shift to Synchronous APIs
Rails' Active Record abstracts database access processes by associating tables with model classes. However, making these APIs asynchronous means constantly being aware of the timing of database access, which hinders abstraction and reduces development efficiency.
Continuing with an asynchronous API to implement the ORM made me realize that achieving the same level of abstraction as Active Record would be difficult, and so would achieving high development efficiency, which was the initial goal.
So, I reconsidered the API design and thought there might be another approach.
I decided to question the assumption that "database access APIs in JavaScript/TypeScript must be asynchronous."
Synchronous API calls without Promises do not require await. If we could implement each API as a synchronous API rather than an asynchronous one, the aforementioned issues would not arise. There would be no need to worry about whether to add await to each method call, allowing a development experience closer to Active Record.
Thus, the next step was to answer the following questions:
- Why do JS/TS database access libraries adopt asynchronous APIs?
- Is it absolutely necessary to use asynchronous APIs?
- Can an ORM be created using synchronous APIs?
Node.js Event Loop and Synchronous Processing
The goal of this ORM library is to provide a development experience similar to Ruby on Rails’ Active Record. Therefore, it is intended for server-side use, not for frontend use. The most common execution environment for server-side TypeScript (JavaScript) is Node.js. So, I investigated the challenges of using synchronous APIs in Node.js.
The most relevant information from Node.js official documentation includes:
Node.js uses a single-threaded, asynchronous I/O model, which achieves high performance by utilizing time waiting for I/O to perform other tasks. JavaScript and Node.js have the concept of an event loop, but synchronous processing can block the event loop, preventing other tasks from being processed during that time. This could potentially degrade system performance compared to using asynchronous processing.
The Downsides of Synchronous Processing and Mitigation Strategies
For example, if a web application handles HTTP requests only with synchronous APIs, it cannot accept other requests until the current one is completed. However, this behavior is common in other languages. In Ruby on Rails, for instance, a process typically cannot handle other requests until one request is completed.
Thus, while using synchronous APIs may lower performance compared to asynchronous Node.js applications, it is not necessarily inferior in performance compared to applications in other languages.
Moreover, this refers to the performance per thread. By running multiple processes or threads in parallel, overall system performance may not significantly decrease. Parallelizing server-side processing with multiple processes is very common in web applications. For instance, in Ruby, it's common to use application servers like Unicorn to run multiple processes.
Even in Node.js, processes can be run in parallel, and there are mechanisms to run certain tasks on separate threads without blocking the event loop . 1
While using synchronous processing in Node.js may reduce performance per thread, system-level performance degradation can potentially be avoided through system architecture.
Additionally, in serverless environments like AWS Lambda, where handling multiple requests concurrently in a single process (container) is uncommon, synchronous processing may not impact performance significantly.
Prioritizing Development Efficiency Over System Performance
Ultimately, my goal is to have a framework for TypeScript with development efficiency comparable to Rails. What I prioritize is development efficiency, not system performance.
Many development environments prioritize development efficiency over system (per-thread) performance. If performance was the only concern, faster languages like C would always be chosen for server-side development. However, languages like PHP and Ruby, which are relatively slower, are also popular. This is because these languages and frameworks are considered to provide a more efficient development environment.
Adopting a Synchronous API in the New ORM
Based on the investigation, I have organized answers to the initial questions:
- Why do JS/TS database access libraries adopt asynchronous APIs?
- To avoid blocking the JavaScript event loop. Blocking the event loop can lead to degraded system performance.
- Is it absolutely necessary to use asynchronous APIs?
- Not necessarily. If the degradation in performance per thread is acceptable, synchronous APIs can be used. (And, as mentioned, there are ways to mitigate this downside through system architecture.)
- Can an ORM be created using synchronous APIs?
- There seems to be no inherent restriction in JavaScript or Node.js preventing this.
If the ORM is designed with synchronous APIs, the main downside would be a degradation in (per-thread) performance. However, as discussed, this can be mitigated through system architecture. Therefore, weighing this downside against the benefit of improved development efficiency for library users, I concluded that the benefits of adopting synchronous APIs outweigh the downsides.
Thus, the new ORM, Accel Record, has adopted synchronous APIs despite being a TypeScript library.
Conclusion
In this article, I explained the background and reasons for adopting synchronous APIs in Accel Record.
First, it was challenging to achieve the ideal ORM interface with asynchronous APIs. Asynchronous APIs restricted the library's interface. This led to investigating whether a synchronous API could be adopted. Although there were concerns about system performance, considering the project's goals, we determined that the benefits (improved development efficiency) of realizing the ideal interface were significant.
To see how Accel Record achieves an ideal interface with synchronous APIs, please check out the Introduction to "Accel Record": A TypeScript ORM Using the Active Record Pattern or the README.
This content originally appeared on DEV Community and was authored by koyopro
koyopro | Sciencx (2024-06-20T16:10:09+00:00) Why We Adopted a Synchronous API for the New TypeScript ORM. Retrieved from https://www.scien.cx/2024/06/20/why-we-adopted-a-synchronous-api-for-the-new-typescript-orm/
Please log in to upload a file.
There are no updates yet.
Click the Upload button above to add an update.