This content originally appeared on Level Up Coding - Medium and was authored by Shivam Mathur
As a software product developer, efficiently managing background processes is crucial for creating robust and reliable applications. Temporal, an open-source platform, offers a powerful solution for orchestrating these processes. Temporal is the successor to Cadence (developed by Uber), created by the same team to address similar needs for workflow orchestration. In this article, we will explore how Temporal can be utilized to manage background processes more effectively, using a PoC KYC System project as a case study.
What is Temporal?
Temporal is a workflow orchestration engine that helps developers build scalable and reliable applications. It provides a framework for managing long-running, stateful workflows and background processes. Temporal abstracts away the complexities of handling retries, state management, and failure recovery, enabling developers to focus on writing business logic.
Key Features of Temporal
- Durable Execution: Ensures that workflows and activities are durable and can survive process failures.
- Stateful Workflows: Temporal allows developers to write workflows as code, managing complex business logic and stateful operations without worrying about infrastructure.
- Scalable Architecture: Supports high scalability and concurrency.
- Monitoring and Visibility: It provides tools for monitoring and debugging workflows in real-time, offering visibility into the execution and performance of applications.
- Fault Tolerance: Automatically handles retries and failures, ensuring robustness.
- Flexible APIs: Provides language-specific SDKs, making it easy to integrate into various environments. (Still waiting on the Rust SDK)
KYC PoC
The motivation behind creating a KYC system stems from personal experience with a complex KYC system, addressing nuanced issues. The system requires the following components:
- Notification on signup to prompt users to submit KYC details.
- Internal systems pinged upon submission of KYC details for preprocessing.
- Preprocessing of submitted KYC data via internal systems.
- Waiting for KYC agents (human input) to update details.
- Processing and notifying the user about actions taken.
- Restarting the KYC process if the user’s KYC is rejected.
These requirements were implemented using temporal workflows, activities, and signals.
Workflows
Temporal workflows are a way to define and manage long-running, stateful processes within the Temporal platform. These workflows allow developers to write business logic that can span days, months, or even years, while Temporal handles the complexities of state persistence, retries, and failure recovery.
// Workflow to manage KYC process
func KYCWorkflow(ctx workflow.Context, user model.User) (string, error) {
// ... workflow code
}
Child Workflows
Child workflows are initiated and managed by another parent workflow. They encapsulate specific tasks or subprocesses within the broader context of a parent workflow, enhancing maintainability and scalability:
// Workflow to manage KYC process
func KYCWorkflow(ctx workflow.Context, user model.User) (string, error) {
// ...
workflow.ExecuteChildWorkflow(ctx, RequestKYCWorkflow, user).Get(ctx, &response)
// ...
}
Activities
Temporal activities represent individual units of work or tasks within a workflow. They are executed asynchronously and reliably by Temporal:
func SendKYCNotification(ctx context.Context, data SendNotificationData) (string, error) {
// ... send notification to User
}
// Workflow to manage KYC process
func KYCWorkflow(ctx workflow.Context, user model.User) (string, error) {
// ...
// Trigger notification for KYC results
var result string
err := workflow.ExecuteActivity(ctx, kyc_activity.SendKYCNotification, request_kyc_notification).Get(ctx, &result)
return result, err
// ...
}Signals
Temporal signals enable communication with running workflows, allowing external processes or users to send asynchronous messages:
// Keep sending notifications to user until we get a KYC submission
func RequestKYCWorkflow(ctx workflow.Context, user model.User) (*model.KYCRequest, error) {
// ...
// set up signal
var kyc_request model.KYCRequest
kyc_signal_channel := workflow.GetSignalChannel(ctx, app.NEW_KYC_SIGNAL)
// following blocks until signal is received
kyc_signal_channel.Receive(ctx, &kyc_request)
//...
}
// KYC API Submitted by User and Signal workflow with new kyc data
func KycSubmit(w http.ResponseWriter, r *http.Request) {
// ... process user input and connection to temporal server
err = client.SignalWorkflow(context.Background(), kyc_request.UserId, "", app.NEW_KYC_SIGNAL, kyc_request)
// ... manage errors and process response
}
Timers
Temporal timers schedule actions or events within workflows, essential for implementing timeouts and scheduling retries:
workflow.NewTimer(ctx, NOTIFICATION_INTERVAL)
Selectors
Selectors in Temporal allow workflows to wait for multiple events concurrently and react to the first event that occurs:
// Keep sending notifications to user until we get a KYC submission
func RequestKYCWorkflow(ctx workflow.Context, user model.User) (*model.KYCRequest, error) {
// ...
kyc_signal_channel := workflow.GetSignalChannel(ctx, app.NEW_KYC_SIGNAL)
// Create selector to wait for signal as well as timer
// Which ever one triggers first will decide what happens in the if check later
selector := workflow.NewSelector(ctx)
selector.AddReceive(kyc_signal_channel, func(c workflow.ReceiveChannel, more bool) {
c.Receive(ctx, &kyc_request)
log.Info("RECEIVED IN CHILD")
})
// Wait for Interval
// If timer triggers first then signal will be received in the next run
selector.AddFuture(workflow.NewTimer(ctx, NOTIFICATION_INTERVAL), func(f workflow.Future) {
err = f.Get(ctx, nil)
})
selector.Select(ctx)
// ...
}
Implementation
The full working implementation of the Proof of Concept is available on GitHub.
Results
In the PoC, everything was orchestrated with retries and a built-in UI to monitor the background system. The developer experience was satisfactory, with efficient implementation and minimal time investment.
Temporal: A better way to manage background processes? was originally published in Level Up Coding on Medium, where people are continuing the conversation by highlighting and responding to this story.
This content originally appeared on Level Up Coding - Medium and was authored by Shivam Mathur
Shivam Mathur | Sciencx (2024-06-30T18:08:42+00:00) Temporal: A better way to manage background processes?. Retrieved from https://www.scien.cx/2024/06/30/temporal-a-better-way-to-manage-background-processes/
Please log in to upload a file.
There are no updates yet.
Click the Upload button above to add an update.