This content originally appeared on Level Up Coding - Medium and was authored by Manoj Chemate
Java 19 Is Here: An Exciting Solution to Your Fear of Scaling Threads — Lightweight Virtual Threads!
Java 19 introduced the exciting preview feature, Virtual Threads! Here, you will understand the internals of java threads and why do we even need virtual threads, and how do they work!
I will get back to that monkey later, let’s get started by understanding existing Java threads, shall we?
What is Java Thread?
- A Thread in Java is equivalent to an operating system thread (Platform thread)
- What does that mean? Let’s understand the thread first.
A thread is just a sequence of instructions executed on the CPU core, and assigning this sequence of instructions to CPU cores is done by the scheduler. So a thread can be thought of as a composition of a sequence of instructions and a scheduler. A sequence of instructions is a continuation.
JVM uses implementations of continuation and scheduler provided by the underlying operating system hence Java thread is considered as equivalent to platform thread.
What is Not Good About Platform Threads?
Scale
- A server can handle a million concurrent user requests but JVM uses the operating system’s threads for its implementation of Java threads, cannot efficiently handle more than a few thousand.
But why can’t we scale platform threads?
- Creating and managing the platform thread has to make system calls, which is slow (that is why we use thread pools)
- OS can only manage a finite number of threads.
- Platform threads take up considerable memory just to exist.
- Whenever there is a blocking I/O(Input/Output) operation like a network call or storage read, the thread has to wait until it gets a response from the network or storage, at this point threads don’t need CPU cores and they go into a waiting state until they receive data, that’s a context switch. When you have lots of I/O, threads are just waiting for data, and the CPU is not utilized to its full capacity.
- Changing thread state from running to waiting means taking the thread context (CPU registers + Program Counter+Stack) out of the CPU core, storing that state, and assigning another thread to the CPU (that’s Context Switch) which takes a significant time and memory.
so,
1. Context switch of platform thread is costly in terms of resources.
2. Platform thread is just waiting there for data doing nothing which could have been utilized for other CPU-bound operations.
3. If you have lot of I/Os, CPU is not utilized fully.
OK, But don’t Reactive Frameworks Like Project Reactor Solve These Problems?
- Yes and No, Reactive programming offers non-blocking I/O by letting a running thread make a blocking call and then immediately go ahead to execute the rest of the program without waiting for data from storage or network.
But then what about that skipped blocking code (network call or storage read)?
- It keeps an event loop that continuously keeps checking whether the socket received data from the network/storage, and once data is received it will be processed by another thread, and hence running thread does not get blocked here.
- But, but, but if you have worked on reactive code, you know it is difficult to read, difficult to debug, difficult to unit test, the coding style does not fit into the existing synchronous code style and it needs reactive model support from data sources as well.
Well, So What Is A Virtual Thread?
- It is a thread managed by JVM scheduler unlike a platform thread managed by OS scheduler. It extends java.lang.Thread class but it is not managed by the OS scheduler.
- Virtual thread is made up of continuation and scheduler both implemented by JVM. Continuation is a sequence of instructions that is assigned to the platform thread by the JVM scheduler. Presently ForkJoinPool is used by JVM as a scheduler.
- A virtual thread can use multiple platform threads to get its work done. A platform thread is also called a carrier thread in Java documentation.
You assign task to the virtual thread → Java scheduler assigns virtual thread to platform thread → virtual thread asks platform thread to get work done.
3. Key point is, if a virtual thread has some blocking I/O operation, it will be detached from the platform thread and another virtual thread will be assigned to the platform thread so that the platform thread stays in a running state. Context of virtual thread will be stored on heap itself, hence blocking and resuming virtual thread is cheap.
4. It is important to note that here the virtual thread is blocked and not the platform thread. Since virtual threads are in an application context, blocking virtual threads is much cheaper as compared to blocking platform threads.
5. So applications having high concurrent requests with fewer CPU operations and threads spending significant time just waiting will have higher throughput with virtual threads. Concurrent requests will be handled by virtual threads and they will go into waiting state without a heavy context switch, meanwhile platform threads will utilize the CPU fully.
6. Application having just blocking calls (say API which just combines the result of two other network calls), can scale well with virtual threads since virtual threads will go in a waiting state until they receive data while platform threads will continuously use the CPU.
Since creating, blocking, and resuming virtual threads is cheaper, they are lightweight!
How To Create Virtual Threads?
Sample code for testing :
What are the Advantages of Virtual Threads?
- Applications written in the simple thread-per-request style can scale with optimal utilization of resources. You can spawn a virtual thread for every incoming request.
- If existing code is written with java.lang.Thread, it is easy to integrate virtual threads.
- You don’t need to create a pool for virtual threads as creating, pausing, and resuming them is cheap.
- Easy troubleshooting, debugging, and profiling of virtual threads with existing JDK tools.
- You can write synchronous and blocking code since blocking operations are cheap with virtual threads.
Current Limitations with Virtual Thread:
- When a virtual thread is in synchronized block or method, it can not be detached from the associated platform thread, since synchronized blocks internally use the address of the stack, as of now it is a limitation but going ahead this might be removed as per official docs. This can be easily solved using Reentrant locks instead of synchronized blocks.
It is introduced as a preview feature, so do try it out, share your feedback, and report bugs, I am excited to see it as a part of language, feel free to express your thoughts in the comments.
that’s it. thanks to Project Loom for working over years on virtual threads, Thanks for reading!
If you learned something new from this article, hit the follow and subscribe button for such content!
Still interested in that monkey?
In my wildest imagination, I think of virtual threads as monkeys riding elephants (heavy platform threads) to get their work done. They can jump from one elephant to other elephant and when there is no work to be done(blocking I/O), just get down and let others ride so that an elephant never sits.
Checkout my other post about stream APIs,
8 Years of Java Stream API, Understand Streams Through 8 Questions!
Java 19 Is Here: An Exciting Solution to Your Fear of Scaling Threads, Lightweight Virtual Threads! was originally published in Level Up Coding on Medium, where people are continuing the conversation by highlighting and responding to this story.
This content originally appeared on Level Up Coding - Medium and was authored by Manoj Chemate
Manoj Chemate | Sciencx (2022-09-26T02:57:29+00:00) Java 19 Is Here: An Exciting Solution to Your Fear of Scaling Threads, Lightweight Virtual Threads!. Retrieved from https://www.scien.cx/2022/09/26/java-19-is-here-an-exciting-solution-to-your-fear-of-scaling-threads-lightweight-virtual-threads/
Please log in to upload a file.
There are no updates yet.
Click the Upload button above to add an update.