Project Loom - A guide to Virtual Threads
19 January 2023
Project Loom has been talked about for quite some time now, what implications does it have on the java language and handling concurrency? Will it really kill reactive programming? And doesn’t Kotlin already have coroutines since the beginning of time so why even bother? Continue to read to find out. The problem with Java’s Concurrency Model Many applications that are written in Java use some form of concurrency, like servers or databases that are required to serve many requests at the same time. Java operates on a Thread-per-request model, which simply means that every request uses a platform Thread which…
Project Loom has been talked about for quite some time now, what implications does it have on the java language and handling concurrency? Will it really kill reactive programming? And doesn’t Kotlin already have coroutines since the beginning of time so why even bother? Continue to read to find out.
The problem with Java’s Concurrency Model
Many applications that are written in Java use some form of concurrency, like servers or databases that are required to serve many requests at the same time. Java operates on a Thread-per-request model, which simply means that every request uses a platform Thread which is then mapped 1:1 to an OS Thread. However, OS Threads are quite expensive and consume memory – so you can only have so many on a server before you reach a limit. Every time a thread is waiting for a database to return some kind of data, the thread remains idle while still using scarce memory resources. Lets look at a simplified example to better understand the problem.
We have our Java Backend system that receives multiple requests from different actors. Each of those requests is mapped to a Platform Thread and lets assume all of these requests need to fetch some kind of information from a Database. While this information from the database is being fetched, all of these threads are idle. This poses following problems:
- Platform threads are expensive, and a thread for every actor, transaction or session is often not feasible.
- These threads need to be synchronized at some point, causing an expensive context switch to happen between OS Threads.
This, in it’s nature is quite inefficient, and that’s one of the reasons for reactive paradigms – to do more with less, specifically processing higher loads of traffic with fewer threads. This is achieved by reactive programming in utilizing the idle OS threads instead of blocking them and send them to work on other tasks while waiting for certain resources to be returned. However, this optimization comes at a cost, most notably:
- Complexity: Most Java developers grew up on traditional imperative programming and introducing a reactive programming paradigm introduces additional complexity not everybody on your team is going to be familiar with. So there is going to be a learning curve associated with introducing a reactive style.
- Debugging: In my experience, it can be harder to debug reactive programs since its more difficult to understand the order in which events are happening and in what ways they are affecting the program. Also, they mostly lack meaningful stacktraces, which kind of sucks.
There has been an introduction of many asynchronous APIS to the java ecosystem in recent years, Spring has even introduced a complete stack based on reactive paradigms. And while there is merit to this approach, it forces programmers to choose between loosing considerably in the scale of concurrency a single server can support, or use constructs to implement concurrency by writing asynchronous code that does not block the thread running it. Or put blunty: You can choose between harder-to-write-maintain-and-debug code, or just waste a significant amount of resources, especially in a cloud setting. So take your shitty pick!
Goals of Project Loom
Now that we have explored the root cause, it is time to use Project Loom and find out what it actually does. As per the oracle blog
Project Loom is an OpenJDK project that aims to enable “easy-to-use, high-throughput lightweight concurrency and new programming models on the Java platform.” The project aims to accomplish this by adding these new constructs:
- Virtual threads
- Delimited continuations
- Tail-call elimination
The key to all of this is virtual threads, which are designed to look to the programmer just like ordinary, familiar threads. However, virtual threads are managed by the Java runtime and are not thin, one-to-one wrappers over OS threads. Instead, virtual threads are implemented in user space by the Java runtime.
So let’s have a look at how to use virtual Threads. After making sure you are using at least JDK19 with preview enabled, it’s as simple as that:
public static void main(String[] args) throws InterruptedException {
Thread.startVirtualThread(() -> System.out.println("Hello World")).join(Duration.ofMillis(2000));
Thread.ofVirtual().start(() -> System.out.println("Hello World #2")).join(Duration.ofMillis(2000));
}
Here we are using the static method .startVirtualThread or the new VirtualThreadBuilder which lets you switch between creating virtual or platform Threads with ease:
Thread.ofPlatform().start(() -> System.out.println("Hello World #2")).join(Duration.ofMillis(2000));
Thread.ofVirtual().start(() -> System.out.println("Hello World #2")).join(Duration.ofMillis(2000));
The following .join() call is made to wait for the threads to finish. So let’s have a look at the performance
public static final int AMOUNT_OF_THREADS = 100_000;
public static void main(String[] args) throws InterruptedException {
Stopwatch timer = Stopwatch.createStarted();
for (int i = 0; i < AMOUNT_OF_THREADS; i++) {
Thread.ofPlatform().start(() -> System.out.println("Hello World " )).join(Duration.ofMillis(2000));
}
System.out.println(("Method took: " + timer.stop()));
}
Running this, I can see a difference of 25 seconds between using virtual or platform Threads. Apart from that, Project Loom will also bring a change of mindset: It will no longer be necessary to create task objects based on Runnable or Callable and handing them to executors backed by thread pools.
Fibers and Continuations
In the recent OpenJDK, a class named Fiber is introdouced along the Thread class. This class differs in the fact that they are able to suspend and resume in the Java runtime instead of the kernel, therefore making switches less expensive.
A continuation, on the other hand, is a coroutine or a sequence of instructions that can yield and be resumed by the caller at a later stage. Therefore every continuation has an entry point and a yield point.
So Loom tries to push you into the direction of using virtual threads instead of platform threads which are cheaper, with the benefit of behaving like todays threads that Java programmers already understand. So this is the end goal of Project Loom – making code look as if it was doing synchronous calls. This has the benefits of being able to continue to use familiar constructs to express our business logic as well as regaining control of the context. Since these new Fibers are natively supported on the JVM, stack traces will once again be meaningful by default. The call stack of each fiber is preserved and then restored when the fiber continues.
Wait a minute.. isn’t this just like Kotlin coroutines?
In short: No. JVM fibers are implemented natively and do in fact have runtime support. Kotlin coroutines are used at compile-time. So if you are using Kotlin coroutines, your code is transformed to a callback-based variant. So basically coroutines are a syntactic construct and are to asynchronous code what Lombok is to to boilerplate code 😉
Conclusion
So, is Project loom going to kill asynchronous programming? In my opinion, it is not the right way to look at this advancement of the Java language. Project Loom seems targeted at your run-of-the-mill web applications that don’t need a FAANG level of performance and scalability. There, Loom shines because it makes it easier for a regular developer team to program asynchronously in a more understandable and readable way using constructs they are familiar with. But fibers and threads are not that different, and everytime you want to run two computations in parallel and combine their results, you still have to instruct your program how to do that. So, can this be considered a step in the right direction? I think for the vast amount of Java developers, it is.
But as always, it is not a silver bullet, and discarding all reactive approaches because Loom is around the corner will probably lead to a headache. But this remains inconclusive until the first applications run with Loom and libraries have adjusted.
If you are interested to further your understanding of this, I’ve included a couple of resources and further reading below which I used in preparation for this article.