Virtual Threads in Java (Project Loom)

Virtual threads are one of the most important innovations in Java for a long time. They were developed in Project Loom and have been included in the JDK since Java 19 as a preview feature (JEP 425).

In this article, you will learn:

Why do we need virtual threads?
What are virtual threads, and how do they work?
How to use virtual threads?
How to create virtual threads?

Let's start with the challenge that led to the development of virtual threads.

Why Do We Need Virtual Threads?

Anyone who has ever maintained a backend application under heavy load knows that threads are often the bottleneck. For every incoming request, a thread is needed to process the request. One Java thread corresponds to one operating system thread, and those are resource-hungry. You should not start more than a few hundred; otherwise, you risk the stability of the entire system.

However, a few hundred are not always enough, especially if it takes longer to process a request because of the need to wait for blocking data structures, such as queues, locks, or external services like databases, microservices, or cloud APIs.

For example, if a request takes two seconds and we limit the thread pool to 100 threads, then a maximum of 50 requests per second could be answered. However, the CPU would be far from being utilized since it would spend most of its time waiting for responses from the external services, even if several threads are served per CPU core.

So far, we have only been able to overcome this problem with reactive programming, as provided by frameworks like RxJava and Project Reactor.

However, anyone who has had to maintain code like the following knows that reactive code is many times more complex than sequential code – and absolutely no fun.

public DeferredResult<ResponseEntity<?>> createOrder(
    CreateOrderRequest createOrderRequest, Long sessionId, HttpServletRequest context) {
  DeferredResult<ResponseEntity<?>> deferredResult = new DeferredResult<>();

  Observable.just(createOrderRequest)
      .doOnNext(this::validateRequest)
      .flatMap(
          request ->
              sessionService
                  .getSessionContainer(request.getClientId(), sessionId)
                  .toObservable()
                  .map(ResponseEntity::getBody))
      .map(
          sessionContainer ->
              enrichCreateOrderRequest(createOrderRequest, sessionContainer, context))
      .flatMap(
          enrichedRequest ->
              orderPersistenceService.persistOrder(enrichedRequest).toObservable())
      .subscribeOn(Schedulers.io())
      .subscribe(
          success -> deferredResult.setResult(ResponseEntity.noContent()),
          error -> deferredResult.setErrorResult(error));

  return deferredResult;
}Code language: Java (java)

Not only is this code hardly readable, but it is also extremely difficult to debug. For example, it would make no sense to set a breakpoint here because the code only defines the reactive flow but does not execute it. The code is executed only after the call to subscribe() (at the end of the method) by the reactive library in a separate thread pool.

In addition, the database drivers and drivers for other external services must also support the reactive model.

What Are Virtual Threads?

Virtual threads solve the problem in a way that again allows us to write easily readable and maintainable code. Virtual threads feel like normal threads from a Java code perspective, but they are not mapped 1:1 to operating system threads.

Instead, there is a pool of so-called carrier threads onto which a virtual thread is temporarily mapped. As soon as the virtual thread encounters a blocking operation, the virtual thread is removed from the carrier thread, and the carrier thread can execute another virtual thread (a new one or a previously blocked one):

Mapping virtual threads to platform threads

Blocking operations thus no longer block the executing thread. This allows us to process a large number of requests in parallel with a small pool of carrier threads.

We could then implement the example use case from above quite simply like this:

public void createOrder(
    CreateOrderRequest createOrderRequest, Long sessionId, HttpServletRequest context) {
  validateRequest(createOrderRequest);

  SessionContainer sessionContainer =
      sessionService
          .getSessionContainer(createOrderRequest.getClientId(), sessionId)
          .execute()
          .getBody();

  EnrichedCreateOrderRequest enrichedCreateOrderRequest =
      enrichCreateOrderRequest(createOrderRequest, sessionContainer, context);

  orderPersistenceService.persistOrder(enrichedCreateOrderRequest);
}Code language: Java (java)

This code is not only easier to write and read but also – like any sequential code – to debug by conventional means.

If your code already looks like this – i.e., you never switched to reactive programming, then I have good news: you can continue to use your code unchanged with virtual threads.

Virtual Threads – Example

We can also demonstrate the power of virtual threads without a backend framework. To do this, we simulate a scenario similar to the one described above: we start 1,000 tasks, each of which waits one second (to simulate access to an external API) and then returns a result (a random number in the example).

First, we implement the task:

public class Task implements Callable<Integer> {

  private final int number;

  public Task(int number) {
    this.number = number;
  }

  @Override
  public Integer call() {
    System.out.printf(
        "Thread %s - Task %d waiting...%n", Thread.currentThread().getName(), number);

    try {
      Thread.sleep(1000);
    } catch (InterruptedException e) {
      System.out.printf(
          "Thread %s - Task %d canceled.%n", Thread.currentThread().getName(), number);
      return -1;
    }

    System.out.printf(
        "Thread %s - Task %d finished.%n", Thread.currentThread().getName(), number);
    return ThreadLocalRandom.current().nextInt(100);
  }
}Code language: Java (java)

Now we measure how long it takes a pool of 100 platform threads (which is how non-virtual threads are referred to) to process all 1,000 tasks:

ExecutorService executor = Executors.newFixedThreadPool(100);

List<Task> tasks = new ArrayList<>();
for (int i = 0; i < 1_000; i++) {
  tasks.add(new Task(i));
}

long time = System.currentTimeMillis();

List<Future<Integer>> futures = executor.invokeAll(tasks);

long sum = 0;
for (Future<Integer> future : futures) {
  sum += future.get();
}

time = System.currentTimeMillis() - time;

System.out.println("sum = " + sum + "; time = " + time + " ms");

executor.shutdown();Code language: Java (java)

The program runs for a little over 10 seconds. That was to be expected:

1,000 tasks divided by 100 threads = 10 tasks per thread

Each platform thread had to process ten tasks sequentially, each lasting about one second.

Next, we test the whole thing with virtual threads. Therefore, we only need to replace the line

ExecutorService executor = Executors.newFixedThreadPool(100);Code language: Java (java)

by:

ExecutorService executor = Executors.newVirtualThreadPerTaskExecutor();Code language: Java (java)

This executor does not use a thread pool but creates a new virtual thread for each task.

After that, the program no longer needs 10 seconds but only just over one second. It can hardly be faster because every task waits one second.

Impressive: even 10,000 tasks can be processed by our little program in just over a second.

Only at 100,000 tasks does the throughput drop noticeably: my laptop needs about four seconds for this – which is still blazingly fast compared to the thread pool, which would need almost 17 minutes for this.

How to Create Virtual Threads?

We have already learned about one way to create virtual threads: An executor service that we create with Executors.newVirtualThreadPerTaskExecutor() creates one new virtual thread per task.

Using Thread.startVirtualThread() or Thread.ofVirtual().start(), we can also explicitly start virtual threads:

Thread.startVirtualThread(() -> {
  // code to run in thread
});

Thread.ofVirtual().start(() -> {
  // code to run in thread
});
Code language: Java (java)

In the second variant, Thread.ofVirtual() returns a VirtualThreadBuilder whose start() method starts a virtual thread. The alternative method Thread.ofPlatform() returns a PlatformThreadBuilder via which we can start a platform thread.

Both builders implement the Thread.Builder interface. This allows us to write flexible code that decides at runtime whether it should run in a virtual or in a platform thread:

Thread.Builder threadBuilder = createThreadBuilder();
threadBuilder.start(() -> {
  // code to run in thread
});
Code language: Java (java)

By the way, you can find out if code is running in a virtual thread with Thread.currentThread().isVirtual().

Summary

Virtual threads deliver what they promise: they allow us to write readable and maintainable code that does not block operating system threads when waiting for locks, blocking data structures, or responses from the file system or external services.

Until the common backend frameworks (in the example above, we used Spring) support virtual threads, we will have to be patient for a while.

I hope you're still as excited as I am and can't wait to use virtual threads in your projects!

If you still have questions, please ask them via the comment function. Do you want to be informed about new tutorials and articles? Then click here to sign up for the HappyCoders.eu newsletter.