One of the major changes that Java 21 brought about is Virtual threads. There is so much hype around it, but let's see a real-world example with some metrics. Spring introduced the ability to create GraalVM native images that use Spring Boot and Java 21's virtual threads(Project Loom).
We will look at virtual threads in detail and the many features that come with them in another article. We will focus on a simple project where we can enable the features of Java Virtual Threads and see the difference in performance gains.
Why virtual threads are faster than normal threads?
Normal threads in Java are tied to OS threads. So, there is a limitation to the actual number of threads it can create. Also, time is lost waiting for blocking I/O Calls. Normal Threads are called Platform Threads. They are an expensive resource that needs to be managed well.
Now, when it comes to virtual threads, they are lightweight constructs. Multiple virtual threads can be tied to a single platform thread which in turn gets tied to an OS thread.
The simple idea behind Virtual Threads
The Spring Project used for benchmarking
The project used in this article can be found here. It has a simple fetch API that returns the Thirukkural based on the ID sent.
Now, by default the http server in Tomcat can run many threads in parallel and this will not let us test out this feature easily. So let's throttle it to 10 max threads by adding the below property in the application.properties file. This will allow Tomcat to only use 10 threads at max.
server.tomcat.threads.max=10 We will mimic a blocking IO call using sleep. We will also log the current thread it's using.
Thread.sleep(1000); log.info("Running on " + Thread.currentThread()); Benchmarking using hey
We will benchmark this using hey tool.
hey -n 200 -c 30 http://localhost:8080/thirukural/1 Summary: Total: 36.1759 secs Slowest: 8.0463 secs Fastest: 2.0080 secs Average: 5.6840 secs Requests/sec: 4.9757 Response time histogram: 2.008 [1] | 2.612 [10] |■■■ 3.216 [0] | 3.819 [0] | 4.423 [11] |■■■ 5.027 [0] | 5.631 [0] | 6.235 [156] |■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■ 6.839 [0] | 7.442 [0] | 8.046 [2] |■ Latency distribution: 10% in 4.0305 secs 25% in 6.0212 secs 50% in 6.0263 secs 75% in 6.0340 secs 90% in 6.0376 secs 95% in 6.0387 secs 99% in 8.0463 secs Details (average, fastest, slowest): DNS+dialup: 0.0007 secs, 2.0080 secs, 8.0463 secs DNS-lookup: 0.0005 secs, 0.0000 secs, 0.0041 secs req write: 0.0000 secs, 0.0000 secs, 0.0003 secs resp wait: 5.6832 secs, 2.0079 secs, 8.0419 secs resp read: 0.0001 secs, 0.0000 secs, 0.0005 secs Status code distribution: [200] 180 responses A few log entries are listed below to see that these requests are using Platform Threads.
2024-06-30T22:07:29.659+05:30 INFO 2120 --- [thriukural] [nio-8080-exec-7] p.v.thriukural.web.ThirukuralController : Running on Thread[#45,http-nio-8080-exec-7,5,main] 2024-06-30T22:07:29.659+05:30 INFO 2120 --- [thriukural] [io-8080-exec-10] p.v.thriukural.web.ThirukuralController : Running on Thread[#48,http-nio-8080-exec-10,5,main] 2024-06-30T22:07:29.659+05:30 INFO 2120 --- [thriukural] [nio-8080-exec-2] p.v.thriukural.web.ThirukuralController : Running on Thread[#40,http-nio-8080-exec-2,5,main] Now, to enable the application to use virtual threads, we just add the below property in application.properties.
spring.threads.virtual.enabled=true Let's run and see the same benchmark results again.
hey -n 200 -c 30 http://localhost:8080/thirukural/1 Summary: Total: 12.2478 secs Slowest: 2.1747 secs Fastest: 2.0086 secs Average: 2.0410 secs Requests/sec: 14.6965 Response time histogram: 2.009 [1] | 2.025 [149] |■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■ 2.042 [0] | 2.058 [0] | 2.075 [0] | 2.092 [0] | 2.108 [0] | 2.125 [0] | 2.141 [0] | 2.158 [0] | 2.175 [30] |■■■■■■■■ Latency distribution: 10% in 2.0115 secs 25% in 2.0129 secs 50% in 2.0158 secs 75% in 2.0188 secs 90% in 2.1724 secs 95% in 2.1739 secs 99% in 2.1747 secs Details (average, fastest, slowest): DNS+dialup: 0.0015 secs, 2.0086 secs, 2.1747 secs DNS-lookup: 0.0014 secs, 0.0000 secs, 0.0093 secs req write: 0.0000 secs, 0.0000 secs, 0.0002 secs resp wait: 2.0392 secs, 2.0085 secs, 2.1656 secs resp read: 0.0003 secs, 0.0000 secs, 0.0026 secs Status code distribution: [200] 180 responses A few log entries to see them running in virtual threads.
2024-06-30T22:12:38.485+05:30 INFO 17700 --- [thriukural] [mcat-handler-48] p.v.thriukural.web.ThirukuralController : Running on VirtualThread[#103,tomcat-handler-48]/runnable@ForkJoinPool-1-worker-6 2024-06-30T22:12:38.485+05:30 INFO 17700 --- [thriukural] [mcat-handler-66] p.v.thriukural.web.ThirukuralController : Running on VirtualThread[#121,tomcat-handler-66]/runnable@ForkJoinPool-1-worker-12 2024-06-30T22:12:38.485+05:30 INFO 17700 --- [thriukural] [mcat-handler-69] p.v.thriukural.web.ThirukuralController : Running on VirtualThread[#124,tomcat-handler-69]/runnable@ForkJoinPool-1-worker-7 Benchmarking results
As evident from the results, we see an increased throughput and better latency as well once virtual threads are enabled.
| Parameter | Without Virtual Threads | With Virtual Threads |
|---|---|---|
| Requests/sec | 4.9757 | 14.6965 |
| 99% Latency | 8.0463 sec | 2.1747 sec |
Summary
A simple parameter change has enabled us with a lot of improved performance and throughput. We will further explore in future articles some of the considerations that need to be taken before using virtual threads.
Originally published at vignesh.page.
Please let me know if any improvements you have experienced using Java Virtual Threads
Top comments (0)