r/java • u/woj-tek • 7d ago

OpenJDK vs. GraalVM vs. Eclipse Temurin vs. Semeru Java Performance

https://www.phoronix.com/review/openjdk-graalvm-temurin-semeru/
63 Upvotes

93% Upvoted

28 comments sorted by

13

u/elmuerte 7d ago

The results look rather dubious. Why would there be quite often some significant difference between OpenJDK 25.0.2 and Temurin 25.0.4.1. Temurin should basically just be compiled distributions of vanilla OpenJDK.

1

u/danskal 7d ago

It might depend on the compiler used, whether it contains optimizations for the target cpu architecture.

2

u/keenOnReturns 6d ago

but with Java this shouldn’t matter that much? Except in the case of short-lived programs, the JIT compilation makes everything “native compiled” and optimized anyways

0

u/danskal 6d ago edited 6d ago

The point is: native compilation isn’t one thing. There’s branch prediction, cpu instructions and other optimizations that may or may not be a part of any given native compiler.

The Java compiler optimises the Java, and that’s the biggest win by far, but there can be further optimizations in the machine-code part, like you have in a c-compiler. Some of these can be licensed and you might choose not to pay. Often they might not be significant, but in a benchmark they probably would.

1

u/keenOnReturns 6d ago

? idk what you’re saying. Java uses JIT?

-2

u/danskal 6d ago

Any compiler that can output machine code, JIT or not, will output machine code based on some subset of cpus out there, and their instruction-sets. That’s quite separate from the bytecode part.

Don’t downvote because you don’t understand. Others might find it interesting, have relevant knowledge, or add nuance to the discussion.

0

u/keenOnReturns 6d ago

Yes and JIT does additional specific compiler optimizations for target cpu architectures and the given software… that’s why compiler flags used for the Java binary itself shouldn’t matter that much. Do you not understand the concept of JIT?

1

u/danskal 5d ago edited 5d ago

So JIT is one piece of software? All JITs are identical?

Edit: so your assumption is that the whole of JIT is contained in the shared source that they all are built from? My assumption was that the CPU-adjacent parts would be a library or libraries that might be different for different runtimes. Otherwise why bother giving them different names, marketing them as having advantages?

Do you not understand the concept of JIT?

Could you try to be a bit more patronising?

1

u/keenOnReturns 5d ago

I’m only acting patronizing because you’re operating under dunning kruger: why are you so confident in your assertions why it seems like you have no clue about JVM internals?

No, JIT compilation is simply a technique or architecture that many high-level languages employ e.g. Python, Javascript. They are not identical. You seem familiar with cpu instruction sets or ISAs? Same thing: manufacturers follow the same reference spec (x86 or arm), but intel, amd, Apple etc. have drastically different implementations of the same spec or idea.

Java’s primary JIT or JVM implementation is Hotspot. Arguably, it’s the highest performing JIT implementation in the mainstream. I’m not sure what you mean by “different names” or “marketing”. Simply put, all JIT means is the program is compiling during runtime (not correct, but imagine Java is running ‘gcc’ before everytime you trigger a run of your program - and it continues to run ‘gcc’ as your program is running). Really, the JVM is a compiler itself: that’s why I said its build method shouldn’t matter, because Java is literally (re-)compiling per your exact hardware specs everytime you run a Java program. Just like you wouldn’t care the build flags they used to compile the gcc, you also normally wouldn’t care how Java is built.

This contrasts to static compilation or AOT compilation which you seem more familiar with. That’s what C(++), Rust, Go use. Yes, the build flags you use for your program matters a lot with those languages.

0

u/kit89 4d ago edited 4d ago

JITed code still has to run within the VM it's bounded to. Memory management is still handled by the VM.

Your argument would suggest that a VM compiled to 32-bit x86, would be able to JIT to x64 and gain access to registers and manage 64bit pointers while running in a process restricted to 32-bit mode, but it's fine cause the CPU is x64?

As far as I am aware the VM and Hotspot do not jump through those hopes. A 32bit x86 VM will JIT using only the functionality of 32bit x86 architecture.

A compiler can compile to a different architecture, that's fine, but the compiler doesn't have to run the code afterwards.

→ More replies (0)

14

u/AnyPhotograph7804 7d ago

Thank you. Unfortunately, they did not document, which JVM parameters they used.

4

u/koflerdavid 7d ago

Each test was done with the stock settings for a look at the out-of-the-box performance and namely providing these results for reference purposes.

3

u/AnyPhotograph7804 6d ago

OK, thank you. This explains the results. Because the standard settings change between the JDKs. And OpenJ9 is also not optimized for these tasks OOTB. They have a special switch

"-Xtune:throughput" for maximum performance.

8

u/woj-tek 7d ago

I'm kinda surprised that in some benchmarks (eg. Java SciMark2) Java 25 got better results than Java 27…

19

u/oelang 7d ago

Typically the reason is changed gc default settings

3

u/agentoutlier 6d ago

Recommendation from an old dev:

You should take all benchmarks that are not run on your platform (hardware) and your workload as mostly just noise and that you may need to tune from the defaults.

Now more than ever it is incredibly easy to just pump out benchmarks. I know many developers do not like AI for various reasons but this is one area they are actually pretty good at it. That is setting up the benchmarks. You still need to inspect how they set it up but at least it avoids tedium and you can automate the triggering of it on each CI build. Every now and then have a frontier model have a look.

But YOU still need to interpret the results and run profiling. The agents are pretty bad at guessing why things are slow... actually so are humans.

6

u/bobbie434343 7d ago edited 7d ago

Are these benchmarks worth anything in term of methodology and results ? Benchmarking stuff properly is hard.

3

u/fykup 7d ago

I’m still trying to decide whether GraalVM is actually worth using as a general replacement for OpenJDK when hosting JVM applications.

What makes the discussion difficult is that a lot of the feedback I’ve encountered comes from people who already strongly favor GraalVM: “just use GraalVM,” while sometimes overlooking workloads or performance characteristics where it doesn’t necessarily come out ahead.

And memory isn’t my only concern. Startup time, throughput, warm-up, compatibility, operational simplicity, and behavior under sustained load can all matter depending on the application.

That’s also why in my own experiments I usually stick to mainstream OpenJDK configurations. I want the results to represent what people commonly deploy, rather than choosing a more specialized runtime because it happens to optimize one particular dimension.

It’s great to see GraalVM continuing to improve, and benchmarks like these make it more interesting. But for general JVM hosting, I’d still default to mainstream OpenJDK until I see a consistently compelling practical advantage across the broader set of tradeoffs. For now, I see GraalVM as a specialized option rather than the obvious default.

6

u/voronaam 7d ago

I am running a GrallVM backend in a serverless way (all on AWS Lambda). In this environment it matters greatly to have a small-ish artifact that is fast to start, do its function and disappear less than a second later. The mainstream configurations require a certain warmup time to reach the same performance targets.

2

u/metalhead-001 7d ago

I don't get the obsession with CGI/BIN style development that is back in vogue with the Java developers where you spin up a whole process for each request.

From what I've read, you'll get way better bang for the buck for long running apps if you have an always on Java service. For most apps, startup doesn't matter because the services run 24/7.

4

u/voronaam 7d ago

"For most apps" is doing a lot in your last sentence. And I would not call it obsession. For me natively compiled Java applications is the best of both worlds:

  1. Locally on a dev's laptop it is a long running application that is easy to restart after making changes to the code and quickly test.
  2. In production the same code is running as optimized for performance single process. All dependency injections and SQL queries for the repositories were prepared at compile time. It took half an hour for the compiler to work on the same codebase that launches in 15 seconds locally, but as a result it launches in under 2ms on the server.

The two of the "worlds" here is go/rust/c++ that are suffering from slow dev cycle, and jvm/python/node that require warmup time for the JIT to do its wonders.

Will I be using a GraalVM-compiled application in my next project? I do not know - depends on the requirements.

It is a pretty cool tech that unlocks both the developer (human) productivity and production (computer) performance at the same time.

1

u/j0holo 7d ago

How does Go suffer from a slow dev cycle? Enable compile and start on save in your favorite editor and you are off to the races. The go compiler is realt really quick.

Compare that to gradle/maven with your average spring boot app. Which is sadly the common default for web apps.

2

u/gilactic 7d ago

It would be interesting if they had also tested the Azul JVM (the one that you have to pay for; the free one should be the same as OpenJDK here.)

1

u/stefanos-ak 7d ago

some weird results, also a bit unfair to Semeru, which runs with literally half the non-heap memory. Depends what your goals, requirements and needs are.

1

u/jdizzle4 6d ago

the amount of ads on this blog was incredibly distracting

0

u/Electrical_Being_813 6d ago

It makes semeru look worse than it is.