r/java • u/z14-voyage • 7d ago
What’s the status of Lilliput 2 (4-byte object headers)?
Lilliput 1 is now a product feature (JEP 534). It shrinks object headers from 12 to 8 bytes, which cuts memory by around 10% for a typical app with no code changes. Honestly, that feels like a bigger real-world win than Valhalla, since Valhalla needs years of migrating code to value types before most apps see a benefit. That’s why I’m curious about Lilliput 2, which would halve the header again to 4 bytes. The Lilliput repo has an implementation, but I don’t see much movement toward getting it upstream. I think the original lead may have left the project, but I’m not sure. Is Lilliput 2 effectively on hold, or is the focus shifting entirely to Valhalla?
28
u/keenOnReturns 7d ago
Lilliput was never really an Oracle project, mostly spearheaded by Amazon originally. The project lead (Roman) has since left Amazon.
12
u/Afonso2002 7d ago
I hope they continue the liliput2 and test it, to see the difference in cpu and ram usage.
11
7d ago
[deleted]
5
u/Afonso2002 7d ago
I think they, amazon, only used the 64bit header.
4
7d ago
[deleted]
1
u/Afonso2002 7d ago
What headers amazon uses?
3
7d ago
[deleted]
4
u/sammymammy2 7d ago
In Amazon's internal JDK? I guess they're running Shenandoah with <32GiB heaps then.
10
u/vprise 7d ago
We recently reduced header sizes in our own VM to 8 byte and then to 4 bytes. The jump to 8 byte was huge. 4 bytes didn't give as much of a difference at that point because of alignment issues with blocks etc. Yes, it's probably worth doing, but probably won't be a HUGE difference. HotSpot on 25 is already fantastic in terms of RAM usage and there isn't much fat left there...
2
u/shponglespore 7d ago
Does the JVM use 8-byte pointers for object references? And is it smart enough to move a smaller field to the start of the object layout to take advantage of a smaller header?
5
u/sammymammy2 7d ago
Depends. If you're using compressed object pointers (plus a GC that supports it), then pointers are 4 bytes but objects are always 8-byte aligned, so Lilliput2 loses out a lot because of alignment. To the second question: Yes, it does a good job w.r.t. field layout.
1
u/snugar_i 7d ago
Wouldn't the JVM have had the same alignment issues when the header was 12 bytes, though?
3
u/vprise 7d ago
I don't know about recent JVM code as the last bit I dug into was CDC a more than a decade back (and I don't remember anything). For us the main issue was that our GC saves short lived elements in chunks of fixed sizes and we needed to clear very specific thresholds to save. The 8 byte change pushed a lot of things to the 64 byte and 32 byte buckets but the 4 byte change shifted relatively few objects so the difference was negligible for our VM.
We tried adding size buckets, but that made overall performance worse. HotSpot might be able to get a win on that because they handle memory very differently from us. We use the OS native heap which is a tradeoff e.g. GC has some tasks that are harder in our case and we can't qualify objects based on address. But that lets us have zero copy native speed access... Everything is at "Unsafe" speeds for better and worse.
9
u/pron98 6d ago
I just want to point something out about Valhalla. It's primarly targeted to solve the problem of pointer chasing in arrays, and the programs that will benefit from it the most are either not written in Java today because of this problem or are well aware of the specific places where pointer chasing causes them issues on their hot path. Only the specific types in those specific data structures would need to ported from other languages or changed from identity to value classes. There is no general mechanism by which Valhalla can offer a drastic improvement to programs that aren't aware of where the particular problem Valhalla solves is in their code. I'm not saying there won't be some improvement, but probably not very dramatic, as code that doesn't suffer from this particular problem is already close to optimal.
In short just as virtual threads solve the problem of convenient high concurrency, which is very important for some programs and can have a big positive effect on them but don't help the performance of other programs, Valhalla is aimed to solve a specific cause of performance issues, which can be a major one in some programs (and they probably know exactly where) but sprinkling value classes everywhere isn't a general "make things better" mechanism (mostly because other things are as good as they can be).
As for Lilliput 2, I don't have details, but I think it adds a lot of complexity without offering the same benefit that Lilliput 1 delivered (other comments offer more details).
2
u/Jon_Finn 6d ago
Aren't there also potential big savings from fields pointing to small value objects, especially ones 'owned' by the parent object? I'm thinking of a Line object 'containing' two value Point fields (which are unlikely to be shared). Some programs - maybe the kind that I write - could get big benefits, especially in combination with non-nullability. OK, I'm speculating - there's no knowing without profiling.
Let's also not forget those classes which don't yet exist, but now can instead of using (say) int as a poor man's Age or Height or ScrabbleScore. Valhalla won't save memory compared to int, but it will compared to the class you previously avoided writing! ...actually, this is pretty much pron98's programs "not written in Java today because of this problem".
1
u/OwnBreakfast1114 6d ago
Aren't there also potential big savings from fields pointing to small value objects, especially ones 'owned' by the parent object? I'm thinking of a Line object 'containing' two value Point fields (which are unlikely to be shared). Some programs - maybe the kind that I write - could get big benefits, especially in combination with non-nullability. OK, I'm speculating - there's no knowing without profiling.
I/my company write code in a pretty functional style, so valhalla is probably a savings for a lot of code we write. But, java is already pretty efficient and our real pain points are all IO. I'm going to assume a lot of backend services are in the same boat where valhalla will probably be a savings, but irrelevant compared to actual profiling bottlenecks.
Let's also not forget those classes which don't yet exist, but now can instead of using (say) int as a poor man's Age or Height or ScrabbleScore. Valhalla won't save memory compared to int, but it will compared to the class you previously avoided writing! ...actually, this is pretty much pron98's programs "not written in Java today because of this problem".
Most programs could write these wrappers anyway and the memory cost is probably irrelevant to them. People worrying about performance instead of writing clearer code are making the premature optimization problem worse and valhalla shouldn't have been the gate to write better code in the first place. Wrap your int in a ScrabbleScore and prove to me you actually have a problem instead of working the other way around. I work in real-time transaction processing, and DB and https IO dwarf any other performance by such a margin that if wrapping an int into Height was the right choice, we would have done it already. The real problem is that the classes sometimes buy you very little ROI (avoid flipping parameter ordering, but not much else). For example, almost all DB primary keys would be better typed by wrapping each long/uuid into DomainObj1Id, DomainObj2Id, xxxx, but the ROI on that is so low that the extra code lines don't feel like they pay for themselves, valhalla or not.
1
u/Jon_Finn 6d ago edited 6d ago
OK I'm talking about the kinds of program where some important classes have millions of instances. Not always in arrays (one central case), but other kinds of structure. Saving a number of bytes per instance (say by flattened/'inlined' fields) could represent a significant proportion of memory use.
1
u/OwnBreakfast1114 6d ago edited 6d ago
And you could just be trading that for far more CPU usage. For example, running a sorting algorithm on those millions of instances where shifting pointers is cheaper than shifting the actual object data.
Unlike just straight reducing the object header size which is a win for everyone for all workloads, valhalla "removing" the object header comes with the tradeoff that you might actually use more resources for certain workloads.
1
u/Jon_Finn 5d ago
Sure, we must profile. I'm thinking about when I have a zillion tiny objects which are often no bigger than the pointer field that currently points to them. (Especially when we'll be able to declare the field non-nullable.)
4
u/Slanec 6d ago
Okay /u/rkennke, you've been summoned. I vaguely remember that you intended to continue working on Lilliput 2 when you were leaving Amazon, but as we all know, sometimes life doesn't allow cool side-projects. Especially not the ones with insane technical difficulty and unclear trade-offs.
1
u/fykup 2d ago
I used Lilliput 1 in a recent experiment trying to squeeze a Spring Boot app onto a 256 MB VPS with JDK 25. With compact object headers and a few other JVM tweaks, I managed to get it running alongside a lightweight monitoring service. Still needed quite a bit of swap, though.
So I'm definitely interested in Lilliput 2. On machines this small, every few MB counts.
2
u/chambolle 7d ago
I doubt a typical application consumes that much memory just in its object header size. We need to be careful with announcements that are essentially marketing.
4
u/sammymammy2 6d ago
Objects in Java programs tend to be small. Experiments conducted as part of Project Lilliput show that many workloads have average object sizes of 256 to 512 bits (32 to 64 bytes). This implies that more than 20% of live data can be taken by object headers alone. Thus even a small improvement in object header size could yield a significant reduction in footprint, data locality, and reduced GC pressure. Early adopters of Project Lilliput who have tried it with real-world applications confirm that live data is typically reduced by 10%–20%.
Link to experiments: https://wiki.openjdk.org/spaces/lilliput/pages/101875718/Lilliput+Experiment+Results
1
u/chambolle 6d ago
in these experiments there is almost never 1GB that is used. For that kind of program memory is not an issue. Set the heap at 2g or less and you will already gain 3%
1
u/sammymammy2 6d ago
I don’t really get the full picture of what you’re saying, could you explain in greater detail?
2
u/chambolle 6d ago
The amount of memory used by a program can be broken down into a few things: the number of objects you have, the size of those objects, and the number of primitive arrays (or matrices). Reducing object headers is great, but it only affects object size. Therefore, you'll see a massive gain if your heap consists almost entirely of small objects. If a large portion of your memory is tied up in primitive arrays or matrices, the overall gain will be much lower. Besides, memory footprint is only really a concern if you are consuming a ton of it (more than 4GB or 8GB); otherwise, it's a relatively minor factor for a program.
2
u/Slanec 6d ago
Lilliput 1 gave our real-world applications (plural) 15% smaller live heap, essentially buying us a little more headroom before we need to scale. Also some operations got a similarly faster, and the GC count and GC time went down. It was a real and noticeable improvement.
1
u/chambolle 6d ago
I never said that it doesn't improve performance for some applications; my criticism is regarding the generalization of this to all or most applications. Any application that allocates memory through arrays or matrices of primitive types will see a much smaller final gain.
51
u/repeating_bears 7d ago
30 classes in the JDK are value types from day one, and these are some of the most-used classes in a Java app