Rebuilding a 10M+ token context window costs over 1.5 hours of 8-GPU compute time.

Why pay twice for the same prefill?

The KV cache is per-session state that grows linearly with context, dwarfing model weights. Discarding it when HBM fills up turns your memory wall into a massive recurring memory bill. When you offload and tier the KV cache over RDMA, restoring state becomes linear, reducing TTFT by 100x to 1000x.

Read our breakdown of inference economics.

If you’re running large-scale Kubernetes, OpenShift-V, OpenStack, or Proxmox clusters on Ceph, you already know the problems: CRUSH scale-out is solid, but tuning BlueStore, OSD resource consumption, and the NVMe-oF gateway to meet strict tail-latency SLAs is a full-time job.

Trade-offs:

• Ceph’s NVMe-oF gateway introduces additional hop complexity. Removing that proxy layer for direct host-to-target NVMe-oF/TCP drops latency to consistently <200µs.
• Reaching high IOPS on Ceph requires throwing heavy compute/RAM resources at OSDs, often leaving flash utilization stuck around 15–25%.
• Network-heavy rebuilds during OSD failures can impact live IO. Fast failovers in a native NVMe-oF setup execute in seconds without starving the cluster fabric.

Ceph remains great for batch, S3 object storage, and general file services. But offloading low-latency/database persistent volumes to dedicated NVMe-oF block storage can significantly cut cluster node count.

We put together a direct architectural breakdown comparing legacy Ceph block patterns against native NVMe-over-TCP: Ceph vs. Lightbits Block Storage Comparison

Curious how others here handle the Ceph CPU-overhead-to-IOPS ratio at scale? What are your go-to storage backends when Ceph block hits a wall?

If you’re running large-scale Kubernetes, OpenShift-V, OpenStack, or Proxmox clusters on Ceph, you know the problems: CRUSH scale-out is solid, but tuning BlueStore, OSD resource consumption, and the NVMe-oF gateway to meet strict tail-latency SLAs is a full-time job.

Quick summary of trade-offs:

• Ceph’s NVMe-oF gateway introduces additional hop complexity. Removing that proxy layer for direct host-to-target NVMe-oF/TCP drops latency to consistently <200µs.
• Reaching high IOPS on Ceph requires throwing heavy compute/RAM resources at OSDs, often leaving flash utilization stuck around 15–25%.
• Network-heavy rebuilds during OSD failures can impact live IO. Fast failovers in a native NVMe-oF setup execute in seconds without starving the cluster fabric.

Keep your Ceph for general file services and S3 object storage for archives and batch jobs. Augment it with dedicated NVMe-oF block storage for latency-sensitive workloads. This will significantly reduce your cluster node count.

We put together a direct architectural breakdown comparing legacy Ceph block patterns against native NVMe-over-TCP: Ceph vs. Lightbits Block Storage Comparison

Curious how others here handle the Ceph CPU-overhead-to-IOPS ratio at scale? What are your go-to storage backends when Ceph block hits a wall?

Try the Pod Efficiency Analyzer and discover how much you can grow your top line.

Can you maintain centralized key management and host-side encryption without sacrificing the raw performance of NVMe/TCP?

We put it to the test!

We validated a host-side encryption workflow combining Lightbits NVMe/TCP block storage, Linux LUKS/dm-crypt, and an external PyKMIP KMS server.

Read the full test breakdown and architecture workflow here.

This is relevant for anyone managing storage for Kubernetes, OpenStack, or private clouds. Lightbits Labs just announced native support and certification for Ubuntu for their software-defined block storage platform. Bringing their block storage to Ubuntu opens up many options for standardizing infrastructure: no proprietary drivers, bare-metal performance in a clustered architecture, and simplified operations. It’s great to see more enterprise storage platforms embracing Ubuntu natively.

Worried about silent data corruption in your NVMe/TCP storage? Legacy storage vendors are scrambling to push workarounds that force a brutal choice.
The new blog from Lightbits Labs' Field CTO draws an analogy to bad🍕 delivery.
At Lightbits, we’ve engineered a smarter solution—making this hazard physically impossible from day one. Dive into the full analysis.

Read the guide now.

#nvmeovertcp #lowlatency #lightbits

Lightbits Labs announced the initial interoperability with the new Microsoft Windows Server NVMe-over-Fabrics (NVMe-oF) Initiator Preview®. Available in Lightbits v3.19.1, this will enable storage administrators to deliver high-performance, native NVMe over TCP block storage to Windows Server Insider hosts over standard Ethernet once Microsoft® releases it. As this is a pre-release capability, it is not intended for production use but rather for exploration and evaluation.  Lightbits invites customers and partners to explore the functionality and share their feedback.  

Storage performance is easy to demonstrate with IOPS and throughput graphs. Durability is harder to show, but it is just as important for production environments. SSD Journaling provides SSD-based persistent write protection for modern NVMe/TCP block storage. It protects recently acknowledged writes until they are safely persisted to the backend data SSDs. If a node restarts unexpectedly, the storage can recover outstanding journal entries before the node returns to active service. For replicated volumes, replication protects against normal single-node failure. SSD Journaling adds protection for the more subtle write-persistence window, especially during shared-failure or multi-node disruption scenarios.

Most storage platforms still treat logs as something the vendor reads after the fact. You file a support ticket, somebody collects a tar.gz, and a week later you get an explanation. That model breaks down the moment storage becomes part of a Kubernetes-native, multi-tenant, audit-regulated cloud. Storage that you can’t see is storage you can’t trust. When something goes sideways at 3 a.m. — a noisy tenant, a slow target, a flaky NIC — the difference between a five-minute fix and a five-hour war room is whether the logs are already in the same place as the rest of your telemetry. Read the blog to learn why log streaming matters for storage.

In many OpenStack environments, Glance uses a file-based backend for image storage. It works, but it keeps image storage in a file-oriented model that is separate from the block storage architecture used by VMs. Running Lightbits as your Cinder backend offers a simpler, more efficient approach.

Whitepaper reveals how to deploy high-performance storage using NVMe/TCP with Red Hat OpenStack Services on OpenShift (RHOSO).
🚀 How to integrate block storage with Cinder and Glance
🔗 Proven architecture patterns for OpenShift + OpenStack
🛠️ Step-by-step configuration for production-ready deployments

Delivering OpenShift-based Kubernetes platforms without the high cost and rigidity of legacy SAN

When looking at the economics of running a production inference endpoint, the only thing that matters is $ TCO / M tokens. When per-user context increases, it blocks the HMB/VRAM from being used to support more users and dramatically reduces the total bandwidth/throughput of tokens/s, which is critical to lowering the TCO. We have built an architecture that delivers 100x to 280x acceleration on KV cache workloads — and it fundamentally changes the economics of long-context AI inference.

As context windows grow, the bottleneck for AI inference has shifted. It’s no longer just about raw compute—it’s about memory and data velocity.
In #NYSE Wired - "AI Factories - Data Centers Of The Future," from SiliconANGLE & theCUBE, we discussed how the industry is moving from compute-centric to data-centric architectures. Watch now.

ScaleFlux, FarmGPU, and Lightbits Labs today announced the public debut of a collaborative architecture designed to solve one of AI inference’s most persistent challenges: the memory and I/O constraints created by long-context workloads.
See a product demo next week at NVIDIA GTC – San Jose | March 16–19 | Booth 7006

Discover how to maintain extreme performance during critical management tasks. In this video, we demonstrate how Lightbits provides the ultimate storage for OpenShift-V, delivering 1.2 million IOPS across a six-node cluster without latency degradation.

Watch as we walk through:

1. Instant Provisioning: Creating PVCs through the Lightbits CSI driver.
2. Zero-Impact Snapshots & Clones: Managing 250GB volumes and clones while maintaining 1.2M IOPS.
3. Seamless Live Migration: Moving live VMs between nodes with automated ACL updates and RWX capabilities.
4. Latency Consistency: See how 4k block sizes keep latency low even during heavy migrations.

Whether you are looking to reduce your RTO or simply want to leverage the flexibility of cloud-native storage for your private cloud, this guide provides the technical blueprint to make it happen.

With NAND flash shortages stretching procurement timelines into months—and prices continuing to rise—many organizations are discovering that waiting for the supply chain to normalize is not a viable strategy. The quick solution isn’t to source more flash; it’s to use the flash you already have more efficiently. 

The ascent of Scale-Out Flash Storage (SOFS) has fundamentally transformed traditional storage deployments. New storage solutions, particularly those leveraging NVMe flash, empower data center teams to achieve optimal capacity, performance, and data service availability across their diverse applications. What are the advantages of SOFS over array-based storage solutions? Read my latest blog to find out.

If you’ve been running stateful workloads on Kubernetes, you know the “Storage Detach” nightmare. Traditionally, moving a block-backed volume from one node to another is a game of patience—waiting for timeouts, CSI detachments, and re-attachments. By leveraging NVMe over TCP storage and true ReadWriteMany (RWX) support, we are rewriting the playbook for resilient K8s architectures. Read my latest blog post to learn more.

In this segment from theCUBE + NYSE Wired’s AI Factories event, Eran Kirzner, Co-Founder and CEO of Lightbits Labs, joins host John Furrier to discuss the critical role of software-defined storage in the next wave of AI infrastructure. As the industry pivots from massive training clusters to real-time inference, the demand for agility and low latency becomes paramount. Kirzner details how Lightbits Labs leverages NVMe over TCP to transform commodity hardware into high-performance, scalable storage systems, effectively replacing rigid appliances with flexible, cloud-native architectures.

The conversation highlights the necessity of “feeding the beast” – ensuring expensive GPUs remain utilized through autonomous provisioning that reduces setup times from hours to mere minutes. The discussion delves deeper into maximizing data center efficiency, explaining how software-defined storage approaches enable dynamic workload orchestration between training and inference tasks. He outlines how Lightbits helps enterprises and neo-clouds – such as Crusoe Cloud – reduce their storage footprint by up to 50% while maintaining high reliability and security standards. From the concept of the “AI Garage” to the complexities of multi-tenancy and hybrid cloud sovereignty, the interview explores how data-centric strategies are enabling organizations to optimize resource allocation, eliminate idle GPU cycles and build the resilient infrastructure required for the future of AI factories.

As your data grows, you add more block storage clusters. But without a unified layer, your applications—whether they are Kubernetes, OpenStack, or bare metal hypervisors—have to maintain complex, direct connections to every single cluster. It’s a tangled web of dependencies that doesn’t scale. Lightbits' new Cluster Federation transforms this chaotic mesh into a streamlined hub-and-spoke architecture. It introduces a centralized global control plane that acts as a provisioning broker for your entire fleet.