# JVM Low-Level I/O - Part 1

Welcome to the first article in our JVM Low-Level I/O series. We're going to tear apart `java.nio.ByteBuffer` — the single most important class for high-performance I/O on the JVM. By the end of this article, you'll understand not just *how* to use ByteBuffer, but *why* it exists and what's happening underneath.

* * *

## 1\. Why ByteBuffer Exists

Before Java NIO (introduced in Java 1.4), all I/O was stream-based. You had `InputStream` and `OutputStream`, which processed data one byte at a time. This was simple but painfully slow for high-throughput scenarios.

The problem? **Every byte had to cross the JVM boundary.** The JVM manages its own heap memory, while the operating system manages native memory. Traditional streams required copying data from native OS buffers → JVM heap → your application. That's at least **two copies of memory** for every read operation.

![](https://cdn.hashnode.com/uploads/covers/637f189ed7d9bcd845996b4b/7adecc59-dc19-4bb6-affa-91a380815f71.png align="center")

ByteBuffer solves this by giving you a **block of memory** that you can read from and write to in bulk — and in some cases, it can live *outside* the JVM heap, eliminating one of those copies entirely.

![](https://cdn.hashnode.com/uploads/covers/637f189ed7d9bcd845996b4b/7a22c068-c394-4615-86ce-a5a167ace464.png align="center")

* * *

## 2\. The Memory Model Behind ByteBuffer

Let's visualise where ByteBuffers actually live in memory:

![](https://cdn.hashnode.com/uploads/covers/637f189ed7d9bcd845996b4b/9912fa41-5aa7-474d-b598-015fecc66956.png align="center")

**Key insight:** A `HeapByteBuffer` is backed by a regular Java `byte[]` array on the garbage-collected heap. A `DirectByteBuffer` lives in native memory — the GC never touches the buffer's *data*, only its Java *wrapper object*.

* * *

## 3\. Creating ByteBuffers

There are three primary ways to create a ByteBuffer:

### 3.1 Heap Buffer (allocate)

```java
// Allocate 1024 bytes on the JVM heap
ByteBuffer heapBuffer = ByteBuffer.allocate(1024);

System.out.println(heapBuffer.isDirect());   // false
System.out.println(heapBuffer.hasArray());   // true — backed by byte[]
System.out.println(heapBuffer.capacity());   // 1024
```

**When to use:** General-purpose work, short-lived buffers, when you need fast allocation, and GC can handle cleanup.

### 3.2 Direct Buffer (allocateDirect)

```java
// Allocate 1024 bytes in native (off-heap) memory
ByteBuffer directBuffer = ByteBuffer.allocateDirect(1024);

System.out.println(directBuffer.isDirect());   // true
System.out.println(directBuffer.hasArray());   // false — no backing byte[]
System.out.println(directBuffer.capacity());   // 1024
```

**When to use:** I/O-heavy operations, large buffers, long-lived buffers that interact with the OS (FileChannel, SocketChannel).

### 3.3 Wrapping an Existing Array (wrap)

```java
byte[] data = {0x48, 0x65, 0x6C, 0x6C, 0x6F}; // "Hello" in ASCII
ByteBuffer wrapped = ByteBuffer.wrap(data);

System.out.println(wrapped.isDirect());    // false
System.out.println(wrapped.hasArray());    // true
System.out.println(wrapped.capacity());    // 5

// IMPORTANT: Changes to the array affect the buffer and vice versa!
data[0] = 0x4A; // changes buffer too
System.out.println((char) wrapped.get(0)); // 'J' (not 'H')
```

**When to use:** When you already have data in a `byte[]` and want to use ByteBuffer's API without copying.

### Allocation Cost Comparison

![](https://cdn.hashnode.com/uploads/covers/637f189ed7d9bcd845996b4b/e81b5b46-3f54-405c-a98a-58c5ed7dd4de.png align="center")

> **Rule of thumb:** Allocate direct buffers once and reuse them. Allocate heap buffers when you need them temporarily.

* * *

## 4\. The Four Core Properties

Every ByteBuffer has four properties that control which part of the buffer you can read from or write to:

| Property | Description | Invariant |
| --- | --- | --- |
| **capacity** | Total size of the buffer (fixed at creation) | Always constant |
| **limit** | First index you *cannot* read/write | limit ≤ capacity |
| **position** | Next index to read/write | position ≤ limit |
| **mark** | A saved position (for reset) | mark ≤ position |

The invariant is always: `0 ≤ mark ≤ position ≤ limit ≤ capacity`

![](https://cdn.hashnode.com/uploads/covers/637f189ed7d9bcd845996b4b/d5b293fd-6957-44f0-b79d-8efeb13693d7.png align="center")

```java
ByteBuffer buf = ByteBuffer.allocate(10);

// After allocation:
// position=0, limit=10, capacity=10
System.out.println("position=" + buf.position()); // 0
System.out.println("limit=" + buf.limit());       // 10
System.out.println("capacity=" + buf.capacity()); // 10

// remaining() = limit - position
System.out.println("remaining=" + buf.remaining()); // 10
```

* * *

## 5\. Reading and Writing Data

ByteBuffer provides two styles of access: **relative** (which uses and advances the position) and **absolute** (which uses an explicit index).

### 5.1 Relative Put/Get

```java
ByteBuffer buf = ByteBuffer.allocate(16);

// Write various types — position advances automatically
buf.put((byte) 0x41);       // 1 byte  → position = 1
buf.putInt(42);              // 4 bytes → position = 5
buf.putLong(123456789L);     // 8 bytes → position = 13
buf.putShort((short) 7);     // 2 bytes → position = 15

System.out.println("position after writes: " + buf.position()); // 15

// To read, we must flip first (see next section)
buf.flip();

byte  b = buf.get();        // reads 0x41, position → 1
int   i = buf.getInt();     // reads 42,   position → 5
long  l = buf.getLong();    // reads 123456789, position → 13
short s = buf.getShort();   // reads 7,    position → 15
```

### 5.2 Absolute Put/Get

```java
ByteBuffer buf = ByteBuffer.allocate(16);

// Write at specific indices — position does NOT change
buf.put(0, (byte) 0x41);
buf.putInt(4, 42);
buf.putLong(8, 123456789L);

System.out.println("position after absolute writes: " + buf.position()); // 0

// Read at specific indices — position does NOT change
byte b = buf.get(0);        // 0x41
int  i = buf.getInt(4);     // 42
long l = buf.getLong(8);    // 123456789

System.out.println("position after absolute reads: " + buf.position());  // 0
```

### 5.3 Bulk Operations

```java
ByteBuffer buf = ByteBuffer.allocate(32);

// Bulk write
byte[] source = "Hello, ByteBuffer!".getBytes(StandardCharsets.UTF_8);
buf.put(source); // copies entire array into buffer

// Bulk read
buf.flip();
byte[] dest = new byte[buf.remaining()];
buf.get(dest); // copies buffer content into array

System.out.println(new String(dest, StandardCharsets.UTF_8)); 
// "Hello, ByteBuffer!"
```

### Type Sizes Reference

![](https://cdn.hashnode.com/uploads/covers/637f189ed7d9bcd845996b4b/7dda7fca-ceef-4fbd-915f-6bc86a658b45.png align="center")

* * *

## 6\. Flip, Clear, Compact — The State Machine

This is where most beginners get confused. ByteBuffer operates like a **state machine** — after writing data, you must prepare it for reading (and vice versa).

### The State Transitions

![](https://cdn.hashnode.com/uploads/covers/637f189ed7d9bcd845996b4b/81e938c0-0c19-40c5-bfe2-22e9b55fb21c.png align="center")

### 6.1 `flip()` — Switch from Writing to Reading

`flip()` sets `limit = position` and then `position = 0`. This makes all the data you just wrote available for reading.

```java
ByteBuffer buf = ByteBuffer.allocate(10);

// WRITE MODE: position=0, limit=10
buf.put((byte) 'H');  // position=1
buf.put((byte) 'i');  // position=2

// Before flip: position=2, limit=10
buf.flip();
// After flip:  position=0, limit=2

// Now we can read exactly the 2 bytes we wrote
while (buf.hasRemaining()) {
    System.out.print((char) buf.get()); // prints "Hi"
}
```

**Visual walkthrough:**

![](https://cdn.hashnode.com/uploads/covers/637f189ed7d9bcd845996b4b/ca571b18-44f2-466f-aefb-5ed77663140e.png align="center")

### 6.2 `clear()` — Reset for Full Rewrite

`clear()` sets `position = 0` and `limit = capacity`. The data is still there, but will be overwritten.

```java
buf.clear();
// position=0, limit=10, capacity=10
// Ready to write fresh data
```

### 6.3 `compact()` — Keep Unread Data, Make Room for More

`compact()` copies unread data to the beginning of the buffer, then sets position after the last copied byte.

```java
ByteBuffer buf = ByteBuffer.allocate(10);
buf.put(new byte[]{'A', 'B', 'C', 'D', 'E'});
buf.flip(); // position=0, limit=5

buf.get();  // reads 'A', position=1
buf.get();  // reads 'B', position=2

// We've read A,B but C,D,E remain unread
buf.compact();
// C,D,E are moved to positions 0,1,2
// position=3, limit=10
// We can now write more data starting at position 3

buf.put((byte) 'F'); // position=4
buf.put((byte) 'G'); // position=5

buf.flip(); // position=0, limit=5
// Buffer now contains: C,D,E,F,G
```

**Visual walkthrough of** `compact()`**:**

![](https://cdn.hashnode.com/uploads/covers/637f189ed7d9bcd845996b4b/569d4e6c-4d36-4e97-85ae-cf9729956ef9.png align="center")

### Decision Flowchart: Which Operation to Use?

![](https://cdn.hashnode.com/uploads/covers/637f189ed7d9bcd845996b4b/1cfd48a2-7c2d-497d-8c4e-c681f4bd8570.png align="center")

* * *

## 7\. Byte Order (Endianness)

Multi-byte values (int, long, double, etc.) can be stored in two orders:

| Order | Description | Example (int 0x01020304) |
| --- | --- | --- |
| **Big-Endian** | Most significant byte first | `01 02 03 04` |
| **Little-Endian** | Least significant byte first | `04 03 02 01` |

![](https://cdn.hashnode.com/uploads/covers/637f189ed7d9bcd845996b4b/462f8766-fa69-496a-bc5a-bf4f6ad47ee2.png align="center")

![](https://cdn.hashnode.com/uploads/covers/637f189ed7d9bcd845996b4b/227d9f10-0f5a-49ff-bc61-404b7c7b5fe1.png align="center")

Java's ByteBuffer defaults to **Big-Endian**. You can change it:

```java
ByteBuffer buf = ByteBuffer.allocate(4);

// Default: Big-Endian
buf.order(ByteOrder.BIG_ENDIAN);
buf.putInt(0x01020304);
buf.flip();
// bytes: 01, 02, 03, 04

buf.clear();

// Switch to Little-Endian
buf.order(ByteOrder.LITTLE_ENDIAN);
buf.putInt(0x01020304);
buf.flip();
// bytes: 04, 03, 02, 01

// Check native platform order
System.out.println("Native: " + ByteOrder.nativeOrder());
// Usually LITTLE_ENDIAN on x86/x64
```

> **When does this matter?**
> 
> *   Network protocols: almost always Big-Endian
>     
> *   Interoperating with C/C++ code: usually Little-Endian on x86
>     
> *   Reading binary file formats: check the spec!
>     
> *   Shared memory IPC: must agree on byte order
>     

* * *

## 8\. View Buffers

You can create a "view" of a ByteBuffer as a buffer of a different type. The view shares the same underlying memory:

```java
ByteBuffer byteBuffer = ByteBuffer.allocate(16);

// Create an IntBuffer view (each int = 4 bytes, so capacity = 4)
IntBuffer intView = byteBuffer.asIntBuffer();
intView.put(100);
intView.put(200);
intView.put(300);

// The ByteBuffer now contains those ints as bytes!
byteBuffer.position(0);
for (int i = 0; i < 12; i++) {
    System.out.printf("%02X ", byteBuffer.get(i));
}
// Output (Big-Endian): 00 00 00 64 00 00 00 C8 00 00 01 2C 
//                       (100)      (200)      (300)
```

![](https://cdn.hashnode.com/uploads/covers/637f189ed7d9bcd845996b4b/56b1f797-1d1d-4ce0-8328-e752feb8434f.png align="center")

Available view types: `asCharBuffer()`, `asShortBuffer()`, `asIntBuffer()`, `asLongBuffer()`, `asFloatBuffer()`, `asDoubleBuffer()`.

* * *

## 9\. Slicing and Duplicating

### 9.1 `slice()` — A View of a Portion

`slice()` creates a new ByteBuffer that shares the same memory but starts at the current position with a capacity equal to remaining:

```java
ByteBuffer original = ByteBuffer.allocate(10);
original.putInt(111);  // position=4
original.putInt(222);  // position=8

original.position(4);
original.limit(8);

ByteBuffer slice = original.slice();
System.out.println(slice.capacity()); // 4
System.out.println(slice.getInt(0));  // 222

// Modifying the slice modifies the original!
slice.putInt(0, 333);
System.out.println(original.getInt(4)); // 333
```

### 9.2 `duplicate()` — Independent Position, Shared Data

```java
ByteBuffer original = ByteBuffer.allocate(10);
original.putInt(42);

ByteBuffer dup = original.duplicate();

// Different position/limit/mark, same data
dup.flip();
System.out.println(dup.getInt()); // 42

// Modifying data through either buffer is visible in both
original.putInt(0, 99);
dup.position(0);
System.out.println(dup.getInt()); // 99
```

![](https://cdn.hashnode.com/uploads/covers/637f189ed7d9bcd845996b4b/25086585-dce0-4156-9976-79e7576d0dcf.png align="center")

* * *

## 10\. Direct vs Heap — When to Use What

Here's a comprehensive comparison:

| Aspect | Heap Buffer | Direct Buffer |
| --- | --- | --- |
| **Location** | JVM heap (`byte[]`) | Native memory |
| **Allocation speed** | Fast (~50ns) | Slow (~1μs) |
| **GC Impact** | Subject to GC | Only wrapper object is GC'd |
| **I/O performance** | Slower (extra copy) | Faster (zero-copy possible) |
| `hasArray()` | `true` | `false` |
| **Max size** | Limited by JVM heap | Limited by `-XX:MaxDirectMemorySize` |
| **Deallocation** | Automatic (GC) | Requires GC of wrapper or `Cleaner` |
| **Best for** | Short-lived, small buffers | Long-lived, I/O-intensive buffers |

### How Direct Buffers Avoid Copies

![](https://cdn.hashnode.com/uploads/covers/637f189ed7d9bcd845996b4b/3cd24867-59fc-4650-9999-3945c0d795f4.png align="center")

### Practical Guidelines

```java
// ❌ BAD: Allocating direct buffer in a hot loop
for (int i = 0; i < 100_000; i++) {
    ByteBuffer buf = ByteBuffer.allocateDirect(1024); // SLOW!
    // ... use and discard
}

// ✅ GOOD: Allocate once, reuse
ByteBuffer reusable = ByteBuffer.allocateDirect(1024);
for (int i = 0; i < 100_000; i++) {
    reusable.clear();
    // ... use
}

// ✅ GOOD: ThreadLocal for thread-safe reuse
private static final ThreadLocal<ByteBuffer> BUFFER = 
    ThreadLocal.withInitial(() -> ByteBuffer.allocateDirect(4096));

public void process() {
    ByteBuffer buf = BUFFER.get();
    buf.clear();
    // ... use
}
```

* * *

## 11\. Common Pitfalls

### Pitfall 1: Forgetting to flip()

```java
// ❌ BUG: Reading without flip
ByteBuffer buf = ByteBuffer.allocate(10);
buf.putInt(42);
int value = buf.getInt(); // BufferUnderflowException! position is at 4, limit is 10
                          // reads garbage bytes 4-7 (all zeros), returns 0

// ✅ FIX: Always flip before reading
buf.flip();
int value = buf.getInt(); // 42 ✓
```

### Pitfall 2: Double flip()

```java
// ❌ BUG: Flipping twice
ByteBuffer buf = ByteBuffer.allocate(10);
buf.putInt(42);      // position=4
buf.flip();          // position=0, limit=4  ← correct
buf.flip();          // position=0, limit=0  ← OOPS! limit is now 0
buf.getInt();        // BufferUnderflowException!
```

### Pitfall 3: Direct Buffer Memory Leak

```java
// ❌ RISK: Direct buffers aren't freed promptly
void processFile() {
    ByteBuffer buf = ByteBuffer.allocateDirect(100_000_000); // 100MB
    // ... use buf
    // buf goes out of scope but native memory may NOT be freed
    // until the next GC cycle
}

// ✅ FIX: Explicit cleanup (Java 9+, or use reflection pre-9)
import sun.misc.Unsafe; // or use Cleaner API

// Better: use try-with-resources pattern around a wrapper
```

### Pitfall 4: Wrong byte order for interop

```java
// ❌ BUG: C program writes little-endian, Java reads big-endian
ByteBuffer buf = ByteBuffer.wrap(dataFromC);
int value = buf.getInt(); // Wrong value!

// ✅ FIX: Match the byte order
buf.order(ByteOrder.LITTLE_ENDIAN);
int value = buf.getInt(); // Correct!
```

## 12\. Summary

![](https://cdn.hashnode.com/uploads/covers/637f189ed7d9bcd845996b4b/81a6a305-bdaf-46ab-8b87-09155af87cc6.png align="center")

**Key takeaways:**

1.  **ByteBuffer** is the foundation of all NIO I/O on the JVM
    
2.  **Heap buffers** live on the GC heap; **direct buffers** live in native memory
    
3.  Always **flip()** before switching from writing to reading
    
4.  Use **compact()** when you've partially consumed a buffer
    
5.  **Byte order** matters for interoperability with native code and network protocols
    
6.  **Allocate direct buffers once and reuse them** — they're expensive to create
    

* * *