JVM Low-Level I/O - Part 3

FileChannel is Java NIO's workhorse for file operations. It's fundamentally different from FileInputStream/FileOutputStream — instead of reading one byte at a time, it operates on blocks of bytes through ByteBuffers. More importantly, it unlocks memory-mapped I/O, one of the most powerful techniques in systems programming.

---

1. FileChannel vs Traditional Streams

Feature	InputStream/OutputStream	FileChannel
API style	Byte-oriented	Block-oriented (ByteBuffer)
Thread safety	Single thread	Concurrent reads
Memory-mapped	❌	✅
Scatter/Gather	❌	✅
Direct transfer	❌	✅ (transferTo/From)
File locking	❌	✅
Random access	❌ (stream = sequential)	✅ (position)

---

2. Opening a FileChannel

There are several ways to obtain a FileChannel:

2.1 From FileInputStream/FileOutputStream (Legacy)

// Read-only channel
FileInputStream fis = new FileInputStream("data.bin");
FileChannel readChannel = fis.getChannel();

// Write-only channel
FileOutputStream fos = new FileOutputStream("data.bin");
FileChannel writeChannel = fos.getChannel();

// Read/Write channel
RandomAccessFile raf = new RandomAccessFile("data.bin", "rw");
FileChannel rwChannel = raf.getChannel();

2.2 Using FileChannel.open() (Modern — Preferred)

import java.nio.channels.FileChannel;
import java.nio.file.*;

// Read-only
FileChannel readChannel = FileChannel.open(
    Path.of("data.bin"),
    StandardOpenOption.READ
);

// Write, create if not exists, truncate existing
FileChannel writeChannel = FileChannel.open(
    Path.of("data.bin"),
    StandardOpenOption.WRITE,
    StandardOpenOption.CREATE,
    StandardOpenOption.TRUNCATE_EXISTING
);

// Read + Write
FileChannel rwChannel = FileChannel.open(
    Path.of("data.bin"),
    StandardOpenOption.READ,
    StandardOpenOption.WRITE,
    StandardOpenOption.CREATE
);

Open Options Reference

---

3. Reading and Writing with FileChannel

3.1 Basic Read

try (FileChannel channel = FileChannel.open(Path.of("data.txt"), 
        StandardOpenOption.READ)) {
    
    ByteBuffer buffer = ByteBuffer.allocate(1024);
    
    // read() fills the buffer and returns bytes read (-1 at EOF)
    int bytesRead;
    while ((bytesRead = channel.read(buffer)) != -1) {
        buffer.flip(); // prepare for reading from buffer
        
        // Process the data
        while (buffer.hasRemaining()) {
            System.out.print((char) buffer.get());
        }
        
        buffer.clear(); // prepare for next read
    }
}

3.2 Basic Write

try (FileChannel channel = FileChannel.open(Path.of("output.txt"),
        StandardOpenOption.WRITE,
        StandardOpenOption.CREATE,
        StandardOpenOption.TRUNCATE_EXISTING)) {

    String text = "Hello, FileChannel!";
    ByteBuffer buffer = ByteBuffer.wrap(text.getBytes(StandardCharsets.UTF_8));
    
    // write() writes from buffer's position to its limit
    while (buffer.hasRemaining()) {
        channel.write(buffer);
    }
}

The Read/Write Cycle

3.3 Read at a Specific Position

// Read 100 bytes starting at file position 500
ByteBuffer buffer = ByteBuffer.allocate(100);
int bytesRead = channel.read(buffer, 500); // absolute position
// Note: channel's position is NOT changed

3.4 Write at a Specific Position

// Write at file position 1000
ByteBuffer data = ByteBuffer.wrap("Hello".getBytes(StandardCharsets.UTF_8));
channel.write(data, 1000); // absolute position
// channel's position is NOT changed

---

4. Scatter/Gather I/O

Scatter/Gather allows you to read into multiple buffers (scatter) or write from multiple buffers (gather) in a single system call. This is extremely useful for protocols with headers and bodies.

Scatter Read

// Read a message with a fixed 12-byte header and variable body
ByteBuffer header = ByteBuffer.allocate(12);
ByteBuffer body   = ByteBuffer.allocate(1024);

ByteBuffer[] buffers = {header, body};

// Single system call fills header first, then body
long bytesRead = channel.read(buffers);

header.flip();
int msgType   = header.getInt();    // bytes 0-3
int bodyLen   = header.getInt();    // bytes 4-7
int checksum  = header.getInt();    // bytes 8-11

body.flip();
byte[] bodyData = new byte[bodyLen];
body.get(bodyData);

Gather Write

// Write a message with header + body in one call
ByteBuffer header = ByteBuffer.allocate(12);
header.putInt(1);           // message type
header.putInt(bodyBytes.length); // body length
header.putInt(checksum);    // checksum
header.flip();

ByteBuffer body = ByteBuffer.wrap(bodyBytes);

ByteBuffer[] buffers = {header, body};

// Single system call writes both
long bytesWritten = channel.write(buffers);

---

5. Channel-to-Channel Transfers

transferTo() and transferFrom() can copy data between channels without intermediate buffers — the OS can perform a zero-copy DMA transfer.

// Copy a file efficiently
try (FileChannel source = FileChannel.open(Path.of("large.bin"), 
         StandardOpenOption.READ);
     FileChannel dest = FileChannel.open(Path.of("copy.bin"),
         StandardOpenOption.WRITE, StandardOpenOption.CREATE)) {

    long size = source.size();
    long transferred = 0;
    
    // transferTo may not transfer everything in one call
    while (transferred < size) {
        transferred += source.transferTo(transferred, size - transferred, dest);
    }
}

Performance note: transferTo can be 3-10x faster than manual buffer copying for large files, because the OS can use DMA (Direct Memory Access) to transfer data between devices without CPU involvement.

---

6. Memory-Mapped I/O (MappedByteBuffer)

This is the most powerful feature of FileChannel. Memory-mapping creates a direct link between a file and virtual memory — the file becomes memory.

How Memory Mapping Works

What happens when you access a memory-mapped byte:

Creating a Memory-Mapped Buffer

try (FileChannel channel = FileChannel.open(Path.of("data.bin"),
        StandardOpenOption.READ, StandardOpenOption.WRITE)) {

    // Map the entire file into memory
    MappedByteBuffer mapped = channel.map(
        FileChannel.MapMode.READ_WRITE,  // mode
        0,                                // file position to start mapping
        channel.size()                    // number of bytes to map
    );

    // Now you can read/write as if it's a regular ByteBuffer!
    int firstInt = mapped.getInt(0);     // reads from file
    mapped.putInt(0, 42);                // writes to file
    
    // Force changes to disk
    mapped.force();
}

Map Modes

Large File Processing

Memory mapping shines for large files — you can map a multi-GB file and the OS handles paging transparently:

Path path = Path.of("huge_dataset.bin");

try (FileChannel channel = FileChannel.open(path, StandardOpenOption.READ)) {
    long fileSize = channel.size();
    
    // For files > 2GB, map in chunks (MappedByteBuffer is limited to ~2GB)
    long chunkSize = Integer.MAX_VALUE; // ~2GB
    long position = 0;
    
    while (position < fileSize) {
        long remaining = fileSize - position;
        long mapSize = Math.min(chunkSize, remaining);
        
        MappedByteBuffer mapped = channel.map(
            FileChannel.MapMode.READ_ONLY,
            position,
            mapSize
        );
        
        // Process this chunk
        processChunk(mapped);
        
        position += mapSize;
    }
}

private void processChunk(MappedByteBuffer buffer) {
    while (buffer.hasRemaining()) {
        // Process data...
        int value = buffer.getInt();
    }
}

---

7. File Locking

FileChannel supports both shared (read) and exclusive (write) locks for coordinating access between processes.

Exclusive Lock (Blocking)

try (FileChannel channel = FileChannel.open(Path.of("shared.dat"),
        StandardOpenOption.READ, StandardOpenOption.WRITE)) {
    
    // Acquire exclusive lock — blocks until available
    try (FileLock lock = channel.lock()) {
        System.out.println("Lock acquired! (exclusive)");
        
        // Safe to read/write
        ByteBuffer buf = ByteBuffer.allocate(4);
        channel.read(buf, 0);
        buf.flip();
        int counter = buf.getInt();
        
        buf.clear();
        buf.putInt(counter + 1);
        buf.flip();
        channel.write(buf, 0);
        
        System.out.println("Counter: " + (counter + 1));
        // Lock released when try block exits
    }
}

Non-Blocking Try-Lock

// Try to acquire lock without blocking
FileLock lock = channel.tryLock();
if (lock != null) {
    try {
        // Got the lock — proceed
    } finally {
        lock.release();
    }
} else {
    System.out.println("File is locked by another process");
}

Region Locking

// Lock only bytes 0-99 (shared/read lock)
FileLock headerLock = channel.lock(0, 100, true); // shared=true

// Lock bytes 100-999 (exclusive/write lock)
FileLock bodyLock = channel.lock(100, 900, false); // shared=false

---

8. Position and Truncation

Seeking (Random Access)

FileChannel channel = FileChannel.open(path, 
    StandardOpenOption.READ, StandardOpenOption.WRITE);

// Get current position
long pos = channel.position(); // 0

// Seek to position 1000
channel.position(1000);

// Read from position 1000
ByteBuffer buf = ByteBuffer.allocate(100);
channel.read(buf); // reads bytes 1000-1099

// Get file size
long size = channel.size();

Truncating a File

// Truncate file to 1000 bytes
channel.truncate(1000);

// If the file was 5000 bytes, it's now 1000 bytes
// If position was > 1000, it's set to 1000

---

9. Force and Metadata

Forcing Data to Disk

By default, writes go to the OS page cache and are flushed to disk asynchronously. Use force() when you need durability guarantees:

// Write critical data
channel.write(buffer);

// Force data to disk
channel.force(false); // false = don't force metadata
channel.force(true);  // true  = force data AND metadata

// MappedByteBuffer also has force()
mapped.force(); // flush mapped region changes to disk

---

10. Performance Comparison

Let's benchmark the different approaches for reading a 100MB file:

import java.nio.*;
import java.nio.channels.*;
import java.nio.file.*;
import java.io.*;

public class FileReadBenchmark {

    private static final String FILE = "test_100mb.bin";
    
    // Method 1: Traditional InputStream (byte-by-byte)
    static long readStream() throws IOException {
        long sum = 0;
        try (FileInputStream fis = new FileInputStream(FILE)) {
            int b;
            while ((b = fis.read()) != -1) {
                sum += b;
            }
        }
        return sum;
    }
    
    // Method 2: Buffered InputStream
    static long readBufferedStream() throws IOException {
        long sum = 0;
        try (BufferedInputStream bis = new BufferedInputStream(
                new FileInputStream(FILE), 65536)) {
            int b;
            while ((b = bis.read()) != -1) {
                sum += b;
            }
        }
        return sum;
    }

    // Method 3: FileChannel + ByteBuffer
    static long readChannel() throws IOException {
        long sum = 0;
        try (FileChannel channel = FileChannel.open(
                Path.of(FILE), StandardOpenOption.READ)) {
            ByteBuffer buf = ByteBuffer.allocateDirect(65536);
            while (channel.read(buf) != -1) {
                buf.flip();
                while (buf.hasRemaining()) {
                    sum += buf.get() & 0xFF;
                }
                buf.clear();
            }
        }
        return sum;
    }

    // Method 4: Memory-Mapped I/O
    static long readMapped() throws IOException {
        long sum = 0;
        try (FileChannel channel = FileChannel.open(
                Path.of(FILE), StandardOpenOption.READ)) {
            MappedByteBuffer mapped = channel.map(
                FileChannel.MapMode.READ_ONLY, 0, channel.size());
            while (mapped.hasRemaining()) {
                sum += mapped.get() & 0xFF;
            }
        }
        return sum;
    }
    
    public static void main(String[] args) throws IOException {
        // Create test file
        createTestFile();
        
        // Warmup
        readChannel();
        
        // Benchmark
        long start, elapsed;
        
        start = System.nanoTime();
        readStream();
        elapsed = System.nanoTime() - start;
        System.out.printf("InputStream:           %,d ms%n", elapsed / 1_000_000);

        start = System.nanoTime();
        readBufferedStream();
        elapsed = System.nanoTime() - start;
        System.out.printf("BufferedInputStream:   %,d ms%n", elapsed / 1_000_000);

        start = System.nanoTime();
        readChannel();
        elapsed = System.nanoTime() - start;
        System.out.printf("FileChannel + Direct: %,d ms%n", elapsed / 1_000_000);
        
        start = System.nanoTime();
        readMapped();
        elapsed = System.nanoTime() - start;
        System.out.printf("Memory-Mapped:        %,d ms%n", elapsed / 1_000_000);
    }
    
    static void createTestFile() throws IOException {
        Path path = Path.of(FILE);
        if (!Files.exists(path)) {
            byte[] data = new byte[100 * 1024 * 1024]; // 100MB
            new java.util.Random(42).nextBytes(data);
            Files.write(path, data);
        }
    }
}

Typical Results (100MB file, SSD)

Method	Time (approx.)	Why
InputStream	~8,000 ms	One byte = one system call
BufferedInputStream	~400 ms	Reads 64KB chunks internally
FileChannel + Direct	~120 ms	Direct buffer + large reads
Memory-Mapped	~80 ms	OS handles paging, no system calls per read

---

11. Practical Patterns

Pattern: Structured Binary File

/**
 * Binary file format:
 * ┌────────────────────────────────────────┐
 * │ Header (16 bytes)                      │
 * │  ├─ magic: 4 bytes (0xCAFEBABE)       │
 * │  ├─ version: 4 bytes                   │
 * │  ├─ record_count: 4 bytes              │
 * │  └─ reserved: 4 bytes                  │
 * ├────────────────────────────────────────┤
 * │ Record[0] (24 bytes)                   │
 * │  ├─ id: 8 bytes (long)                │
 * │  ├─ value: 8 bytes (double)            │
 * │  └─ timestamp: 8 bytes (long)          │
 * ├────────────────────────────────────────┤
 * │ Record[1] ...                          │
 * └────────────────────────────────────────┘
 */
public class BinaryFileWriter {
    
    private static final int MAGIC = 0xCAFEBABE;
    private static final int VERSION = 1;
    private static final int HEADER_SIZE = 16;
    private static final int RECORD_SIZE = 24;
    
    public static void write(Path path, List<Record> records) throws IOException {
        int fileSize = HEADER_SIZE + records.size() * RECORD_SIZE;
        
        try (FileChannel channel = FileChannel.open(path,
                StandardOpenOption.WRITE,
                StandardOpenOption.CREATE,
                StandardOpenOption.TRUNCATE_EXISTING)) {
            
            ByteBuffer buffer = ByteBuffer.allocate(fileSize);
            buffer.order(ByteOrder.BIG_ENDIAN);
            
            // Write header
            buffer.putInt(MAGIC);
            buffer.putInt(VERSION);
            buffer.putInt(records.size());
            buffer.putInt(0); // reserved
            
            // Write records
            for (Record r : records) {
                buffer.putLong(r.id());
                buffer.putDouble(r.value());
                buffer.putLong(r.timestamp());
            }
            
            buffer.flip();
            channel.write(buffer);
        }
    }
    
    public static List<Record> read(Path path) throws IOException {
        try (FileChannel channel = FileChannel.open(path, StandardOpenOption.READ)) {
            MappedByteBuffer mapped = channel.map(
                FileChannel.MapMode.READ_ONLY, 0, channel.size());
            mapped.order(ByteOrder.BIG_ENDIAN);
            
            // Read & validate header
            int magic = mapped.getInt();
            if (magic != MAGIC) {
                throw new IOException("Invalid file format: bad magic number");
            }
            
            int version = mapped.getInt();
            int recordCount = mapped.getInt();
            mapped.getInt(); // skip reserved
            
            // Read records
            List<Record> records = new ArrayList<>(recordCount);
            for (int i = 0; i < recordCount; i++) {
                long id = mapped.getLong();
                double value = mapped.getDouble();
                long timestamp = mapped.getLong();
                records.add(new Record(id, value, timestamp));
            }
            
            return records;
        }
    }
    
    record Record(long id, double value, long timestamp) {}
}

Pattern: Append-Only Log

public class AppendLog implements AutoCloseable {
    private final FileChannel channel;
    private final ByteBuffer headerBuf = ByteBuffer.allocate(8);
    
    public AppendLog(Path path) throws IOException {
        this.channel = FileChannel.open(path,
            StandardOpenOption.WRITE,
            StandardOpenOption.CREATE,
            StandardOpenOption.APPEND);
    }
    
    /**
     * Append a record: [4-byte length][data bytes][4-byte CRC32]
     */
    public synchronized void append(byte[] data) throws IOException {
        // Calculate checksum
        CRC32 crc = new CRC32();
        crc.update(data);
        int checksum = (int) crc.getValue();
        
        // Prepare header
        headerBuf.clear();
        headerBuf.putInt(data.length);
        headerBuf.putInt(checksum);
        headerBuf.flip();
        
        // Gather write: header + data
        ByteBuffer dataBuf = ByteBuffer.wrap(data);
        channel.write(new ByteBuffer[]{headerBuf, dataBuf});
        
        // Ensure durability
        channel.force(false);
    }
    
    @Override
    public void close() throws IOException {
        channel.close();
    }
}

---

12. Summary

Key takeaways:

1. FileChannel operates on ByteBuffers — block I/O, not byte I/O

2. Memory-mapped I/O is the fastest way to access file data

3. Scatter/Gather lets you read/write structured data efficiently

4. transferTo/transferFrom enables zero-copy file operations

5. File locking coordinates access between processes

6. force() ensures data durability when you need it