Java ByteBuffer: How to use flip() and compact()

In this article, I show you, using an example, how the Java ByteBuffer works, and what precisely the Methoden flip() and compact() do.

The article answers the following questions:

• What is a ByteBuffer, and what do you need it for?
• How do you create a ByteBuffer?
• What do the values position, limit, and capacity mean?
• How do I write into the ByteBuffer, how do I read from it?
• What exactly do the methods flip() and compact() do?

What is a ByteBuffer, and what do you need it for?

You need a ByteBuffer to write data to or read data from a file, a socket, or another I/O component using a so-called Channel. This article is mainly about the ByteBuffer itself. To learn how to read and write files with ByteBuffer and FileChannel, read this article.

A ByteBuffer is a wrapper around a byte array and provides methods for conveniently writing to and reading from the byte array. The ByteBuffer internally stores the read/write position and a so-called “limit”.

You can find out exactly what this means in the following example – step-by-step.

You can find the code written for this article in my GitLab Repository.

How to create a ByteBuffer

First, you have to create a ByteBuffer with a given size (“capacity”). There are two methods for this:

• ByteBuffer.allocate(int capacity)
• ByteBuffer.allocateDirect(int capacity)

The parameter capacity specifies the size of the buffer in bytes.

The allocate() method creates the buffer in the Java heap memory, where the Garbage collector will remove it after use.

allocateDirect(), on the other hand, creates the buffer in native memory, i.e., outside the heap. Native memory has the advantage that read and write operations can be performed faster. The reason is that the corresponding operating system operations can access this memory area directly, and data does not have to be exchanged between the Java heap and the operating system first. The disadvantage of this method is higher allocation and deallocation costs.

We create a ByteBuffer with a size of 1,000 bytes as follows:

var buffer = ByteBuffer.allocate(1000);

Then we have a look at the buffer’s metrics – position, limit and capacity:

System.out.printf("position = %4d, limit = %4d, capacity = %4d%n",
    buffer.position(), buffer.limit(), buffer.capacity());

(Since we will print these metrics repeatedly throughout the example, we extract the System.out.println() command into a printMetrics(buffer) method right away.)

We see the following output:

position = 0, limit = 1000, capacity = 1000

Here is a graphical representation so that you can better imagine the buffer. The light yellow area is empty and can subsequently be filled.

ByteBuffer position, limit, and capacity

The meaning of the displayed metrics:

• position is the read/write position. It is always 0 for a new buffer.
• limit has two meanings: When we write to the buffer, limit indicates the position up to which we can write. When we read from the buffer, limit indicates up to which position the buffer contains data. Initially, a ByteBuffer is always in write mode, and limit is equal to capacity – we can fill the empty buffer up to the end.
• capacity indicates the size of the buffer. Its value 1,000 corresponds to the 1,000 bytes that we passed to the allocate() method. It will not change during the lifetime of the buffer.

The ByteBuffer read-write cycle

Writing to the ByteBuffer using put()

For writing into the ByteBuffer, there are several put() methods to write single bytes, a byte array, or other primitive types (like char, double, float, int, long, short) into the buffer. First, we write 100 times the value 1 into the buffer, and then we look at the buffer metrics again:

for (int i = 0; i < 100; i++) {
  buffer.put((byte) 1);
}

printMetrics(buffer);

After running the program, we see the following output:

position = 100, limit = 1000, capacity = 1000

The position has moved 100 bytes to the right; the buffer now looks as follows:

Next, we write 200 times a two in the buffer. This time we use a different method: We first fill a byte array and copy it into the buffer. Finally, we print the metrics again:

byte[] twos = new byte[200];
Arrays.fill(twos, (byte) 2);
buffer.put(twos);

printMetrics(buffer);

Now we see:

position = 300, limit = 1000, capacity = 1000

The position has shifted another 200 bytes to the right; the buffer looks like this:

Switching to read mode with Buffer.flip()

For reading from the buffer, there are corresponding get() methods. These are invoked, for example, when writing to a channel using Channel.write(buffer).

Since position indicates not only the write position but also the read position, we must set position back to 0.

At the same time, we set limit to 300 to indicate that a maximum of 300 bytes can be read from the buffer.

In the program code, we do this as follows:

buffer.limit(buffer.position());
buffer.position(0);

Since these two lines are needed every time you switch from write to read mode, there is a method that does the same:

buffer.flip();

Invoking printMetrics() now shows the following values:

position = 0, limit = 300, capacity = 1000

So the position pointer has returned to the beginning of the buffer, and limit points to the end of the filled area:

Reading from the ByteBuffer with get()

Let’s assume that the channel we want to write to can currently only take 200 of the 300 bytes. We can simulate this by supplying the ByteBuffer.get() method with a 200-byte-sized byte array in which the buffer should write its data:

buffer.get(new byte[200]);

printMetrics() now displays the following:

position = 200, limit = 300, capacity = 1000

The read position has shifted to the right by 200 bytes – i.e., to the end of the data already read, which is equal to the beginning of the data that we still need to read:

Switching to write mode – how not to do it

To write back to the buffer now, you could make the following mistake: You set position to the end of the data, i.e., 300, and limit back to 1000, which brings us back to precisely the state we were in after writing the ones and twos:

Let’s assume that we would now write 300 more bytes into the buffer. The buffer would then look like this:

If we would now use flip() to switch back to read mode, position would be back to 0:

Now, however, we would read the first 200 bytes, which we’ve already read, once more.

This approach is, therefore, wrong. The following section explains how to do it correctly.

Switching to write mode with Buffer.compact()

Instead, we must proceed as follows when switching to write mode:

• We calculate the number of remaining bytes: remaining = limit - position, in the example, this results in 100.
• We move the remaining bytes to the beginning of the buffer.
• We set the write position to the end of the bytes shifted left, that’s 100 in the example.
• We set limit to the end of the buffer.

ByteBuffer also provides a convenience method for this:

buffer.compact();

After invoking compact(), printMetrics() prints the following:

position = 100, limit = 1000, capacity = 1000

In the diagram, the compact() process looks like this:

The next cycle

Now we can write the next 300 bytes into the buffer:

byte[] threes = new byte[300];
Arrays.fill(threes, (byte) 3);
buffer.put(threes);

printMetrics() now displays the following values:

position = 400, limit = 1000, capacity = 1000

After writing the threes, position has shifted to the right by 300 bytes:

Now we can easily switch back to read mode using flip():

buffer.flip();

A final call to printMetrics() prints the following values:

position = 0, limit = 400, capacity = 1000

The reading position is at the beginning of the buffer, to where the compact() method shifted the remaining 100 twos. So we can now continue reading at precisely the position where we stopped before.

Summary

This article explained the functionality of the Java ByteBuffer and its flip() and compact() methods.

If this article has helped you understand ByteBuffer better, feel free to share it using one of the share buttons below.

You are also welcome to join my e-mail list, and you will be informed as soon as I publish a new article.