A custom java.io.InputStream implementation for testing

Marcio EndoMarcio EndoFeb 4, 2024

edit 2024-02-05: add a link to the GitHub repo

Suppose you are writing a HTTP/1.1 request parser in Java. For the purposes of this blog post, the API is the following:

try (Socket socket  = serverSocket.accept()) {  MyHttpParser parser;  parser = new MyHttpParser();  InputStream is;  is = socket.getInputStream();  Request request;  request = parser.parse(is);  // proceed to handle request  }

So, in order to test the parse method, we need to provide a java.io.InputStream instance.

We could use one of the JDK provided implementations. For example, there's the ByteArrayInputStream.

But, as we will discuss, a custom implementation will be a better fit to our requirements.

So, in this blog post, we will talk about the java.io.InputStream implementation I've used to test the Objectos HTTP request parser.

Let's begin.

TestingInputStream

The name of our class is TestingInputStream. It is a direct subclass of java.io.InputStream.

In this section we will discuss its requirements through a series of test cases. And, at the end of the section, we will present our implementation.

00: it should support buffered reads

Our parser has an internal buffer which it uses to read from the input stream. So it only invokes the read(byte[], int, int) method of the input stream.

As a result, our read() method implementation only throws an exception:

@Overridepublic final int read() throws IOException {  throw new UnsupportedOperationException(      "Please use the read(byte[], int, int) method"  );}

On the other hand, our implementation must override the read(byte[], int, int) method.

In our tests, we will use the read(byte[]) method. Here's the implementation from InputStream:

public int read(byte[] b) throws IOException {  return read(b, 0, b.length);}

So it delegates to the method we will actually implement.

01: it should be easy to use strings as input

The HTTP/1.1 protocol is text based. So we want it to be easy to write our test data using string literals.

Using a ByteArrayInputStream we would have to write an auxiliary method:

private InputStream inputStreamOf(String request) {  byte[] bytes = request.getBytes(StandardCharsets.UTF_8);  return new ByteArrayInputStream(bytes);}

Instead, we should be able to simply pass the test data directly to the constructor.

So, if we were to test the happy path where the buffer is the exact size of the input data, we would like to write it like so:

@Testpublic void testCase01() {  try (InputStream is = new TestingInputStream("GET")) {    byte[] buf;    buf = new byte[3];    assertEquals(is.read(buf), 3);    assertEquals(utf8(buf), "GET");    assertEquals(is.read(buf), -1);    assertEquals(is.read(buf), -1);  } catch (IOException e) {    Assert.fail("It should not have thrown", e);  }}private String utf8(byte[] bytes) {  return new String(bytes, StandardCharsets.UTF_8);}

The first read operation fills the entire buffer. We verify that it returns the correct number of bytes read.

Next, we verify that the buffer contents corresponds to the UTF-8 string "GET".

Then, we verify that any subsequent read operation returns -1 indicating that the end of the stream was reached.

02: it should return the number of bytes read

To be sure, our read method implementation should return the number of bytes that were actually read. In other words, it shouldn't return e.g. the size of the buffer.

So we write a test where the buffer is larger than the data:

@Testpublic void testCase02() {  try (InputStream is = new TestingInputStream("GET")) {    byte[] buf;    buf = new byte[10];    assertEquals(is.read(buf), 3);    assertEquals(utf8(buf, 0, 3), "GET");    assertEquals(is.read(buf), -1);    assertEquals(is.read(buf), -1);  } catch (IOException e) {    Assert.fail("It should not have thrown", e);  }}private String utf8(byte[] bytes, int offset, int len) {  return new String(bytes, offset, len, StandardCharsets.UTF_8);}

But, as our buffer is larger than the data, in the second assertion, we use only test the contents of its initial section.

03: it should support partial reads

Our input stream should support the case when the buffer is smaller than the data.

In other words, multiple reads are required to fully exhaust the stream:

@Testpublic void testCase03() {  try (InputStream is = new TestingInputStream("abc123")) {    byte[] buf;    buf = new byte[3];    assertEquals(is.read(buf), 3);    assertEquals(utf8(buf), "abc");    assertEquals(is.read(buf), 3);    assertEquals(utf8(buf), "123");    assertEquals(is.read(buf), -1);    assertEquals(is.read(buf), -1);  } catch (IOException e) {    Assert.fail("It should not have thrown", e);  }}

So, somehow, our implementation must remember that, after the first read, 3 bytes are still left to be read.

04: simulating slow clients

We will use our TestingInputStream to test a HTTP/1.1 request parser.

In production, we will use our InputStream to read the data that is being sent by a remote network client.

It could be that a client cannot (or will not) send all of the request data at once. Reasons could be:

  • it is a POST request, and the client sends the headers first and then the body;

  • the request has a very large body which is sent in chunks; or

  • due to network conditions, data arrives in fragments and slowly.

Up to this point, we could have used a ByteArrayInputStream for our tests.

But with a vanilla ByteArrayInputStream we can't easily simulate the situations listed above. In particular, we want to express in that the data may arrive in chunks:

@Testpublic void testCase04() {  try (InputStream input = new TestingInputStream("abc", "123")) {    byte[] buf;    buf = new byte[10];    assertEquals(input.read(buf), 3);    assertEquals(utf8(buf, 0, 3), "abc");    assertEquals(input.read(buf), 3);    assertEquals(utf8(buf, 0, 3), "123");    assertEquals(input.read(buf), -1);    assertEquals(input.read(buf), -1);  } catch (IOException e) {    Assert.fail("It should not have thrown", e);  }}

It means that, even though our buffer is large enough to hold the complete message, our parser must be aware that the first read operation might only return a part of the full message.

05: the buffer might not be large enough to fit a whole chunk

The current scenario is a combination of the two previous ones:

@Testpublic void testCase05() {  try (InputStream is = new TestingInputStream("abcde", "12345")) {    byte[] buf;    buf = new byte[3];    assertEquals(is.read(buf), 3);    assertEquals(utf8(buf, 0, 3), "abc");    assertEquals(is.read(buf), 2);    assertEquals(utf8(buf, 0, 2), "de");    assertEquals(is.read(buf), 3);    assertEquals(utf8(buf, 0, 3), "123");    assertEquals(is.read(buf), 2);    assertEquals(utf8(buf, 0, 2), "45");    assertEquals(is.read(buf), -1);    assertEquals(is.read(buf), -1);  } catch (IOException e) {    Assert.fail("It should not have thrown", e);  }}

In other words:

  • the data arrives in chunks; and

  • each chunk will require multiple read operations.

06: simulate an error during a read operation

Our parser implementation must properly handle errors that might arise during a read operation.

Additionally, the error might not occur in the first read operation:

@Testpublic void testCase06() {  IOException readError;  readError = new IOException("On read");  try (InputStream is = new TestingInputStream("abc", readError)) {    byte[] buf;    buf = new byte[3];    assertEquals(is.read(buf), 3);    assertEquals(utf8(buf, 0, 3), "abc");    assertEquals(is.read(buf), 3);    Assert.fail("It should have thrown");  } catch (IOException e) {    assertSame(e, readError);  }}

So here we create an instance of our input stream with two elements:

  • the string "abc"; and

  • an IOException instance representing an error during the read operation.

We expect the second read operation to fail with an IOException. So, if the execution continues normally after the method invocation, the test should fail. We indicate it by adding the Assert::fail at the end of the try block.

Finally, in the catch block, we verify that the specified IOException was thrown.

07: simulate an error during a close operation

We should be able to simulate an IOException error occurring during the close operation:

@Testpublic void testCase07() {  IOException closeError;  closeError = new IOException("On close");  try (TestingInputStream is = new TestingInputStream("abc", "123")) {    is.onClose(closeError);    byte[] buf;    buf = new byte[3];    assertEquals(is.read(buf), 3);    assertEquals(utf8(buf), "abc");    assertEquals(is.read(buf), 3);    assertEquals(utf8(buf), "123");    assertEquals(is.read(buf), -1);    assertEquals(is.read(buf), -1);  } catch (IOException e) {    assertSame(e, closeError);  }}

Notice the TestingInputStream::onClose invocation at the top of the try block.

In the catch block, we verify that the exception is the exact same instance we created at the beginning of the test.

08: it should support plain byte arrays

Finally, some of the HTTP methods allow for a message body. It could be arbitrary binary data.

So our InputStream should allow byte arrays as input data:

@Testpublic void testCase08() {  byte[] world;  world = "World".getBytes(StandardCharsets.UTF_8);  try (InputStream is = new TestingInputStream("Hello", world)) {    byte[] buf;    buf = new byte[5];    assertEquals(is.read(buf), 5);    assertEquals(utf8(buf), "Hello");    assertEquals(is.read(buf), 5);    assertEquals(utf8(buf), "World");    assertEquals(is.read(buf), -1);  } catch (IOException e) {    Assert.fail("It should not have thrown", e);  }}

Notice that we should also be able to mix it with string values.

Implementation

And here's our TestingInputStream implementation:

public class TestingInputStream extends InputStream {  private final Object[] data;  private int index;  private IOException closeError;  public TestingInputStream(Object... data) {    this.data = data.clone();  }  public final void onClose(IOException error) {    closeError = error;  }  @Override  public final int read() throws IOException {    throw new UnsupportedOperationException(        "Please use the read(byte[], int, int) method"    );  }  @Override  public final int read(byte[] b, int off, int len) throws IOException {    if (index == data.length) {      return -1;    }    Object next;    next = data[index++];    if (next instanceof String s) {      next = s.getBytes(StandardCharsets.UTF_8);    }    if (next instanceof byte[] bytes) {      int lengthToCopy;      lengthToCopy = bytes.length;      if (lengthToCopy > len) {        int lengthRemaining;        lengthRemaining = lengthToCopy - len;        byte[] remaining;        remaining = new byte[lengthRemaining];        System.arraycopy(bytes, len, remaining, 0, lengthRemaining);        data[--index] = remaining;        lengthToCopy = len;      }      System.arraycopy(bytes, 0, b, off, lengthToCopy);      return lengthToCopy;    }    else if (next instanceof IOException ioe) {      throw ioe;    }    else {      throw new UnsupportedOperationException(          "Implement me :: type=" + next.getClass()      );    }  }  @Override  public final void close() throws IOException {    if (closeError != null) {      throw closeError;    }  }}

It supports all of the discussed scenarios.

Conclusion

I try to avoid creating abstractions if I can. So, if a test needs a file, I'll just use a regular file. I will not create an abstraction just so I can write a test that does not requires a file.

The same principle goes for when a test requires a java.io.InputStream class.

This has not always been the case, I'll be the first to admit.

And, just like production code, I try to write maintainable testing code. It is OK and natural to fail at times.

But, as a general guideline:

  • I prefer not to use assertion libraries. I just use the one provided by TestNG;

  • I prefer not to use mocking libraries. I code a custom implementation if necessary; and

  • I prefer to test exceptions explicitly.

You can find the source code of the examples in this GitHub repository.