Generating Java source code with String Templates

Marcio Endo
March 3, 2024

We can use String Templates in JDK 21 as a preview feature. One good use case I've found is to generate Java source code.

So, in this blog post, we will write a String Template processor suited for generating Java source code. Additionally, it should serve as an introduction to String Templates to the Java developer.

Be warned of the following caveats regarding the implementation presented in this blog post:

In short, it is an interpolator with side-effects. And, if I understand it correctly, it is not what the JEP 430 designers had in mind as the best use of it.

You have been warned.

Let's continue.

Iteration 01: a template without expressions

A string template lets us embed regular Java expressions in it. In the following example:

String what = "Hi!";
String greeting = STR."Say \{what}";

The string template contains the \{what} embedded expression. It refers to the what local variable declared immediately before.

A string template can also have no embedded expressions:

String greeting = STR."Say Hi!";

It's probably not the best use for a string template. Regardless, our processor implementation must work when we do not declare embedded expressions.

Iteration 01: test case

We start with the following test case:

@Test(description = "no embedded expressions")
public void testCase01() {
  Code code;
  code = new Code();

  String java;
  java = code."public class TestCase01 {}";

  assertEquals(java, "public class TestCase01 {}");
}

First, we create a instance of our processor by invoking its constructor. Our template processor implementation is named Code.

Next, using our processor instance, we evaluate a string template. The template does not contain any embedded expressions. Our processor produces a String value.

Finally, at the bottom of the method, we verify the produced value. We expect the produced value to be equal to the template as if it were a plain string.

Iteration 01: implementation

First, our Code class must be a StringTemplate.Processor instance:

public class Code 
    implements 
    StringTemplate.Processor<String, RuntimeException> {

  @Override
  public final String process(
  	  StringTemplate template) throws RuntimeException {
    ...
  }
  
}

It is a functional interface that declares two type parameters.

The first type parameter is the type of the result of the processor. From our test case we want our processor to produce a String value.

The second type parameter is the type of the exception thrown by the process method. From our test case it is not clear what type of exception our processor should throw. But we must declare a type. So, for now, we have declared it will simply throw a RuntimeException.

Iteration 01: the process method

And here's our process method implementation:

@Override
public final String process(StringTemplate template) throws RuntimeException {
  List<String> fragments;
  fragments = template.fragments();

  int fragmentsSize;
  fragmentsSize = fragments.size();

  if (fragmentsSize == 1) {
    return fragments.getFirst();
  }

  return process0(template, fragments);
}

protected String process0(StringTemplate template, List<String> fragments) {
  throw new UnsupportedOperationException("Implement me");
}

At runtime, a string template is broken down into fragments.

The number of fragments is a function of the number of embedded expressions declared in the template:

Our particular test case falls in the first case. So we simply return the first and only fragment of the runtime template.

Iteration 02: a single embedded expression

Let's handle templates with embedded expressions.

We will start with a single embedded expression but the code should work for any number of expressions.

Iteration 02: test case

Here's our test case:

@Test(description = "single embedded expression")
public void testCase02() {
  Code code;
  code = new Code();

  String message;
  message = "Hello world!";

  String java;
  java = code."""
  public class TestCase02 {
    public static void main(String[] args) {
      System.out.println("\{message}");
    }
  }
  """;

  assertEquals(java, """
  public class TestCase02 {
    public static void main(String[] args) {
      System.out.println("Hello world!");
    }
  }
  """);
}

Our template contains the \{message} embedded expression. It refers to the message local variable.

And we verify that the generated result correctly contains the value of the message local variable.

Iteration 02: implementation

Here's an implementation that makes our test pass:

public class Code extends iter01.Code {
  @Override
  protected final String process0(StringTemplate template, List<String> fragments) {
    Object[] values;
    values = values(template);

    StringBuilder out;
    out = new StringBuilder();

    for (int idx = 0, len = values.length; idx < len; idx++) {
      String fragment;
      fragment = fragments.get(idx);

      out.append(fragment);

      Object value;
      value = values[idx];

      out.append(value);
    }

    String lastFragment;
    lastFragment = fragments.getLast();

    out.append(lastFragment);

    return out.toString();
  }

  protected Object[] values(StringTemplate template) {
    List<Object> values;
    values = template.values();

    return values.toArray();
  }
}

It extends the implementation of the previous iteration and overrides the process0 method.

It starts by getting all of the values from the template. The StringTemplate::values method returns a list of objects. Each object is the result of evaluating, in encounter order, each of the embedded expressions.

Next, in a for loop, it appends one fragment and one value to the resulting string.

Finally, the last fragment is appended and the result is returned.

So, in summary, it simply interpolates the fragments and values.

Iteration 03: import declarations

I think that, to be useful, a Java source code generator should automatically generate the import declarations.

So let's add such facility to our string template processor.

Additionally, we should test a template having more than one embedded expression.

Iteration 03: test case

That's how we want our import facility to work:

@Test
public void testCase03() {
  Code code;
  code = new Code();

  ClassName LIST;
  LIST = ClassName.of("java.util", "List");

  String packageName;
  packageName = "com.example";

  ImportList imports;
  imports = code.importList(packageName);

  String java;
  java = code."""
  package \{packageName};
  \{imports}
  public class TestCase02 {
    public static void main(String[] args) {
      \{LIST}<String> msgs = \{LIST}.of("Foo", "Bar");
      for (var msg : msgs) {
        System.out.println(msg);
      }
    }
  }
  """;

  assertEquals(java, """
  package com.example;

  import java.util.List;

  public class TestCase02 {
    public static void main(String[] args) {
      List<String> msgs = List.of("Foo", "Bar");
      for (var msg : msgs) {
        System.out.println(msg);
      }
    }
  }
  """);
}

First we introduce a ClassName class. It represents the fully qualified name of a type that is referenced in the generated Java code.

Next, we introduce the ImportList class. It represents a (possibly empty) list of import declarations. It is bound to a particular instance of our processor. It is also bound to particular package name; it shouldn't generate a import declaration for a type that is from the same package.

Additionally, as noted in the assertion, the ImportList should generate empty lines above and below the import declarations.

Iteration 03: implementation

And here's an implementation that makes our test pass:

public class Code extends iter02.Code {
  private final ImportList importList = new ImportList();

  public final ImportList importList(String packageName) {
    Objects.requireNonNull(packageName, "packageName == null");
    importList.set(packageName);
    return importList;
  }

  @Override
  protected final Object[] values(StringTemplate template) {
    List<Object> values;
    values = template.values();

    int size;
    size = values.size();

    Object[] result;
    result = new Object[size];

    for (int idx = 0; idx < size; idx++) {
      Object value;
      value = values.get(idx);

      if (value instanceof ClassName className) {
        value = importList.process(className);
      } else {
        value = processValue(value);
      }

      result[idx] = value;
    }

    return result;
  }

  protected Object processValue(Object value) {
    return value;
  }
}

It extends the implementation of the previous iteration and overrides the values method.

It starts by declaring an ImportList private instance. The importList(String) method returns the instance and, at the same time, binds it to the specified package name.

The values method iterates over the templates' values. When a particular value is a ClassName instance it is further processed by the ImportList instance. If the value of a different type then the value is left as it is.

Next, let's take a closer look at the ImportList class.

Iteration 03: the ImportList class

The following test case specifies how the ImportList class works:

@Test(description = "Happy path")
public void testCase01() {
  ImportList imports;
  imports = new ImportList("com.example");

  ClassName list;
  list = ClassName.of("java.util", "List");

  ClassName awt;
  awt = ClassName.of("java.awt", "List");

  ClassName inputStream;
  inputStream = ClassName.of("java.io", "InputStream");

  ClassName foo;
  foo = ClassName.of("com.example", "Foo");

  assertEquals(imports.process(list), "List");
  assertEquals(imports.process(awt), "java.awt.List");
  assertEquals(imports.process(inputStream), "InputStream");
  assertEquals(imports.process(foo), "Foo");
  assertEquals(imports.toString(), """

  import java.io.InputStream;
  import java.util.List;
  """);
}

First, an ImportList instance is bound to a specific package name.

The java.util.List type is processed to its simple name List and it is imported.

The java.awt.List type is processed to its full name and is not imported.

The java.io.InputStream type is processed to its simple name InputStream and is imported.

The com.example.Foo type is processed to its simple name Foo and is not imported.

Iteration 04: string literals

The following would produce invalid Java code:

String s = """
The message has "quotes".
And new lines
""";

String invalid;
invalid = code."String s = \"\{s}\";"

We should properly escape string values that will serve as string literals in the generated Java code.

Due to time constraints, this has been left as an exercise to the reader. Sorry about that.

Conclusion

In this blog post we have developed a String Template processor for generating Java source code.

As noted in the introduction, one should be aware of the implementation caveats:

Still, it should have worked as an introduction to string templates. In particular:

It does not cover all of topics related to implementing a template processor. In particular, it does mention the Linkage interface.

You can find the source code of the examples in this GitHub repository.