Jamal Groovy integration module

Using this integration module you can mix Jamal macro text with Groovy code snippets. To use this module you have to add the dependency to you Maven project, as:

<dependency>
    <groupId>com.javax0.jamal</groupId>
    <artifactId>jamal-groovy</artifactId>
    <version>2.8.3-SNAPSHOT</version>
</dependency>

Following that you can use the macros

groovy:eval
groovy:import
groovy:shell
groovy:property
groovy:closer

macros.

Macros implemented in the package

The macros implemented in this package make it possible to embed Groovy scripts into the Jamal source code. Groovy is a more or less Java source-compatible JVM language with compelling features. Using Groovy as a scripting language may be an alternative to using the Java provided and Jamal core integrated JShell. In some cases, an embedded Groovy script may also serve your purpose instead of writing a Java built-in macro.

In this chapter, we will discuss every macro defined in the package with examples. The examples are automatically generated into the Asciidoc file, similarly as the samples are generated in the other documentation files. The difference is that in the case of the main README.adoc and the other files, the conversion uses two Java-defined macros. In this case, these macros are not used. Instead, Groovy scripts perform the tasks eliminating the need for writing a separate class to implement the functionality for the user-defined macros sample and output.

The next chapter, Sample describes this process as a non-trivial example for using the Groovy engine.

The package defines 5 macros.

The macro groovy:eval evaluates a simple expression. groovy:import can import Groovy code. groovy:property can set or get a Groovy property. groovy:shell can execute more complex Groovy code. Finally, groovy:closer makes it possible to implement closer scripts.

The different macros use GroovyShell objects to execute the Groovy scripts. These shell objects are automatically created and named. Their use is detailed in the chapter Using Multiple Shell Objects If you do not want to use multiple, separated Groovy shells, you do not need to worry about the shell objects. One will be created upon the first use of one of the Groovy macros and used throughout the Jamal file.

i. `groovy:eval`

The macro groovy:eval uses its whole input as a groovy expression and evaluates it. For example the following macro call:

{@groovy:eval 1+1}

will result:

ii. `groovy:import`

This macro can import a Groovy file into the shell. The macro has two forms:

{@groovy:import fileName}, and
the format

{@groovy:import

}

In the first case the named file can be imported. The name of the file can be relative to the input file that contains the import macro, can be absolute, or can have the prefix res: or https://. The file names starting with the prefix res: are resource files and have to be available on the classpath. The files with the prefix https://` are downloaded from the net and are cached.

The name resolution and file access, download and caching is done exactly the same way as in the case of the built-in macro import. For more information about that please read the main readme file of Jamal.

In the second case, if there are no characters denoting a file name on the first line after the macro name, the content of the macro will be imported.

The content will be evaluated in the shell. This evaluation tries to create a return value, which is ignored in the case of the groovy:import macro. However, if the content does not have an evaluable return value then the shell will throw an exception. This exception is hard to ignore, because the Jamal macro implementation cannot comprehend if this is a final value evaluation error or a real Groovy error. If there is a real Groovy error, then the macros enclose it into a Jamal syntax error and report it that way.

To avoid the final evaluation errors the macro appends a ;'' string after the Groovy script to be imported. This will result an empty string, which is, what the macro groovy:import results.

Use groovy:import if you do not want to include the result of the script into your output or if the script is in a separate Groovy file.

iii. `groovy:shell`

You can execute script using groovy:shell and get the result as a string into the Jamal output. The format of the macro is

{@groovy:shell scriptname.groovy
script content
}

The scriptname.groovy part is optional. This name is only used in error reports. When you get an error message it makes simpler to find which Groovy script fragment was throwing up. If this part is missing then the name of the actual Groovy shell will be used with an appended .groovy extension. Again: this is not a reference to any real file name, it is only for the error messages. Using something.groovy is just a reasonable convention that you may like to follow.

The script content will be evaluated, and the final result will be the result of the macro. Use this macro in case your script is longer and not just a simple one liner, in which case you can use the macro groovy:eval.

iv. `groovy:property`

This macro sets and gets a Groovy shell property. Groovy shells can communicate with the embedding application, in this case this is Jamal, via so called bindings. A "bindings" is map that assignes values to a string. In the Groovy script you can access these bindings as global variables. For example the following sample

{@groovy:property myProp=(int)55}
{@groovy:eval myProp+45}

and it will result

You can also fetch the value of a property that was set before by the Groovy script or by the groovy:property macro:

{@groovy:eval yourProp=133}
{@groovy:property yourProp}

will result

133
133

once by the result of the groovy:eval and once as the groovy:property also fetched this value.

Setting the value you can specify the type of the property. The possible types are limited to

int
long
double
float
boolean
short
byte
char
YamlString

The casting type has to be enclosed between ( and ) characters, the same way as casting usually is in Java. The default is to set the property to be a string. The casting (string) is available in case you want to emphasize that the value should be handled as a string. It may also happen that you want to pass a string that starts with the characters (int) or something similar.

v. `groovy:closer`

Using the macro groovy:closer you can create a so-called closer script. The script can be used to modify the whole output after the processing of Jamal has finished.

The format of the macro is

{@groovy:closer groovy script}

The only argument to the macro is the closer Groovy string. I can be multi line and it is executed after the processing of the whole Jamal file has finished. Before starting the script the shell will get the bindings result. It means that the global variable result can be used in the closer. The content of the global variable result is the StringBuilder object that holds the final output of Jamal.

The script can either modify this StringBuilder object and return null, or the original StringBuilder object itself, or it should return something that can be converted to a string calling toString().

If the script returns null or the original StringBuilder object then the macro will tell Jamal to use the original result object. Returning the "original" object means that the returned object is the same as it was assigned to the global variable result. The content of the StringBuilder may be modified, and these modification will be used. This is the most effective and optimized way to modify the final result in a post processing step.

If the return value is neither null, nor the original object then the Jamal StringBuilder object holding the result up to now is deleted and the returned value is put into the result. This approach needs more memory creating and copying the result.

You can specify any number of closer scripts using the different or the same Groovy shell. The scripts will all be invoked one after the other in the order as they were defined in the Jamal source.

Using Multiple Shell Objects

If you do not specify any shell object it will be created automatically using the name :groovyShell.

Groovy shell objects are stored along with the user defined macros. This has two consequences.

If there is a user defined name with the same name as the Groovy shell name, then the one defined later will overwrite the other.
The Groovy shell objects are available only within their scopes exactly the same way as user defined objects. They can also be exported.

Note that the default name starts with : therefore this is a global name, available in all scopes. This is a feature to ease the use of the Groovy shells when you have only one. It will be created and be available everywhere in the Jamal file even if the first use of Groovy was in a local scope.

The name of the shell can be overwritten defining the user defined macro groovyShell or using macro options.

It can be done using the usual built-in macro define, as in the example

{@groovy:eval z = 13}
{@define groovyShell=myLocalShell}
{@try! {@groovy:eval z}}

will result the output:

13

Error evaluating groovy script using eval

The reason for this is that the first evaluation is executed in a shell named :groovyShell. The second evaluation, however runs in a different shell, named myLocalShell.

Note

Note that the try macro is used as {@try…} and NOT {try…}. When we want the content of a built-in macro to be evaluated before the macro is invoked we have to use the character. In case of the try macro we want the content to be evaluated, but NOT BEFORE the try macro is invoked. If we use the macro in the form {#try…} then the content is evaluated before starting the macro try. If there is any error the macro try has no possibility to catch it, because it has not started yet. On the other hand using {@try…} will pass the content unevaluated, and the macro try will evaluate it and catch the errors.

This is not Groovy module specific, but it is a very common mistake.

There is a resource file named groovy.jim. You can import this file and then use the macros defined in it. The previous example will look the following:

{@import res:groovy.jim}
{@groovy:eval z = 13}
{shell=myLocalShell}
{@try! {@groovy:eval z}}

will result the output:

13

Error evaluating groovy script using eval

This is the same as the previous one, not surprisingly.

All Groovy macros are inner scope dependent, which means that you can define the macro groovyShell inside the Groovy macro call. In that case the definition, following the Jamal rules will be local to the Groovy macro.

For example

{@import res:groovy.jim}
{@groovy:eval z = 13}
{@try! {#groovy:eval {shell=myLocalShell}z}}
{@groovy:eval z = 13}

will result the output:

13
Error evaluating groovy script using eval
13

The second evaluation is performed in a different shell, but the definition of the shell name is local to the macro groovy:eval. (What is more, it is local to the try macro.)

The simplest way (starting with version 1.9.0) is to specify the shell name using an option. The macro groovyShell reads the option named :groovyShell which also has the alias shell. Note that when macros use options the name of the option can also be used to name macro that holds the value of the macro. Aliases are not checked as macro names.

The above example using options will look as the following:

{@groovy:eval z = 13}
{@try! {#groovy:eval (shell=myLocalShell) z}}
{@groovy:eval z = 13}

will result the same output:

13
Error evaluating groovy script using eval
13

Sample Application, Converting this `README.adoc`

In this chapter, I will tell the story and the technology used to maintain this documentation file. Several macros are used during the maintenance of the documentation to ensure that the documentation is correct and up-to-date. This particular document’s processing uses Groovy scripts, which are used instead of some built-in macros for demonstration purposes.

The documentation of Jamal is a series of Asciidoc files. The Asciidoc format was invented to be a documentation source format that is easy to read and edit. At the same time, Jamal can also convert it to many different output formats. Asciidoc, however, provides only limited possibility to eliminate redundancy and to ensure consistency. This is where Jamal comes into play.

Jamal’s documentation is maintained in xxx.adoc.jam files, and they are converted to xxx.adoc files. With this workflow, the Asciidoc files are not source files. They are intermediate files along the conversion path. Jamal define macros are used to eliminate text repetition, redundancy whenever it is possible. The Jamal snippet library macros are used to keep the sample codes included in the document up-to-date.

Note

When reading this part of the documentation, you are probably familiar with the basic functionalities of Jamal. If you need to refresh the memory, then read the documentation in the project’s root folder. Snippet macros are documented in the Snippet README.adoc file. It is unnecessary to know and understand how the snippet macros work to read this chapter, but it is a recommended read in general.

Technical documentation using Jamal and the snippet macros usually generates the documentation in multiple steps.

Run the tests, including the sample code, and capture the sample output in one or more output files.
Process the Jamal source of the documentation and include from the source code and the generated sample output files the samples.

For example, a Java application can support the documentation with unit test samples. Some of the unit tests serve the purpose of testing only, while others are there to document specific code parts. The output of the documentation purposed tests is captured into output files. The test file jamal-groovy/src/test/java/javax0/jamal/groovy/TestGroovyMacros.java contains

// snippet sample_snippet
@Test
@DisplayName("Test a simple groovy eval")
void testSimpleEval() throws Exception {
    TestThat.theInput("{@groovy:eval 6+3}").results("9");
}
// end snippet

To get this content into the document what we have to write is the following:

[source,java]
----
// snippet sample_snippet
{%@snip sample_snippet %}\
// end snippet
----

The output generated (none in this case) can also be included using the snip macro.

It is logical to run the tests and generate the test output in an initial step in the case of Java. However, when we test and document Jamal processing, it is a logical idea to use the Jamal environment, which is converting the documentation. The external approach with an initial step is also possible, but it is not needed.

The sample Jamal code can be included in the documentation as a code sample. Using Jamal macros, Jamal can also convert it to the corresponding output, which can also be included in the resulting document without saving it into an intermediate file.

To do that, the Jamal Snippet package unit test file jamal-snippet/src/test/java/javax0/jamal/documentation/TestConvertReadme.java uses a built-in macro, implemented in the file:

jamal-snippet/src/test/java/javax0/jamal/documentation/Output.java

This Java implemented macro is available on the classpath when the unit test runs.

Note

Executing the Jamal processing of a Java software package documentation via the unit tests has other advantages. The macros java:class and java:method can check that the class and method names referenced in the document are valid. Class and method names may change during refactoring. The documentation many times does not follow this change and becomes stale. When the classes and methods are referenced using these macros, they throw an exception if the class or method does not exist.

This class is very simple:

public class Output implements Macro {
    final Processor localProc = new javax0.jamal.engine.Processor("{", "}");

    @Override
    public String evaluate(Input in, Processor processor) throws BadSyntax {
        return localProc.process(new javax0.jamal.tools.Input(in.toString(), in.getPosition()));
    }
}

It creates a single Jamal processor instance and uses it to evaluate the input passed to it. This macro runs a Jamal processor separate from the Jamal processor that is converting the document. However, the two Jamal processors run in the same JVM, and one is invoking the other through this built-in macro.

To simplify the use, there is a readmemacros.jim macro import file that defines the user-defined macro sample and output. (A built-in macro can have the same name as a user-defined.) The macro sample results in its content in Asciidoc code sample format, adding [source]\n---- before and ---- after the sample code. At the same time, it also saves the sample code in a user-defined variable called lastCode. The macro output uses the lastCode and using the built-in output from the Output.java displays the calculated result as a code block.

It is very similar when we are using Groovy, but in this case, we do not need the built-in macro output. When Jamal converts this document, the readmemacros.jim` inside the jamal-groovy directory contains some Groovy scripts instead of the built-in macros.

The unit test code that invokes the Jamal processor to convert this document is the following:

final var processor = new Processor("{%", "%}");
final var in = FileTools.getInput(directory + "/" + fileName + "." + ext + ".jam", processor);
final var shell = Shell.getShell(processor, Shell.DEFAULT_GROOVY_SHELL_NAME);
shell.property("processor", new Processor("{", "}"));
processor.defineGlobal(shell);
final var result = processor.process(in);

It is almost a standard invocation of the Jamal processor. The only difference is that it creates a Groovy shell using the default shell name and injects a Jamal processor instance into the Groovy bindings with the name processor. When Jamal runs any Groovy code running in the same shell will be able to access the processor.

Using this possibility the user defined macros sample and output are simply the following:

sample

    {%@define sample(code)=[source]
    ----
    {%#trimLines
    {%@groovy:property lastCode=(string)code%}{%@groovy:shell
    lastCode.replaceAll('^\\n+','').replaceAll('\\n+$','')
    %}%}
    ----%}

This macro displays the sample as a code block in Asciidoc format. At the same time it also saves the sample text into a Groovy bindings property.

output

    {%@define output=[source]
    ----
    {%#trimLines
    {%@groovy:shell
    processor.process(new javax0.jamal.tools.Input(lastCode))%}%}
    ----
    %}

This macro uses the saved property lastCode to access to the text of the last sample. It converts the text to a Jamal Input objects and then invokes the processor. The result value of the macro is the output of the processor.

In this chapter we discussed how documentations should be "programs" to avoid redundacy in the source and to support consistency. After that we made a short detour discussing the Jamal snippets, which have a full documentation in the file Snippet README. We also discussed how the documentation conversion works with snippets and Jamal samples in the Snippet module. Finally, we had a look at how simpler it is using the Groovy integration.

Note	None of the sample codes in the source `README.adoc.jam` was manually copied.

This clearly demonstrates the power and flexibility of Jamal enhanced with the Groovy integration. If you like the idea, but Groovy is not your favourite scripting language have a look at the Ruby Integration documentation and give it a try.

Loading the macros

Starting with the version 2.0.0 the library is not configured to be on the class path of the command line version or the Asciidoctor preprocessor. The reason is security. The interpreter, just as well as the Ruby and ScriptBasic interpreters, can execute arbitrary code. If you want to use the ScriptBasic interpreter you have to

modify the property maven.load.include and maven.load.exclude in the file ~/.jamal/settings.properties to include the groovy module. For example:
```
maven.load.include=com.javax0.jamal:jamal-groovy:2.8.3-SNAPSHOT
```
add the line
```
{@maven:load com.javax0.jamal:jamal-groovy:2.8.3-SNAPSHOT}
```
to the Jamal file where you want to use the ScriptBasic interpreter.
To include the resource file groovy.jim you have to add the line
```
{@import maven:com.javax0.jamal:jamal-groovy:2.8.3-SNAPSHOT::groovy.jim}
```
instead importing it as a resource.

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

README.adoc

README.adoc

Jamal Groovy integration module

Macros implemented in the package

i. `groovy:eval`

ii. `groovy:import`

iii. `groovy:shell`

iv. `groovy:property`

v. `groovy:closer`

Using Multiple Shell Objects

Sample Application, Converting this `README.adoc`

Loading the macros

Files

README.adoc

Latest commit

History

README.adoc

File metadata and controls

Jamal Groovy integration module

Macros implemented in the package

i. groovy:eval

ii. groovy:import

iii. groovy:shell

iv. groovy:property

v. groovy:closer

Using Multiple Shell Objects

Sample Application, Converting this README.adoc

Loading the macros

i. `groovy:eval`

ii. `groovy:import`

iii. `groovy:shell`

iv. `groovy:property`

v. `groovy:closer`

Sample Application, Converting this `README.adoc`