# Apache ANT as JCL #

[Apache Ant](http://ant.apache.org/) is a Java library and command-line tool whose mission is to drive processes described in build files as targets and extension points dependent upon each other. The main known usage of [Apache Ant](http://ant.apache.org/) is the build of Java applications. [Apache Ant](http://ant.apache.org/) supplies a number of built-in tasks allowing to compile, assemble, test and run Java applications. [Apache Ant](http://ant.apache.org/) can also be used effectively to build non Java applications, for instance C or C++ applications. More generally, Ant can be used to pilot any type of process which can be described in terms of targets and tasks.

[Apache Ant](http://ant.apache.org/) is written in Java. Users of [Apache Ant](http://ant.apache.org/) can develop their own "antlibs" containing [Apache Ant](http://ant.apache.org/) tasks and types.

[Apache Ant](http://ant.apache.org/) is extremely flexible and does not impose coding conventions or directory layouts to the Java projects which adopt it as a build tool.
JEM uses [Apache Ant](http://ant.apache.org/) as main job control language, developing all management utilities on it (only for _Enterprise Edition_).

# Installation of JEM-ANT module #

[Apache Ant](http://ant.apache.org/) integration in JEM is provided out of the box inside of JEM distribution and installation file.

See [here](ConfiguringJEMCluster#jem-envxml) how [Apache Ant](http://ant.apache.org/) is configured.

## Apache Ant Factory, additional configuration ##

The Ant factory, before the job execution, goes through a pre-process phase to validate the content of the JCL. This phase, is customizable by the user adding a custom validator that uses XSLT.
To activate the validator add a property to the factory.
```xml
    <factories>
        <factory className="org.pepstock.jem.ant.AntFactory">
            <!-- if needed you can set property here and call them on JCL as ${property.name} -->
            <properties>
                <!-- <property name="property.name" value="property.value"/> -->
                <property name="jem.xslt.project.validator" value="d:\JEM\HelloWorldValidate.xsl"/>
            </properties>
        </factory>
    </factories>
```

This is an example of a XSLT validator document:
```xml
<xsl:stylesheet version="2.0"
    xmlns:xsl="http://www.w3.org/1999/XSL/Transform"
    xmlns:xs="http://www.w3.org/2001/XMLSchema"
    xmlns:fn="http://www.w3.org/2005/xpath-functions"
    xmlns:info="xalan://org.apache.xalan.lib.NodeInfo"
    xmlns:exception="org.pepstock.jem.ant.validator.transformer.ValidationExceptionThrower"
    extension-element-prefixes="exception">
    <xsl:param name="jem.classpath" />
    <xsl:template name="validate" match="/">
        <xsl:for-each select="/project/property[@name='jem.job.environment']">
            <xsl:choose>
                <xsl:when test="@value">
                    <xsl:if test="@value=''">
                        <xsl:value-of select="exception:throwValidateException('jem.job.environment is empty', info:lineNumber( ), info:columnNumber( ))" />
                    </xsl:if>
                </xsl:when>
                <xsl:otherwise>
                <xsl:value-of select="exception:throwValidateException('jem.job.environment not set', info:lineNumber( ), info:columnNumber( ))" />
                </xsl:otherwise>
            </xsl:choose>
        </xsl:for-each>
    </xsl:template>
</xsl:stylesheet>
```

To enable XSLT to raise exception, you must declare the following namespace and the extension prefixes:
```
xmlns:exception="org.pepstock.jem.ant.validator.transformer.ValidationExceptionThrower"
extension-element-prefixes="exception"
```

To enable XSLT to reporting the text coordinates, you must declare the following namespace:
```
xmlns:info="xalan://org.apache.xalan.lib.NodeInfo"
```

In the XSLT you may reference all JEM properties as XSL parameter. This may be useful to implement some logic.

Finally, to analyze your JCL, you may use XPath and, if there are some inconsistencies, rise an exception:
```
<xsl:value-of select="exception:throwValidateException('my custom description', info:lineNumber( ), info:columnNumber( ))" />
```


# Properties definition #

To use [Apache Ant](http://ant.apache.org/) inside JEM, ANT XML JCL file must contain the mandatory and optional properties that JEM needs, described in _JCL Reference_, as following:

  * **Job Name** is optional string property, called for ANT `jem.job.name` . If missing, name attribute of project element will be used. If boths are missing, an exception occurs.
  * **Environment** is optional string property, called for ANT `jem.job.environment`.If missing, the JEM node environment definition is used.
  * **Domain** is optional string property, called for ANT `jem.job.domain`. If missing, default value (`***`) will be used.
  * **Affinity** is optional string property, called for ANT `jem.job.affinity`. If missing, default value (`***`) will be used.
  * **User** is optional string property, called for ANT `jem.job.user`. If missing, default value (`null`) will be used. An exception occurs if the user, who submitted the job, is not authorized to change the user job execution.
  * **Locking Scope** is optional string property, called for ANT `jem.job.lockingScope`. If missing, default value (`job`) will be used. If the value is not equals to one the possible values (`job, step , task`), an exception occurs.
  * **Hold** is optional boolean property, called for ANT `jem.job.hold`. If missing, default value (`false`) will be used.
  * **Priority** is optional integer property, called for ANT `jem.job.priority`. If missing, default value (`10`) will be used (highest priority is 1).
  * **Memory** is optional integer property, called for ANT `jem.job.memory`. If missing, default value (`128`) will be used. Be aware the unit is MegaBytes.
  * **Classpath** is optional string property, called for ANT `jem.job.classPath`. Defining a classpath, it will be able to import ANT task definitions of 3rd parties. The value is a string and the files are separated by semicolons `;`. If the file doesn't represent an absolute path, JEM will use relative position from `jem.classpath` folder. You could use variables that JEM substitutes in string value. If missing, default value (`null`) will be used. The paths will be added at the end of default classpath, build by JEM.
  * **Prior Classpath** is optional string property, called for ANT `jem.job.priorClassPath`. Defining a classpath, it will be able to import ANT task definitions of 3rd parties. The value is a string and the files are separated by semicolons `;`. If the file doesn't represent an absolute path, JEM will use relative position from `jem.classpath` folder. You could use variables that JEM substitutes in string value. If missing, default value (`null`) will be used. The paths will be added at the beginning of default classpath, build by JEM.
  * **Emails Notification** is optional string property, called for ANT `jem.job.emailsNotification`. If missing, default value (`null`) will be used.
  * **Java** is optional string property, called for ANT `jem.job.java`. If missing, JEM will use the JRE used by JEM node.

Here is the sample about the ANT properties definition:

```xml
<property name="jem.job.name" value="JOB1"></property>
<property name="jem.job.environment" value="ENV1"></property>
<property name="jem.job.domain" value="domain"></property>
<property name="jem.job.affinity" value="classA"></property>
<property name="jem.job.priority" value="99"></property>
<property name="jem.job.user" value="newUSER"></property>
<property name="jem.job.lockingScope" value="task"></property>
<property name="jem.job.hold" value="true"></property>
<property name="jem.job.memory" value="1024"></property>
<property name="jem.job.classPath" value="/test/ant-taskdefs-lib.jar"></property>
<property name="jem.job.priorClassPath" value="/test/commons.jar"></property>
<property name="jem.job.emailsNotification" value="m1@pepstock.org;m2@pepstock.org"/>
<property name="jem.job.java" value="jdk7"/>
```

# Data Descriptions and Datasets in ANT #

Data descriptions and datasets are implemented by specific tags. Data descriptions could be defined as following:

```xml
<dataDescription name="name-of-data-description"
                 sysout="true/false"
                 disposition="NEW/MOD/OLD/SHR">
</dataDescription>
```

Data description needs a mandatory `name` attribute. This name is used inside the business logic to access to datasets by JNDI so this name must be unique in the task definition. The `sysout` attribute is optional one that if there is means that the data description is a sysout.
The `disposition` attribute is optional and must be a valid value, one of the following string enumeration: NEW, MOD, OLD or SHR.

Datasets are children of a data description definition and identifies a file or data necessary for business logic. Datasets could be defined as following:

```xml
<dataSet name="file-name" resource="datasource-name">
.......
</dataSet>
```

Datasets could have the `name` attribute which could represent:
  * file name (absolute or relative path), composed by ANT properties if necessary. If it's a relative path, JEM adds the content of `jem.data` variable, used to identify the global file system with all data.
  * GDG file name, following the same rules of a normal file name (see previous item).
  * Reference to another data set in another data description, not in the same task as well.
  * Temporary prefix file name

If the `name` attribute is missing, a temporary file is created, putting the content `dataset` element.

Datasets could have the `resource` attribute which could represent a relation with a data source, previously defined. It works only the FTP resources and the name of dataset represents the file to manage by FTP.

# Data sources in ANT #

Data sources are implemented by specific tags. They could be defined as following:

```xml
<!-- DB datasource reference -->
<dataSource name="my-db" resource="resource-db"></dataSource>
<!-- FTP datasource reference -->
<dataSource name="my-ftp" resource="resource-ftp">
    <property name="binary">true</property>
</dataSource>
```

Data source needs a mandatory `name` attribute. This name is used inside the business logic to access to data source (for every kind of data source, as jdbc or ftp) by JNDI so this name must be unique in the task definition. Data source needs a mandatory `resource` attribute. This name is used to locate the common-resource, defined previously in JEM. If resource is not defined, an exception occurs.

Data source element can contain one or more property element to override at run-time the value of some properties defined to create a data source. The property element needs a mandatory `name` attribute. An exception could occur if you try to override a property created in JEM with `override="false"` typically used for password, user-id or URL.

# Locks in ANT #

Locks are implemented by specific tags. They could be defined as following:

```xml
<lock name="lock-name"/>
```

Lock needs a mandatory `name` attribute. This name is used to create a lock inside the JEM environment in exclusive. This name must be unique in the task definition.

# JEM ANT extensions #

JEM extends ANT providing some custom tasks that you could use to benefit of special JEM features, as data descriptions, datasets or locks. It extends **java** and **exec**  tasks.

**Pay attention** that JEM ANT tasks MUST NOT be defined inside of any TaskContainer (as Parallel or Sequential tasks).

To defined all JEM ANT tasks in your ANT file, you can insert the taskdef related to the properties file with all tasks, as following:

```xml
 <taskdef resource="org/pepstock/jem/ant/tasks/JemAntTasks.properties"/>
```

With this taskdef, you define the JEM tasks with the following names:

```properties
step-java=org.pepstock.jem.ant.tasks.StepJava
step-exec=org.pepstock.jem.ant.tasks.StepExec
wrapper=org.pepstock.jem.ant.tasks.WrapperTask
procedure=org.pepstock.jem.ant.tasks.Procedure
procdef=org.pepstock.jem.ant.tasks.ProcedureDefinition
abend=org.pepstock.jem.ant.tasks.utilities.AbendTask
copy=org.pepstock.jem.ant.tasks.utilities.CopyTask
sort=org.pepstock.jem.ant.tasks.utilities.SortTask
null=org.pepstock.jem.ant.tasks.utilities.NullTask
wait=org.pepstock.jem.ant.tasks.utilities.WaitTask
bash=org.pepstock.jem.ant.tasks.utilities.scripts.BashScriptTask
win-cmd=org.pepstock.jem.ant.tasks.utilities.scripts.WindowsScriptTask
perl=org.pepstock.jem.ant.tasks.utilities.scripts.PerlScriptTask
python=org.pepstock.jem.ant.tasks.utilities.scripts.PythonScriptTask
ps=org.pepstock.jem.ant.tasks.utilities.scripts.PowerShellScriptTask
resources=org.pepstock.jem.ant.tasks.utilities.CommonResourcesTask
gdg=org.pepstock.jem.ant.tasks.utilities.GDGTask
nodes=org.pepstock.jem.ant.tasks.utilities.NodesTask
roles=org.pepstock.jem.ant.tasks.utilities.RolesTask
stats=org.pepstock.jem.ant.tasks.utilities.StatsCollectTask
archive=org.pepstock.jem.ant.tasks.utilities.JobOutputArchiveTask
```

## STEP-JAVA task ##

`<step-java>` is a custom Ant task (extends [Java Task](http://ant.apache.org/manual/Tasks/java.html)) adding a set of data descriptions, data sources and locks functions. Using this way, the programmer is able to develop own job batch programs, avoiding strong references with physical files but logical ones, by JNDI context.

```xml
<taskdef name="step-java" classname="org.pepstock.jem.ant.tasks.StepJava"/>
... ...
<step-java classname="org.pepstock.test.Main">
... ...
</step-java>
```

Inside of `<step-java>` task you could add all data descriptions, data sources or locks necessary to complete the business logic.

Here is a JCL sample:

```xml
<taskdef name="step-java" classname="org.pepstock.jem.ant.tasks.StepJava"/>

<step-java classname="org.pepstock.test.Main">
<!-- Single file data description (a normal file, cnf.xml), in read mode -->
    <dataDescription name="my-single" disposition="SHR">
        <dataSet name="my-file.dat"></dataSet>
    </dataDescription>
</step-java>
```

and the java main program which can read and display the file content:

```java
Hashtable<String, String> env = new Hashtable<String, String>();
env.put(Context.INITIAL_CONTEXT_FACTORY, "org.pepstock.jem.node.tasks.jndi.JemContextFactory");

try {
    InitialContext context = new InitialContext(env);
    /** single file input stream **/
    Object object = (Object) context.lookup("my-single");
    FileInputStream inputStream = (FileInputStream) object;
    
    /** reads input stream **/
    Scanner sc = new Scanner(inputStream);
    sc.useDelimiter("\n");
    while (sc.hasNext()) {
        String record = sc.next().toString();
        System.out.println(record);
    }
    sc.close();

} catch (Exception ex){
    ex.printStackTrace();
}
```

Here is the complete sample with many possible implementation (JCL and java main program):

**JCL**
```xml
<step-java classname="org.pepstock.test.Main">
    <classpath>
        <pathelement path="${java.class.path}"></pathelement>
    </classpath>
    
    <!-- DB datasource reference -->
    <dataSource name="my-db" resource="resource-db"></dataSource>
    
    <!-- FTP datasource reference, overriding binary attribute -->
    <dataSource name="my-ftp" resource="resource-ftp">
        <property name="binary">true</property>
    </dataSource>

    <!-- Sysout data description -->
    <dataDescription name="my-sysout" sysout="true" disposition="NEW"></dataDescription>

    <!-- Single file data description (a normal file, cnf.xml), in read mode -->
    <dataDescription name="my-single" disposition="SHR">
        <dataSet name="cnf.xml"></dataSet>
    </dataDescription>

    <!-- Single file data description using a FTP datasource, in read mode -->
    <dataDescription name="my-ftp-data" disposition="SHR">
        <dataSet name="remote-file.data" resource="my-ftp"></dataSet>
    </dataDescription>

    <!-- Multi files data description (a GDG file and normal file), in read mode -->
    <dataDescription name="my-multi" disposition="SHR">
        <dataSet name="my.data.gdg(0)"></dataSet>
        <dataSet name="cnf.xml"></dataSet>
    </dataDescription>

    <!-- Single file data description (a inline file), in read mode -->
    <dataDescription name="my-inline">
        <dataSet>
            Here I am
            This is a DD
        </dataSet>
    </dataDescription>

    <!-- Single file data description (a temporary file), in write mode -->
    <dataDescription name="my-temp" disposition="NEW">
        <dataSet name="@@mytemp"></dataSet>
    </dataDescription>

    <!-- Single file data description (a reference file), in read mode -->
    <dataDescription name="my-reference" disposition="SHR">
        <dataSet name="*.main.step-java.single"></dataSet>
    </dataDescription>
</step-java>
```

**MAIN program**
```java
Hashtable<String, String> env = new Hashtable<String, String>();
env.put(Context.INITIAL_CONTEXT_FACTORY, "org.pepstock.jem.node.tasks.jndi.JemContextFactory");

try {
    InitialContext context = new InitialContext(env);
    /**
    * database reference
    */
    Object object1 = (Object) context.lookup("my-db");
    DataSource datasource = (DataSource) object1;
    /**
    * syslog output stream
    */
    Object object2 = (Object) context.lookup("my-sysout");
    FileOutputStream fos = (FileOutputStream) object2;
    /**
    * single file input stream
    */
    Object object3 = (Object) context.lookup("my-single");
    FileInputStream fis = (FileInputStream) object3;
    /**
    * FTP input stream
    */
    Object object4 = (Object) context.lookup("my-ftp-data");
    InputStream ftpis = (InputStream) object4;
    /**
    * sequence input stream (multi input stream)
    */
    Object object5 = (Object) context.lookup("my-multi");
    SequenceInputStream sis = (SequenceInputStream) object5;
    /**
    * Inline input stream
    */
    Object object6 = (Object) context.lookup("my-inline");
    FileInputStream fis1 = (FileInputStream) object6;
    /**
    * Referback file input stream
    */
    Object object7 = (Object) context.lookup("my-reference");
    FileInputStream fis2 = (FileInputStream) object7;
    /**
    * Temporary file output stream
    */
    Object object8 = (Object) context.lookup("my-temp");
    FileOutputStream fis3 = (FileOutputStream) object8;

} catch (NamingException e) {
    e.printStackTrace();
}
```

As sample explains, to use the JNDI context, you should set as context factory a JEM context implementation, by `org.pepstock.jem.node.tasks.jndi.JemContextFactory` class.

## STEP-EXEC task ##

`<step-exec>` is a custom Ant task (extends [Exec task](http://ant.apache.org/manual/Tasks/exec.html)) adding a set of data descriptions and locks functions (not data sources, only for java). Using this way, the programmer is able to develop own job batch programs, avoiding strong references with physical files but logical ones, by _environment variables_. To reduce the risk to use variables already used on system, all variables names have **`DD_`** prefix.

`<step-exec>` is able to execute every kind of executable program, without taking care about programming language (for instance COBOL, Perl, C or C++, Python, Bash scripts).

Here is a simple sample how to execute a COBOL program which can read a data description copying records to another data description:

**JCL**
```xml
<taskdef name="step-exec" classname="org.pepstock.jem.ant.tasks.StepExec"></taskdef>

<step-exec executable="cmd" dir="CobolDirectory">
    <arg value="/c"></arg>
    <arg value="myCobol.exe"></arg>

    <dataDescription name="FILEOUT" disposition="NEW">
        <dataSet name="gdg.jemtest(1)"></dataSet>
    </dataDescription>

    <dataDescription name="FILEIN" disposition="SHR">
        <dataSet name="gdg.jemtest(0)"></dataSet>
    </dataDescription>
</step-exec>
```

**COBOL main program**
```cobol
IDENTIFICATION DIVISION.
PROGRAM-ID. MYCOBOL.
AUTHOR.
ENVIRONMENT DIVISION.
CONFIGURATION SECTION.
SPECIAL-NAMES. C01 IS CANALE1
    DECIMAL-POINT IS COMMA.
SOURCE-COMPUTER. IBM.
INPUT-OUTPUT SECTION.
FILE-CONTROL.
    SELECT FILEIN ASSIGN DD_FILEIN.
    SELECT FILEOUT ASSIGN DD_FILEOUT.
DATA DIVISION.
FILE SECTION.
FD FILEIN
    LABEL RECORD STANDARD
    RECORDING MODE IS F
    BLOCK 0.
01 REC-FILEIN PIC X(100).
FD FILEOUT
    LABEL RECORD STANDARD
    RECORDING MODE IS F
    BLOCK 0.
01 REC-FILEOUT PIC X(100).
01 SW-EOF PIC X(01) VALUE SPACE.
  88 EOF VALUE SPACE.
  88 NOT-EOF VALUE 'X'.
PROCEDURE DIVISION.
    OPEN INPUT FILEIN
        OUTPUT FILEOUT.
    READ FILEIN
        AT END
            DISPLAY 'EMPTY FILE'
            SET EOF TO TRUE
        NOT AT END
            SET NOT-EOF TO TRUE
    END-READ.
    PERFORM UNTIL EOF
        WRITE REC-FILEOUT FROM REC-FILEIN
        READ FILEIN
            AT END
                SET EOF TO TRUE
            NOT AT END
                SET NOT-EOF TO TRUE
        END-READ
    END-PERFORM.
    FINE.
    CLOSE FILEIN.
    CLOSE FILEOUT.
STOP RUN.
```

## WRAPPER task ##

`<wrapper>` is a custom Ant task adding a set of data descriptions functions.
Using this way, the programmer is able to use all already prepared ANT tasks in own job batch programs, avoiding strong references with physical files
but logical ones, by _local ANT properties_. To reduce the risk to use properties already used in the ANT project, all variables names have **`DD_`** prefix.

`<wrapper>` is able to execute a nested ANT task, passing and substituting properties with values of data descritpions.

Here is a simple sample how to execute ANT task COPY to copy a file into antoher:

**JCL**
```xml
<taskdef name="wrapper" classname="org.pepstock.jem.ant.tasks.WrapperTask" />

<target name="copyFile">
   <wrapper>
      <dataDescription name="OUTPUT" disposition="NEW">
         <dataSet name="gdg/jemtest(+1)"/>
      </dataDescription>

      <dataDescription name="INPUT" disposition="SHR">
         <dataSet name="gdg2/jemtest"/>
      </dataDescription>

      <copy file="${DD_INPUT}" todir="${DD_OUTPUT}"/>

   </wrapper>
</target>
```

## Utilities ##

With JEM, you could find some common tasks which are helpful for your JCLs. [Apache Ant](http://ant.apache.org/) has got a lot of tasks already prepared and ready to use for many purposes but all of them doesn't have any integration with data descriptions, data sources and locks.

For these reasons, JEM provides:

  * **`NullTask`**  : do nothing! Just exiting in RC=0!
  * **`WaitTask`** : waits for seconds (passed as argument) and ends in RC=0
  * **`CopyTask`** : copies one or more files into another one
  * **`SortTask`** : sorts a file into another one
  * **`ShellScriptTask`** : executes the script commands defined in task. JEM provides some OOTB language based task, extending the generic `ShellScriptTask`, as:
    * **`BashScriptTask`** : executes BASH script commands
    * **`WindowsScriptTask`** : executes WINDOWS script code
    * **`PerlScriptTask`** : executes Perl code
    * **`PythonScriptTask`** : executes Python code
  * **`Procedure`**  and **`ProcedureDefinition`** : definitions of a task which contains other tasks. It can be imported and override from task reference in the target

### NULL task ###

_`NullTask`_ is a ANT task that doesn't do anything, without any statement in its main method. It could be useful because it performs all locks for datasets defined in the task, as `<step-java>` does.

Here is a sample:

```xml
<!--
ANT task definition
-->
<taskdef name="iefbr14" classname="org.pepstock.jem.ant.tasks.utilities.NullTask"/>
<!--
NULL: does nothing
-->
<target name="null">
   <iefbr14/>
</target>
```

### WAIT task ###

_`WaitTask`_ is a ANT task that wait for some seconds before to end in return code 0. The amount of seconds to wait must be passed as argument to main program. If the parameter is missing, `WaitTask` waits for forever and just canceling it could end. It could be useful because it performs all locks for datasets defined in the task, as step-java does.

Here is a sample:

```xml
<!--
ANT task definition
-->
<taskdef name="wait" classname="org.pepstock.jem.ant.tasks.utilities.WaitTask" />
<!--
WAIT: waits for 60 seconds before ending
-->
<target name="wait">
   <wait>
      <arg value="600" />
   </wait>
</target>
```

### COPY task ###

_`CopyTask`_ is a ANT task that copies one or more datasets into another one. The input datasets must be defined in a data description called **INPUT** and the output dataset must be defined in a data description called **OUTPUT**.

Here is a sample:

```xml
<!--
ANT task definition
-->
<taskdef name="icegener" classname="org.pepstock.jem.ant.tasks.utilities.CopyTask"/>
<!--
ICEGENER: copy a GDG generation 0 in a new one
-->
<target name="icegener">
   <icegener>
      <dataDescription name="OUTPUT" disposition="NEW">
         <dataSet name="gdg/jemtest(1)"></dataSet>
      </dataDescription>
      
      <dataDescription name="INPUT" disposition="SHR">
         <dataSet name="gdg/jemtest(0)"></dataSet>
      </dataDescription>
   </icegener>
</target>
```

### SORT task ###

_`SortTask`_ is a ANT task that sorts the content of one or more datasets coping into another one. The input datasets must be defined in a data description called **INPUT** and the output dataset must be defined in a data description called **OUTPUT**.

The utility can accept `class` parameter to indicate a `Comparator<String>` implementation to use to sort the data. If `class` misses, it uses the default comparator which can parse a list of order-by statements, found inside a data description called **COMMAND** (be careful because is case sensitive). The format is:

```xml
( start [, length] [, ASC|DESC] ),...
```

It's mandatory to indicate the start point in the string. Length is optional (default is **0**) and, if missing or 0 or less than 0, the length is all string starting from start character. The type of sort is optional as well (default is **ASC**) and if indicated must both `ASC` (for ascending order) or `DESC` (for descending one).

You can have many order-by statements (comma separated), used in case of the previous statements comapres equals values.

Here is a sample:

```
(10,4,ASC),(5,6),(18,DESC) 
```

It sorts:

  1. first statement `(10,4,ASC)` sorts the records starting from character at index 10 for 4 characters, ascending;
  1. if the previous comparing returns equals values, uses the statement `(5,6)` sorts the records starting from character at index 5 for 6 characters, ascending;
  1. if the previous comparing returns equals values, uses the statement `(18,DESC)` sorts the records starting from character at index 18 till end of record, descending;

If **COMMAND** misses or there is any error parsing order-by statements, it uses `String.compareTo` method to sort the data.

Here is a sample:

```xml
<!--
ANT task definition
-->
<taskdef name="sort" classname="org.pepstock.jem.ant.tasks.utilities.SortTask"/>
<!--
SORT: sort a GDG generation 0 into a new generation
-->
<target name="sort">
   <sort>
     <arg value="-class"/>
     <arg value="org.pepstock.jem.test.MyComparator"/>

     <classpath>
        <pathelement path="${java.class.path}" />
        <pathelement path="${jem.classpath}/test/jem-test.jar" />
     </classpath>


     <dataDescription name="COMMAND" disposition="SHR">
         <dataSet>
           (10,4,ASC),(5,6),(18,DESC)
         </dataSet>
     </dataDescription>

      <dataDescription name="OUTPUT" disposition="NEW">
         <dataSet name="${jem.data}/gdg1/jemtest(1)"></dataSet>
      </dataDescription>

      <dataDescription name="INPUT" disposition="SHR">
         <dataSet name="${jem.data}/gdg1/jemtest(0)"></dataSet>
      </dataDescription>
   </sort>
</target>
```

### Shell Script task ###

_`ShellScript`_ is a ANT task that is able to execute code in script language, put in task definition.

Here is a sample:

```xml
<!--
ANT task definition
-->
<taskdef name="shellscript" classname="org.pepstock.jem.ant.tasks.utilities.ShellScriptTask"/>
<!--
SHELL-SCRIPT: display the jem.data directory content
-->
<target name="step">
   <shellscript shell="cmd.exe" suffix=".bat">
      <arg value="/c"></arg>
      <arg value="call"></arg>
      dir ${jem.data}
   </shellscript>
</target>
```

### Windows task ###
_`Windows`_ is a ANT task that is able to execute Windows scripting code, put in task definition.

Here is a sample:

```xml
<!--
ANT task definition
-->
<taskdef name="windows" classname="org.pepstock.jem.ant.tasks.utilities.scripts.WindowsScriptTask"/>
<!--
COPY a GDG file, defined in data description,
from INPUT to OUTPUT by "copy' command
-->
<target name="step1">
   <windows>
      <dataDescription name="OUTPUT" disposition="NEW">
         <dataSet name="gdg/jemtest(1)"></dataSet>
      </dataDescription>
      <dataDescription name="INPUT" disposition="SHR">
         <dataSet name="gdg/jemtest(0)"></dataSet>
      </dataDescription>
      copy %DD_INPUT% %DD_OUTPUT%
   </windows>
</target>
<!--
SORT a GDG file, defined in data description by a reference to previous step,
from INPUT to OUTPUT by "sort' command
-->
<target name="step2" depends="step1">
   <windows>
      <dataDescription name="OUTPUT" disposition="NEW" sysout="true">
      </dataDescription>
      <dataDescription name="INPUT" disposition="SHR">
         <dataSet name="*.step1.windows.OUTPUT"></dataSet>
      </dataDescription>
      sort %DD_INPUT% /O %DD_OUTPUT% /+10
   </windows>
</target>
```

### Procedure and Procedure Definition task ###

_`Procedure Definition`_ is a ANT task that is able to define a task inside of ANT project (by ANT import task as well). This task is not executed but only defined and it can be related to a task in a target to be executed. It can have a task definition so if more than one task is necessary, is enough to
define all tasks inside of [ANT Sequential](http://ant.apache.org/manual/Tasks/sequential.html) task.
`Procedure Definition` must have a `name` attribute, unique inside of ANT project. This is used by `Procedure` to create the link with definitions.

_`Procedure`_ is a ANT task that is related to a `Procedure Definition` (by `name`) and is able to execute all tasks defined in. It can also override data description definition, where the tasks are data description container (`StepJava` or `StepExec` and their extensions).

![http://www.pepstock.org/resources/warning.png](http://www.pepstock.org/resources/warning.png) Pay attention that the procedures usage needs job _lockingScope_ set to **job**.

Here is a sample of Procedure Definition:

```xml
<!--
ANT task definition
-->
<taskdef name="procdef" classname="org.pepstock.jem.ant.tasks.ProcedureDefinition"></taskdef>
<!--
WAIT is just a wait task, locking a resource
-->
<procdef name="wait">
   <wait>
      <arg value="15"></arg>
      <lock name="block"></lock>
   </wait>
</procdef>
<!--
ECHOANDCOPY is procedure with some echos tasks and 2 copies by
Windows task
-->
<procdef name="echoandcopy">
   <sequential>
      <echo> Embed another:${user.name} </echo>
      <echo> Embed another:${jem.classpath} </echo>
      <windows name="first">
         <dataDescription name="OUTPUT" disposition="NEW">
            <dataSet name="gdg/jemtest(1)"></dataSet>
         </dataDescription>
         <dataDescription name="INPUT" disposition="SHR">
            <dataSet name="${jem.data}/gdg/jemtest(0)"></dataSet>
         </dataDescription>
         copy %DD_INPUT% %DD_OUTPUT%
      </windows>

      <windows id="second">
         <dataDescription name="OUTPUT" disposition="NEW">
            <dataSet name="gdg/jemtest(1)"></dataSet>
         </dataDescription>
         <dataDescription name="INPUT" disposition="SHR">
            <dataSet name="${jem.data}/gdg/jemtest(0)"></dataSet>
         </dataDescription>
         copy %DD_INPUT% %DD_OUTPUT%
      </windows>
   </sequential>
</procdef>
```

Here is a sample of Procedure, which is related to previous sample:

```xml
<!--
ANT task definition
-->
<taskdef name="procedure" classname="org.pepstock.jem.ant.tasks.Procedure"></taskdef>
<!--
CALL to "wait" procedure definition
-->
<target name="step1">
   <procedure name="wait"></procedure>
</target>
<!--
CALL to "echoandcopy" procedure definition, overriding
some data description definition
-->
<target name="step2" depends="step1">
   <procedure name="echoandcopy">
      <windows name="first">
         <dataDescription name="INPUT" disposition="SHR">
            <dataSet name="gdg/jemtest(-1)"></dataSet>
         </dataDescription>
      </windows>
      <windows id="second">
         <dataDescription name="INPUT" disposition="SHR">
            <dataSet name="gdg/jemtest(-1)"></dataSet>
         </dataDescription>
      </windows>
   </procedure>
</target>
```