Raster Filename Parsing

Scientific data is commonly serialized and exchanged in common raster data formats such as GeoTIFF and ASC. While these file types are ubiquitous and relatively easy to explore, they lack the self-describing quality of other scientific data formats such as NetCDF and HDF. As such, meta data such as variable type, units, and dates are often added to the filename, following no particular convention, as a way to describe the contents of the file.

It is a challenge for a tool like Vortex to reasonably handle all of the possibilities when tasked with parsing unconventional file names. There are a few common data sources that use non-self-describing raster formats, for which specialized file name parsing logic has been added to Vortex. These sources include:

• PRISM (ASC and BIL files)
• ATLAS 14 (ASC files)
• SNODAS (DAT files)

If there are other common data sources that you believe would provide benefit to the hydrologic modeling community given specialized parsing logic, feel free to suggest them.

For importing uncommon or unconventional data sources, you may consider using the HEC-DSS C, Java, Jython, or Python APIs directly. Alternatively you can use Vortex, keeping a few things in mind:

1. Vortex will attempt to parse dates in the filename. The dates must use the ISO 8601 standard with minimal variation. If two compliant dates are provided, the first will be applied to the DSS pathname D-Part and the second will be applied to the DSS pathname E-Part. If one compliant date is provided the date will apply to both the DSS pathname D-Part and E-Part.
   
   For example, if the filename is: precip_2021-12-07T1200_to_2021-12-07T1300.tif
   the DSS D-part will be 07DEC2021:1200,
   and the DSS E-Part will be 07DEC2021:1300.
   
   For example, if the filename is: 2021-12-07T1200_TEMPERATURE.tif
   the DSS D-part will be 07DEC2021:1200,
   and the DSS E-Part will be 07DEC2021:1200.
   
   The only variations from the ISO 8601 standard is that the separator between year and month, and month and day, can optionally be an underscore rather than a hyphen and the separator between date and time can optionally be an underscore or hyphen, rather than "T".

2. Vortex will attempt to parse the filename for variable type. You can also specify the C-Part as a DSS write option using the partC argument. For more, see Batch Import Options.

3. Vortex does not parse the filename for units. You can specify the units as a DSS write option using the units argument. For more, see Batch Import Options.