General interface design

Commands

The only executable of Azulero is azul. It supports a variety of so-called commands, such as retrieve to download datafiles or process to render color images. All commands follow the same pattern:

$ azul [global_options] <command> <inputs> [options]

with:

[global_options]: Optional global arguments (e.g. --log DEBUG).
<command>: The name of the command (e.g. retrieve).
<input>: The space-separated list of inputs (e.g. UGC11116 PGC61356). If the list is empty, then stdin is read (see Standard streams section below and Pipelines page).
[options]: Optional command arguments (e.g. -r 1m).

Global options, common to all commands, are passed between azul and the command name, and command options are passed after the command name, before or after inputs. Option -o <output> exists for all commands to specify the output paths.

Here is an example command line with global and command options, a list of inputs and an output specification:

$ azul --log DEBUG retrieve -r 1m -n 1 UGC11116 PGC61356 -o {target}

Workspace, input and output paths

Azulero defines three types of directories:

Workspace: A parent directory under which all inputs and outputs are located by default.
Tiledirs: Directories inside the workspace which contain data for a single tile. Tiledirs are named after the tile index.
Workdirs: Directories inside the workspace, generally inside tiledirs, which contain data for a single target.

By default, all paths passed as input of a command are assumed to be relative to the workspace, unless they are absolute paths. Output paths are often specified as templates with curly brace-enclosed placeholders. The placeholders are replaced with values depending on the command parameters. For example, most output path start with placeholder {workspace}, which will be rendered as the actual workspace path.

Sky and image coordinates

Most command take coordinates as parameters:

Sky coordinates: They are specified as RA/Dec in the International Celestial Reference Frame (ICRF).
Image coordinates: They are given from left to right and from bottom to top. Integral values are the pixel centers.

Global options

Global options are:

--log: The log level, either DEBUG, INFO (the default), WARNING or ERROR.
--workspace: The path to the workspace. Defaults to the current directory.
--channels: The way the four Euclid bands are uniquely identified in the paths to MER data, e.g. NIR-J. Unless you generate MER-like data yourself, you should not use this option.
--input: The glob pattern used to find Euclid bands in a workdir. It must contain placeholder {channel}, which is rendered accordingly to the values of option --channels. For example, *_{channel}_*.fits would accept files with .fits extension which contain the band name (e.g. NIR-J) separated from other chunks by underscores.

Standard streams

Azulero commands read and write different types of messages from and to the different standard streams:

stdin: When no inputs are provided to the command line, they are read from stdin. If stdin is also empty, then the command stops early.
stdout: The results of the command (e.g. path to workdir or rendered image) are written to stdout for further use by commands down a pipeline.
stderr: Azulero commands log to stderr.

For concrete usage examples, see Pipelines.

Named options

For pipelining and batch processing, Azulero offers an environment-level mechanism for setting named options like global option --log or azul retrieve option --from. Each named option can be read from an environment variable named as follows:

AZUL<COMMAND>_<OPTION>

with:

<COMMAND>: The uppercase command name, if any, such as RETRIEVE or PROCESS, or nothing for global options.
<OPTION>: The uppercase long-from option name, such as FROM for --from or RADIUS for --radius but not R for -r.

For example:

$ export AZUL_LOG=DEBUG
$ export AZULRETRIEVE_FROM=pdr
$ export AZULRETRIEVE_RADIUS=1m

sets the log level to DEBUG for all commands, and sets the data provider to pdr and crop radius to 1 arcmin for azul retrieve. These parameters are overloaded by command line arguments.

Within the above environment, the following lines are equivalent:

$ azul retrieve NGC6505 UGC11116 | azul process
$ azul --log DEBUG retrieve NGC6505 UGC11116 -r 1m --from pdr | azul --log DEBUG process

The variables can be set in a .env file in the directory from which the command is triggered.

Precedence

The decreasing precedence order is:

Command line arguments
Environment variables
.env file
Default values

$ azul retrieve PGC61356
  # Logs at INFO level (default)

$ echo "AZUL_LOG=DEBUG" > .env

$ azul retrieve PGC61356
  # Logs at DEBUG level (.env overwrites default)

$ export AZUL_LOG=WARNING

$ azul retrieve PGC61356
  # Logs at WARNING level (variable overwrites default and .env)

$ AZUL_LOG=ERROR azul retrieve PGC61356
  # Logs at ERROR level (local variable overwrites exported variable)

$ AZUL_LOG=ERROR azul --log CRITICAL retrieve PGC61356
  # Logs at CRITICAL level (argument overwrites everything)

Command line help

We tried our best to write a complete-enough documentation. However, we may have omitted a few details, such as the default option values. For an exhaustive description of global and command options, azul and each of its commands feature an option -h, or --help:

$ azul -h
$ azul process -h

In these pages, only the short forms of the options are given when they exist. Check the help messages in order to find the long-from options – typically, to know the environment variable names.

Finding errors?

If you find discrepancies between the documentation and help messages, or any other kind of mistakes down here, please open issues or contact us. This really helps!