3.3. DeDop Shell¶
3.3.1. Overview¶
DeDop Shell comprises a single command-line executable, which is called dedop and is available after installing
the DeDop Shell on your computer. With this tool, users can have a complete interaction with the input data, processor,
configuration, and finally the output products. If you have not done so, see section Setup to install and
initialize DeDop Shell.
In this section, you will find a complete manual on how to use DeDop Shell. This can be sub-divided into the following parts:
3.3.2. Workspace Management¶
3.3.2.1. Add a new workspace¶
When DeDop Shell runs for the first time, there is no workspace available. An automatic creation of default workspace can
be triggered by running dedop input add some/path/to/your/L1A.nc or dedop run command as described
here.
To add a new workspace is by running the following command:
$ dedop w add workspace_name
Upon successful operation, the following responses shall be returned:
$ created workspace "workspace_name"
$ current workspace is "workspace_name"
This means that the new workspace has been successfully created and made the current workspace, which means that, unless explicitly changed, whatever operations being performed after this will happen inside this workspace.
3.3.2.2. Remove a workspace¶
To remove a workspace, run one of the following commands:
$ dedop w remove # CASE 1
$ dedop w remove workspace_name # CASE 2
In both cases, a confirmation prompt will appear to really ensure that this action is intentional. To disable this prompt,
use the optional argument --yes (eg. dedop w remove --yes).
In CASE 1, the tool will remove the current workspace. It will then change the current workspace to the next workspace. When no more workspaces exist, it will just be assumed that there is no current workspace.
In CASE 2, the tool will remove the specified workspace. If the specified name is also the current workspace name, the above scenario will be in effect.
remove has an alias rm.
3.3.2.3. Copy a workspace¶
To copy a workspace, run one of the following commands:
$ dedop w copy # CASE 1
$ dedop w copy workspace_to_be_copied # CASE 2
$ dedop w copy workspace_to_be_copied new_workspace # CASE 3
In CASE 1, without specifying which workspace to be copied, it will default to the current workspace. The new workspace
name is also not specified, so in this case a new name with the format of current_workspace_name + _ + unique_number
is created. The unique number will be incremented until a unique combination is constructed. For example, for a current
workspace workspace1, it will create a new workspace named workspace1_1.
In CASE 2, the workspace to be copied is specified, but not the name of the new workspace. It will then follow the
same rule as in CASE 1, by creating a new workspace with a name that is of format workspace_to_be_copied + _ + unique_number.
In CASE 3, both the workspace to be copied and the name of the new workspace are specified. It is very clear that it will copy the specified workspace to a new workspace with the specified name.
One thing that is worth mentioning is that copying a workspace does not automatically change the current workspace.
copy has an alias cp.
3.3.2.4. Rename a workspace¶
To rename a workspace, run one of the following commands:
$ dedop w rename new_workspace_name # CASE 1
$ dedop w rename workspace_to_be_renamed new_workspace_name # CASE 2
In CASE 1, the current workspace will be renamed to new_workspace_name. The current workspace will be automatically
changed after the renaming.
In CASE 2, the specified workspace will be renamed to new_workspace_name. There is NO change on the current workspace.
rename has an alias rn.
3.3.2.5. Get current workspace¶
It is sometimes useful to know in which workspace we are working on at the moment. To get that information, run the following:
$ dedop w current
If there is a current workspace, the name of the current workspace will be returned. Otherwise, no current workspace
will be returned.
current has an alias cur.
3.3.2.6. List workspaces¶
To list available workspaces, run the following command:
dedop w list
list has an alias ls.
3.3.3. Source File Management¶
After creating a workspace, the next step is to manage the L1A source files.
3.3.3.1. Add new L1A source file¶
To add a new L1A file(s) into this workspace, run the following command:
$ dedop i add /path/to/file1 /path/to/file2 /path/to/file3
What this command does is copying those files into the current workspace directory. When successful, those files will be
located inside inputs directory under the current workspace directory. Every workspace will have its own inputs
directory, which in the end can be used as a source file for running multiple processes with different configurations.
3.3.3.2. Remove L1A source file¶
To remove the previously-added L1A file(s), run one of the following commands:
$ dedop i remove # CASE 1
$ dedop i remove file_name1 file_name2 # CASE 2
$ dedop i remove -w workspace_name # CASE 3
$ dedop i remove -w workspace_name file_name1 file_name2 # CASE 4
In all cases, a confirmation prompt will appear to really ensure that this action is intentional. To disable this prompt,
use the optional argument --quiet (eg. dedop i remove --quiet).
In CASE 1, all previously-added source files in the current workspace will be removed.
In CASE 2, the specified files in the current workspace will be removed.
In CASE 3, all previously-added source in the specified workspace will be removed.
In CASE 4, the specified files in the specified workspace will be removed.
remove has an alias rm.
3.3.3.3. List all L1A source files¶
To list all source files that have been added, run one of the following commands:
$ dedop i list # CASE 1
$ dedop i list -w other_workspace # CASE 2
$ dedop i list L1A* # CASE 3
In CASE 1, the tool will return a list of all source files in the current workspace.
In CASE 2, the tool will return a list of all source files in the specified workspace.
In CASE 3, the tool will return a list of all source files that match the given regex L1A*.
list has an alias ls.
3.3.4. Processor Configuration Management¶
The next step before running an actual process is to manage the configurations.
3.3.4.1. Add a new configuration¶
To add a new configuration, run one of the following commands:
$ dedop c add new_config_name # CASE 1
$ dedop c add -w other_workspace new_config_name # CASE 2
$ dedop c add --cryosat-adapted new_config_name # CASE 3
In all cases, a new folder named new_config_name is created under a workspace and it consists of three default configuration
files CHD.json, CNF.json, and CST.json. The generated configurations are by default for Sentinel-3 processing
unless when --cryosat-adapted is specified.
In CASE 1, a new configuration will be created under the current workspace directory.
In CASE 2, a new configuration will be created under the specified workspace directory.
In CASE 3, a new configuration suited for Adapted Cryosat-2 FBR data will be created under the current workspace directory.
3.3.4.2. Remove a configuration¶
To remove a configuration, run one of the following commands:
$ dedop c remove # CASE 1
$ dedop c remove config_name # CASE 2
$ dedop c remove -w other_workspace config_name # CASE 3
In all cases, a confirmation prompt will appear to really ensure that this action is intentional. To disable this prompt,
use the optional argument --yes (eg. dedop c remove --yes). Removing a configuration means deleting a configuration
folder including its contents (all the CHD, CNF, and CST files).
In CASE 1, the current configuration in the current workspace will be removed. It will then change the current configuration to the next configuration. When none left, it will go into a state where there are no current configurations.
In CASE 2, the specified configuration in the current workspace will be removed. There is no change of current configuration if it does not involve current configuration.
In CASE 3, the specified configuration inside a specified workspace will be removed.
remove has an alias rm.
3.3.4.3. Modify a configuration¶
To modify a configuration, run one of the following commands:
$ dedop c edit # CASE 1
$ dedop c edit config_name # CASE 2
$ dedop c edit -w other_workspace config_name # CASE 3
In all cases, it will launch a text editor and open all three configuration files. The text editor to be launched is OS-dependent and it is configurable on the Tool Configuration with the key name launch_editor_command.
In CASE 1, the text editor will open all the configuration files of the current configuration under the current workspace.
In CASE 2, the text editor will open all the configuration files of the specified configuration under the current workspace.
In CASE 3, the text editor will open all the configuration files of the specified configuration under the specified workspace.
When you are finished, just save the files and close the editor.
edit has an alias ed.
3.3.4.4. Copy a configuration¶
To copy a configuration, run one of the following commands:
$ dedop c copy # CASE 1
$ dedop c copy config_name_to_be_copied # CASE 2
$ dedop c copy config_name_to_be_copied new_config_name # CASE 3
$ dedop c copy -w other_workspace config_name_to_be_copied new_config_name # CASE 4
In CASE 1, neither the configuration to be copied nor the new configuration name is specified, so in this case a new
name with the format of current_config_name + _copy_ + unique_number is created. The unique number will be incremented
until a unique combination is constructed. For example, for a current config config1, it will create a new config
named config1_copy, config1_copy_2, config1_copy_3, and so on.
In CASE 2, the configuration to be copied is specified, but not the name of the new config. It will then follow the
same rule as in CASE 1, by creating a new config with a name that is of format
current_config_name + _copy_ + unique_number.
In CASE 3, the specified configuration will be copied as new_config_name inside the current workspace
In CASE 4, the specified configuration will be copied as new_config_name inside the specified workspace
As in workspace management, copying a configuration does NOT automatically change the current configuration.
copy has an alias cp.
3.3.4.5. Rename a configuration¶
To rename a configuration, run one of the following commands:
$ dedop c rename new_config_name # CASE 1
$ dedop c rename config_to_be_renamed new_config_name # CASE 2
$ dedop c rename -w other_workspace config_to_be_renamed new_config_name # CASE 3
In CASE 1, the current config name will be renamed to new_config_name. The current configuration will also be
changed to new_config_name.
In CASE 2, the specified config name in the current workspace will be renamed to new_config_name.
In CASE 3, the specified config name in the specified workspace will be renamed to new_config_name.
rename has an alias rn.
3.3.4.6. Show configuration info¶
To display information about the configuration such as current configuration path, list of files, as well as the file sizes, run the following command:
$ dedop c info # CASE 1
$ dedop c info other_config # CASE 2
$ dedop c info -w other_workspace config_name # CASE 3
In CASE 1, information for the current configuration in the current workspace will be displayed.
In CASE 2, information for the specified configuration in the current workspace will be displayed.
In CASE 3, information for the specified configuration in the specified workspace will be displayed.
info has an alias i.
3.3.4.7. Get current configuration¶
To get the current configuration name, run the following:
$ dedop c current
If there is a current configuration, the name of the current configuration will be returned. Otherwise,
no current DDP configuration will be returned.
It is also possible to get the current configuration in the other workspace by adding this parameter
-w other_workspace_name in the command.
current has an alias cur.
3.3.4.8. List configurations¶
To list available configurations, run one of the following commands:
$ dedop c list
As before, to list available configurations in the other workspace, just add -w other_workspace_name in the command.
list has an alias ls.
3.3.4.9. Upgrade configurations¶
A new version of DeDop Core sometimes comes with new versions of configuration files. In order to update your configurations, run the following command:
$ dedop c upgrade
Failure to use the latest version of configurations may result in processing errors.
upgrade has an alias up.
3.3.4.10. Show configuration version¶
To display the current configuration version, run the following command:
$ dedop c version
version has an alias v.
3.3.5. Running the Processor¶
Once the L1A source files have been added and configurations have been created, it is time to run the processing. To do that, use the following command:
$ dedop run
This command calls a processor to process L1A files to L1B (and possible L1BS). More information on how the processor
works, go to here. By default, the command above will process every single L1A files
inside the inputs directory under the current workspace, unless --inputs [L1A_FILE [L1A_FILE ...]] flag is specified.
The default behaviour is that the current configuration is used for processing. However, when --all-configs
flag is set, it will process the input files with all available configurations in the current workspace, producing an
output for each configuration. The output products will be located inside outputs directory under each configuration
directory. To specify other locations for the outputs, the flag --output DIR can be used.
When the flag --skip-l1bs is added to the command above, the process will generate only L1B files.
3.3.6. Analyse the Results¶
After the processing has been finished, we can now compare the L1B outputs in an interactive Jupyter Notebook:
$ dedop output compare -C default L1B_myconf.nc L1B_default.nc
When you pass just file names to the dedop output compare command, DeDop must know to which configurations they
refer to. The first filename corresponds to the current DDP configuration or the one given by the -c option.
The second filename corresponds to a DDP configuration given by the -C (upper case!) option.
You can also pass file paths to the dedop output compare command in which case the configuration names are ignored.
3.3.7. Tool Configuration¶
3.3.7.1. Configuration File¶
When DeDop is run for the first time it will create a file config.py in the directory .dedop of the
current user’s home directory. All DeDop tools use this file to read special software configuration parameters.
This is not to be confused with the processor configurations referred to in the dedicated section above.
Unixes and Darwin: On Unixes and Darwin (OS X), the full path to the DeDop tools configuration file is usually:
/home/<username>/.dedop/config.py
where /home/<username> is also given by ~ or $HOME in a terminal or shell.
Windows: On Windows 7+, the full path to the DeDop tools configuration file is usually:
C:/Users/<username>/.dedop/config.py
where C:/Users/<username> is also given by %USERPROFILE% on the Windows command-prompt.
To force writing a new DeDop tools configuration file use:
$ dedop --new-conf
This may be useful after DeDop software updates. It will ensure that you get the latest configuration parameters supported by a given DeDop version.
3.3.7.2. Configuration Parameters¶
Given here are the current DeDop tools configuration parameters:
| Parameter name | Description | Default value |
|---|---|---|
workspaces_dir |
Path where the DeDop Shell stores your workspaces. | '~/.dedop/workspaces' |
launch_notebook_command |
An OS-specific shell command string used to launch a new Jupyter notebook server. | OS-specific |
launch_notebook_in_new_terminal |
Whether launching the notebook creates a new terminal window. | False |
launch_editor_command |
An OS-specific shell command string used to launch a text editor for the processor configuration files. | OS-specific |
3.3.8. Command Reference¶
The following examples shall help you understand the basic concepts behind the various dedop commands.
Delay Doppler Altimeter Data (DeDop) command-line interface, version 1.5.1.dev1
usage: dedop [-h] [--version] [--traceback] [--license] [--docs] COMMAND ...
3.3.8.1. Positional Arguments¶
| COMMAND | Possible choices: workspace, w, config, c, input, i, output, o, run, r, status, s, notebook, man, copyright, license One of the following commands. Type “COMMAND -h” to get command-specific help. |
3.3.8.2. Named Arguments¶
| --version | show program’s version number and exit |
| --traceback | show (Python) stack traceback for the last error Default: False |
| --license | show software license and exit Default: False |
| --docs | show software documentation in a browser window Default: False |
3.3.8.3. Sub-commands:¶
3.3.8.3.1. workspace (w)¶
Manage DeDop workspaces.
dedop workspace [-h] COMMAND ...
3.3.8.3.1.1. Positional Arguments¶
| COMMAND | Possible choices: add, remove, rm, copy, cp, rename, rn, current, cur, list, ls One of the following commands. Type “COMMAND -h” for help. |
3.3.8.3.1.2. Sub-commands:¶
3.3.8.3.1.2.1. add¶
Add new workspace
dedop workspace add [-h] WORKSPACE
| WORKSPACE | Name of the workspace |
3.3.8.3.1.2.2. remove (rm)¶
Remove workspace
dedop workspace remove [-h] [-y] [WORKSPACE]
| WORKSPACE | Name of the workspace |
| -y, --yes | Do not ask for confirmation. Default: False |
3.3.8.3.1.2.3. copy (cp)¶
Copy workspace
dedop workspace copy [-h] [WORKSPACE] [NEW_NAME]
| WORKSPACE | Name of the workspace |
| NEW_NAME | Name of the new workspace |
3.3.8.3.1.2.4. rename (rn)¶
Rename workspace
dedop workspace rename [-h] [WORKSPACE] NEW_NAME
| WORKSPACE | Name of the workspace |
| NEW_NAME | New name of the workspace |
3.3.8.3.2. config (c)¶
Manage DeDop DDP configurations.
dedop config [-h] COMMAND ...
3.3.8.3.2.1. Positional Arguments¶
| COMMAND | Possible choices: add, remove, rm, edit, ed, copy, cp, rename, rn, info, i, current, cur, list, ls, upgrade, up, version, v One of the following commands. Type “COMMAND -h” for help. |
3.3.8.3.2.2. Sub-commands:¶
3.3.8.3.2.2.1. add¶
Add new DDP configuration
dedop config add [-h] [-w WORKSPACE] [--cryosat-adapted] CONFIG
| CONFIG | Name of the DDP configuration |
| -w, --workspace | |
| Name of the workspace | |
| --cryosat-adapted | |
Initialise configuration with parameters for processing reconditioned Cryosat-2 FBR data Default: False | |
3.3.8.3.2.2.2. remove (rm)¶
Remove DDP configuration
dedop config remove [-h] [-y] [-w WORKSPACE] [CONFIG]
| CONFIG | Name of the DDP configuration |
| -y, --yes | Do not ask for confirmation. Default: False |
| -w, --workspace | |
| Name of the workspace | |
3.3.8.3.2.2.3. edit (ed)¶
Edit DDP configuration
dedop config edit [-h] [-w WORKSPACE] [CONFIG]
| CONFIG | Name of the DDP configuration |
| -w, --workspace | |
| Name of the workspace | |
3.3.8.3.2.2.4. copy (cp)¶
Copy DDP configuration
dedop config copy [-h] [-w WORKSPACE] [CONFIG] [NEW_NAME]
| CONFIG | Name of the DDP configuration |
| NEW_NAME | Name of the new DDP configuration |
| -w, --workspace | |
| Name of the workspace | |
3.3.8.3.2.2.5. rename (rn)¶
Rename DDP configuration
dedop config rename [-h] [-w WORKSPACE] [CONFIG] NEW_NAME
| CONFIG | Name of the DDP configuration |
| NEW_NAME | New name of the DDP configuration |
| -w, --workspace | |
| Name of the workspace | |
3.3.8.3.2.2.6. info (i)¶
Show DDP configuration info
dedop config info [-h] [-w WORKSPACE] [CONFIG]
| CONFIG | Name of the DDP configuration |
| -w, --workspace | |
| Name of the workspace | |
3.3.8.3.2.2.7. current (cur)¶
Current DDP configuration
dedop config current [-h] [-w WORKSPACE] [CONFIG]
| CONFIG | Name of the DDP configuration |
| -w, --workspace | |
| Name of the workspace | |
3.3.8.3.2.2.8. list (ls)¶
List DDP configurations
dedop config list [-h] [-w WORKSPACE]
| -w, --workspace | |
| Name of the workspace | |
3.3.8.3.3. input (i)¶
Manage L1A inputs.
dedop input [-h] COMMAND ...
3.3.8.3.3.1. Positional Arguments¶
| COMMAND | Possible choices: add, remove, rm, list, ls One of the following commands. Type “COMMAND -h” for help. |
3.3.8.3.3.2. Sub-commands:¶
3.3.8.3.3.2.1. add¶
Add new inputs
dedop input add [-h] [-w WORKSPACE] [-q] L1A_FILE [L1A_FILE ...]
| L1A_FILE | L1A input file to add to workspace. |
| -w, --workspace | |
| Name of the workspace | |
| -q, --quiet | Suppress output of progress information. Default: False |
3.3.8.3.3.2.2. remove (rm)¶
Remove inputs
dedop input remove [-h] [-w WORKSPACE] [-q] [L1A_FILE [L1A_FILE ...]]
| L1A_FILE | L1A input file to add to workspace. |
| -w, --workspace | |
| Name of the workspace | |
| -q, --quiet | Suppress output of progress information. Default: False |
3.3.8.3.4. output (o)¶
Manage and analyse L1B outputs.
dedop output [-h] COMMAND ...
3.3.8.3.4.1. Positional Arguments¶
| COMMAND | Possible choices: clean, cl, list, ls, open, op, inspect, ins, compare, cmp One of the following commands. Type “COMMAND -h” for help. |
3.3.8.3.4.2. Sub-commands:¶
3.3.8.3.4.2.1. clean (cl)¶
Clean outputs folder of current configuration or CONFIG
dedop output clean [-h] [-w WORKSPACE] [-c CONFIG] [-q]
| -w, --workspace | |
| Name of the workspace. | |
| -c, --config | Name of the DDP configuration. |
| -q, --quiet | Suppress output of progress information. Default: False |
3.3.8.3.4.2.2. list (ls)¶
List outputs folder of current configuration or CONFIG
dedop output list [-h] [-w WORKSPACE] [-c CONFIG] [WC]
| WC | Wildcard pattern. |
| -w, --workspace | |
| Name of the workspace. | |
| -c, --config | Name of the DDP configuration. |
3.3.8.3.4.2.3. open (op)¶
Open outputs folder of current configuration or CONFIG
dedop output open [-h] [-w WORKSPACE] [-c CONFIG]
| -w, --workspace | |
| Name of the workspace. | |
| -c, --config | Name of the DDP configuration. |
3.3.8.3.4.2.4. inspect (ins)¶
Inspect L1B product
dedop output inspect [-h] [-w WORKSPACE] [-c CONFIG] L1B_FILENAME
| L1B_FILENAME | The filename or path of the a L1B product. If only a filename is given, it must exist in outputs of the given workspace/configuration. |
| -w, --workspace | |
| Name of the workspace. | |
| -c, --config | Name of the DDP configuration. |
3.3.8.3.4.2.5. compare (cmp)¶
Compare two L1B products
dedop output compare [-h] [-w WORKSPACE] [-c CONFIG] [-W WORKSPACE_2]
[-C CONFIG_2]
L1B_FILENAME_1 [L1B_FILENAME_2]
| L1B_FILENAME_1 | The filename or path of the first L1B product. If only a filename is given, it must exist in outputs of the given workspace/configuration. |
| L1B_FILENAME_2 | The filename or path of the second L1B product. If only a filename is given, it must exist in outputs of the given second or first workspace/configuration. If omitted, the first filename or path is used. |
| -w, --workspace | |
| Name of the workspace. | |
| -c, --config | Name of the DDP configuration. |
| -W, --workspace-2 | |
| The workspace of the second L1B product. | |
| -C, --config-2 | The DDP configuration of the second L1B product |
3.3.8.3.5. run (r)¶
Run the Delay Doppler Processor (DDP).
dedop run [-h] [-s] [-q] [-w WORKSPACE] [-c CONFIG]
[-i [L1A_FILE [L1A_FILE ...]]] [-o DIR] [-a]
3.3.8.3.5.1. Named Arguments¶
| -s, --skip-l1bs | |
Skip generation of L1B-S files. Default: False | |
| -q, --quiet | Suppress output of progress information. Default: False |
| -w, --workspace | |
| Use WORKSPACE, defaults to current workspace. | |
| -c, --config | Use CONFIG in workspace, defaults to current DDP configuration. |
| -i, --inputs | L1A input files. Defaults to all L1A files in workspace. |
| -o, --output | Alternative output directory. |
| -a, --all-configs | |
Run all DDP configurations in workspace. Cannot be used with option -c Default: False | |