Test execution via the command line and --noUI support
Requirements
The following components are required for using the new CLI syntax:
CODESYS V3.5 SP19 Patch 3 or newer
CODESYS Test Manager 5.2.0.0 or newer
Support of the --noUI CLI option requires CODESYS V3.5 SP22 or newer instead. In addition, the --noUI option is supported only with the new --runtest CLI syntax.
Tip
In the Test Manager view in CODESYS, the Copy Command Line to Clipboard command is available in the context menu of test scripts. This command copies the CLI call for the selected test script in the new CLI syntax to the clipboard.
Tip
Arguments which contain spaces, hyphens, or slashes should be enclosed in double quotation marks: "<argument>"
Tip
Credentials might be logged to stdout in plain text.
New CLI syntax --runtest
CLI option | Description | Optional | Values | Note |
|---|---|---|---|---|
| Entry point for every CLI test execution. | No | - | Without this command, all subsequent options are ignored. |
| The absolute path of the top-level directory of the test repository. Example: | No | The path must point to a valid test repository. A test repository which is not yet known in | |
| The qualified name of the test script within the test repository. The folder and script are separated by a period. Example: | No | The syntax for the path follows the notation in the IDE, not the file path. | |
| The absolute path to the directory where the test reports are saved. Example: | No | The path must be absolute and valid. Directories in the path which do not exist yet are created. | |
| Version of the test script. | Yes | An invalid or nonexistent version causes an error during test execution. If no version is specified, then the newest available version of the test script is executed. | |
| The name which is listed in the test report. | Yes | Default: Name of the user of the current OS login session | |
| The format in which the test report(s) are saved. Example: | Yes | Possible values:
Default: | Multiple options can be selected. They must be separated by commas without spaces. If no value, an invalid value, or only With the combination of |
| The channel for the output of the test progress. | Yes | Possible values:
Default UI: Default | The default for a test execution with UI includes the |
| A reduced test report is generated. Variable values are not listed. | Yes | - | |
| CODESYS is not closed after the test execution. | Yes | - | |
| The test execution is started in debug mode. | Yes | Possible values:
Default: | Multiple options can be selected. They must be separated by commas without spaces. A combination with any of the following options results in an error:
|
| The test execution is started in calibration mode. | Yes | - | A combination with any of the following options results in an error:
|
| General CLI option of CODESYS CODESYS is started without a graphical user interface. However, this operating mode still requires a Windows desktop session. Depending on the CODESYS functionality which is used in the script, modal dialogs may still appear. | Yes |
Compatibility of the legacy CLI syntax --executecommand="TestManager ExecuteScript"
The legacy syntax is still supported. However, the introduction of the new CLI command has resulted in some changes.
Important
It is recommended that you switch to the new syntax.
The --noUI CLI option is still not supported for the legacy syntax, regardless of the CODESYS version.
New and old syntax cannot be mixed.
Legacy CLI option | Change as of CODESYS Test Manager version 5.2.0.0 |
|---|---|
| - |
| Omitted As of version 5.2.0.0 of the CODESYS Test Manager, test repositories are no longer identified by their name. |
| - |
| - |
| - |
| - |
| - |
| Omitted As of version 5.2.0.0 of the CODESYS Test Manager, the internal report repository is no longer supported. |
| Omitted As of version 5.2.0.0 of the CODESYS Test Manager, the internal report repository is no longer supported. |
| Omitted As of version 5.2.0.0 of the CODESYS Test Manager, old test reports are overwritten by default. |
| The option to specify a log file (absolute path) is no longer supported. In the CI/CD context, working with |
| - |
Exit codes
The following exit codes replace the previous exit codes because the previous exit codes no longer correspond to the implementation of CODESYS Test Manager version 5.2.0.0.
Important
When you start the CODESYS executable directly from the command line, the console returns control immediately by default. The console does not wait until the CODESYS process has ended. This is because CODESYS is a desktop application and not a console application. To make the console wait for the CODESYS process to end, use the Windows command start /wait <command line>. Alternatively, you could also call the command line from a batch file. Use one of these methods to reliably read the exit code.
Exit code | Meaning for the new CLI command |
|---|---|
| No error has occurred. |
| During execution in |
| The |
| The XML test report cannot be saved. |
| The HTML test report cannot be saved. |
| An error occurred while processing the specified |
| The execution did not return a test result. |
| No valid license has been found for the CODESYS Test Manager. No test report has been created. |
| The combination of the selected |
| The combination of |
| The selected |
| The test run has been completed with failed test steps. |