Run Automation Test

Release Task Inputs

After adding the Kobiton Automation Test Executor task to your pipeline, you'll need to add some information:

Required Information:

• Display name: The Kobiton API Credentials created in the previous section. (More credentials can be added, but for now select from the dropdown list).

• Kobiton executor server credentials: The Kobiton Executor Server credentials in the service connection store.

• Git repository URL: Git URL that contains test scripts. If the repository is private, use the SSH URL.

• Git repository branch: Git branch that will execute automation test script.

• Git repository SSH key: The SSH Key to access a private Git repository. In this document, we will use Kobiton's public sample repository and will leave this field empty.

Desired Capabilities

• Upload New App Version:

  By enabling, you'll need to upload a new version of your existing app on the Kobiton App Repository.
  App file path: Select an .apk or .ia in the release artifact.

• App ID: The App ID of your existing app on the Kobiton App Repository, which can be found on Kobiton's app repository. 

As an example, once the app uploading process is complete, a value of 4058 will be parsed to your KOBITON_SESSION_APPLICATION environmental variables. 

If this field is left unchecked, in the App ID of the .apk file, you can enter your existing App ID or URL. This value will be directly passed to your KOBITON_SESSION_APPLICATION environmental variable.

• Use Custom Device: This will check if you want to define the device related parameters in your desiredCaps via environmental variables.

If left unchecked, the extension will ignore environmental variables.

If checked, the following fields are required:

• Kobiton Cloud Device Name: KOBITON_DEVICE_NAME
• Device platform version: KOBITON_SESSION_PLATFORM_VERSION
• Device platform: KOBITON_DEVICE_PLATFORM

Execution Inputs

• Root Directory: The path to your Git repository.

• Commands: The command lines to execute your test script. These commands will be called at the root directory of your Git repository. The commands will depend on your test project. In this example, we will use the command “mvn test” to build the project and run automation test script.

• Wait for execution:

  • If unchecked, the release will continue to the next release task. The execution job ID of this task will be printed, but not on the console logs or report URL.

  • If checked, the task will wait for the execution to be either COMPLETED (commands ran successfully) or FAILED (cannot clone git repository, server error,...) then print the console log and report URL (if it existed).

    • Log Type: Users can pick a type of log to be printed. Combined means the output logs and error logs will be printed in chronological order. The Separated option allows you to choose one of the 2 types of logs.
      • It is recommended to log your test case result with appropriate log type. For example, in Java you can use System.err.println to print test case errors, this will be in the error logs while System.out.println will return texts in output logs.
    • Terminate release pipeline on failed job: If checked, once the execution job status is FAILED, an error will be returned and the whole release is terminated (All the following release tasks below this task will be skipped).

    
    
    

2. Execute the release

• Once everything is set up, click on the Save button on the right.

  

• Leave fields blank, then click OK.

  

• On the right, click Create Release button to create a new release.

  

• A new window will appear, leave everything blank ,and click Create.

  

• On the notification, click Release-1 to view the release status.

  

• When the release in completed, you can check the release log by clicking the Logs button below the release Stage.

  

• Click on Kobiton Automation Test Executor task to view its logs

  

  

  If you chose to Wait for execution, in cases with no server error and all test cases pass, you can see the test result, execution job ID ,and report URL

  

  

  The Job ID is formatted with prefix “rN” where N is the release ID of your Azure DevOps project. Clicking on the Report URL will lead you to the report interface.

• Please contact us for the necessary username, password and domain.

• After the session is successfully created, look up the session in the Kobiton Portal.

  

3. Overall

This sample case demonstrates how to trigger the automation test script on Kobiton devices.  

• Filling in the inputs

  

  

  • Kobiton public git repository

  • In Root directory enter the execution task. Change directory to test scripts before running the test. ./java/java_testng_junit

  • In the command input of the execution task, run the test: $ mvn test -Dtest=azureTestNG

  • Because this demo repository is public, leave the SSH Key field blank and use HTTPS URL in the Kobiton Azure DevOps Release Task.

• Preparing the automation script

  The script will set default values for these following capabilities:

   

   

  • • KOBITON_USERNAME: the Kobiton Username set in Azure Service connection
    • KOBITON_API_KEY: The Kobiton API Key set in Azure Service connection
    • KOBITON_DEVICE_NAME: The Kobiton API Key set in Azure Service connection
    • KOBITON_SESSION_APPLICATION: Value of the app in Kobiton Extension Task