1. How can I open the failed and passed revisit sessions?
In the execution list, you can directly click on a revisit session name to access the session detail of any failed or passed revisit session. Sometimes, you will see the error icon on a revisit session and a revisit name is unable to click, which means there were some errors occurred and leads to the cannot initiate the revisit. Therefore, there are no created sessions at that time.
On the other hand, you can access the session list and search by typing your preferred manual session. At a result, you will see the list of revisit sessions that related to the giving manual session.
2. Why did all of my revisit sessions fail but I was not able to see them on the sessions page?
Scriptless Automation has a process called “exploring” the manual session and it will reproduce your Manual session step by step before executing revisit sessions, the purpose is to ensure that all the actions in the manual session are qualified for running Scriptless Automation. So if you see all of your revisit sessions failed, it is mostly because the “exploring” process has not run successfully, you can hit the re-run icon to execute the "exploring" process again.
3. In my test case (manual session), I performed 20 test steps but only saw only 10 test steps in my scriptless rerun sessions. Why?
The reason might be that during the manual session, you have violated some current Scriptless Automation unsupported actions (guardrails). Kobiton Scriptless records the session up to the step that violates unsupported actions/endpoints. All the steps after the unsupported actions would still be displayed in Session Detail but would not be recorded for scriptless reruns.
If you access the session details again you will see the purple test steps are represented for the qualified steps for Scriptless Automation and others will be considered as normal test steps and will not be included in the Scriptless Automation process.
4. What can make the Scriptless Automation “exploring” process failed?
The “exploring” process failed mostly because the user has violated Scriptless Automation unsupported actions or encountered Scriptless Automation limitations (see Limitations section). Below is the list of use cases that cause the failures:
-
Perform a test action too fast: Please take a waiting time about 3 seconds between 2 test actions, this will ensure the screen has loaded completely when you jump to a next test.
-
Tap on a button that is covered by the virtual keyboard: This may impact the element recognition ability of Scriptless Automation. This is a case that we will handle in the future. For now, there is a way to go through this problem. Before you click on a button, if the virtual keyboard currently appears, please press “Back” button on the manual toolbar then click the button as normal (Android only).
-
Tap outside a pop-up to close it: This is an action that the user normally does when they want to close a pop-up. Unfortunately, Scriptless Automation can not do this for now. You are recommended to close a pop-up by clicking on the buttons inside the pop-up properly.
-
Tap on dynamic elements: Buttons and texts that change over the time like news or weather forecast are the limitations of Scriptless Automation. Don't worry, a team is addressing this!
-
Tap on buttons on device’s navigation bar (device's softkey)
- There are some problems with devices that are used for running Scriptless Automation (device availability, network, etc.)
5. Can I trigger Scriptless Automation on latest devices ?
Android devices
Technically you can, but right now, there are some issues that currently makes Scriptless Automation unstable on on latest devices like S10, S10+, etc. Some devices that we recommend for the manual:
- Galaxy Note 5
- Galaxy Note 8
- Galaxy Note 9
- Galaxy S7
- Galaxy S8
- Galaxy S8+
- Moto X Play
- Moto Z (2) Play
iOS devices
You can run any latest iOS devices like iPhone X, iPhone XS Max, iPhone 11, etc. with verion 11+. But please aware that right now Scriptless Automation does not support iPad or iPod.
6. Does Scriptless Automation use the same version of the application as in the manual session when I have many versions of the app in the app repository?
Yes, Scriptless Automation will use exact application version in the giving manual as well as any test actions, ensuring your test case will play backed properly on multiple devices.
7. Is there any way to select which Android versions/devices to run the tests on instead of selecting the recommended random 5/10/20 device options?
For now, we only provide a default bundle likes 5/10/20 devices bundles. In the near future, we will provide you more choices by giving an ability to customer your bundles based on your need.
8. why does a revisit session automatically has the scroll actions in my test case although I did not scroll or swipe in my manual session?
Don't worry, those are the scrolling actions that Scriptless Automation performed. The reason is because an elements on any specific resolution might have different locations especially on the list view, therefore, to ensure an elements will be selected properly, Scriptless Automation will try to find an element by scrolling a couple of times.
9. I selected Top 20 recommended but the tests were en-queued on 18 devices only.
This is because these devices are offline at that time. At a result, Scriptless Automation does not always “revisit” on full of 20 devices. In the near future, we will provide you an ability to properly cover a range of devices for the revisit.
10. What is the difference between a Failed revisit and a Blocked revisit?
Scriptless Automation uses deep learning to "know" which element in the Revisit session matches the element used in the Manual session. Unlike web testing where the HTML is standard across browsers, on mobile how the element is referenced differs by device type, manufacturer and, in some cases, test run. That's right, some apps dynamically generate the XML!
For the cases where the deep learning is unable to find an element, Scriptless Automation presents a Blocker Remediation screen to allow the tester to "guide" the deep learning to ensure the continuation of the test case.
Blocked Test: The deep learning could not find an element on the screen during a Revisit session. You can “remediate” the blocker and the test will rerun without encounter an issue again.
Failed Test: This typically happens when one of the restricted user actions have been performed during the manual session, or the reason might come from the device issues like network, high CPU, etc. We are improving the user experience related to this and continuing to implement more gestures / user actions to eliminate these scenarios long-term.
11. Why did my rerun sessions fail?
-
Be aware whether the device's processor chip is compatible with your app (CPU architecture compatibility). If there is a device in the device bundle with a processor that your app doesn’t support (eg. either 32bit or 64bit), that session running in that device will be failed.
-
The rerun device went offline unexpectedly.
-
Network issue: The connection to the device sometimes is not stable, it could make Scriptless Automation fail unexpectedly.
12. How long would it take for Scriptless Automation to execute a manual session and how will I know when this task is completed? Will there be a notification?
It takes about an hour on average for Scriptless Automation to run on 20 devices with a manual session that has 20 steps. Either when Scriptless Automation was kicked-off or Scriptless Automation was completed, you will receive an email for the notification.