5. Kobiton Cloud Installation - Troubleshooting

Common issues & troubleshooting steps

1. Cannot install the iOS app on Kobiton portal

First, go to Settings > Other Settings > Kobiton Re-signing Process: Make sure App signing using Kobiton Certificate (iOS) check-box is selected.Screen_Shot_2022-09-06_at_5.46.56_PM.png

Second, re-sign issue on the iOS devices:

  1. Check the app installed:
    • Go to App repo of Kobiton portal
    • Search app name on the search bar
    • Click on footer of app card > observe the Bundle ID (example: com.equinoxfitness.Equinox.qa)
    • Go to Gigafox portal > System > iOS Management: check iOS Provisioning Status
    • Go to Entitlements of the provisioning profile of the checked device
    • Observe “application-identifier”:
      • “86B8H7MB89.com.*" -> that means, this provisioning profile has teamID = 86B8H7MB89 and this device can install all app has bundle ID starting with "com", the others ("org",..) can not.
      • “DVD8TV3C54.*" -> that means, this provisioning profile has teamID = DVD8TV3C54 and this device can install all app has bundle ID starting with any word.
  2. Check provisioning profile + cert for iOS devices:
    • Go to Gigafox portal > System > iOS Management: check iOS Provisioning Status, they should be green. If cert or provisioning profile have any issue (such as: incorrect, profile not included in cert, cert or profile is expired), please contact our DataCenter team to update.
    • This step needs action from Kobiton team: Go to mac mini which has been installed deviceShare 
      • Check the list of provisioning profiles in usr/local/kobiton/deviceshare/provisioning_profiles: the provisioning profile of checking deviceConnect server should be included in this folder
      • Check the list of the certs in KeyChain of deviceShare server: the associated certs should be included in deviceshare and system keychain
      • If there is no provisioning profile or cert: focus step 2.2.1 and 2.2.2 on this
      • deviceConnect/deviceShare (aka Gigafox) | 2.2.-Configuration
      • After the update is done, please focus these steps to re-load deviceShare:
        1. Go to /usr/local/kobiton/deviceshare from deviceShare server’s terminal
        2. Stop the deviceShare: sudo /bin/launchctl unload -w /Library/LaunchDaemons/com.kobiton.deviceshare.signing.plist
        3. Start the deviceShare: sudo /bin/launchctl load -w /Library/LaunchDaemons/com.kobiton.deviceshare.signing.plist
    • This step needs action from Kobiton team: Check the TeamID on table OrganizationAppleTeams: 
  3. Check the dc.ini file of deviceConnect:
    • Make sure Kobiton.UseInternalLegacyAppResigning=false is on dc.ini file.
      • If not, please add it.
      • Then restart deviceConnect and deviceShare again (if you still cannot install app, please help to reboot the Macmini - it should be the last option for us)

2. Cannot use the iOS devices (launch manual session or install app)

  • Re-check the provisioning profile + cert for iOS devices per the step 2b above.
  • Observe the device log:
    • On GF UI, download device log of device: System > Select device name > Click on “Save Logs” button
    • Open the zip file and go to folder “usr_local_deviceconnect_Logs”
    • Open file “MobileLabs.DeviceConnect.Services.log”
    • Filter by keyword “DeviceState <UDID>“
    • Observe each results in file.
    • Steps to verify:
    • What is the issue: Log file has this error
    • 1Due to an internal issue on the device the developer image was unable to be mounted. Please reboot the device and try again.
    • Why: Current Xcode version does not support this OS version of iOS device.
    • How to fix: Please ask DC team for upgrade Xcode into latest version.

3. Auto-lock is not turned off on iOS devices

1. iOS Devices

  • New iOS phones: X, XS, XS Max, 11, 12, and above.
  • Apple ID should be logged in on the devices
  • Follow the same guide as Kobiton device:iOS Setup and Troubleshooting
  • Turn off auto-lock: Go to Settings -> Display -> Auto-Lock and turn it off.

2. Android Devices

  • New android phones: Galaxy S20, S21, Galaxy Note 10, and above.
  • Samsung ID/Google ID/ Xiaomi ID… should be set up for all Android devices to protect our devices
  • Follow the same guide as Kobiton device:Android Setup and Troubleshooting

4. Video is not available after the manual session is ended

This could be the result of the old GEM firmware installed. We should refer to the latest firmware to deploy correctly. Please contact our Support at support@kobiton.com to know the latest firmware version.

5. Devices are available in Gigafox portal but not online on KOBITON portal

Sometimes, when DC team set up a new Mac mini for deviceConnect or switches devices between Mac minis, some devices should be available in GigaFox portal but not displayed in both KOBITON portal or KOBITON database. Here are some possible causes:

#1. Devices slots are full

Why:

  • Clients only purchased for some device slots (e.g: 4) but DataCenter team plugged more on Mac mini (e.g: 10)
  • So, there are 4 devices are displayed in Kobiton portal after the setting completed.

How to Fix (if clients request/quick fix to increase slots):

  • To verify the number of slots: https://asgard.kobiton.com/question/15?userId=<UserID>Screen_Shot_2022-09-06_at_5.51.06_PM.png
  • To show all devices (or increased the number of slots), please help to contact the Technical support team.
  • After the device slots are updated, back to Mac mini, open Terminal and run the below command to register and restart the services:
    bash register-consul.sh
    sudo sv restart dc-web dc-services dc-webviewer dc-watchdog

#2: Slots are not full. Devices are not recorded as ‘ACTIVATED’ in database

Why:

  1. Logic: When a new device is plugged into Mac mini, the cleanup policy is triggered, the device will need to go through those cleanup processes, then deviceConnect will send full device info to the backend to register the new hosting account.
  2. Problem: If the first time cleanup after plugging the device is successful, the device should be online via Kobiton portal. If not, the device cannot be online as they are stuck with the old hosting account.

How to Fix:

  • Open Asgard:https://asgard.kobiton.com/browse/4
  • Open table Devices
  • Filter data by Udid of devices in Mac miniScreen_Shot_2022-09-06_at_5.54.35_PM.png
  • If devices have State = ACTIVATED → Devices are online in KOBITION portal. If not, devices are not displayed in Device list.
  • Go to mac mini, update file dc.ini to turn off cleanup policy:
    # Enable cleanup policies
    HealthService.EnableCleanup=false
  • Open Terminal and run: 
    bash register-consul.sh
    sudo sv restart dc-web dc-services dc-webviewer dc-watchdog
  • Observe the change in the Devices tableScreen_Shot_2022-09-06_at_5.59.19_PM.png
  • Observe the device list and device management in Kobiton portal
  • Go to mac mini, update file dc.ini to turn on cleanup policy
    # Enable cleanup policies
    HealthService.EnableCleanup=true
  • Open Terminal and run:
    bash register-consul.sh
    sudo sv restart dc-web dc-services dc-webviewer dc-watchdog

#3: Account OpenVPN is out of available device slots

Scenario: although user has configured the setting in dc.ini and register-consul.sh correctly: when running the command bash register-consul.sh to register to an environment (test/staging), deviceConnect server can’t be health-checked by Consul → devices will not be displayed in Kobiton portal because Consul doesn’t allow deviceConnect service to access Kobiton portal.

Why:

  • deviceConnect server uses OpenVpn account kobiton-<test/staging>-openvpn to create an IPv4 address for routing and registering Consul.
  • The account has a limitation of devices (= deviceConnect server). If the deviceConnect server is out of the limitation → OpenVPN will block the network in/out of that deviceConnect server → it will be deregistered on Consul after 180 seconds.
  • At this time, the available device slot in account kobiton-test-openvpn is 10 and kobiton-staging-openvpn is 3.

How to fix: 

  • Click LastPass then search/type “openvpn” → launch the account you want to see the device slot. For example: if you are registering deviceConnect server to test environment then launch the option openvpn-kobiton-test.
    Screen_Shot_2022-09-06_at_6.03.12_PM.png
  • Select OpenVPN Cloud → Owner Portal → login with the corresponding account (kobiton-test/kobiton-staging)
  • From the left sidebar, expand Users tab -> click Devices. ASK YOUR LEAD OR DEVOPS TEAM FOR PERMISSION BEFORE PROCESSING THIS: remove any devices whose status is offline (red dot) then count remain devices on screen. If your device’s IPv4 is in the list and its order number is below 10 (for test) or 3 (staging) → now your deviceConnect server can be registered to Consul → devices of deviceConnect server now can be observed on Kobiton portal.Screen_Shot_2022-09-06_at_6.03.56_PM.png

6. Lightning mode is NOT working

When there is a Lightning mode error in UI and the Chrome Console shows a timeout error like below.Screen_Shot_2022-09-06_at_6.04.48_PM.png

When running a session and waiting for about 15 seconds and the session does not automatically switch to lightning mode (even though there is a GEM installed for that environment) there can be many reasons, but the most common is that the GEM is not yet configured correctly or networking has problem.

Verify that:

  • The browser uses Chrome, Lightning mode doesn’t run on other browser types
  • The WebRTC traffic is allowed at customer side, sendwebrtc-connectivity-check/README.md at main · kobiton/webrtc-connectivity-check  to them for checking
  • In GEM, at file /ml/var/config/gigacap-options , examine IP at line --public-address. For example, the value of “public-address” should be "173.213.234.182" (depending on GEM) not "xxx".Screen_Shot_2022-09-06_at_6.10.08_PM.png If the Kobiton is OP system, make sure the IP is the GEM IP. If the Kobiton is AWS system (e.g. hosted environment),
    • Make sure the IP is the Internet-resoled IP generated by Data Center team. How to verify:
      • Go to any MacOS that locates outside of Data Center, execute traceroute 173.213.234.149 (replace your IP)
      • If the IP is configured correctly, the output is like below:
        ... truncated
        13  nash-bb1-link.ip.twelve99.net (62.115.137.44)  247.473 ms  248.405 ms *
        14  * nash-bb1-link.ip.twelve99.net (62.115.137.44)  245.843 ms  245.619 ms
        15  * atl-b24-link.ip.twelve99.net (62.115.134.247)  240.310 ms *
        16  atl-b24-link.ip.twelve99.net (62.115.134.247)  240.758 ms  239.930 ms
           databank-ic341824-atl-b24.ip.twelve99-cust.net (62.115.170.173)  246.546 ms
        17  databank-ic341824-atl-b24.ip.twelve99-cust.net (62.115.170.173)  248.240 ms * *
        18  * 173.213.234.149 (173.213.234.149)  244.335 ms *
        19  173.213.234.149 (173.213.234.149)  245.427 ms  246.895 ms  261.342 ms

        Otherwise, if the IP is configured incorrectly, the output is like below:

        ... truncated
        13  nash-bb1-link.ip.twelve99.net (62.115.137.44)  245.932 ms  255.318 ms *
        14  atl-bb1-link.ip.twelve99.net (62.115.137.54)  239.082 ms
           nash-bb1-link.ip.twelve99.net (62.115.137.44)  266.446 ms *
        15  atl-b24-link.ip.twelve99.net (62.115.134.247)  240.768 ms  238.744 ms  242.510 ms
        16  databank-ic341824-atl-b24.ip.twelve99-cust.net (62.115.170.173)  247.439 ms  246.995 ms *
        17  databank-ic341824-atl-b24.ip.twelve99-cust.net (62.115.170.173)  245.267 ms *  244.702 ms
        18  * * *
        19  * * *
        20  * * *
        21  * *^C
    • Make sure the traffic routing for the IP is correct that the IP would route the traffic to that GEM. How to verify:
      • Go to GEM, execute nc -u -l 50000
      • Go to any MacOS that locates outside of Data Center (i.e. your house or Kobiton office), execute echo "hello" > /dev/udp/<GEM PUBLIC IP>/50000 for instance, echo "a" > /dev/udp/173.213.234.184/50000.
      Back to the GEM where the above nc execution, it should print “hello”. Otherwise, there is something wrong at the Internet-resolved IP that it doesn’t route the traffic to that GEM, contact Data Center team for solving this issue.

7. An Appium command in the script receives error response 

7.1. The script expects the device has network connectivity and services.log reports TaskCanceledException on the execution command

Conventions:When saying services.log, it refers to the file MobileLabs.DeviceConnect.Services.log

The error in services.log is like below:

4034 2021-12-07 22:50:33.4020 E/WebDriverController [162] [WebDriver 9daece06] Exception
System.Threading.Tasks.TaskCanceledException: A task was canceled.
  at MobileLabs.DeviceConnect.Framework.IO.RemoteWebDebuggingProtocolClient.SynchronousCommand(String methodName, Object args, ChromeExecutionContext executionContext, CancellationToken cancel) in /tmp/build/9674af12/ml2/src/deviceconnect/MobileLabs.DeviceConnect.Framework/IO/RemoteWebDebuggingClient.cs:line 344
  at MobileLabs.DeviceConnect.Framework.IO.RemoteWebDebuggingProtocolClient.PageNavigate(String url, ChromeExecutionContext executionContext, CancellationToken cancel) in /tmp/build/9674af12/ml2/src/deviceconnect/MobileLabs.DeviceConnect.Framework/IO/RemoteWebDebuggingClient.cs:line 493
  at MobileLabs.DeviceConnect.Services.Sessions.SessionDevice.WebNavigateTo(String url) in /tmp/build/9674af12/ml2/src/deviceconnect/MobileLabs.DeviceConnect.Services/Sessions/SessionDevice.cs:line 672
  at MobileLabs.DeviceConnect.Web.Controllers.WebDriverController.GoToUrl(String sessionId) in /tmp/build/9674af12/ml2/src/deviceconnect/MobileLabs.DeviceConnect.Services/Web/Controllers/WebDriverController.cs:line 1603
   at MobileLabs.DeviceConnect.Web.Controllers.RoutingAirStreamController.MobileLabs.DeviceConnect.Framework.IO.Mvc.IHandleRouting.HandleRequestAsync(IRequestContext context) in /tmp/build/9674af12/ml2/src/deviceconnect/MobileLabs.DeviceConnect.Services/Web/Controllers/RoutingAirStreamController.cs:line 83

It might be because the device became disconnected on wifi or the wifi access point has a problem with the upstream network node. Please verify by looking at the wifi status on the command screenshot or the session video.

7.2. services.log reports PortableNetworkException on the execution command

The error in services.log is like below: 

4034 2021-12-07 22:53:26.8819 E/WebDriverController [17] [WebDriver fac0a226] Exception
MobileLabs.DeviceConnect.Framework.IO.PortableNetwork.PortableNetworkException: Closing set EPIPE on 285
  at MobileLabs.DeviceConnect.Framework.WebSocketServer.WebSocketFrame.SendFrame(Stream writer, AsyncLock writeLock, CancellationToken cancel) in /tmp/build/9674af12/ml2/src/deviceconnect/MobileLabs.DeviceConnect.Framework/WebSocketServer/WebSocketFrame.cs:line 234
  at MobileLabs.DeviceConnect.Framework.WebSocketServer.PortableClientWebSocket.SendAsync(ArraySegment`1 buffer, WebSocketMessageType messageType, Boolean endOfMessage, CancellationToken cancel) in /tmp/build/9674af12/ml2/src/deviceconnect/MobileLabs.DeviceConnect.Framework/WebSocketServer/PortableClientWebSocket.cs:line 183
  at MobileLabs.DeviceConnect.Framework.IO.ChromeDebuggingProtocolClient.Command(String methodName, Int32 id, Object args, ChromeExecutionContext executionContext, CancellationToken cancel) in /tmp/build/9674af12/ml2/src/deviceconnect/MobileLabs.DeviceConnect.Framework/IO/ChromeDebuggingProtocolClient.cs:line 67
  at MobileLabs.DeviceConnect.Framework.IO.RemoteWebDebuggingProtocolClient.SynchronousCommand(String methodName, Object args, ChromeExecutionContext executionContext, CancellationToken cancel) in /tmp/build/9674af12/ml2/src/deviceconnect/MobileLabs.DeviceConnect.Framework/IO/RemoteWebDebuggingClient.cs:line 344
  at MobileLabs.DeviceConnect.Framework.IO.RemoteWebDebuggingProtocolClient.RuntimeEvaluate(String expression, ChromeExecutionContext executionContext, CancellationToken cancel) in /tmp/build/9674af12/ml2/src/deviceconnect/MobileLabs.DeviceConnect.Framework/IO/RemoteWebDebuggingClient.cs:line 462
  at MobileLabs.DeviceConnect.Services.Sessions.SessionDevice.WebEvaluateCore(WebContextInfo context, String fullExpression, CancellationToken token, Boolean shouldLog) in /tmp/build/9674af12/ml2/src/deviceconnect/MobileLabs.DeviceConnect.Services/Sessions/SessionDevice.cs:line 649
  at MobileLabs.DeviceConnect.Services.Sessions.SessionDevice.WebEvaluate(WebContextInfo context, String expression, String arguments, CancellationToken token, Boolean shouldLog) in /tmp/build/9674af12/ml2/src/deviceconnect/MobileLabs.DeviceConnect.Services/Sessions/SessionDevice.cs:line 632
  at MobileLabs.DeviceConnect.Web.WebDriver.WebDriverSession.FindWebElements(WebDriverSearchBuilder builder, CancellationToken cancel) in /tmp/build/9674af12/ml2/src/deviceconnect/MobileLabs.DeviceConnect.Services/Web/WebDriver/WebDriverSession.cs:line 685
  at MobileLabs.DeviceConnect.Web.WebDriver.WebDriverSession.FindWithRetry(Func`2 search) in /tmp/build/9674af12/ml2/src/deviceconnect/MobileLabs.DeviceConnect.Services/Web/WebDriver/WebDriverSession.cs:line 666
  at MobileLabs.DeviceConnect.Web.WebDriver.WebDriverSession.Find(Object searchParameters, String rootElementId, Boolean findOne) in /tmp/build/9674af12/ml2/src/deviceconnect/MobileLabs.DeviceConnect.Services/Web/WebDriver/WebDriverSession.cs:line 635
  at CallSite.Target(Closure , CallSite , Object )
  at MobileLabs.DeviceConnect.Web.Controllers.WebDriverController.FindElement(String sessionId) in /tmp/build/9674af12/ml2/src/deviceconnect/MobileLabs.DeviceConnect.Services/Web/Controllers/WebDriverController.cs:line 656
   at MobileLabs.DeviceConnect.Web.Controllers.RoutingAirStreamController.MobileLabs.DeviceConnect.Framework.IO.Mvc.IHandleRouting.HandleRequestAsync(IRequestContext context) in /tmp/build/9674af12/ml2/src/deviceconnect/MobileLabs.DeviceConnect.Services/Web/Controllers/RoutingAirStreamController.cs:line 83

Check the session video or the command screenshot, there maybe unexpected foreground activities happens in parallel while the script is executing which corrupts the Appium session then.

7.3. The command session creation (e.g. POST /wd/hub/session/) takes long time to receive the success response

The error in services.log is like below:

# this line means Device connect received create Appium session
4034 2021-12-07 22:52:40.8146 I/WebDriverController [148] [WebDriver fac0a226] Request POST /webdriver/session?: {"desiredCapabilities":{"app":null,"appUrl":null,"appVersionId":null,"autoWebview":true,"automationName":"Appium","browserName":"chrome","captureScreenshots":true,"deviceGroup":"ORGANIZATION","deviceName":"Pixel 5","deviceOrientation":"portrait","fullReset":false,"gigafox:existingSession":null,"kobiton-internal:app-binary-key":null,"kobiton-internal:app-download-url":null,"kobiton:session":"2027","networkActivity":false,"newCommandTimeout":null,"noReset":true,"packageName":null,"platformName":"Android","platformVersion":"11","sessionDescription":"","sessionName":"Not Automation test session","udid":"09261FDD400557"}}
# after a lot of long, we see the log line for response
4034 2021-12-07 22:53:05.9146 I/WebDriverController [148] [WebDriver fac0a226] Session created.
4034 2021-12-07 22:53:05.9146 I/WebDriverController [148] [WebDriver fac0a226] Response POST /webdriver/session?, {"sessionId":"fac0a226-bf42-4ad8-80fd-5f0224857f97","status":0,"value":{"app":null,"appUrl":null,"appVersionId":null,"autoWebview":true,"automationName":"Appium","browserName":"chrome","captureScreenshots":true,"deviceGroup":"ORGANIZATION","deviceName":"Google_09261FDD400557","deviceOrientation":"portrait","fullReset":false,"gigafox:existingSession":null,"kobiton-internal:app-binary-key":null,"kobiton-internal:app-download-url":null,"kobiton:session":"2027","networkActivity":false,"newCommandTimeout":null,"noReset":true,"packageName":null,"platformName":"Android","platformVersion":"11","ses... (+86b)

The middle of session create and response, we see like below:

4034 2021-12-07 22:52:41.1403 D/WebDriverSession [152] [WebDriver fac0a226] Browser launched.  PID 5975.
4034 2021-12-07 22:52:41.1403 D/SessionDevice [152] [WebDriver fac0a226] Agent request: GET /session/info.
4034 2021-12-07 22:52:41.1538 D/IOSDeviceController: 00008020 [126] Writing _forInstalldeviceConnect-Services-chpqr3ar.bxm from /tmp/deviceConnect-Services-chpqr3ar.bxm
4034 2021-12-07 22:52:41.1584 D/AirStreamHubService [170]  H15304: Ignoring registered application 'com.mobilelabsinc.deviceControl'
4034 2021-12-07 22:52:41.1711 D/SessionDevice [158] [WebDriver fac0a226] Agent response: GET /session/info, {"agentVersion":"9.0.9950.1129","application":"deviceControl","automationStatus":"ok","density":440,"deviceModel":"Pixel 5","deviceName":"Google","hasPrerotatedInput":true,"hasPrerotatedScreenshots":true,"logicalResolutionHeight":2340,"logicalResolutionWidth":1080,"nativeAutomation":true,"orientation":0,"protocolVersion":"1.1.1.0","scale":1.0,"screenHeight":2340.0,"screenWidth":1080.0,"sessionId":"sessionless","sidewaysOrientationReversed":false,"systemName":"AndroidOS","systemVersion":"11 (REL)"}.
4034 2021-12-07 22:52:41.1711 D/SessionDevice [158] [WebDriver fac0a226] Agent request: POST /session/setup, {"adjust_element_id":false}.
4034 2021-12-07 22:52:41.1742 D/IOSDeviceController: 00008101 [35] Writing _forInstalldeviceConnect-Services-fsvfu30h.j55 from /tmp/deviceConnect-Services-fsvfu30h.j55
4034 2021-12-07 22:52:41.1742 D/IOSDeviceController: 00008030 [169] Writing _forInstalldeviceConnect-Services-xp5hdq4j.hb5 from /tmp/deviceConnect-Services-xp5hdq4j.hb5
4034 2021-12-07 22:52:41.1742 D/IOSDeviceController: 00008101 [139] Writing _forInstalldeviceConnect-Services-fsvfu30h.j55 from /tmp/deviceConnect-Services-fsvfu30h.j55
4034 2021-12-07 22:52:41.1844 D/AndroidDeviceCleanup [155] [DeviceState d7793d7b] Set the orientation to PORTRAIT
4034 2021-12-07 22:52:41.1983 D/AirStreamHubService [32]  H15324: Ignoring registered application 'com.mobilelabsinc.deviceControl'
4034 2021-12-07 22:52:41.2188 D/SessionDevice [170] [WebDriver fac0a226] Agent response: POST /session/setup, {"status":"ok"}.
4034 2021-12-07 22:52:41.2209 D/AndroidDeviceCleanup [70] [DeviceState 09261FDD400557 Session ID 2027] Cleaning up with policies { "resetLocale": "en-US", "browsersToCleanup": [ "com.android.chrome", "com.chrome.beta", "org.mozilla.firefox", "org.mozilla.firefox_beta", "com.opera.browser", "com.opera.mini.native", "com.UCMobile.intl", "mobi.mgeek.TunnyBrowser", "com.android.browser", "com.sec.android.app.sbrowser", "com.asus.browser", "com.htc.sense.browser" ], "closeRecentApps": true, "removeInstalledApps": true, "removeInstalledAppData": true, "removeSignedInAccounts": true, "clearKeyboardCaches": true, "turnOnNavigationBar": true, "enableWifi": true, "resetGeoLocation": true, "resetTimeZone": true, "resetZoom": true }
4034 2021-12-07 22:52:41.2274 D/AndroidDeviceCleanup [98] [DeviceState d7793d7b] Remove new installed applications
4034 2021-12-07 22:52:41.2311 D/IOSDeviceController: c99ca122 [171] Writing _forInstalldeviceConnect-Services-33xuoopj.un5 from /tmp/deviceConnect-Services-33xuoopj.un5
4034 2021-12-07 22:52:41.2625 D/IOSDeviceController: 00008020 [163] Writing _forInstalldeviceConnect-Services-chpqr3ar.bxm from /tmp/deviceConnect-Services-chpqr3ar.bxm
4034 2021-12-07 22:52:41.3029 D/AndroidDeviceCleanup [163] [DeviceState d7793d7b] Clear browser caches
4034 2021-12-07 22:52:41.3091 D/AirStreamHubService [98]  H15326: Ignoring registered application 'com.mobilelabsinc.deviceControl'
4034 2021-12-07 22:52:41.3556 D/AirStreamHubService [169]  H15328: Ignoring registered application 'com.mobilelabsinc.deviceControl'
4034 2021-12-07 22:52:41.3738 D/AndroidUsbDeviceBusMonitor [35]  G30: Setting pid -1
4034 2021-12-07 22:52:41.3738 D/AndroidUsbDeviceBusMonitor [35]  G30: Setting verbosity Boolean: True
4034 2021-12-07 22:52:41.3768 D/IOSDeviceController: 00008020 [70] Writing _forInstalldeviceConnect-Services-chpqr3ar.bxm from /tmp/deviceConnect-Services-chpqr3ar.bxm
4034 2021-12-07 22:52:41.4215 I/ChromeDebugProtocol [94] Connecting to Chrome Debug Protocol virtual service on device 09261FDD400557, PID 0.
4034 2021-12-07 22:52:41.4256 D/IOSDeviceController: 00008020 [171] Writing _forInstalldeviceConnect-Services-chpqr3ar.bxm from /tmp/deviceConnect-Services-chpqr3ar.bxm
4034 2021-12-07 22:52:41.4489 D/IOSDeviceController: 00008101 [150] Installing PublicStaging/_forInstalldeviceConnect-Services-fsvfu30h.j55
4034 2021-12-07 22:52:41.4746 D/IOSDeviceController: 00008101 [35] Installing PublicStaging/_forInstalldeviceConnect-Services-fsvfu30h.j55
4034 2021-12-07 22:52:41.4986 D/IOSDeviceController: 00008020 [62] Installing PublicStaging/_forInstalldeviceConnect-Services-chpqr3ar.bxm
4034 2021-12-07 22:52:41.5048 D/IOSDeviceController: 00008101 [62] Installing PublicStaging/_forInstalldeviceConnect-Services-fsvfu30h.j55
4034 2021-12-07 22:52:41.5178 D/IOSDeviceController: 00008030 [51] Installing PublicStaging/_forInstalldeviceConnect-Services-xp5hdq4j.hb5
4034 2021-12-07 22:52:41.5523 D/IOSDeviceController: c99ca122 [139] Installing PublicStaging/_forInstalldeviceConnect-Services-33xuoopj.un5
4034 2021-12-07 22:52:41.5714 D/IOSDeviceController: 00008020 [66] Installing PublicStaging/_forInstalldeviceConnect-Services-chpqr3ar.bxm
4034 2021-12-07 22:52:41.6410 D/AndroidDeviceCleanup [51] [DeviceState RF8N82M4N0E] Reset device timezone
4034 2021-12-07 22:52:41.6729 D/AndroidDeviceCleanup [24] [DeviceState RF8N82M4N0E] Reset device geolocation
4034 2021-12-07 22:52:41.6825 I/ChromeDebugProtocol [146] Connecting to Chrome Debug Protocol virtual service on device 09261FDD400557, PID 0.
4034 2021-12-07 22:52:41.6831 D/IOSDeviceController: 00008020 [94] Installing PublicStaging/_forInstalldeviceConnect-Services-chpqr3ar.bxm
4034 2021-12-07 22:52:41.7061 D/IOSDeviceController: 00008020 [48] Installing PublicStaging/_forInstalldeviceConnect-Services-chpqr3ar.bxm
4034 2021-12-07 22:52:41.7229 I/IOSUsbDeviceBusMonitor [153]  G19: Launching 'com.mobilelabsinc.deviceControl
4034 2021-12-07 22:52:41.7244 I/IOSUsbDeviceBusMonitor [98]  G14: Launching 'com.mobilelabsinc.deviceControl'
4034 2021-12-07 22:52:41.7372 W/CambrionixHubService [48] Gigacap Airstream disconnected while requesting hubs?
4034 2021-12-07 22:52:41.7871 I/HealthService [158] [DeviceState RF8N82M4N0E] Clean up finished.
4034 2021-12-07 22:52:41.7871 I/HealthService [158] [DeviceState RF8N82M4N0E] Android 10 device is now ready for use.
4034 2021-12-07 22:52:41.8059 D/AndroidDeviceCleanup [163] [DeviceState d7793d7b] Remove signed in accounts
4034 2021-12-07 22:52:41.8519 I/IOSUsbDeviceBusMonitor [163]  G28: Launching 'com.mobilelabsinc.deviceControl'
4034 2021-12-07 22:52:41.8782 D/IOSDeviceController: 00008101 [158] Not mounting developer disk image because it appears to already be mounted.
4034 2021-12-07 22:52:41.8836 D/IOSDeviceController: 00008101 [152] Not mounting developer disk image because it appears to already be mounted.
4034 2021-12-07 22:52:41.9778 I/IOSUsbDeviceBusMonitor [48]  G25: Launching 'com.mobilelabsinc.deviceControl'
4034 2021-12-07 22:52:42.0138 D/IOSDeviceController: 00008101 [40] Not mounting developer disk image because it appears to already be mounted.
4034 2021-12-07 22:52:42.0187 W/HealthService [62] Skipping network health check due to unconfigured upstream health check address.
4034 2021-12-07 22:52:42.0408 D/IOSUsbDeviceBusMonitor [155]  G14: Environment [{'NSUnbufferedIO', 'YES'}, {'TRUST_LISTEN_PORT', '10151'}, {'TRUST_LAUNCH_APPLICATION', 'com.mobilelabsinc.deviceControl'}, {'TRUST_SESSION_ID', 'sessionless'}]
4034 2021-12-07 22:52:42.0408 D/InstrumentsClient [155] New InstrumentsClient<IOSDeviceContext>
4034 2021-12-07 22:52:42.0477 D/IOSUsbDeviceBusMonitor [40]  G19: Environment [{'NSUnbufferedIO', 'YES'}, {'TRUST_LISTEN_PORT', '10151'}, {'TRUST_LAUNCH_APPLICATION', 'com.mobilelabsinc.deviceControl'}, {'TRUST_SESSION_ID', 'sessionless'}]
4034 2021-12-07 22:52:42.0477 D/InstrumentsClient [40] New InstrumentsClient<IOSDeviceContext>
4034 2021-12-07 22:52:42.0693 I/IOSUsbDeviceBusMonitor [40]  G17: Launching 'com.mobilelabsinc.deviceControl'
4034 2021-12-07 22:52:42.0765 I/IOSUsbDeviceBusMonitor [163]  G8: Launching 'com.mobilelabsinc.deviceControl'
4034 2021-12-07 22:52:42.0919 D/AsyncService [66] Starting service 'InstrumentsClient.AsyncMessageService<00008101-000E6CA801A0001E>'.
4034 2021-12-07 22:52:42.0956 D/AsyncService [158] Starting service 'InstrumentsClient.AsyncMessageService<00008101-000A20382620001E>'.
4034 2021-12-07 22:52:42.1002 E/DisposalCounter [148] MobileLabs.DeviceConnect.Framework.DeviceControl.IOS.LockdownClient [id:4995 undisposed:55 leaked:0] Disposed with unknown ID. Double disposed?
4034 2021-12-07 22:52:42.1002 E/DisposalCounter [148] MobileLabs.DeviceConnect.Framework.DeviceControl.IOS.PlistServiceClient [id:7629 undisposed:59 leaked:0] Disposed with unknown ID. Double disposed?
4034 2021-12-07 22:52:42.1005 D/InstrumentsClient [148] New InstrumentsClient<IOSDeviceController Device Info Client>
4034 2021-12-07 22:52:42.1052 E/DisposalCounter [153] MobileLabs.DeviceConnect.Framework.DeviceControl.IOS.LockdownClient [id:4997 undisposed:56 leaked:0] Disposed with unknown ID. Double disposed?
4034 2021-12-07 22:52:42.1052 E/DisposalCounter [153] MobileLabs.DeviceConnect.Framework.DeviceControl.IOS.PlistServiceClient [id:7633 undisposed:60 leaked:0] Disposed with unknown ID. Double disposed?
4034 2021-12-07 22:52:42.1055 D/InstrumentsClient [153] New InstrumentsClient<IOSDeviceController Device Info Client>
4034 2021-12-07 22:52:42.1279 I/IOSUsbDeviceBusMonitor [169]  G21: Launching 'com.mobilelabsinc.deviceControl'
4034 2021-12-07 22:52:42.1848 D/AsyncService [163] Starting service 'InstrumentsClient.AsyncMessageService<00008101-000E6CA801A0001E>'.
4034 2021-12-07 22:52:42.1864 D/AsyncService [155] Stopping service 'InstrumentsClient.AsyncMessageService<00008101-000E6CA801A0001E>'...
4034 2021-12-07 22:52:42.1864 D/TestManagerRunService: 00008101 [155] Create
System.IO.EndOfStreamException: Attempted to read past the end of the stream.
  at MobileLabs.DeviceConnect.Framework.Extensions.StreamEx.ReadFully(Stream stream, Byte[] buf, Int32 offset, Int32 count, CancellationToken cancel) in /tmp/build/9674af12/ml2/src/deviceconnect/MobileLabs.DeviceConnect.Framework/Extensions/StreamEx.cs:line 30
  at MobileLabs.DeviceConnect.Framework.DeviceControl.IOS.InstrumentsClient.PacketHeader.Read(Stream stream, CancellationToken cancel) in /tmp/build/9674af12/ml2/src/deviceconnect/MobileLabs.DeviceConnect.Framework/DeviceControl/IOS/InstrumentsClient.Message.cs:line 170
  at MobileLabs.DeviceConnect.Framework.DeviceControl.IOS.InstrumentsClient.ReadMessage(CancellationToken cancel) in /tmp/build/9674af12/ml2/src/deviceconnect/MobileLabs.DeviceConnect.Framework/DeviceControl/IOS/InstrumentsClient.cs:line 368
  at MobileLabs.DeviceConnect.Framework.DeviceControl.IOS.InstrumentsClient.AsyncMessageService.Process(CancellationToken cancel) in /tmp/build/9674af12/ml2/src/deviceconnect/MobileLabs.DeviceConnect.Framework/DeviceControl/IOS/InstrumentsClient.cs:line 856
  at MobileLabs.DeviceConnect.Framework.Collections.AsyncService.Process(CancellationTokenSource cts) in /tmp/build/9674af12/ml2/src/deviceconnect/MobileLabs.DeviceConnect.Framework/Collections/AsyncService.cs:line 147
  at MobileLabs.DeviceConnect.Framework.DeviceControl.IOS.InstrumentsClient.CreateService[T](String serviceName, CancellationToken cancel) in /tmp/build/9674af12/ml2/src/deviceconnect/MobileLabs.DeviceConnect.Framework/DeviceControl/IOS/InstrumentsClient.cs:line 711
  at MobileLabs.DeviceConnect.Services.Gateway.IOS.TestManagerRunService.Create(CancellationToken cancel) in /tmp/build/9674af12/ml2/src/deviceconnect/MobileLabs.DeviceConnect.Services/Gateway/IOS/TestManagerRunService.cs:line 207
  at MobileLabs.DeviceConnect.Services.Gateway.IOS.TestManagerRunService.RestartCore(CancellationToken cancel) in /tmp/build/9674af12/ml2/src/deviceconnect/MobileLabs.DeviceConnect.Services/Gateway/IOS/TestManagerRunService.cs:line 123

4034 2021-12-07 22:52:42.1874 D/AsyncService [103] Starting service 'InstrumentsClient.AsyncMessageService<00008101-000A20382620001E>'.
4034 2021-12-07 22:52:42.1881 E/HealthService [155] [DeviceState 00008101-000E6CA801A0001E] Could not initialize deviceControl: Attempted to read past the end of the stream.
4034 2021-12-07 22:52:42.1881 D/InstrumentsClient [155] InstrumentsClient.AsyncMessageService<00008101-000E6CA801A0001E>: Connection to instruments service closed: EndOfStreamException
4034 2021-12-07 22:52:42.1881 D/AsyncService [155] Process terminated: 'InstrumentsClient.AsyncMessageService<00008101-000E6CA801A0001E>'.
4034 2021-12-07 22:52:42.1881 D/AsyncService [83] Stopping service 'InstrumentsClient.AsyncMessageService<00008101-000E6CA801A0001E>'...
4034 2021-12-07 22:52:42.1881 D/AsyncService [83] Process terminated: 'InstrumentsClient.AsyncMessageService<00008101-000E6CA801A0001E>'.
4034 2021-12-07 22:52:42.1892 D/AsyncService [150] Stopping service 'InstrumentsClient.AsyncMessageService<00008101-000A20382620001E>'...
4034 2021-12-07 22:52:42.1892 D/TestManagerRunService: 00008101 [150] Create
System.IO.EndOfStreamException: Attempted to read past the end of the stream.
  at MobileLabs.DeviceConnect.Framework.Extensions.StreamEx.ReadFully(Stream stream, Byte[] buf, Int32 offset, Int32 count, CancellationToken cancel) in /tmp/build/9674af12/ml2/src/deviceconnect/MobileLabs.DeviceConnect.Framework/Extensions/StreamEx.cs:line 30
  at MobileLabs.DeviceConnect.Framework.DeviceControl.IOS.InstrumentsClient.PacketHeader.Read(Stream stream, CancellationToken cancel) in /tmp/build/9674af12/ml2/src/deviceconnect/MobileLabs.DeviceConnect.Framework/DeviceControl/IOS/InstrumentsClient.Message.cs:line 170
  at MobileLabs.DeviceConnect.Framework.DeviceControl.IOS.InstrumentsClient.ReadMessage(CancellationToken cancel) in /tmp/build/9674af12/ml2/src/deviceconnect/MobileLabs.DeviceConnect.Framework/DeviceControl/IOS/InstrumentsClient.cs:line 368
  at MobileLabs.DeviceConnect.Framework.DeviceControl.IOS.InstrumentsClient.AsyncMessageService.Process(CancellationToken cancel) in /tmp/build/9674af12/ml2/src/deviceconnect/MobileLabs.DeviceConnect.Framework/DeviceControl/IOS/InstrumentsClient.cs:line 856
  at MobileLabs.DeviceConnect.Framework.Collections.AsyncService.Process(CancellationTokenSource cts) in /tmp/build/9674af12/ml2/src/deviceconnect/MobileLabs.DeviceConnect.Framework/Collections/AsyncService.cs:line 147
  at MobileLabs.DeviceConnect.Framework.DeviceControl.IOS.InstrumentsClient.CreateService[T](String serviceName, CancellationToken cancel) in /tmp/build/9674af12/ml2/src/deviceconnect/MobileLabs.DeviceConnect.Framework/DeviceControl/IOS/InstrumentsClient.cs:line 711
  at MobileLabs.DeviceConnect.Services.Gateway.IOS.TestManagerRunService.Create(CancellationToken cancel) in /tmp/build/9674af12/ml2/src/deviceconnect/MobileLabs.DeviceConnect.Services/Gateway/IOS/TestManagerRunService.cs:line 207
  at MobileLabs.DeviceConnect.Services.Gateway.IOS.TestManagerRunService.RestartCore(CancellationToken cancel) in /tmp/build/9674af12/ml2/src/deviceconnect/MobileLabs.DeviceConnect.Services/Gateway/IOS/TestManagerRunService.cs:line 123

4034 2021-12-07 22:52:42.1902 E/HealthService [150] [DeviceState 00008101-000A20382620001E] Could not initialize deviceControl: Attempted to read past the end of the stream.
4034 2021-12-07 22:52:42.1902 D/InstrumentsClient [150] InstrumentsClient.AsyncMessageService<00008101-000A20382620001E>: Connection to instruments service closed: EndOfStreamException
4034 2021-12-07 22:52:42.1902 D/AsyncService [150] Process terminated: 'InstrumentsClient.AsyncMessageService<00008101-000A20382620001E>'.
4034 2021-12-07 22:52:42.1902 D/AsyncService [103] Stopping service 'InstrumentsClient.AsyncMessageService<00008101-000A20382620001E>'...
4034 2021-12-07 22:52:42.1902 D/AsyncService [103] Process terminated: 'InstrumentsClient.AsyncMessageService<00008101-000A20382620001E>'.
4034 2021-12-07 22:52:42.1934 D/IOSUsbDeviceBusMonitor [153]  G28: Environment [{'NSUnbufferedIO', 'YES'}, {'TRUST_LISTEN_PORT', '10151'}, {'TRUST_LAUNCH_APPLICATION', 'com.mobilelabsinc.deviceControl'}, {'TRUST_SESSION_ID', 'sessionless'}]
4034 2021-12-07 22:52:42.1934 D/InstrumentsClient [153] New InstrumentsClient<IOSDeviceContext>
4034 2021-12-07 22:52:42.2152 W/HealthService [94] Skipping network health check due to unconfigured upstream health check address.
4034 2021-12-07 22:52:42.2199 D/SessionDevice [48] [WebDriver fac0a226] Agent request: POST /system/view/wait-for-idle, {"timeout":5000}.
4034 2021-12-07 22:52:42.2273 D/IOSDeviceController: 00008020 [48] Not mounting developer disk image because it appears to already be mounted.
4034 2021-12-07 22:52:42.2418 D/AsyncService [150] Starting service 'InstrumentsClient.AsyncMessageService<00008101-001C3CA63640001E>'.
4034 2021-12-07 22:52:42.2437 D/SessionDevice [163] [WebDriver fac0a226] Agent response: POST /system/view/wait-for-idle, {}.
4034 2021-12-07 22:52:42.2437 I/ChromeDebugProtocol [163] Connecting to Chrome Debug Protocol virtual service on device 09261FDD400557, PID 5975.
4034 2021-12-07 22:52:42.2538 E/DisposalCounter [170] MobileLabs.DeviceConnect.Framework.DeviceControl.IOS.LockdownClient [id:4998 undisposed:53 leaked:0] Disposed with unknown ID. Double disposed?
4034 2021-12-07 22:52:42.2538 E/DisposalCounter [170] MobileLabs.DeviceConnect.Framework.DeviceControl.IOS.PlistServiceClient [id:7635 undisposed:56 leaked:0] Disposed with unknown ID. Double disposed?
4034 2021-12-07 22:52:42.2538 D/InstrumentsClient [170] New InstrumentsClient<IOSDeviceController Device Info Client>
4034 2021-12-07 22:52:42.2677 D/WebDriverSession [153] [WebDriver fac0a226] No valid webviews found, waiting...


There are 02 possible reasons:
The middle of session create and response, we see like below: 

  • With the line No valid webviews found, waiting... , deviceConnect can’t create the Appium session yet and still waiting until 25 seconds later
  • Some foreground activities happens while the deviceConnect prepares for the new Appium session, let’s verify from the session video or command screenshot

8. Manual sessions fail to launch

 

The error is as below:Screen_Shot_2022-09-06_at_6.22.28_PM.png

Possibility #1: Verify the device is setup properly: 

iOS devices:

  • New iOS phones: X, XS, XS Max, 11, 12, and above.
  • Apple ID should be logged in on the devices
  • Follow the same guide as Kobiton device:iOS Setup and Troubleshooting
  • Turn off auto-lock: Go to Settings -> Display -> Auto-Lock and turn it off. 

Android devices:

  • New android phones: Galaxy S20, S21, Galaxy Note 10, and above.
  • Samsung ID/Google ID/ Xiaomi ID… should be set up for all Android devices to protect our devices
  • Follow the same guide as Kobiton device:Android Setup and Troubleshooting

Possibility #2: Verify that iOS doesn’t terminate the deviceControl

In services.log it’s like below. Note: 00008020-00141C603421002E is the device udid

D/AsyncService [97] Starting service 'InstrumentsClient.AsyncMessageService<00008020-00141C603421002E>'.

... ( a lot of normal message, no error on that device )

D/InstrumentsClient [44] InstrumentsClient.AsyncMessageService<00008020-00141C603421002E>: Connection to instruments service closed: EndOfStreamException

This means there is something wrong with the device while it’s running. We will look at the device log to find out what happened there. Go to Gigafox UI, pick the device and download its log (below screenshot).Screen_Shot_2022-09-06_at_6.24.54_PM.png

If the device log has the line similar as below, it means iOS kills the deviceControl app (the agent app) which will break the Manual flow as well. The resolution is to go to Gigafox and restart the device.

[2021-12-13 09:26:16.1342] Dec 13 09:26:15 iPhone runningboardd(RunningBoard)[29] <Notice>: Received termination request from [daemon<com.apple.mobile.installd>:49] on application<com.mobilelabsinc.deviceControl> with context <RBSTerminateContext: 0x102471290; code: 0xDEAD6502; explanation: FBSApplicationTerminationAssertion; reportType: 0; maxRole: 0; maxTerminationResistance: 3> attrs = {
   <RBSPreventLaunchLimitation: 0x1024597e0>;
}
Was this article helpful?
0 out of 0 found this helpful