ci: flakey tests in BrowserStack

[gkalpak]

The tests on BrowserStack have been extra flakey lately. Most of the times, it is iPhone 5S on iOS 7.0 that fails to be captured. This seems inline with what @mgol suggests in angular/angular.js#15892 (that BrowserStack does no longer support iOS 7.0).

So, I assume that removing this line and possibly updating browserstack-tunnel to the latest version would make the flakey-ness go away.

[mgol]

I was talking about 8.0, not 7.0. BrowserStack often updates versions to the latest minor/patch on a given major so they e.g. replaced 8.0 with 8.3 so requesting 8.0 fails. iOS 7.0 is still in the API, though.

[mgol]

Angular config uses tunnelIdentifier for BrowserStack which no longer works - old browserstacktunnel-wrapper config options that changed in the newest version won't work even when browserstacktunnel-wrapper is not updated as the parameters in the BrowserStack API changed and they don't maintain the old one.

browserstacktunnel-wrapper should be updated, indeed and tunnelIdentifier should be changerd to localIdentifier; not sure if that changes anything wrt. to iOS 7.0 failures, though.

[gkalpak]

I was talking about 8.0, not 7.0. BrowserStack often updates versions to the latest minor/patch on a given major so they e.g. replaced 8.0 with 8.3 so requesting 8.0 fails. iOS 7.0 is still in the API, though.

Oh, I see. Thx for clarifying.

[mgol]

I run AngularJS (1.x) tests in iOS 7.0, 8.3, 9.3 & 10.0 via BrowserStack and 8.3, 9.3 & 10.0 finished successfully while 7.0 was timing out constantly.

It seems this browser is just broken on BrowserStack right now. I suggest either trying to use it via Sauce Labs or contacting BrowerStack about the problem. Or just dropping it, its market share is miniscule right now (it was <1% of all iOS versions a few months ago).

[sinstein]

Hi @gkalpak, @mgol could you share a Travis/CircleCI build where you could observe this behaviour?

I set up the angular test suite on Travis(Check builds 9 - 11) and ran a whole bunch of sessions on iOS7 on BrowserStack and did not face any issue of this kind. Is there a different setup you are using?

[gkalpak]

@sinstein, you can see one occurrence here, but it is admittedly happening much less often than before.

[sinstein]

@gkalpak We made some changes to the IOS7 platform that should take care of this issue for good now. Could you please verify and let me know if things work fine at your end?

[gkalpak]

@sinstein, I haven't seen the job flaking in a while, so I assume it's all good now 😃
Thanks for looking into it ❤️

[angular-automatic-lock-bot]

This issue has been automatically locked due to inactivity.
Please file a new issue if you are encountering a similar or related problem.

Read more about our automatic conversation locking policy.

_{This action has been performed automatically by a bot.}