[Benchmark Issue]IPHONE MAX PRO 15 failure investigation [May 22 2025]

[yangw-dev]

🐛 Describe the bug

Dug doc: https://docs.google.com/document/d/1ykNbU0p_1X-U1_kJH4OD4l7meBlX0p6AcklChy8zURk/edit?tab=t.0

Issue details

Missing the benchmark result for iPhone 15 Pro Max (private) since 5/10: hud link

Investigate

The ios private job fail since two weeks ago,
Example: https://github.com/pytorch/executorch/actions/workflows/apple-perf-private-device-experiment.yml?page=4

Some jobs are auto cancelled for unknown reason

When investigating the family farm, it seems like the Iphone 15 Pro max failed at setup suite step, with error such as logs .

Potential Issues & next steps

Yang Wang May 22, 2025
It seems like this might be the setup issue on AWS side, need to contact the customer support to see what is the root cause

Error logs from aws device farm:

https://www.internalfb.com/phabricator/paste/view/P1819807342

Versions

issue for investigation tracking

cc @guangy10 @huydhn @kirklandsign @shoumikhin

[huydhn]

As it looks like the device is having issue, I have removed it out of the pool. Let me submit a high priority issue to AWS support on this

[huydhn]

I'm chatting with AWS support and the case has been escalated to Device Farm service team