Skip to content

Conversation

@mikehardy
Copy link
Collaborator

@mikehardy mikehardy commented Jan 29, 2026

Description

Our Simulator runtimes were just deprecated and removed: actions/runner-images#13570

Also, there are intermittent failures on iOS runners related to the build failing which seem to be related to simulators not being consistently available. Works most of the time, fails sometimes.

Per discussion in the upstream repo as linked in the comment in this diff, simply listing the simulators as a step before the ones that fail may be a viable workaround, although there are many indications that it does not 100% fix the issue.

Notes:
1- we may need to download the runtime but this may apparently add a lot of CI time actions/runner-images#12758 (comment)
2- it may be that the runtime already exists but is simply not loaded, and reloading it may be done without adding too much CI time - https://github.com/getsentry/sentry-cocoa/pull/6837/changes#diff-0e892ebad2a2b4f630bce02ab66a0a7ead71801904e24dd1cc30b94ce907e598
3- it may be that we still need to create the simulator even then - example https://github.com/siteline/swiftui-introspect/pull/482/changes
4- it may be that switching from a -destination <simulator name> in build to a -target <iphoneos> and using any simulator would work best though there are reports that there are zero simulators available at times [edit: we already do this]

Of those options,

  • 1 appears to be overkill but is also easy to implement
  • 2 appears to be supported by evidence and the most lightweight solution. It's in use by Sentry. But it only helps if the runtime already exists, and in our case it does not
  • 3 appears unnecessary if we know a simulator is supposed to exist based on installed software and only intermittent failure.
  • 4 is done but doesn't help,

So, option 1 it is - I download the runtime now if it is not available. I skip the download if it isn't.

This adds ~5mins to CI but lets us whatever runtime we want.

This is important because the current runtimes - 26.2 - are awful. So slow with performance problems that they flake all the time. 26.0.1 was the last one that performed decently. But in order to use it we have to download the runtime now.

👀 - while I was in there I tweaked coverage as well. It is not useful to have codecov project level stats give us a failing CI run status when we don't gate PRs on that status. So I set it to informational mode. I want to see the coverage, but not have it fail the status checks.

Related issues

Release Summary

All test

Checklist

  • I read the Contributor Guide and followed the process outlined there for submitting PRs.
    • Yes
  • My change supports the following platforms;
    • Android
    • iOS
    • Other (macOS, web)
  • My change includes tests;
    • e2e tests added or updated in packages/\*\*/e2e
    • jest tests added or updated in packages/\*\*/__tests__
  • I have updated TypeScript types that are affected by my change.
  • This is a breaking change;
    • Yes
    • No

Test Plan

This was developed while watching the iOS e2e test carefully.

I'll run a 30-iteration flake hammer on it as well to make sure it didn't regress from prior to the runtimes disappearing


Think react-native-firebase is great? Please consider supporting the project with any of the below:

@vercel
Copy link

vercel bot commented Jan 29, 2026

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
react-native-firebase Ready Ready Preview, Comment Jan 31, 2026 0:30am

Request Review

@mikehardy
Copy link
Collaborator Author

Indeed, that did not work.

xcodebuild: error: Found no destinations for the scheme 'testing' and action build.

Will try the runtime reloading script next

@russellwheatley
Copy link
Member

russellwheatley commented Jan 30, 2026

@mikehardy - this worked for my callable stream PR: 9fce0be

edit- I should add that I have only ran once so maybe it's intermittent like existing.

@mikehardy
Copy link
Collaborator Author

@russellwheatley that will help because as I understand it you will now be in sync with the current installed tuples of Xcode/SDK/Simulator-Runtime

But it still may be intermittent on Runtime availability, or on performance of the runtime (the current ones have problems and caused sim boot / timeout flakes - which is why I pinned Xcode to a lower version before).

If we want to stray at all from the recently-shrunk officially supported version windows of the Xcode/SDK/Runtime tuple then we'll have the problem again

I'm looking for something more like what sentry implemented where they can specify arbitrary tuples and it works with no availability flakes

...then hopefully the performance flakes in the newer runtimes are fixed (apparently Apple is working on it, per note from Github runner image maintainers)

It's all just messy.

github actions runners do not always have the SimRuntime installed
for the Xcode version we select, for disk space reasons

If you are using latest-stable that won't affect you but if you stray
from that version at all it might

Detect this case and install the runtime if needed
simulator performance gets worse with each macOS/iOS version
@mikehardy mikehardy force-pushed the @mikehardy/workaround-ci-simulator-unavailability branch from 4829612 to eadb033 Compare January 30, 2026 23:54
maintain comments showing coverage, maintain fail on patch coverage
issues, but the whole project fails frequently because uploads don't
send or aren't processed - those have little value at this point
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

3 participants