-
Notifications
You must be signed in to change notification settings - Fork 2.3k
test(ios): workaround simulator availability issues #8865
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Conversation
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
|
Indeed, that did not work.
Will try the runtime reloading script next |
|
@mikehardy - this worked for my callable stream PR: 9fce0be edit- I should add that I have only ran once so maybe it's intermittent like existing. |
|
@russellwheatley that will help because as I understand it you will now be in sync with the current installed tuples of Xcode/SDK/Simulator-Runtime But it still may be intermittent on Runtime availability, or on performance of the runtime (the current ones have problems and caused sim boot / timeout flakes - which is why I pinned Xcode to a lower version before). If we want to stray at all from the recently-shrunk officially supported version windows of the Xcode/SDK/Runtime tuple then we'll have the problem again I'm looking for something more like what sentry implemented where they can specify arbitrary tuples and it works with no availability flakes ...then hopefully the performance flakes in the newer runtimes are fixed (apparently Apple is working on it, per note from Github runner image maintainers) It's all just messy. |
dff754c to
a38df91
Compare
a38df91 to
21cf08b
Compare
2a8bb92 to
335b64e
Compare
53b86b5 to
13d2c22
Compare
13d2c22 to
4829612
Compare
github actions runners do not always have the SimRuntime installed for the Xcode version we select, for disk space reasons If you are using latest-stable that won't affect you but if you stray from that version at all it might Detect this case and install the runtime if needed
simulator performance gets worse with each macOS/iOS version
4829612 to
eadb033
Compare
maintain comments showing coverage, maintain fail on patch coverage issues, but the whole project fails frequently because uploads don't send or aren't processed - those have little value at this point
eadb033 to
a28c4d5
Compare
Description
Our Simulator runtimes were just deprecated and removed: actions/runner-images#13570
Also, there are intermittent failures on iOS runners related to the build failing which seem to be related to simulators not being consistently available. Works most of the time, fails sometimes.
Per discussion in the upstream repo as linked in the comment in this diff, simply listing the simulators as a step before the ones that fail may be a viable workaround, although there are many indications that it does not 100% fix the issue.
Notes:
1- we may need to download the runtime but this may apparently add a lot of CI time actions/runner-images#12758 (comment)
2- it may be that the runtime already exists but is simply not loaded, and reloading it may be done without adding too much CI time - https://github.com/getsentry/sentry-cocoa/pull/6837/changes#diff-0e892ebad2a2b4f630bce02ab66a0a7ead71801904e24dd1cc30b94ce907e598
3- it may be that we still need to create the simulator even then - example https://github.com/siteline/swiftui-introspect/pull/482/changes
4-
it may be that switching from a[edit: we already do this]-destination <simulator name>in build to a-target <iphoneos>and using any simulator would work best though there are reports that there are zero simulators available at timesOf those options,
So, option 1 it is - I download the runtime now if it is not available. I skip the download if it isn't.
This adds ~5mins to CI but lets us whatever runtime we want.
This is important because the current runtimes - 26.2 - are awful. So slow with performance problems that they flake all the time. 26.0.1 was the last one that performed decently. But in order to use it we have to download the runtime now.
👀 - while I was in there I tweaked coverage as well. It is not useful to have codecov project level stats give us a failing CI run status when we don't gate PRs on that status. So I set it to informational mode. I want to see the coverage, but not have it fail the status checks.
Related issues
Release Summary
All test
Checklist
AndroidiOSOther(macOS, web)e2etests added or updated inpackages/\*\*/e2ejesttests added or updated inpackages/\*\*/__tests__Test Plan
This was developed while watching the iOS e2e test carefully.
I'll run a 30-iteration flake hammer on it as well to make sure it didn't regress from prior to the runtimes disappearing
Think
react-native-firebaseis great? Please consider supporting the project with any of the below:React Native FirebaseandInvertaseon Twitter