Skip to content

tests: Run tagged tests with --changed when shared contexts change - #22471

Merged
issyl0 merged 2 commits into
mainfrom
brew-tests-changed-understands-test-helpers
May 30, 2026
Merged

tests: Run tagged tests with --changed when shared contexts change#22471
issyl0 merged 2 commits into
mainfrom
brew-tests-changed-understands-test-helpers

Conversation

@issyl0

@issyl0 issyl0 commented May 30, 2026

Copy link
Copy Markdown
Member

  • Have you followed the guidelines in our Contributing document?
  • Have you checked to ensure there aren't other open Pull Requests for the same change?
  • Have you added an explanation of what your changes do and why you'd like us to include them? Performance claims (e.g. "this is faster") must include Hyperfine benchmarks.
  • Have you written new tests (excluding integration tests) for your changes? Here's an example.
  • Have you successfully run brew lgtm (style, typechecking and tests) with your changes locally?

  • AI was used to generate or assist with generating this PR.
  • Assisted by GPT-5.3 Codex (Medium reasoning). Human review, tests and tweaks.

  • When a shared context file changes, the tests that include that shared context should be run when brew tests --changed is invoked. This ensures that relevant test failures are caught.
  • I found this bug while working on sorbet: Bump test/support/helper files to typed: true #22469 where I changed shared contexts, ran brew lgtm which ran brew tests --changed which lulled me into a false sense of security that the changes were fine, until CI ran the full test suite1.
  • Add some standard behaviour tests for brew tests --changed as well as there was zero coverage before.
- When a shared context file changes, the tests that include that shared
  context should be run when `brew tests --changed` is invoked. This
  ensures that relevant test failures are caught.
- I found this bug while working on PR 22469 where I changed shared
  contexts, ran `brew lgtm` which ran `brew tests --changed` which
  lulled me into a false sense of security that the changes were fine,
  until CI ran the full test suite[1].
- Add some standard behaviour tests for `brew tests --changed` as well as
  there was zero coverage before.

[1]: #22469 (comment)
@github-code-quality

github-code-quality Bot commented May 30, 2026

Copy link
Copy Markdown

Code Coverage Overview

Languages: Ruby

Ruby / code-coverage/simplecov

The overall coverage remains at 78%, unchanged from the main branch.


Updated May 30, 2026 18:46 UTC
Code Coverage is in Public Preview. Learn more and provide us with your feedback.

@issyl0
issyl0 marked this pull request as ready for review May 30, 2026 16:08
Copilot AI review requested due to automatic review settings May 30, 2026 16:08

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR updates brew tests --changed so changes to shared RSpec context files trigger related tagged specs, improving confidence when shared test helpers are modified.

Changes:

  • Converts changed-test resolution to support one changed file mapping to multiple specs.
  • Adds mappings for integration_test and homebrew_cask shared contexts.
  • Adds specs covering direct spec changes, source-to-spec mapping, and shared-context expansion.
Show a summary per file
File Description
Library/Homebrew/dev-cmd/tests.rb Expands --changed test selection for shared context files based on tags.
Library/Homebrew/test/dev-cmd/tests_spec.rb Adds coverage for changed_test_files behavior.

Copilot's findings

Tip

Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

  • Files reviewed: 2/2 changed files
  • Comments generated: 1
Comment thread Library/Homebrew/dev-cmd/tests.rb Outdated
issyl0 added a commit that referenced this pull request May 30, 2026
- Avoid matching on arbitrary symbol usage (e.g. let(:cask)), per
  #22471 (comment).
@issyl0
issyl0 enabled auto-merge May 30, 2026 16:41

@MikeMcQuaid MikeMcQuaid left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks! One comment, don't need rereview after.

Comment thread Library/Homebrew/dev-cmd/tests.rb Outdated
- Avoid matching on arbitrary symbol usage (e.g. let(:cask)), per
  #22471 (comment).
@issyl0
issyl0 force-pushed the brew-tests-changed-understands-test-helpers branch from a81ebb9 to 4ed673a Compare May 30, 2026 18:25
@issyl0
issyl0 added this pull request to the merge queue May 30, 2026
Merged via the queue into main with commit f15fc02 May 30, 2026
38 checks passed
@issyl0
issyl0 deleted the brew-tests-changed-understands-test-helpers branch May 30, 2026 19:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

3 participants