Skip to content

test: reset ExternalIDGenServiceMock between tests to fix flaky external-state ITs#3387

Merged
csviri merged 2 commits into
operator-framework:mainfrom
Dennis-Mircea:test/reset-external-id-gen-service-mock
May 29, 2026
Merged

test: reset ExternalIDGenServiceMock between tests to fix flaky external-state ITs#3387
csviri merged 2 commits into
operator-framework:mainfrom
Dennis-Mircea:test/reset-external-id-gen-service-mock

Conversation

@Dennis-Mircea
Copy link
Copy Markdown
Contributor

@Dennis-Mircea Dennis-Mircea commented May 28, 2026

Summary

Fixes an intermittent CI failure in three integration tests (an example can be found here: https://github.com/operator-framework/java-operator-sdk/actions/runs/26563951370/job/78253684721?pr=3383):

  • ExternalStateIT.reconcilesResourceWithPersistentState
  • ExternalStateDependentIT.reconcilesResourceWithPersistentState (extends ExternalStateTestBase)
  • ExternalStateBulkIT.reconcilesResourceWithPersistentState

Symptoms in CI:

Expected size: 1 but was: 2 in:
[io.javaoperatorsdk.operator.support.ExternalResource@4f234290,
 io.javaoperatorsdk.operator.support.ExternalResource@3370ff56] within 10 seconds.

The same ExternalResource@3370ff56 instance hash showed up in all three failures, which is the giveaway: an ExternalResource object was outliving the test that created it.

Root cause

operator-framework/src/test/java/io/javaoperatorsdk/operator/support/ExternalIDGenServiceMock.java is a JVM-lifetime singleton with a mutable ConcurrentHashMap field:

public class ExternalIDGenServiceMock {
  private static final ExternalIDGenServiceMock serviceMock = new ExternalIDGenServiceMock();
  private final Map<String, ExternalResource> resourceMap = new ConcurrentHashMap<>();
  // ...
  public static ExternalIDGenServiceMock getInstance() { return serviceMock; }
}

Each external-state test creates resources through getInstance().create(...) (entered into resourceMap) and expects the reconciler's cleanup path to remove them after the custom resource is deleted. Cleanup is asynchronous, so when the next test runs there is a race window in which the previous test's ExternalResource is still in the map. The next test creates its own resource, sees two entries, and Awaitility times out after 10 s.

Fix

Three small, test-only changes:

  1. Add a reset() method to ExternalIDGenServiceMock that clears the internal map.
  2. Call it from a @BeforeEach in ExternalStateTestBase (covers both ExternalStateIT and ExternalStateDependentIT).
  3. Call it from a @BeforeEach in ExternalStateBulkIT (which does not extend the base).

Concurrency note

JOSDK does not enable JUnit 5 parallel execution (no junit-platform.properties, no junit.jupiter.execution.parallel.* properties in any pom.xml). The flake addressed here is a sequential test-ordering issue, not a parallel-execution one, and the BeforeEachCallback is sufficient given that constraint. If parallel execution is ever enabled, the singleton would need to be replaced by per-test instances rather than relying on reset().

Test plan

  • mvn -pl operator-framework verify -Dit.test=ExternalStateIT,ExternalStateDependentIT,ExternalStateBulkIT passes locally on a clean checkout.
  • No other tests use ExternalIDGenServiceMock.

Copilot AI review requested due to automatic review settings May 28, 2026 09:36
@openshift-ci openshift-ci Bot requested review from csviri and xstefank May 28, 2026 09:36
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Note

Copilot was unable to run its full agentic suite in this review.

Adds explicit reset hooks to a JVM-lifetime singleton test mock and invokes them before each test to prevent cross-test state leakage.

Changes:

  • Added reset() to ExternalIDGenServiceMock to clear internal state.
  • Added @BeforeEach setup in external-state tests to reset the singleton before every test run.

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 2 comments.

File Description
operator-framework/src/test/java/io/javaoperatorsdk/operator/support/ExternalIDGenServiceMock.java Adds a reset() method to clear the singleton mock’s internal state between tests.
operator-framework/src/test/java/io/javaoperatorsdk/operator/dependent/externalstate/externalstatebulkdependent/ExternalStateBulkIT.java Resets the singleton mock before each test to avoid state leakage.
operator-framework/src/test/java/io/javaoperatorsdk/operator/dependent/externalstate/ExternalStateTestBase.java Resets the singleton mock before each test in the shared base test class.

Comment on lines +39 to +42
@BeforeEach
void resetExternalService() {
externalService.reset();
}
Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Centralised the reset into ExternalServiceResetExtension (BeforeEachCallback) and dropped the duplicated @BeforeEach blocks. Both ExternalStateTestBase and ExternalStateBulkIT now declare @ExtendWith(ExternalServiceResetExtension.class).

@xstefank
Copy link
Copy Markdown
Collaborator

Hi @Dennis-Mircea, thank you for the PR. Please run the spotless plugin to format the code in order for CI to pass.

@Dennis-Mircea
Copy link
Copy Markdown
Contributor Author

Hi @Dennis-Mircea, thank you for the PR. Please run the spotless plugin to format the code in order for CI to pass.

Resolved!

…nal-state ITs

Signed-off-by: Dennis-Mircea Ciupitu <dennis.mircea.ciupitu@gmail.com>
Signed-off-by: Dennis-Mircea Ciupitu <dennis.mircea.ciupitu@gmail.com>
@Dennis-Mircea Dennis-Mircea force-pushed the test/reset-external-id-gen-service-mock branch from 0a6a5fe to 233cb9f Compare May 29, 2026 07:53
Copy link
Copy Markdown
Collaborator

@csviri csviri left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

thank you @Dennis-Mircea

@csviri csviri merged commit 442b0cb into operator-framework:main May 29, 2026
46 of 47 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants