MON-3775: [WIP] add tests for CMO collection profiles

[rexagod]

Refer: openshift/enhancements#1298

Signed-off-by: Pranshu Srivastava rexagod@gmail.com

[openshift-ci-robot]

@rexagod: This pull request references MON-3775 which is a valid jira issue.

Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the task to target the "4.16.0" version, but no target version was set.

In response to this:

The tests cover the following cases:

• Apply every collection profile, and check if appropriate metrics are
  available (or unavailable).
• CMO configuration reconciliation after a collection profile is applied
  is checked within the CMO tests itself.
• Prometheus *monitor selectors' reconciliation after a collection
  profile is applied is checked within the CMO tests itself.

Refer: openshift/enhancements#1298

Signed-off-by: Pranshu Srivastava rexagod@gmail.com

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

[openshift-ci]

Skipping CI for Draft Pull Request.
If you want CI signal for your change, please convert it to an actual PR.
You can still manually trigger a test run with /test all

[openshift-ci]

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: rexagod
Once this PR has been reviewed and has the lgtm label, please assign deads2k for approval. For more information see the Kubernetes Code Review Process.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

• OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[openshift-ci-robot]

@rexagod: This pull request references MON-3775 which is a valid jira issue.

Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the task to target the "4.16.0" version, but no target version was set.

In response to this:

The tests cover the following cases:

• Apply every collection profile, and check if the expected
  ServiceMonitors are available.
• CMO configuration reconciliation after a collection profile is applied
  is checked within the CMO tests itself.
• Prometheus *monitor selectors' reconciliation after a collection
  profile is applied is checked within the CMO tests itself.

Refer: openshift/enhancements#1298

Signed-off-by: Pranshu Srivastava rexagod@gmail.com

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

[openshift-trt-bot]

Job Failure Risk Analysis for sha: e449b02

Job Name

Failure Risk

pull-ci-openshift-origin-master-e2e-aws-ovn-single-node-serial

High [sig-network-edge][Feature:Idling] Unidling [apigroup:apps.openshift.io][apigroup:route.openshift.io] should handle many TCP connections by possibly dropping those over a certain bound [Serial] [Suite:openshift/conformance/serial] This test has passed 98.57% of 70 runs on jobs ['periodic-ci-openshift-release-master-nightly-4.17-e2e-aws-ovn-single-node-serial' 'periodic-ci-openshift-release-master-nightly-4.16-e2e-aws-ovn-single-node-serial'] in the last 14 days.

[rexagod]

Ah, nevermind, it seems my go switch never worked.

┌[rexagod@nebuchadnezzar] [/dev/ttys000] [3775] 
└[~/repositories/work/origin]> gvm use 1.21.11
go version go1.21.11 darwin/arm64
┌[rexagod@nebuchadnezzar] [/dev/ttys000] [3775] 
└[~/repositories/work/origin]> go version     
go version go1.22.3 darwin/arm64

EDIT: This still seems to be the case, looking into it.

┌[rexagod@nebuchadnezzar] [/dev/ttys005] [3775]
└[~/repositories/work/origin]> make verify -k --warn-undefined-variables # https://github.com/openshift/release/blob/master/ci-operator/config/openshift/origin/openshift-origin-master.yaml#L87C5-L87C46
Running `gofmt -s -l` on 716 file(s).
go vet -mod=vendor ./...
ln -s /Users/rexagod/repositories/work/origin/vendor "/var/folders/lt/fkdznpv57qjfcvm4p2psgcj00000gn/T/tmp.2ZwH3Y4PVb/src"
export GO111MODULE=off && export GOPATH=/var/folders/lt/fkdznpv57qjfcvm4p2psgcj00000gn/T/tmp.2ZwH3Y4PVb && export GOBIN=/var/folders/lt/fkdznpv57qjfcvm4p2psgcj00000gn/T/tmp.2ZwH3Y4PVb/bin && go install "./vendor/github.com/go-bindata/go-bindata/..."
/var/folders/lt/fkdznpv57qjfcvm4p2psgcj00000gn/T/tmp.2ZwH3Y4PVb/bin/go-bindata -nocompress -nometadata -prefix "testextended" -pkg "testdata" -o "/var/folders/lt/fkdznpv57qjfcvm4p2psgcj00000gn/T/tmp.KO8bkJDuge/test/extended/testdata/bindata.go" -ignore "OWNERS" -ignore ".*\.(go|md)$" examples/db-templates examples/image-streams examples/sample-app examples/quickstarts/... examples/hello-openshift examples/jenkins/... examples/quickstarts/cakephp-mysql.json test/extended/testdata/... e2echart && gofmt -s -w "/var/folders/lt/fkdznpv57qjfcvm4p2psgcj00000gn/T/tmp.KO8bkJDuge/test/extended/testdata/bindata.go" && diff -Naup {.,/var/folders/lt/fkdznpv57qjfcvm4p2psgcj00000gn/T/tmp.KO8bkJDuge}/test/extended/testdata/bindata.go
hack/verify-jsonformat.sh
hack/verify-generated.sh
SUCCESS after 33.000s: hack/verify-generated.sh:13: executing '/Users/rexagod/repositories/work/origin/hack/update-generated.sh' expecting success
SUCCESS after 0.000s: hack/verify-generated.sh:14: executing 'git diff --exit-code /Users/rexagod/repositories/work/origin/test/extended/util/annotate/generated/' expecting success
SUCCESS after 0.000s: hack/verify-generated.sh:15: executing 'git diff --exit-code /Users/rexagod/repositories/work/origin/test/extended/util/image/zz_generated.txt' expecting success
SUCCESS after 0.000s: hack/verify-generated.sh:16: executing 'git diff --exit-code /Users/rexagod/repositories/work/origin/zz_generated.manifests' expecting success
[INFO] hack/verify-generated.sh exited with code 0 after 00h 00m 34s
hack/verify-tls-ownership.sh
hack/verify-tls-ownership.sh took 9 seconds
┌[rexagod@nebuchadnezzar] [/dev/ttys005] [3775]
└[~/repositories/work/origin]> go version
go version go1.21.11 darwin/arm64

EDIT: I was able to finally reproduce this by generating everything again. I believe the artifacts generated using 1.22.x conflicted with the generating process of 1.21.x locally, since removing them and generating again seems to resolve the issue.

[openshift-trt-bot]

Job Failure Risk Analysis for sha: be65e96

Job Name	Failure Risk
pull-ci-openshift-origin-master-e2e-aws-ovn-ipsec-serial	High [sig-arch] events should not repeat pathologically for ns/openshift-authentication-operator This test has passed 100.00% of 55 runs on jobs ['periodic-ci-openshift-release-master-nightly-4.17-e2e-aws-ovn-serial' 'periodic-ci-openshift-release-master-ci-4.17-e2e-aws-ovn-serial'] in the last 14 days.

[rexagod]

Looking at a different CI error, but it seems that the line is already generated?

[openshift-trt-bot]

Job Failure Risk Analysis for sha: b356051

Job Name	Failure Risk
pull-ci-openshift-origin-master-e2e-aws-ovn-upgrade	IncompleteTests Tests for this run (18) are below the historical average (820): IncompleteTests (not enough tests ran to make a reasonable risk analysis; this could be due to infra, installation, or upgrade problems)
pull-ci-openshift-origin-master-e2e-aws-ovn-single-node-upgrade	IncompleteTests Tests for this run (17) are below the historical average (2132): IncompleteTests (not enough tests ran to make a reasonable risk analysis; this could be due to infra, installation, or upgrade problems)
pull-ci-openshift-origin-master-e2e-aws-ovn-ipsec-serial	IncompleteTests Tests for this run (18) are below the historical average (458): IncompleteTests (not enough tests ran to make a reasonable risk analysis; this could be due to infra, installation, or upgrade problems)

[openshift-trt-bot]

Job Failure Risk Analysis for sha: b356051

Job Name	Failure Risk
pull-ci-openshift-origin-master-e2e-aws-ovn-upgrade	IncompleteTests Tests for this run (18) are below the historical average (820): IncompleteTests (not enough tests ran to make a reasonable risk analysis; this could be due to infra, installation, or upgrade problems)
pull-ci-openshift-origin-master-e2e-aws-ovn-single-node-upgrade	IncompleteTests Tests for this run (17) are below the historical average (2132): IncompleteTests (not enough tests ran to make a reasonable risk analysis; this could be due to infra, installation, or upgrade problems)
pull-ci-openshift-origin-master-e2e-aws-ovn-ipsec-serial	IncompleteTests Tests for this run (18) are below the historical average (456): IncompleteTests (not enough tests ran to make a reasonable risk analysis; this could be due to infra, installation, or upgrade problems)

[openshift-trt-bot]

Job Failure Risk Analysis for sha: ade9ab4

Job Name	Failure Risk
pull-ci-openshift-origin-master-e2e-aws-ovn-serial	IncompleteTests Tests for this run (25) are below the historical average (871): IncompleteTests (not enough tests ran to make a reasonable risk analysis; this could be due to infra, installation, or upgrade problems)

[openshift-trt-bot]

Job Failure Risk Analysis for sha: ac09570

Job Name	Failure Risk
pull-ci-openshift-origin-master-e2e-metal-ipi-ovn-ipv6	IncompleteTests Tests for this run (23) are below the historical average (1562): IncompleteTests (not enough tests ran to make a reasonable risk analysis; this could be due to infra, installation, or upgrade problems)
pull-ci-openshift-origin-master-e2e-aws-ovn-upgrade	IncompleteTests Tests for this run (17) are below the historical average (809): IncompleteTests (not enough tests ran to make a reasonable risk analysis; this could be due to infra, installation, or upgrade problems)
pull-ci-openshift-origin-master-e2e-aws-ovn-single-node-serial	IncompleteTests Tests for this run (23) are below the historical average (805): IncompleteTests (not enough tests ran to make a reasonable risk analysis; this could be due to infra, installation, or upgrade problems)
pull-ci-openshift-origin-master-e2e-aws-ovn-ipsec-serial	IncompleteTests Tests for this run (27) are below the historical average (448): IncompleteTests (not enough tests ran to make a reasonable risk analysis; this could be due to infra, installation, or upgrade problems)

[openshift-trt-bot]

Job Failure Risk Analysis for sha: ac09570

Job Name	Failure Risk
pull-ci-openshift-origin-master-e2e-metal-ipi-ovn-ipv6	IncompleteTests Tests for this run (23) are below the historical average (1551): IncompleteTests (not enough tests ran to make a reasonable risk analysis; this could be due to infra, installation, or upgrade problems)
pull-ci-openshift-origin-master-e2e-aws-ovn-upgrade	IncompleteTests Tests for this run (17) are below the historical average (800): IncompleteTests (not enough tests ran to make a reasonable risk analysis; this could be due to infra, installation, or upgrade problems)
pull-ci-openshift-origin-master-e2e-aws-ovn-single-node-serial	IncompleteTests Tests for this run (23) are below the historical average (798): IncompleteTests (not enough tests ran to make a reasonable risk analysis; this could be due to infra, installation, or upgrade problems)
pull-ci-openshift-origin-master-e2e-aws-ovn-ipsec-serial	IncompleteTests Tests for this run (27) are below the historical average (448): IncompleteTests (not enough tests ran to make a reasonable risk analysis; this could be due to infra, installation, or upgrade problems)

[openshift-ci]

@rexagod: The following tests failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name	Commit	Details	Required	Rerun command
ci/prow/e2e-gcp-ovn-techpreview-serial	ac235e1	link	false	/test e2e-gcp-ovn-techpreview-serial
ci/prow/verify	24f7bb0	link	true	/test verify

Full PR test history. Your PR dashboard.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

[rexagod]

/test e2e-gcp-ovn-techpreview