Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[FailingTest] [sig-node] [NodeFeature:SidecarContainers] Containers Lifecycle should terminate sidecars simultaneously if prestop doesn't exit #124743

Closed
saschagrunert opened this issue May 8, 2024 · 5 comments · Fixed by cri-o/cri-o#8162
Closed

[FailingTest] [sig-node] [NodeFeature:SidecarContainers] Containers Lifecycle should terminate sidecars simultaneously if prestop doesn't exit #124743

saschagrunert opened this issue May 8, 2024 · 5 comments · Fixed by cri-o/cri-o#8162
Assignees
@saschagrunert
Labels
kind/failing-test Categorizes issue or PR as related to a consistently or frequently failing test. sig/node Categorizes an issue or PR as relevant to SIG Node. triage/accepted Indicates an issue or PR is ready to be actively worked on.
Projects
SIG Node CI/Test Board

Comments

@saschagrunert
Copy link
Member

saschagrunert commented May 8, 2024 •
edited

Which jobs are failing?

Multiple, for example:

Which tests are failing?

E2eNode Suite: [It] [sig-node] [NodeFeature:SidecarContainers] Containers Lifecycle should terminate sidecars simultaneously if prestop doesn't exit

Since when has it been failing?

Since e798b9c

Testgrid link

See above

Reason for failure (if possible)

The test tolerations are now to strict:

Expected
    <int64>: 29372
to be within 1000 of ~
    <int>: 32000
In [It] at: k8s.io/kubernetes/test/e2e_node/container_lifecycle_test.go:2876 @ 05/08/24 05:42:29.367
}

Anything else we need to know?

No response

Relevant SIG(s)

/sig node
@saschagrunert saschagrunert added the kind/failing-test Categorizes issue or PR as related to a consistently or frequently failing test. label May 8, 2024
@k8s-ci-robot k8s-ci-robot added the sig/node Categorizes an issue or PR as relevant to SIG Node. label May 8, 2024
@k8s-ci-robot
Copy link
Contributor

This issue is currently awaiting triage.

If a SIG or subproject determines this is a relevant issue, they will accept it by applying the triage/accepted label and provide further guidance.

The triage/accepted label can be added by org members by writing /triage accepted in a comment.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

@k8s-ci-robot k8s-ci-robot added the needs-triage Indicates an issue or PR lacks a `triage/foo` label and requires one. label May 8, 2024
@saschagrunert saschagrunert mentioned this issue May 8, 2024
Merged
@saschagrunert
Copy link
Member Author

cc @kubernetes/sig-node-cri-o-test-maintainers

@haircommander
Copy link
Contributor

I think it's from kubernetes/test-infra#32576
if you check the cri-o versions the succeessful one was efff37a2900e5039e5af554c3196633df25890dc and the failing was d519447937c4711db9ad3e783ab2b8121b39b996

saschagrunert added a commit to saschagrunert/cri-o that referenced this issue May 10, 2024
@saschagrunert
Kill exec PIDs after main container exited
fc46700 
Before applying this patch we killed the exec PIDs right away on
container stop which leads into the failing e2e test:

```
[sig-node] [NodeFeature:SidecarContainers] Containers Lifecycle should terminate sidecars simultaneously if prestop doesn't exit
```

This regression is now fixed by killing the exec PIDs after the main
container as well as in the same thread.

Fixes kubernetes/kubernetes#124743
Follow-up on cri-o#7937

Needs a cherry-pick since the enhancement got already backported into
supported release branches.

Signed-off-by: Sascha Grunert <sgrunert@redhat.com>
@saschagrunert saschagrunert mentioned this issue May 10, 2024
Merged
@saschagrunert saschagrunert self-assigned this May 10, 2024
@saschagrunert saschagrunert added triage/accepted Indicates an issue or PR is ready to be actively worked on. and removed needs-triage Indicates an issue or PR lacks a `triage/foo` label and requires one. labels May 10, 2024
@saschagrunert
Copy link
Member Author

Working on a fix in cri-o/cri-o#8162

@pacoxu pacoxu added this to Triage in SIG Node CI/Test Board May 14, 2024
@hshiina hshiina mentioned this issue May 14, 2024
Open
@gjkim42
Copy link
Member

gjkim42 commented May 14, 2024

cc

openshift-cherrypick-robot pushed a commit to openshift-cherrypick-robot/cri-o that referenced this issue May 15, 2024
@saschagrunert
Kill exec PIDs after main container exited
ea87bd6 
Before applying this patch we killed the exec PIDs right away on
container stop which leads into the failing e2e test:

```
[sig-node] [NodeFeature:SidecarContainers] Containers Lifecycle should terminate sidecars simultaneously if prestop doesn't exit
```

This regression is now fixed by killing the exec PIDs after the main
container as well as in the same thread.

Fixes kubernetes/kubernetes#124743
Follow-up on cri-o#7937

Needs a cherry-pick since the enhancement got already backported into
supported release branches.

Signed-off-by: Sascha Grunert <sgrunert@redhat.com>
openshift-cherrypick-robot pushed a commit to openshift-cherrypick-robot/cri-o that referenced this issue May 15, 2024
@saschagrunert
Kill exec PIDs after main container exited
84fc117 
Before applying this patch we killed the exec PIDs right away on
container stop which leads into the failing e2e test:

```
[sig-node] [NodeFeature:SidecarContainers] Containers Lifecycle should terminate sidecars simultaneously if prestop doesn't exit
```

This regression is now fixed by killing the exec PIDs after the main
container as well as in the same thread.

Fixes kubernetes/kubernetes#124743
Follow-up on cri-o#7937

Needs a cherry-pick since the enhancement got already backported into
supported release branches.

Signed-off-by: Sascha Grunert <sgrunert@redhat.com>
openshift-cherrypick-robot pushed a commit to openshift-cherrypick-robot/cri-o that referenced this issue May 15, 2024
@saschagrunert
Kill exec PIDs after main container exited
f350711 
Before applying this patch we killed the exec PIDs right away on
container stop which leads into the failing e2e test:

```
[sig-node] [NodeFeature:SidecarContainers] Containers Lifecycle should terminate sidecars simultaneously if prestop doesn't exit
```

This regression is now fixed by killing the exec PIDs after the main
container as well as in the same thread.

Fixes kubernetes/kubernetes#124743
Follow-up on cri-o#7937

Needs a cherry-pick since the enhancement got already backported into
supported release branches.

Signed-off-by: Sascha Grunert <sgrunert@redhat.com>
openshift-cherrypick-robot pushed a commit to openshift-cherrypick-robot/cri-o that referenced this issue May 15, 2024
@saschagrunert
Kill exec PIDs after main container exited
bab13b4 
Before applying this patch we killed the exec PIDs right away on
container stop which leads into the failing e2e test:

```
[sig-node] [NodeFeature:SidecarContainers] Containers Lifecycle should terminate sidecars simultaneously if prestop doesn't exit
```

This regression is now fixed by killing the exec PIDs after the main
container as well as in the same thread.

Fixes kubernetes/kubernetes#124743
Follow-up on cri-o#7937

Needs a cherry-pick since the enhancement got already backported into
supported release branches.

Signed-off-by: Sascha Grunert <sgrunert@redhat.com>
openshift-cherrypick-robot pushed a commit to openshift-cherrypick-robot/cri-o that referenced this issue May 15, 2024
@saschagrunert
Kill exec PIDs after main container exited
48e4681 
Before applying this patch we killed the exec PIDs right away on
container stop which leads into the failing e2e test:

```
[sig-node] [NodeFeature:SidecarContainers] Containers Lifecycle should terminate sidecars simultaneously if prestop doesn't exit
```

This regression is now fixed by killing the exec PIDs after the main
container as well as in the same thread.

Fixes kubernetes/kubernetes#124743
Follow-up on cri-o#7937

Needs a cherry-pick since the enhancement got already backported into
supported release branches.

Signed-off-by: Sascha Grunert <sgrunert@redhat.com>
openshift-cherrypick-robot pushed a commit to openshift-cherrypick-robot/cri-o that referenced this issue May 15, 2024
@saschagrunert
Kill exec PIDs after main container exited
ed33c3e 
Before applying this patch we killed the exec PIDs right away on
container stop which leads into the failing e2e test:

```
[sig-node] [NodeFeature:SidecarContainers] Containers Lifecycle should terminate sidecars simultaneously if prestop doesn't exit
```

This regression is now fixed by killing the exec PIDs after the main
container as well as in the same thread.

Fixes kubernetes/kubernetes#124743
Follow-up on cri-o#7937

Needs a cherry-pick since the enhancement got already backported into
supported release branches.

Signed-off-by: Sascha Grunert <sgrunert@redhat.com>
SIG Node CI/Test Board automation moved this from Triage to Done May 15, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@saschagrunert saschagrunert

Labels
kind/failing-test Categorizes issue or PR as related to a consistently or frequently failing test. sig/node Categorizes an issue or PR as relevant to SIG Node. triage/accepted Indicates an issue or PR is ready to be actively worked on.
Projects
SIG Node CI/Test Board
Done
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Kill exec PIDs after main container exited saschagrunert/cri-o
4 participants
@saschagrunert @haircommander @gjkim42 @k8s-ci-robot