Commits on May 30, 2024

1. container: always wait for containerd's deletion API call …

   On a heavily loaded host, we were experiencing long container(d) deletion
   times:
   
       containerd[8907]: time="2024-03-25T13:47:45.938479195Z" level=debug msg="event forwarded" ns=moby topic=/tasks/exit type=containerd.events.TaskExit
       # our control plane logic deletes the successfully exited container via
       # the docker API, and...
       containerd[8907]: time="2024-03-25T13:47:47.202055216Z" level=debug msg="failed to delete task" error="context deadline exceeded" id=a618057629b35e3bfea82d5ce4cbb057ba979498496428dfe6935a1322b94add
   
   Before 4bafaa0 ("Refactor libcontainerd to minimize c8d RPCs") when
   this happens, the docker API reports a 255 exit code and no error:
   
       0a7ddd027c0497d5a titus-executor-[900884]: Processing msg from a docker: main container exited with code 255
   
   which is especially confusing. After 4bafaa0, the behavior has changed
   to report the container's real exit code, although there is still a hard
   coded timeout after which containerd will (try to) stop cleaning up.
   
   After some discussion, it seems best to explicitly wait for as long as this
   takes so we do not leak containerd resources. So let's change this 30s
   timeout to wait forever.
   
   Reported-by: Hechao Li <hli@netflix.com>
   Signed-off-by: Tycho Andersen <tandersen@netflix.com>

   

   tych0 committed May 30, 2024
   Configuration menu

   • View commit details
   • Copy full SHA for e81b867
   • Browse repository at this point

   Copy the full SHA

   e81b867 View commit details

   Browse the repository at this point in the history