How to make sure docker daemon sucessfully started even through PID file existed

[botieking98]

Description

Now docker daemon will start failed with error process with PID xxx is still running when /var/run/docker.pid file existed and process is alive.

The associated code is below:

// Read reads the "PID file" at path, and returns the PID if it contains a
// valid PID of a running process, or 0 otherwise. It returns an error when
// failing to read the file, or if the file doesn't exist, but malformed content
// is ignored. Consumers should therefore check if the returned PID is a non-zero
// value before use.
func Read(path string) (pid int, err error) {
	pidByte, err := os.ReadFile(path)
	if err != nil {
		return 0, err
	}
	pid, err = strconv.Atoi(string(bytes.TrimSpace(pidByte)))
	if err != nil {
		return 0, nil
	}
	if pid != 0 && process.Alive(pid) {
		return pid, nil
	}
	return 0, nil
}

// Write writes a "PID file" at the specified path. It returns an error if the
// file exists and contains a valid PID of a running process, or when failing
// to write the file.
func Write(path string, pid int) error {
	if pid < 1 {
		// We might be running as PID 1 when running docker-in-docker,
		// but 0 or negative PIDs are not acceptable.
		return fmt.Errorf("invalid PID (%d): only positive PIDs are allowed", pid)
	}
	oldPID, err := Read(path)
	if err != nil && !os.IsNotExist(err) {
		return err
	}
	if oldPID != 0 {
		return fmt.Errorf("process with PID %d is still running", oldPID)
	}
	return os.WriteFile(path, []byte(strconv.Itoa(pid)), 0o644)
}

Reproduce

We should do this:

1. Let /var/run/docker.pid file redundant
2. Reboot machine
3. Start docker and PID of docker daemon is same to PID file content

docker daemon failed started occasionally.

Expected behavior

docker daemon started sucessfully even through PID file existed and is same to PID of dockerd

docker version

ALL version

docker info

ALL version

Additional Info

No response

[thaJeztah]

This behavior is intentional, and is to prevent multiple instances from the daemon to be started (running multiple instances of the daemon that interact with the same data-store can cause corruption of data).

On a normal shutdown of the docker daemon service, the PID-file should be removed;

cat /var/run/docker.pid
101388

systemctl stop docker.service
Warning: Stopping docker.service, but it can still be activated by:
  docker.socket

cat /var/run/docker.pid
cat: /var/run/docker.pid: No such file or directory

Reboot machine

Did this lead to an unclean shutdown? Is the docker daemon running as a systemd unit in your setup, or is it started through other means? (See above; the PID-file should be removed on a clean shut-down).

[thaJeztah]

As I don't think there's a bug here, I'll move this to a discussion