Existing photos / faces are not associated with a person during the creation of a person.

[esclear]

The bug

I'm raising this as a bug, as this behavior is, in my opinion unexpected. It tripped me up, at least:

When the person service creates a new person upon reaching the minFaces threshold (currently 3), only the newest (minFaces-th) face and thus picture is associated with the newly created person.

This leads to cases where there are more than minFaces present for a person, but the first minFaces-1 pictures that person is visible on are not associated to that person.

When running the face detection / face recognition tasks manually for missing images, the older pictures are also assigned to the person, but that requires manual intervention by an instance admin.

The OS that Immich Server is running on

Official Docker container

Version of Immich Server

v1.105.1

Version of Immich Mobile App

n/a

Platform with the issue

• Server
• Web
• Mobile

Your docker-compose.yml content

n/a

Your .env content

n/a

Reproduction steps

1. Upload two images of a new person (that doesn't have any faces present in Immich yet)
2. Observe that none of the images has assigned a person and that no person was created yet
3. Upload a third image (the person service will create a new person)
4. Check the newly created person and the images: Only the third one image is associated with the person.

Relevant log output

No response

Additional information

I'm not familiar with the Immich code base, but the fix could be as simple as using the (up to minFaces) matches in the already present pictures and assigning them to the person during creation of the new person.

[mertalev]

There are two parts to facial recognition.

What you described is the first part where new faces are matched to a new or existing person. The second step runs as a nightly job to go through existing unmatched faces and try to find a match for them. This separation is done for performance reasons since the second step is more expensive.

The most typical case is that the other faces of the person are also queued and hence assigned the person. In cases where this isn’t true, the nightly job cleans this up.

It’s possible to assign the group of matched faces, but it messes with the clustering algorithm somewhat.

[esclear]

I am not sure that we are talking about the same thing.

If I am uploading three images of the same person, then one person is created with the last image/face being assigned to this person.

The matching has already happened at this point, without having to wait for a nightly background run, as Immich was able to detect that at least minFaces images of that person have been uploaded - otherwise, the person wouldn't have been created.

Matching the third picture to the person is instant, but why should people have to wait for some nightly job to have the first and second image associated as well?

The matches variable already contains the faces that ought to be assigned to the newly created person.

That is, you wouldn't have to rely on that second step at all and get the same effect instantly.

[mertalev]

The other matched faces go through their own recognition jobs to determine a) if they're core or non-core (that is, if other faces should be able to match these faces) and b) if they have a closer match.

[mertalev]

One thing I noticed is that the first step is too eager to run. It waits until thumbnail generation and face detection queues are empty, but in the case of network speeds etc. there can be a brief lapse where it prematurely starts recognition.

If it waited a bit longer, then it would avoid the issue you described (uploading three images and only having one face assigned to a person). This is because faces that have fewer than minFaces matches are deferred to the end of the queue to see if there's a person they can be assigned to.