Inquiry Regarding Advanced Object Lifecycle Management Features in Minio

[okgame]

Dear Minio Development Team,

I have noticed that Minio currently supports the feature of automatically deleting files X days after they have been uploaded. However, it appears there is no feature to delete files X days from the last access.

I am wondering if Minio has the capability to down-tier or delete objects based on the number of inactive days? https://docs.aws.amazon.com/AmazonS3/latest/userguide/intelligent-tiering-overview.html.

With this approach, data that is frequently accessed would always remain in the hot tier without ever being moved or deleted. I believe this could help optimize both performance and storage costs.

Looking forward to your response.

Best regards,

[harshavardhana]

@okgame last access is very hard to capture on a filesystem, it requires repeated chatty I/O to the disks. It is something that we are not going to implement at any time in the near future.

[okgame]

Would it not be necessary to record this in real time on the disk. Could we instead capture this information in RAM and update it later?
For example, the node responsible for delivering the first byte of a file to an endpoint could be the one to log the last access time. Subsequently, we could have nodes synchronizing this last access time data every 30 to 60 seconds.
Would such an approach be feasible?

[klauspost]

It may be feasible, but not for MinIO.

You are welcome to build a system that keeps a bloom filter of accesses per day, but it isn't really something we are actively looking to integrate into MinIO.

[Scapal]

A Bloom filter wouldn't be the solution in this case. It could only be used to have an indication if an object actually exists in a bucket.

I agree that to have a meaningful lifecycle management, the last access time is essential. In fact, the last access day could be enough, no need to update it if it has already been accessed the same day.
Another optimisation/compromise, this doesn't need to be at the object level but only at a prefix level.
It has to be managed by MinIO to avoid the need for a perf-hurting gateway on the network.

[klauspost]

https://blog.min.io/enterprise-catalog/