Recommended block size for Borg backup storage backend

[NielsH]

Hello,

We're in the process of moving our backups to a different storage, from an ext4 backend to ZFS. Ext4 has a blocksize of 4KB (and has some limitations preventing it from becoming bigger). ZFS has 128KB by default. I was wondering, are there any recommendations on block size (recordsize in ZFS terms) for the Borg backup storage backend?
If it matters, it will host ~500 repo's totalling about 100TB.

Thanks!

[ThomasWaldmann]

Borg by default creates files of about 500MiB size, so there is no need for a small fs blocksize.

Of course, the larger you make the blocksize, the more loss by overhead you produce. But due to the large file size, only the last block will contribute to the overhead and the overall amount of files is relatively low.

Guess I'ld just keep the ZFS default and disable zfs compression and zfs deduplication for the borg repo fs.

[NielsH]

Cool, good to hear defaults should be just fine. We indeed have compression/dedup off!
Cheers!

[j77h]

Of course, the larger you make the blocksize, the more loss by overhead you produce.

Apparently this is not so.
"The recordsize property sets the maximum logical size of blocks in an OpenZFS dataset."
Each file that is smaller than recordsize has a block-size that is the lowest power of 2 that can fit all the data.
https://klarasystems.com/articles/tuning-recordsize-in-openzfs/

just keep the ZFS default

It's now recommended to increase it to 1M (1 MiB).
You might first need to turn on the 'large blocks' feature.

[p4tpr0]

This thread eludes 2 important points about ZFS:

1- As @j77h wrote, when you set recordsize to 1M (for example) it means your maximum block size is 1M, smaller files will use smaller block size (lowest ^2 that can hold the data), BUT, by design, all blocks for a given file will have the same size. So technically you could have a non negligible overhead. Not something to worry about for 500MB files, though.

2- The file size is not necessarily what you have to take into account. For example, DBs or VM disks can be stored in massive files, but the way they are used matters a lot. PostgreSQL for example uses 8KB block size when it uses its DB file. So even if your DB file is enormous you must set recordsize to 8KB in order to maximize performance and minimize read/write amplification.

This leads to a complementary question: how does Borg use those ~500MB files? Are these «write once» or «append only» files or are these like DB files where Borg will read/write small portions inside?
In the first case, a 1M recordsize is perfectly fine. In the later case you’ll have to set recordsize to the size of these IO.

edit:
The doc (as of 1.2.8) states «Those files are strictly append-only and modified only once.», hence 1M recordsize is a fine choice.