Change the encryption/compression information in the FAQ

[ramondeklein]

Using transformations (such as encryption/compression) will have an effect on the changed blob and all the blobs following them. Suppose you change the first byte in a 1GB file, then all 200 blobs will be different and need to be reuploaded. When the compression is done on the blob level, then only the changed part needs to be reuploaded.

This change would be a bit more difficult to implement, because of the following changes:

• Transformations should be done from within s3git, so you need a (pluggable) module to do this.
• Slightly more overhead, because each blob is compressed/encrypted on its own.
• A single encryption key should be used for the entire repository.

This encryption should only be used to encrypt the repository, because the underlying storage layer cannot be trusted. It cannot be used to encrypt individual files (with different keys), because each blob should have the same key. If you do need per-file encryption, then you need to encrypt the file using the traditional methods.

The data will be stored with the hash of the transformed data. Because during the encryption of a block a randomized initialization vector is used, the encrypted data cannot use deduplication anymore. If you do want to use deduplication, then you have two options:

• Use the hash of the untransformed data as the IV (potentially unsecure, because IVs should be random).
• Store the file in the underlying storage using the untransformed hash. The drawback of this is that you cannot scrub each block to check if it hasn't been altered (without knowing the key).