--files-from doesn't seem to play nicely with --links

[tlemo]

When used together with --links, the --files-from filtering is demonstrating an unfortunate behavior: symlinks paths will be ignored. It appears that the filtering is applied after the .rclonelink mapping of the original names (so filtering on the "augmented" name works).

It's not clear if this behavior is intended or simply a bug, but it is both surprising as is unfortunate, since it requires clients to be aware of the difference between regular files and symlinks. Moreover, it's fragile in the face of changing a regular file to a symlink.

Ideally, the --files-from filtering should work on the original file paths.

To reproduce the problem, simply create a directory with a file and a symlink pointing to it, for example:

test/
  bar.txt -> foo.txt
  foo.txt

and a files with the content:

foo.txt
bar.txt

rclone sync --log-level DEBUG --progress --links --files-from files test x:/test

rclone v1.66.0
- os/version: ubuntu 18.04 (64 bit)
- os/kernel: 4.15.0-213-generic (x86_64)
- os/type: linux
- os/arch: amd64
- go/version: go1.22.1
- go/linking: static
- go/tags: none

[ncw]

The way to refer to a symlink in a --files-from when --links is active is this

foo.txt
bar.txt.rclonelink

Those are the names as rclone sees them if you use the --links parameter and all filtering works on the names that rclone sees.

If you really aren't sure whether bar.txt is a symlink or not you could do something like --include "/bar.txt{,.rclonelink}"

[tlemo]

Thanks for the suggestions. I have implemented a workaround (unfortunately boiling down to replacing rclone's filtering altogether) - I opened this issue hoping that it might provide useful feedback for improving rclone's filtering.

If you really aren't sure whether bar.txt is a symlink or not you could do something like --include "/bar.txt{,.rclonelink}"

Every path could potentially be a symlink (or change between a regular to a symlink). Wouldn't make more sense to abstract this away instead of forcing this awkward pattern matching for every path? (I don't have a chance to try it right now, does the pattern matching even work with --files-from?)

Those are the names as rclone sees them if you use the --links parameter

This describes an implementation detail. Is there a fundamental reason why the users have to be exposed to it?