OAuth tokens fail to refresh when they are provided as a JSON blob (for native-app clients)

[mikix]

What is the problem you are having with rclone?

When providing Rclone an OAuth token registered to an open-source or native-app-style client ID (i.e. one that doesn't use the client-secret OAuth flow), Rclone will always fail to refresh that token.

That is, when using a config flag like --drive-token , Rclone will always fail to refresh the provided token itself, even if expiry and refresh_token fields are defined in the blob and a client ID is provided.

This is because the default client secret is also sent along to the server when a token refresh is attempted, and the server will reject the refresh attempt.

It's not possible to stop Rclone from sending a client secret in backends that define a default id/secret (like drive and onedrive do), since Rclone ignores attempts to clear those defaults out with empty string config overrides.

I've proposed a fix in #7809, but I'm also filing this issue for extra find-ability in case others hit this or in case that PR is not suitable for whatever reason. Or if folks know of workarounds.

To reproduce

This is a little tricky, but if you have a client flow that does the whole auth flow of "open a web site for the user, get a code back from a redirect URL, use that code to get a refresh token" - save that token.

Now if that token doesn't have an expiry field set, add it (and for testing purposes, manually set it to now or in a minute so it will need to be refreshed). This is something Go's oauth library would normally do for you, but if you're mucking with tokens manually or in another language, you will need to define it yourself.

You'll also want to ensure that the token blob you are looking at has a refresh_token field (which it might not if you just used the refresh token to get a new access token).

Then run Rclone with your client ID and edited token, and watch it fail to refresh the token.

What is your rclone version (output from rclone version)

rclone v1.67.0-DEV

• os/version: ubuntu 24.04 (64 bit)
• os/kernel: 6.8.0-22-generic (x86_64)
• os/type: linux
• os/arch: amd64
• go/version: go1.22.2
• go/linking: dynamic
• go/tags: none

Which OS you are using and how many bits (e.g. Windows 7, 64 bit)

Ubuntu 24.04, 64 bit

Which cloud storage system are you using? (e.g. Google Drive)

Google Drive & Microsoft OneDrive mainly

The command you were trying to run (e.g. rclone copy /tmp remote:tmp)

Any of them

A log from the command with the -vv flag (e.g. output from rclone -vv copy /tmp remote:tmp)

I get a message like this when it fails:

2024/05/02 17:38:38 ERROR : locks/xxx Delete request remove error: Delete "https://www.googleapis.com/drive/v3/files/xxx?alt=json&fields=&prettyPrint=false&supportsAllDrives=true": couldn't fetch token: invalid_client: if you're using your own client id/secret, make sure they're properly set up following the docs
Fatal: unable to save snapshot: server response unexpected: 500 Internal Server Error (500)

or

2024/05/02 14:06:52 DEBUG : xxx: got fatal oauth error: oauth2: "invalid_client" "Unauthorized"
2024/05/02 14:06:52 Failed to create file system for "xxx": couldn't find root directory ID: Get "https://www.googleapis.com/drive/v3/files/root?alt=json&fields=id&prettyPrint=false&supportsAllDrives=true": couldn't fetch token: invalid_client: if you're using your own client id/secret, make sure they're properly set up following the docs

Thanks

Thanks for your work on Rclone btw! It's been great using it aside from this hiccup.

How to use GitHub

• Please use the 👍 reaction to show that you are affected by the same issue.
• Please don't comment if you have no relevant information to add. It's just extra noise for everyone subscribed to this issue.
• Subscribe to receive notifications on status change and new comments.

[ncw]

Thanks for the very well explained issue and the fix and apologies on the delay getting round to it.

I've merged this to master now which means it will be in the latest beta in 15-30 minutes and released in v1.67

Thank you :-)

[mikix]

No worries! Thanks for your work, and I’m glad this did manage to squeeze into 1.67 😄