Object storage
Using rclone with S3-compatible storage: endpoint, credentials and verification
rclone supports many object-storage backends and is useful for migrations, scheduled copies and verification. The important part of an S3-compatible configuration is being explicit about the provider endpoint, region and credential scope. AWS defaults are not safe assumptions for a third-party endpoint.
Create a dedicated rclone credential
Give the rclone job a service-specific access key that can reach only the intended bucket or project. Do not place the provider control-plane credential into an rclone configuration simply because it works.
Protect the rclone config file and consider its password-obscuring feature an accidental-display safeguard, not strong secret encryption. Filesystem access still matters.
Set the custom endpoint
Choose the S3 backend, select the compatible-provider mode as appropriate and enter the HTTPS endpoint published for the service. Keep the region value consistent with the credential signing configuration.
An endpoint typo can produce confusing signature or redirect errors. Test a simple bucket listing before starting a multi-terabyte transfer.
Start with a small checksum-verifiable dataset
Copy several files with known hashes and different sizes. List them from another client and retrieve them to a separate directory.
Compare checksums after download. Object presence and object integrity are different checks, especially when migration tools transform metadata or multipart ETags.
Tune transfers only after correctness
rclone exposes transfer concurrency, checkers and multipart-related options. Higher values can increase throughput but can also saturate the source disk, WAN or provider request path.
Measure with realistic files and watch errors before raising concurrency. Stable sustained throughput is more useful than a brief peak followed by retries.
Use sync with caution
`rclone sync` can delete destination objects to make it match the source. Use `copy` or a dry-run while designing a migration, and understand retention before enabling destructive synchronization.
For backup use, prefer a backup application or naming/retention scheme that deliberately preserves recovery history explicitly; sync alone does not create backup retention.