How to resume an interrupted SFTP transfer on a bad connection
You can resume an SFTP transfer because SFTP lets a client open a remote file at any byte offset. After a dropped connection, a client that remembers how far it got can reconnect, seek to that offset, and send or fetch only the remainder. A 4 GB upload that dies at 3.6 GB costs the last 400 MB, not the whole file again. This post covers how resuming works over SFTP, FTP and S3, how to do it from the command line, and how to make sure a resumed file is actually intact.
How resuming works, protocol by protocol
SFTP
SFTP is a file-access protocol, not just a copy command. A client opens a remote file, then reads or writes at whatever offsets it chooses. To resume a download, it opens the remote file, seeks to the size of the partial local copy and reads from there, appending locally. To resume an upload, it opens the remote file without truncating it, seeks to the offset and continues writing. Nothing special is needed on the server.
FTP and FTPS
FTP resumes with the REST command, which sets a restart offset for the next transfer command. REST 1048576 followed by RETR file.zip downloads from the 1 MiB mark onwards; REST followed by STOR continues an upload. The behaviour is defined for stream mode in RFC 3659. Most servers support it for downloads. Upload restart is sometimes disabled in server configuration, and the server answers REST with an error rather than 350.
Amazon S3 and S3-compatible storage
Downloads resume with an HTTP Range header (Range: bytes=1048576-), which S3 supports. Uploads are different. A plain PUT replaces the whole object, so an interrupted single-request upload can't be continued; it has to start again. The S3 way to make large uploads resumable is a multipart upload, where the file is sent in independently retried parts.
How to resume an SFTP transfer from the command line
OpenSSH's sftp has resume built in. The reget and reput commands continue a partial download or upload, and get -a and put -a do the same:
sftp deploy@example.com
sftp> reput build/release.tar.gz /srv/releases/release.tar.gz
sftp> reget /var/backups/db.sql.gz
The catch is that these trust the existing partial file. If the local file changed since the first attempt, or the partial copy on the other end is from a different version, the result is a file made of two different sources, and nothing warns you. Resume only a transfer of the same unchanged file.
Check the file after a resume
A resumed transfer is only as good as the bytes that were already there. For anything that matters, compare a checksum on both ends. On a Mac:
shasum -a 256 release.tar.gz
On most Linux servers:
sha256sum /srv/releases/release.tar.gz
If the two hashes match, the file is identical. If they don't, delete the remote copy and transfer it fresh. This takes seconds and removes any doubt about a transfer that was interrupted, paused or retried.
Making bad connections less bad
Resuming limits the damage from a drop. A few settings make drops less likely in the first place:
- Keepalives. Home routers, hotel Wi-Fi and NAT gateways often drop connections that look idle. SSH keepalives send a small message periodically to keep the mapping alive. With the
sshandsftpcommands, setServerAliveInterval 30in~/.ssh/config. - Fewer parallel connections. Several simultaneous transfers on a weak link compete with each other and make every one of them more likely to time out. One or two at a time is often faster overall.
- A bandwidth cap. A transfer that saturates the uplink also starves everything else, including the acknowledgements that keep the connection alive. Capping it a little below the line's capacity can make it more reliable.
- Upload to a temporary name. For files that something else picks up (a deploy watcher, an import job), upload as
release.tar.gz.partand rename when complete, so a half-finished upload is never mistaken for a finished one.
How Wharf resumes transfers
Wharf records how many bytes each transfer has moved, and continues from that offset rather than starting again. What happens after an interruption depends on the protocol:
| Protocol | Downloads | Uploads |
|---|---|---|
| SFTP | Resume from the offset | Resume from the offset |
| FTP and FTPS | Resume with REST | Resume with REST, where the server allows it |
| S3 and compatible | Resume with a Range request | Start again (single PUT) |
Automatic retries. When an SFTP transfer fails because the connection dropped or timed out, Wharf discards the broken connection, opens a new one and carries on from where the transfer got to, without you doing anything. The number of attempts is set under Settings › Transfers as "Retry a failed transfer" (three by default, up to ten). Only failures worth retrying are retried: a permission error or a missing file stops straight away, since trying again would give the same answer.
Manual retry and pause. A transfer that has used up its attempts, or failed for another reason, stays in the Transfers list with its error. Its retry button keeps the offset, so retrying after the network comes back continues rather than restarts. The same applies to pausing: pause a transfer, or the whole queue, and resuming picks up at the byte it stopped on.
Keepalives and limits. SFTP connections send SSH keepalives every 30 seconds, which helps through NAT and idle timeouts during long transfers. Settings › Transfers also has a bandwidth limit shared across all transfers, and a per-site Maximum connections setting in the site editor under Advanced, for links or hosts that cope badly with parallel transfers.
Progress you can trust. Each transfer shows its speed and time remaining, and the queue header estimates how long the whole queue will take. The features page has more on dual-pane transfers.
Wharf doesn't checksum a file after a resume, so for large or critical files the shasum check above is still worth the few seconds. If the transfer is part of a regular deployment, a sync-based deploy compares sizes and dates and only re-sends files that differ, which is a natural second pass after a bad connection. The support page lists what to include if a server behaves oddly when resuming.