rsync keeps previous partial file when resuming
up vote
1
down vote
favorite
I often transfer large files from a remote server using rsync with the following command :
rsync -rvhm
--progress
--size-only
--stats
--partial-dir=.rsync-partial
"user@server::module/some_path" "some_path"
That way, even if the transfer fails, I can resume it later and I know that I'll only have complete files in some_path on the destination, since all partial transfers will stay in some_path/.rsync-partial.
When a transfer resumes, rsync first checks the partial one to determine where exactly to resume (I guess) and I'm fine with that. The problem is that when it's done with this check, the partial file gets copied outside of the .partial-rsync folder for resume. Therefore, I'm left with a partial transfer (that will be replaced or deleted at the next pause or when the transfer finishes) along with the ongoing one.
This is inconvenient since :
- I don't have much free space on the destination ;
- The files are quite large ;
- If the partial transfer went "far enough", I might not be able to resume it since rsync will try to copy it first and will complain that there isn't enough space available to resume ;
- There is no reason that I can think of to keep a copy of the partial file to resume the transfer : the partial file itself should be used.
Is there a way to avoid this behavior or is this by design ? And if so, why would we want it to work this way ?
rsync file-copy file-transfer
asked Dec 2 at 22:42
Scentle5S
1061
add a comment |
up vote
1
down vote
favorite
I often transfer large files from a remote server using rsync with the following command :
rsync -rvhm
--progress
--size-only
--stats
--partial-dir=.rsync-partial
"user@server::module/some_path" "some_path"
That way, even if the transfer fails, I can resume it later and I know that I'll only have complete files in some_path on the destination, since all partial transfers will stay in some_path/.rsync-partial.
When a transfer resumes, rsync first checks the partial one to determine where exactly to resume (I guess) and I'm fine with that. The problem is that when it's done with this check, the partial file gets copied outside of the .partial-rsync folder for resume. Therefore, I'm left with a partial transfer (that will be replaced or deleted at the next pause or when the transfer finishes) along with the ongoing one.
This is inconvenient since :
- I don't have much free space on the destination ;
- The files are quite large ;
- If the partial transfer went "far enough", I might not be able to resume it since rsync will try to copy it first and will complain that there isn't enough space available to resume ;
- There is no reason that I can think of to keep a copy of the partial file to resume the transfer : the partial file itself should be used.
Is there a way to avoid this behavior or is this by design ? And if so, why would we want it to work this way ?
rsync file-copy file-transfer
asked Dec 2 at 22:42
Scentle5S
1061
add a comment |
up vote
1
down vote
favorite
up vote
1
down vote
favorite
I often transfer large files from a remote server using rsync with the following command :
rsync -rvhm
--progress
--size-only
--stats
--partial-dir=.rsync-partial
"user@server::module/some_path" "some_path"
That way, even if the transfer fails, I can resume it later and I know that I'll only have complete files in some_path on the destination, since all partial transfers will stay in some_path/.rsync-partial.
When a transfer resumes, rsync first checks the partial one to determine where exactly to resume (I guess) and I'm fine with that. The problem is that when it's done with this check, the partial file gets copied outside of the .partial-rsync folder for resume. Therefore, I'm left with a partial transfer (that will be replaced or deleted at the next pause or when the transfer finishes) along with the ongoing one.
This is inconvenient since :
- I don't have much free space on the destination ;
- The files are quite large ;
- If the partial transfer went "far enough", I might not be able to resume it since rsync will try to copy it first and will complain that there isn't enough space available to resume ;
- There is no reason that I can think of to keep a copy of the partial file to resume the transfer : the partial file itself should be used.
Is there a way to avoid this behavior or is this by design ? And if so, why would we want it to work this way ?
rsync file-copy file-transfer
asked Dec 2 at 22:42
Scentle5S
1061
I often transfer large files from a remote server using rsync with the following command :
rsync -rvhm
--progress
--size-only
--stats
--partial-dir=.rsync-partial
"user@server::module/some_path" "some_path"
That way, even if the transfer fails, I can resume it later and I know that I'll only have complete files in some_path on the destination, since all partial transfers will stay in some_path/.rsync-partial.
When a transfer resumes, rsync first checks the partial one to determine where exactly to resume (I guess) and I'm fine with that. The problem is that when it's done with this check, the partial file gets copied outside of the .partial-rsync folder for resume. Therefore, I'm left with a partial transfer (that will be replaced or deleted at the next pause or when the transfer finishes) along with the ongoing one.
This is inconvenient since :
- I don't have much free space on the destination ;
- The files are quite large ;
- If the partial transfer went "far enough", I might not be able to resume it since rsync will try to copy it first and will complain that there isn't enough space available to resume ;
- There is no reason that I can think of to keep a copy of the partial file to resume the transfer : the partial file itself should be used.
Is there a way to avoid this behavior or is this by design ? And if so, why would we want it to work this way ?
rsync file-copy file-transfer
rsync file-copy file-transfer
asked Dec 2 at 22:42
Scentle5S
1061
asked Dec 2 at 22:42
Scentle5S
1061
asked Dec 2 at 22:42
Scentle5S
1061
asked Dec 2 at 22:42
Scentle5S
1061
asked Dec 2 at 22:42
Scentle5S
1061
1061
add a comment |
add a comment |
active
oldest
votes
Your Answer
StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "106"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});
function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
convertImagesToLinks: false,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: null,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});
}
});
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2funix.stackexchange.com%2fquestions%2f485577%2frsync-keeps-previous-partial-file-when-resuming%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
active
oldest
votes
active
oldest
votes
active
oldest
votes
active
oldest
votes
Thanks for contributing an answer to Unix & Linux Stack Exchange!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Some of your past answers have not been well-received, and you're in danger of being blocked from answering.
Please pay close attention to the following guidance:
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2funix.stackexchange.com%2fquestions%2f485577%2frsync-keeps-previous-partial-file-when-resuming%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
