On copying, which is considered: actual or apparent size?
I was wondering if I have a file that has apparent size of 1 GiB but actual size of 0 B can get copied to, for example, a USB flash stick with 512 kiB free space?
You can create a file as such using:
dd if=/dev/null of=big-file bs=1 seek=1GiB
Now you can see the apparent and actual sizes:
du --apparent-size -hs big-file
Note: remove the --apparent-size option to get the actual size.
So, the main question is: which size is taken into consideration when copying a file to some directory, be that a hard disk drive, USB flash stick, DVD, etc?
file-copy size
edited Dec 31 '18 at 10:21
SouravGhosh
481311
asked Oct 3 '18 at 14:52
jokerjoker
21717
add a comment |
I was wondering if I have a file that has apparent size of 1 GiB but actual size of 0 B can get copied to, for example, a USB flash stick with 512 kiB free space?
You can create a file as such using:
dd if=/dev/null of=big-file bs=1 seek=1GiB
Now you can see the apparent and actual sizes:
du --apparent-size -hs big-file
Note: remove the --apparent-size option to get the actual size.
So, the main question is: which size is taken into consideration when copying a file to some directory, be that a hard disk drive, USB flash stick, DVD, etc?
file-copy size
edited Dec 31 '18 at 10:21
SouravGhosh
481311
asked Oct 3 '18 at 14:52
jokerjoker
21717
2
It depends on if filesystem supports fallocate or not.
– Ipor Sircer
Oct 3 '18 at 14:55
4
That depends very much on the filesystem of the target device, and if it supports sparse files.
– dirkt
Oct 3 '18 at 14:55
add a comment |
I was wondering if I have a file that has apparent size of 1 GiB but actual size of 0 B can get copied to, for example, a USB flash stick with 512 kiB free space?
You can create a file as such using:
dd if=/dev/null of=big-file bs=1 seek=1GiB
Now you can see the apparent and actual sizes:
du --apparent-size -hs big-file
Note: remove the --apparent-size option to get the actual size.
So, the main question is: which size is taken into consideration when copying a file to some directory, be that a hard disk drive, USB flash stick, DVD, etc?
file-copy size
edited Dec 31 '18 at 10:21
SouravGhosh
481311
asked Oct 3 '18 at 14:52
jokerjoker
21717
I was wondering if I have a file that has apparent size of 1 GiB but actual size of 0 B can get copied to, for example, a USB flash stick with 512 kiB free space?
You can create a file as such using:
dd if=/dev/null of=big-file bs=1 seek=1GiB
Now you can see the apparent and actual sizes:
du --apparent-size -hs big-file
Note: remove the --apparent-size option to get the actual size.
So, the main question is: which size is taken into consideration when copying a file to some directory, be that a hard disk drive, USB flash stick, DVD, etc?
file-copy size
file-copy size
edited Dec 31 '18 at 10:21
SouravGhosh
481311
asked Oct 3 '18 at 14:52
jokerjoker
21717
edited Dec 31 '18 at 10:21
SouravGhosh
481311
asked Oct 3 '18 at 14:52
jokerjoker
21717
edited Dec 31 '18 at 10:21
SouravGhosh
481311
edited Dec 31 '18 at 10:21
SouravGhosh
481311
edited Dec 31 '18 at 10:21
SouravGhosh
481311
481311
asked Oct 3 '18 at 14:52
jokerjoker
21717
asked Oct 3 '18 at 14:52
jokerjoker
21717
asked Oct 3 '18 at 14:52
jokerjoker
21717
21717
2
It depends on if filesystem supports fallocate or not.
– Ipor Sircer
Oct 3 '18 at 14:55
4
That depends very much on the filesystem of the target device, and if it supports sparse files.
– dirkt
Oct 3 '18 at 14:55
add a comment |
2
It depends on if filesystem supports fallocate or not.
– Ipor Sircer
Oct 3 '18 at 14:55
4
That depends very much on the filesystem of the target device, and if it supports sparse files.
– dirkt
Oct 3 '18 at 14:55
2
2
It depends on if filesystem supports fallocate or not.
– Ipor Sircer
Oct 3 '18 at 14:55
It depends on if filesystem supports fallocate or not.
– Ipor Sircer
Oct 3 '18 at 14:55
4
4
That depends very much on the filesystem of the target device, and if it supports sparse files.
– dirkt
Oct 3 '18 at 14:55
That depends very much on the filesystem of the target device, and if it supports sparse files.
– dirkt
Oct 3 '18 at 14:55
add a comment |
1 Answer
1
active
oldest
votes
Actual size.
Of the destination file.
The simplest implementation is to not try to predict this in advance.
In addition to that, a small fraction is required as overhead. This overhead would be even more complex to predict.
Support for sparse files varies, depending on both the filesystem, and the program making the copy. Most notably, FAT filesystems do not support sparse files.
GNU cp automatically detects and handles sparse files. It does not check for space in advance. (Based on the documentation for GNU coreutils 8.29)
By default, sparse SOURCE files are detected by a crude heuristic and
the corresponding DEST file is made sparse as well. That is the behavior selected by--sparse=auto. Specify--sparse=alwaysto create a
sparse DEST file whenever the SOURCE file contains a long enough
sequence of zero bytes.
Notice that filesystems can have different block sizes. This can cause the amount of space used by the sparse file to increase, or even decrease!
( Writing file data generally also requires internal filesystem structures which take up space of their own. This is not shown in the file's size - neither the actual nor the apparent size. In traditional UNIX filesystem design, you also need a free inode. The space available for inodes is shown by df -i. And most obviously, you need some space to store the file name :-) ; this is often shown in the size of the parent directory. This is discussed specifically in the question Can I run out of disk space by creating a very large number of empty files? )
answered Oct 3 '18 at 15:51
sourcejedisourcejedi
23.2k437102
"Support for sparse files varies, depending on both the filesystem, and the program making the copy." -- This could be in bold as the first thing here... And since we're talking USB sticks, chances are it's formatted as FAT32, which I don't think will support sparse files.
– ilkkachu
Oct 3 '18 at 17:00
add a comment |
Your Answer
StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "106"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});
function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: false,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: null,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});
}
});
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2funix.stackexchange.com%2fquestions%2f473010%2fon-copying-which-is-considered-actual-or-apparent-size%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
1 Answer
1
active
oldest
votes
1 Answer
1
active
oldest
votes
active
oldest
votes
active
oldest
votes
Actual size.
Of the destination file.
The simplest implementation is to not try to predict this in advance.
In addition to that, a small fraction is required as overhead. This overhead would be even more complex to predict.
Support for sparse files varies, depending on both the filesystem, and the program making the copy. Most notably, FAT filesystems do not support sparse files.
GNU cp automatically detects and handles sparse files. It does not check for space in advance. (Based on the documentation for GNU coreutils 8.29)
By default, sparse SOURCE files are detected by a crude heuristic and
the corresponding DEST file is made sparse as well. That is the behavior selected by--sparse=auto. Specify--sparse=alwaysto create a
sparse DEST file whenever the SOURCE file contains a long enough
sequence of zero bytes.
Notice that filesystems can have different block sizes. This can cause the amount of space used by the sparse file to increase, or even decrease!
( Writing file data generally also requires internal filesystem structures which take up space of their own. This is not shown in the file's size - neither the actual nor the apparent size. In traditional UNIX filesystem design, you also need a free inode. The space available for inodes is shown by df -i. And most obviously, you need some space to store the file name :-) ; this is often shown in the size of the parent directory. This is discussed specifically in the question Can I run out of disk space by creating a very large number of empty files? )
answered Oct 3 '18 at 15:51
sourcejedisourcejedi
23.2k437102
"Support for sparse files varies, depending on both the filesystem, and the program making the copy." -- This could be in bold as the first thing here... And since we're talking USB sticks, chances are it's formatted as FAT32, which I don't think will support sparse files.
– ilkkachu
Oct 3 '18 at 17:00
add a comment |
Actual size.
Of the destination file.
The simplest implementation is to not try to predict this in advance.
In addition to that, a small fraction is required as overhead. This overhead would be even more complex to predict.
Support for sparse files varies, depending on both the filesystem, and the program making the copy. Most notably, FAT filesystems do not support sparse files.
GNU cp automatically detects and handles sparse files. It does not check for space in advance. (Based on the documentation for GNU coreutils 8.29)
By default, sparse SOURCE files are detected by a crude heuristic and
the corresponding DEST file is made sparse as well. That is the behavior selected by--sparse=auto. Specify--sparse=alwaysto create a
sparse DEST file whenever the SOURCE file contains a long enough
sequence of zero bytes.
Notice that filesystems can have different block sizes. This can cause the amount of space used by the sparse file to increase, or even decrease!
( Writing file data generally also requires internal filesystem structures which take up space of their own. This is not shown in the file's size - neither the actual nor the apparent size. In traditional UNIX filesystem design, you also need a free inode. The space available for inodes is shown by df -i. And most obviously, you need some space to store the file name :-) ; this is often shown in the size of the parent directory. This is discussed specifically in the question Can I run out of disk space by creating a very large number of empty files? )
answered Oct 3 '18 at 15:51
sourcejedisourcejedi
23.2k437102
"Support for sparse files varies, depending on both the filesystem, and the program making the copy." -- This could be in bold as the first thing here... And since we're talking USB sticks, chances are it's formatted as FAT32, which I don't think will support sparse files.
– ilkkachu
Oct 3 '18 at 17:00
add a comment |
Actual size.
Of the destination file.
The simplest implementation is to not try to predict this in advance.
In addition to that, a small fraction is required as overhead. This overhead would be even more complex to predict.
Support for sparse files varies, depending on both the filesystem, and the program making the copy. Most notably, FAT filesystems do not support sparse files.
GNU cp automatically detects and handles sparse files. It does not check for space in advance. (Based on the documentation for GNU coreutils 8.29)
By default, sparse SOURCE files are detected by a crude heuristic and
the corresponding DEST file is made sparse as well. That is the behavior selected by--sparse=auto. Specify--sparse=alwaysto create a
sparse DEST file whenever the SOURCE file contains a long enough
sequence of zero bytes.
Notice that filesystems can have different block sizes. This can cause the amount of space used by the sparse file to increase, or even decrease!
( Writing file data generally also requires internal filesystem structures which take up space of their own. This is not shown in the file's size - neither the actual nor the apparent size. In traditional UNIX filesystem design, you also need a free inode. The space available for inodes is shown by df -i. And most obviously, you need some space to store the file name :-) ; this is often shown in the size of the parent directory. This is discussed specifically in the question Can I run out of disk space by creating a very large number of empty files? )
answered Oct 3 '18 at 15:51
sourcejedisourcejedi
23.2k437102
Actual size.
Of the destination file.
The simplest implementation is to not try to predict this in advance.
In addition to that, a small fraction is required as overhead. This overhead would be even more complex to predict.
Support for sparse files varies, depending on both the filesystem, and the program making the copy. Most notably, FAT filesystems do not support sparse files.
GNU cp automatically detects and handles sparse files. It does not check for space in advance. (Based on the documentation for GNU coreutils 8.29)
By default, sparse SOURCE files are detected by a crude heuristic and
the corresponding DEST file is made sparse as well. That is the behavior selected by--sparse=auto. Specify--sparse=alwaysto create a
sparse DEST file whenever the SOURCE file contains a long enough
sequence of zero bytes.
Notice that filesystems can have different block sizes. This can cause the amount of space used by the sparse file to increase, or even decrease!
( Writing file data generally also requires internal filesystem structures which take up space of their own. This is not shown in the file's size - neither the actual nor the apparent size. In traditional UNIX filesystem design, you also need a free inode. The space available for inodes is shown by df -i. And most obviously, you need some space to store the file name :-) ; this is often shown in the size of the parent directory. This is discussed specifically in the question Can I run out of disk space by creating a very large number of empty files? )
answered Oct 3 '18 at 15:51
sourcejedisourcejedi
23.2k437102
edited Oct 3 '18 at 17:57
answered Oct 3 '18 at 15:51
sourcejedisourcejedi
23.2k437102
answered Oct 3 '18 at 15:51
sourcejedisourcejedi
23.2k437102
answered Oct 3 '18 at 15:51
sourcejedisourcejedi
23.2k437102
23.2k437102
"Support for sparse files varies, depending on both the filesystem, and the program making the copy." -- This could be in bold as the first thing here... And since we're talking USB sticks, chances are it's formatted as FAT32, which I don't think will support sparse files.
– ilkkachu
Oct 3 '18 at 17:00
add a comment |
"Support for sparse files varies, depending on both the filesystem, and the program making the copy." -- This could be in bold as the first thing here... And since we're talking USB sticks, chances are it's formatted as FAT32, which I don't think will support sparse files.
– ilkkachu
Oct 3 '18 at 17:00
"Support for sparse files varies, depending on both the filesystem, and the program making the copy." -- This could be in bold as the first thing here... And since we're talking USB sticks, chances are it's formatted as FAT32, which I don't think will support sparse files.
– ilkkachu
Oct 3 '18 at 17:00
"Support for sparse files varies, depending on both the filesystem, and the program making the copy." -- This could be in bold as the first thing here... And since we're talking USB sticks, chances are it's formatted as FAT32, which I don't think will support sparse files.
– ilkkachu
Oct 3 '18 at 17:00
add a comment |
Thanks for contributing an answer to Unix & Linux Stack Exchange!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2funix.stackexchange.com%2fquestions%2f473010%2fon-copying-which-is-considered-actual-or-apparent-size%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
2
It depends on if filesystem supports fallocate or not.
– Ipor Sircer
Oct 3 '18 at 14:55
4
That depends very much on the filesystem of the target device, and if it supports sparse files.
– dirkt
Oct 3 '18 at 14:55