Are files always cached in $BLOCKSIZE increments?
Given a 4KB block size, when a 1KB file is read from disk and cached in main memory, is the entire 4KB block that contains the file cached?
Similarly, if opening a 100KB file, seeking 50KB in and then reading 1KB of data, is only the 4KB block that contains the 1KB of data read cached in main memory and not the entire file?
I am assuming ext4 filesystem, but not stuck to it.
filesystems memory cache
asked Dec 19 '18 at 20:37
mjb2kmn
388114
add a comment |
Given a 4KB block size, when a 1KB file is read from disk and cached in main memory, is the entire 4KB block that contains the file cached?
Similarly, if opening a 100KB file, seeking 50KB in and then reading 1KB of data, is only the 4KB block that contains the 1KB of data read cached in main memory and not the entire file?
I am assuming ext4 filesystem, but not stuck to it.
filesystems memory cache
asked Dec 19 '18 at 20:37
mjb2kmn
388114
1
Thanks for the feedback. I meant this as one question, as stated in the title. The body offers two different illustrations of the same question. I am not sure quite yet how I can rephrase it to make that clearer.
– mjb2kmn
Dec 20 '18 at 14:42
add a comment |
Given a 4KB block size, when a 1KB file is read from disk and cached in main memory, is the entire 4KB block that contains the file cached?
Similarly, if opening a 100KB file, seeking 50KB in and then reading 1KB of data, is only the 4KB block that contains the 1KB of data read cached in main memory and not the entire file?
I am assuming ext4 filesystem, but not stuck to it.
filesystems memory cache
asked Dec 19 '18 at 20:37
mjb2kmn
388114
Given a 4KB block size, when a 1KB file is read from disk and cached in main memory, is the entire 4KB block that contains the file cached?
Similarly, if opening a 100KB file, seeking 50KB in and then reading 1KB of data, is only the 4KB block that contains the 1KB of data read cached in main memory and not the entire file?
I am assuming ext4 filesystem, but not stuck to it.
filesystems memory cache
filesystems memory cache
asked Dec 19 '18 at 20:37
mjb2kmn
388114
asked Dec 19 '18 at 20:37
mjb2kmn
388114
edited Dec 20 '18 at 19:34
asked Dec 19 '18 at 20:37
mjb2kmn
388114
asked Dec 19 '18 at 20:37
mjb2kmn
388114
asked Dec 19 '18 at 20:37
mjb2kmn
388114
388114
1
Thanks for the feedback. I meant this as one question, as stated in the title. The body offers two different illustrations of the same question. I am not sure quite yet how I can rephrase it to make that clearer.
– mjb2kmn
Dec 20 '18 at 14:42
add a comment |
1
Thanks for the feedback. I meant this as one question, as stated in the title. The body offers two different illustrations of the same question. I am not sure quite yet how I can rephrase it to make that clearer.
– mjb2kmn
Dec 20 '18 at 14:42
1
1
Thanks for the feedback. I meant this as one question, as stated in the title. The body offers two different illustrations of the same question. I am not sure quite yet how I can rephrase it to make that clearer.
– mjb2kmn
Dec 20 '18 at 14:42
Thanks for the feedback. I meant this as one question, as stated in the title. The body offers two different illustrations of the same question. I am not sure quite yet how I can rephrase it to make that clearer.
– mjb2kmn
Dec 20 '18 at 14:42
add a comment |
1 Answer
1
active
oldest
votes
Yep.
The authoritative Linux Page Cache Basics states
File blocks are written to the Page Cache not just during writing, but also when reading files.
and
Automatically storing file blocks in the Page Cache is generally quite advantageous.
This applies to block-based filesystems, i.e., ccfs, ext?, gfs, ifs, xfs, as shown in his diagram. It's also confirmed by Page Cache, the Affair Between Memory and Files, which explicitly states:
If you read a single byte from a file, the whole 4KB chunk containing the byte you asked for is read from disk and placed into the page cache.
and the third paragraph of that web page confirms only the 4KB block that contains the 1KB of data read cached in main memory and not the entire file.
answered Dec 19 '18 at 22:09
K7AAY
380319
1
This page is very general and uses examples of large files, I am specifically looking for details on how small files or small reads of large files are cached.
– mjb2kmn
Dec 20 '18 at 14:59
Added another citation which directly addresses your second question.
– K7AAY
Dec 20 '18 at 19:25
1
The Page Cache, the Affair Between Memory and Files post by Gustavo Duarte explains it perfectly.
– mjb2kmn
Dec 20 '18 at 19:36
add a comment |
Your Answer
StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "106"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});
function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: false,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: null,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});
}
});
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2funix.stackexchange.com%2fquestions%2f490008%2fare-files-always-cached-in-blocksize-increments%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
1 Answer
1
active
oldest
votes
1 Answer
1
active
oldest
votes
active
oldest
votes
active
oldest
votes
Yep.
The authoritative Linux Page Cache Basics states
File blocks are written to the Page Cache not just during writing, but also when reading files.
and
Automatically storing file blocks in the Page Cache is generally quite advantageous.
This applies to block-based filesystems, i.e., ccfs, ext?, gfs, ifs, xfs, as shown in his diagram. It's also confirmed by Page Cache, the Affair Between Memory and Files, which explicitly states:
If you read a single byte from a file, the whole 4KB chunk containing the byte you asked for is read from disk and placed into the page cache.
and the third paragraph of that web page confirms only the 4KB block that contains the 1KB of data read cached in main memory and not the entire file.
answered Dec 19 '18 at 22:09
K7AAY
380319
1
This page is very general and uses examples of large files, I am specifically looking for details on how small files or small reads of large files are cached.
– mjb2kmn
Dec 20 '18 at 14:59
Added another citation which directly addresses your second question.
– K7AAY
Dec 20 '18 at 19:25
1
The Page Cache, the Affair Between Memory and Files post by Gustavo Duarte explains it perfectly.
– mjb2kmn
Dec 20 '18 at 19:36
add a comment |
Yep.
The authoritative Linux Page Cache Basics states
File blocks are written to the Page Cache not just during writing, but also when reading files.
and
Automatically storing file blocks in the Page Cache is generally quite advantageous.
This applies to block-based filesystems, i.e., ccfs, ext?, gfs, ifs, xfs, as shown in his diagram. It's also confirmed by Page Cache, the Affair Between Memory and Files, which explicitly states:
If you read a single byte from a file, the whole 4KB chunk containing the byte you asked for is read from disk and placed into the page cache.
and the third paragraph of that web page confirms only the 4KB block that contains the 1KB of data read cached in main memory and not the entire file.
answered Dec 19 '18 at 22:09
K7AAY
380319
1
This page is very general and uses examples of large files, I am specifically looking for details on how small files or small reads of large files are cached.
– mjb2kmn
Dec 20 '18 at 14:59
Added another citation which directly addresses your second question.
– K7AAY
Dec 20 '18 at 19:25
1
The Page Cache, the Affair Between Memory and Files post by Gustavo Duarte explains it perfectly.
– mjb2kmn
Dec 20 '18 at 19:36
add a comment |
Yep.
The authoritative Linux Page Cache Basics states
File blocks are written to the Page Cache not just during writing, but also when reading files.
and
Automatically storing file blocks in the Page Cache is generally quite advantageous.
This applies to block-based filesystems, i.e., ccfs, ext?, gfs, ifs, xfs, as shown in his diagram. It's also confirmed by Page Cache, the Affair Between Memory and Files, which explicitly states:
If you read a single byte from a file, the whole 4KB chunk containing the byte you asked for is read from disk and placed into the page cache.
and the third paragraph of that web page confirms only the 4KB block that contains the 1KB of data read cached in main memory and not the entire file.
answered Dec 19 '18 at 22:09
K7AAY
380319
Yep.
The authoritative Linux Page Cache Basics states
File blocks are written to the Page Cache not just during writing, but also when reading files.
and
Automatically storing file blocks in the Page Cache is generally quite advantageous.
This applies to block-based filesystems, i.e., ccfs, ext?, gfs, ifs, xfs, as shown in his diagram. It's also confirmed by Page Cache, the Affair Between Memory and Files, which explicitly states:
If you read a single byte from a file, the whole 4KB chunk containing the byte you asked for is read from disk and placed into the page cache.
and the third paragraph of that web page confirms only the 4KB block that contains the 1KB of data read cached in main memory and not the entire file.
answered Dec 19 '18 at 22:09
K7AAY
380319
edited Dec 20 '18 at 19:24
answered Dec 19 '18 at 22:09
K7AAY
380319
answered Dec 19 '18 at 22:09
K7AAY
380319
answered Dec 19 '18 at 22:09
K7AAY
380319
380319
1
This page is very general and uses examples of large files, I am specifically looking for details on how small files or small reads of large files are cached.
– mjb2kmn
Dec 20 '18 at 14:59
Added another citation which directly addresses your second question.
– K7AAY
Dec 20 '18 at 19:25
1
The Page Cache, the Affair Between Memory and Files post by Gustavo Duarte explains it perfectly.
– mjb2kmn
Dec 20 '18 at 19:36
add a comment |
1
This page is very general and uses examples of large files, I am specifically looking for details on how small files or small reads of large files are cached.
– mjb2kmn
Dec 20 '18 at 14:59
Added another citation which directly addresses your second question.
– K7AAY
Dec 20 '18 at 19:25
1
The Page Cache, the Affair Between Memory and Files post by Gustavo Duarte explains it perfectly.
– mjb2kmn
Dec 20 '18 at 19:36
1
1
This page is very general and uses examples of large files, I am specifically looking for details on how small files or small reads of large files are cached.
– mjb2kmn
Dec 20 '18 at 14:59
This page is very general and uses examples of large files, I am specifically looking for details on how small files or small reads of large files are cached.
– mjb2kmn
Dec 20 '18 at 14:59
Added another citation which directly addresses your second question.
– K7AAY
Dec 20 '18 at 19:25
Added another citation which directly addresses your second question.
– K7AAY
Dec 20 '18 at 19:25
1
1
The Page Cache, the Affair Between Memory and Files post by Gustavo Duarte explains it perfectly.
– mjb2kmn
Dec 20 '18 at 19:36
The Page Cache, the Affair Between Memory and Files post by Gustavo Duarte explains it perfectly.
– mjb2kmn
Dec 20 '18 at 19:36
add a comment |
Thanks for contributing an answer to Unix & Linux Stack Exchange!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Some of your past answers have not been well-received, and you're in danger of being blocked from answering.
Please pay close attention to the following guidance:
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2funix.stackexchange.com%2fquestions%2f490008%2fare-files-always-cached-in-blocksize-increments%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
1
Thanks for the feedback. I meant this as one question, as stated in the title. The body offers two different illustrations of the same question. I am not sure quite yet how I can rephrase it to make that clearer.
– mjb2kmn
Dec 20 '18 at 14:42