Does linux systems flush dirty pages to disk one block by one block?
up vote
0
down vote
favorite
As far as I know, in linux systems, when modifying the content of a file, the pages containing the content of the file in page cache will be marked dirty, and will be flushed to disk eventually.
What I'm wondering is that, when these pages are flushed to disk, are they flushed in blocks?
For example, if the block size is 4kB and I need to flush some content of 1024kB, is the disk written 1024 / 4 = 256 times?
linux disk io
New contributor
TsReaper is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
add a comment |
up vote
0
down vote
favorite
As far as I know, in linux systems, when modifying the content of a file, the pages containing the content of the file in page cache will be marked dirty, and will be flushed to disk eventually.
What I'm wondering is that, when these pages are flushed to disk, are they flushed in blocks?
For example, if the block size is 4kB and I need to flush some content of 1024kB, is the disk written 1024 / 4 = 256 times?
linux disk io
New contributor
TsReaper is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
add a comment |
up vote
0
down vote
favorite
up vote
0
down vote
favorite
As far as I know, in linux systems, when modifying the content of a file, the pages containing the content of the file in page cache will be marked dirty, and will be flushed to disk eventually.
What I'm wondering is that, when these pages are flushed to disk, are they flushed in blocks?
For example, if the block size is 4kB and I need to flush some content of 1024kB, is the disk written 1024 / 4 = 256 times?
linux disk io
New contributor
TsReaper is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
As far as I know, in linux systems, when modifying the content of a file, the pages containing the content of the file in page cache will be marked dirty, and will be flushed to disk eventually.
What I'm wondering is that, when these pages are flushed to disk, are they flushed in blocks?
For example, if the block size is 4kB and I need to flush some content of 1024kB, is the disk written 1024 / 4 = 256 times?
linux disk io
linux disk io
New contributor
TsReaper is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
New contributor
TsReaper is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
New contributor
TsReaper is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
asked Nov 23 at 8:35
TsReaper
1033
1033
New contributor
TsReaper is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
New contributor
TsReaper is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
TsReaper is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
add a comment |
add a comment |
1 Answer
1
active
oldest
votes
up vote
2
down vote
accepted
That's a pretty complex topic, and depends on the disk, the disk controller and kernel settings.
In general, the kernel will attempt to be as efficient as it can. For example, if you update the same block multiple times within an adjustable time window (typically 30 seconds or so), and don't explicitly force syncing all the way to the disk each time, most of your write operations will only update the data in the cache and only the ultimate result will actually go to the disk.
If you write a long series of consecutive blocks, the kernel will certainly attempt to execute it in as few and as large chunks as the storage controller and the disk itself will allow.
The kernel's I/O scheduler may also optimize the ordering of disk operations to achieve most efficient disk access. This optimization can be mostly irrelevant in virtual machines and on SSDs, and so it can be switched off. (SSDs are plenty fast even if you access random blocks in a shotgun fashion; on virtual machines, the hypervisor will usually redo the optimization based on the entire set of VMs and all their disk operations anyway, so trying to micro-optimize on the level of a single VM is wasted effort.)
Some disks may have restrictions or recommendations on I/O operation sizes:
# fdisk -l /dev/sdb
Disk /dev/sdb: 1.8 TiB, 2000398934016 bytes, 3907029168 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes
For example, this HDD internally uses 4k sector size, although it emulates traditional 512-byte disk sectors. As a result, a minimum I/O size of 4k is specified.
add a comment |
1 Answer
1
active
oldest
votes
1 Answer
1
active
oldest
votes
active
oldest
votes
active
oldest
votes
up vote
2
down vote
accepted
That's a pretty complex topic, and depends on the disk, the disk controller and kernel settings.
In general, the kernel will attempt to be as efficient as it can. For example, if you update the same block multiple times within an adjustable time window (typically 30 seconds or so), and don't explicitly force syncing all the way to the disk each time, most of your write operations will only update the data in the cache and only the ultimate result will actually go to the disk.
If you write a long series of consecutive blocks, the kernel will certainly attempt to execute it in as few and as large chunks as the storage controller and the disk itself will allow.
The kernel's I/O scheduler may also optimize the ordering of disk operations to achieve most efficient disk access. This optimization can be mostly irrelevant in virtual machines and on SSDs, and so it can be switched off. (SSDs are plenty fast even if you access random blocks in a shotgun fashion; on virtual machines, the hypervisor will usually redo the optimization based on the entire set of VMs and all their disk operations anyway, so trying to micro-optimize on the level of a single VM is wasted effort.)
Some disks may have restrictions or recommendations on I/O operation sizes:
# fdisk -l /dev/sdb
Disk /dev/sdb: 1.8 TiB, 2000398934016 bytes, 3907029168 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes
For example, this HDD internally uses 4k sector size, although it emulates traditional 512-byte disk sectors. As a result, a minimum I/O size of 4k is specified.
add a comment |
up vote
2
down vote
accepted
That's a pretty complex topic, and depends on the disk, the disk controller and kernel settings.
In general, the kernel will attempt to be as efficient as it can. For example, if you update the same block multiple times within an adjustable time window (typically 30 seconds or so), and don't explicitly force syncing all the way to the disk each time, most of your write operations will only update the data in the cache and only the ultimate result will actually go to the disk.
If you write a long series of consecutive blocks, the kernel will certainly attempt to execute it in as few and as large chunks as the storage controller and the disk itself will allow.
The kernel's I/O scheduler may also optimize the ordering of disk operations to achieve most efficient disk access. This optimization can be mostly irrelevant in virtual machines and on SSDs, and so it can be switched off. (SSDs are plenty fast even if you access random blocks in a shotgun fashion; on virtual machines, the hypervisor will usually redo the optimization based on the entire set of VMs and all their disk operations anyway, so trying to micro-optimize on the level of a single VM is wasted effort.)
Some disks may have restrictions or recommendations on I/O operation sizes:
# fdisk -l /dev/sdb
Disk /dev/sdb: 1.8 TiB, 2000398934016 bytes, 3907029168 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes
For example, this HDD internally uses 4k sector size, although it emulates traditional 512-byte disk sectors. As a result, a minimum I/O size of 4k is specified.
add a comment |
up vote
2
down vote
accepted
up vote
2
down vote
accepted
That's a pretty complex topic, and depends on the disk, the disk controller and kernel settings.
In general, the kernel will attempt to be as efficient as it can. For example, if you update the same block multiple times within an adjustable time window (typically 30 seconds or so), and don't explicitly force syncing all the way to the disk each time, most of your write operations will only update the data in the cache and only the ultimate result will actually go to the disk.
If you write a long series of consecutive blocks, the kernel will certainly attempt to execute it in as few and as large chunks as the storage controller and the disk itself will allow.
The kernel's I/O scheduler may also optimize the ordering of disk operations to achieve most efficient disk access. This optimization can be mostly irrelevant in virtual machines and on SSDs, and so it can be switched off. (SSDs are plenty fast even if you access random blocks in a shotgun fashion; on virtual machines, the hypervisor will usually redo the optimization based on the entire set of VMs and all their disk operations anyway, so trying to micro-optimize on the level of a single VM is wasted effort.)
Some disks may have restrictions or recommendations on I/O operation sizes:
# fdisk -l /dev/sdb
Disk /dev/sdb: 1.8 TiB, 2000398934016 bytes, 3907029168 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes
For example, this HDD internally uses 4k sector size, although it emulates traditional 512-byte disk sectors. As a result, a minimum I/O size of 4k is specified.
That's a pretty complex topic, and depends on the disk, the disk controller and kernel settings.
In general, the kernel will attempt to be as efficient as it can. For example, if you update the same block multiple times within an adjustable time window (typically 30 seconds or so), and don't explicitly force syncing all the way to the disk each time, most of your write operations will only update the data in the cache and only the ultimate result will actually go to the disk.
If you write a long series of consecutive blocks, the kernel will certainly attempt to execute it in as few and as large chunks as the storage controller and the disk itself will allow.
The kernel's I/O scheduler may also optimize the ordering of disk operations to achieve most efficient disk access. This optimization can be mostly irrelevant in virtual machines and on SSDs, and so it can be switched off. (SSDs are plenty fast even if you access random blocks in a shotgun fashion; on virtual machines, the hypervisor will usually redo the optimization based on the entire set of VMs and all their disk operations anyway, so trying to micro-optimize on the level of a single VM is wasted effort.)
Some disks may have restrictions or recommendations on I/O operation sizes:
# fdisk -l /dev/sdb
Disk /dev/sdb: 1.8 TiB, 2000398934016 bytes, 3907029168 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes
For example, this HDD internally uses 4k sector size, although it emulates traditional 512-byte disk sectors. As a result, a minimum I/O size of 4k is specified.
edited yesterday
answered Nov 23 at 9:41
telcoM
14.6k11842
14.6k11842
add a comment |
add a comment |
TsReaper is a new contributor. Be nice, and check out our Code of Conduct.
TsReaper is a new contributor. Be nice, and check out our Code of Conduct.
TsReaper is a new contributor. Be nice, and check out our Code of Conduct.
TsReaper is a new contributor. Be nice, and check out our Code of Conduct.
Thanks for contributing an answer to Unix & Linux Stack Exchange!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Some of your past answers have not been well-received, and you're in danger of being blocked from answering.
Please pay close attention to the following guidance:
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2funix.stackexchange.com%2fquestions%2f483620%2fdoes-linux-systems-flush-dirty-pages-to-disk-one-block-by-one-block%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown