Does linux systems flush dirty pages to disk one block by one block?











up vote
0
down vote

favorite












As far as I know, in linux systems, when modifying the content of a file, the pages containing the content of the file in page cache will be marked dirty, and will be flushed to disk eventually.



What I'm wondering is that, when these pages are flushed to disk, are they flushed in blocks?



For example, if the block size is 4kB and I need to flush some content of 1024kB, is the disk written 1024 / 4 = 256 times?










share|improve this question







New contributor




TsReaper is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
























    up vote
    0
    down vote

    favorite












    As far as I know, in linux systems, when modifying the content of a file, the pages containing the content of the file in page cache will be marked dirty, and will be flushed to disk eventually.



    What I'm wondering is that, when these pages are flushed to disk, are they flushed in blocks?



    For example, if the block size is 4kB and I need to flush some content of 1024kB, is the disk written 1024 / 4 = 256 times?










    share|improve this question







    New contributor




    TsReaper is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
    Check out our Code of Conduct.






















      up vote
      0
      down vote

      favorite









      up vote
      0
      down vote

      favorite











      As far as I know, in linux systems, when modifying the content of a file, the pages containing the content of the file in page cache will be marked dirty, and will be flushed to disk eventually.



      What I'm wondering is that, when these pages are flushed to disk, are they flushed in blocks?



      For example, if the block size is 4kB and I need to flush some content of 1024kB, is the disk written 1024 / 4 = 256 times?










      share|improve this question







      New contributor




      TsReaper is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
      Check out our Code of Conduct.











      As far as I know, in linux systems, when modifying the content of a file, the pages containing the content of the file in page cache will be marked dirty, and will be flushed to disk eventually.



      What I'm wondering is that, when these pages are flushed to disk, are they flushed in blocks?



      For example, if the block size is 4kB and I need to flush some content of 1024kB, is the disk written 1024 / 4 = 256 times?







      linux disk io






      share|improve this question







      New contributor




      TsReaper is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
      Check out our Code of Conduct.











      share|improve this question







      New contributor




      TsReaper is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
      Check out our Code of Conduct.









      share|improve this question




      share|improve this question






      New contributor




      TsReaper is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
      Check out our Code of Conduct.









      asked Nov 23 at 8:35









      TsReaper

      1033




      1033




      New contributor




      TsReaper is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
      Check out our Code of Conduct.





      New contributor





      TsReaper is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
      Check out our Code of Conduct.






      TsReaper is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
      Check out our Code of Conduct.






















          1 Answer
          1






          active

          oldest

          votes

















          up vote
          2
          down vote



          accepted










          That's a pretty complex topic, and depends on the disk, the disk controller and kernel settings.



          In general, the kernel will attempt to be as efficient as it can. For example, if you update the same block multiple times within an adjustable time window (typically 30 seconds or so), and don't explicitly force syncing all the way to the disk each time, most of your write operations will only update the data in the cache and only the ultimate result will actually go to the disk.



          If you write a long series of consecutive blocks, the kernel will certainly attempt to execute it in as few and as large chunks as the storage controller and the disk itself will allow.



          The kernel's I/O scheduler may also optimize the ordering of disk operations to achieve most efficient disk access. This optimization can be mostly irrelevant in virtual machines and on SSDs, and so it can be switched off. (SSDs are plenty fast even if you access random blocks in a shotgun fashion; on virtual machines, the hypervisor will usually redo the optimization based on the entire set of VMs and all their disk operations anyway, so trying to micro-optimize on the level of a single VM is wasted effort.)



          Some disks may have restrictions or recommendations on I/O operation sizes:



          # fdisk -l /dev/sdb
          Disk /dev/sdb: 1.8 TiB, 2000398934016 bytes, 3907029168 sectors
          Units: sectors of 1 * 512 = 512 bytes
          Sector size (logical/physical): 512 bytes / 4096 bytes
          I/O size (minimum/optimal): 4096 bytes / 4096 bytes


          For example, this HDD internally uses 4k sector size, although it emulates traditional 512-byte disk sectors. As a result, a minimum I/O size of 4k is specified.






          share|improve this answer























            Your Answer








            StackExchange.ready(function() {
            var channelOptions = {
            tags: "".split(" "),
            id: "106"
            };
            initTagRenderer("".split(" "), "".split(" "), channelOptions);

            StackExchange.using("externalEditor", function() {
            // Have to fire editor after snippets, if snippets enabled
            if (StackExchange.settings.snippets.snippetsEnabled) {
            StackExchange.using("snippets", function() {
            createEditor();
            });
            }
            else {
            createEditor();
            }
            });

            function createEditor() {
            StackExchange.prepareEditor({
            heartbeatType: 'answer',
            convertImagesToLinks: false,
            noModals: true,
            showLowRepImageUploadWarning: true,
            reputationToPostImages: null,
            bindNavPrevention: true,
            postfix: "",
            imageUploader: {
            brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
            contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
            allowUrls: true
            },
            onDemand: true,
            discardSelector: ".discard-answer"
            ,immediatelyShowMarkdownHelp:true
            });


            }
            });






            TsReaper is a new contributor. Be nice, and check out our Code of Conduct.










            draft saved

            draft discarded


















            StackExchange.ready(
            function () {
            StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2funix.stackexchange.com%2fquestions%2f483620%2fdoes-linux-systems-flush-dirty-pages-to-disk-one-block-by-one-block%23new-answer', 'question_page');
            }
            );

            Post as a guest















            Required, but never shown

























            1 Answer
            1






            active

            oldest

            votes








            1 Answer
            1






            active

            oldest

            votes









            active

            oldest

            votes






            active

            oldest

            votes








            up vote
            2
            down vote



            accepted










            That's a pretty complex topic, and depends on the disk, the disk controller and kernel settings.



            In general, the kernel will attempt to be as efficient as it can. For example, if you update the same block multiple times within an adjustable time window (typically 30 seconds or so), and don't explicitly force syncing all the way to the disk each time, most of your write operations will only update the data in the cache and only the ultimate result will actually go to the disk.



            If you write a long series of consecutive blocks, the kernel will certainly attempt to execute it in as few and as large chunks as the storage controller and the disk itself will allow.



            The kernel's I/O scheduler may also optimize the ordering of disk operations to achieve most efficient disk access. This optimization can be mostly irrelevant in virtual machines and on SSDs, and so it can be switched off. (SSDs are plenty fast even if you access random blocks in a shotgun fashion; on virtual machines, the hypervisor will usually redo the optimization based on the entire set of VMs and all their disk operations anyway, so trying to micro-optimize on the level of a single VM is wasted effort.)



            Some disks may have restrictions or recommendations on I/O operation sizes:



            # fdisk -l /dev/sdb
            Disk /dev/sdb: 1.8 TiB, 2000398934016 bytes, 3907029168 sectors
            Units: sectors of 1 * 512 = 512 bytes
            Sector size (logical/physical): 512 bytes / 4096 bytes
            I/O size (minimum/optimal): 4096 bytes / 4096 bytes


            For example, this HDD internally uses 4k sector size, although it emulates traditional 512-byte disk sectors. As a result, a minimum I/O size of 4k is specified.






            share|improve this answer



























              up vote
              2
              down vote



              accepted










              That's a pretty complex topic, and depends on the disk, the disk controller and kernel settings.



              In general, the kernel will attempt to be as efficient as it can. For example, if you update the same block multiple times within an adjustable time window (typically 30 seconds or so), and don't explicitly force syncing all the way to the disk each time, most of your write operations will only update the data in the cache and only the ultimate result will actually go to the disk.



              If you write a long series of consecutive blocks, the kernel will certainly attempt to execute it in as few and as large chunks as the storage controller and the disk itself will allow.



              The kernel's I/O scheduler may also optimize the ordering of disk operations to achieve most efficient disk access. This optimization can be mostly irrelevant in virtual machines and on SSDs, and so it can be switched off. (SSDs are plenty fast even if you access random blocks in a shotgun fashion; on virtual machines, the hypervisor will usually redo the optimization based on the entire set of VMs and all their disk operations anyway, so trying to micro-optimize on the level of a single VM is wasted effort.)



              Some disks may have restrictions or recommendations on I/O operation sizes:



              # fdisk -l /dev/sdb
              Disk /dev/sdb: 1.8 TiB, 2000398934016 bytes, 3907029168 sectors
              Units: sectors of 1 * 512 = 512 bytes
              Sector size (logical/physical): 512 bytes / 4096 bytes
              I/O size (minimum/optimal): 4096 bytes / 4096 bytes


              For example, this HDD internally uses 4k sector size, although it emulates traditional 512-byte disk sectors. As a result, a minimum I/O size of 4k is specified.






              share|improve this answer

























                up vote
                2
                down vote



                accepted







                up vote
                2
                down vote



                accepted






                That's a pretty complex topic, and depends on the disk, the disk controller and kernel settings.



                In general, the kernel will attempt to be as efficient as it can. For example, if you update the same block multiple times within an adjustable time window (typically 30 seconds or so), and don't explicitly force syncing all the way to the disk each time, most of your write operations will only update the data in the cache and only the ultimate result will actually go to the disk.



                If you write a long series of consecutive blocks, the kernel will certainly attempt to execute it in as few and as large chunks as the storage controller and the disk itself will allow.



                The kernel's I/O scheduler may also optimize the ordering of disk operations to achieve most efficient disk access. This optimization can be mostly irrelevant in virtual machines and on SSDs, and so it can be switched off. (SSDs are plenty fast even if you access random blocks in a shotgun fashion; on virtual machines, the hypervisor will usually redo the optimization based on the entire set of VMs and all their disk operations anyway, so trying to micro-optimize on the level of a single VM is wasted effort.)



                Some disks may have restrictions or recommendations on I/O operation sizes:



                # fdisk -l /dev/sdb
                Disk /dev/sdb: 1.8 TiB, 2000398934016 bytes, 3907029168 sectors
                Units: sectors of 1 * 512 = 512 bytes
                Sector size (logical/physical): 512 bytes / 4096 bytes
                I/O size (minimum/optimal): 4096 bytes / 4096 bytes


                For example, this HDD internally uses 4k sector size, although it emulates traditional 512-byte disk sectors. As a result, a minimum I/O size of 4k is specified.






                share|improve this answer














                That's a pretty complex topic, and depends on the disk, the disk controller and kernel settings.



                In general, the kernel will attempt to be as efficient as it can. For example, if you update the same block multiple times within an adjustable time window (typically 30 seconds or so), and don't explicitly force syncing all the way to the disk each time, most of your write operations will only update the data in the cache and only the ultimate result will actually go to the disk.



                If you write a long series of consecutive blocks, the kernel will certainly attempt to execute it in as few and as large chunks as the storage controller and the disk itself will allow.



                The kernel's I/O scheduler may also optimize the ordering of disk operations to achieve most efficient disk access. This optimization can be mostly irrelevant in virtual machines and on SSDs, and so it can be switched off. (SSDs are plenty fast even if you access random blocks in a shotgun fashion; on virtual machines, the hypervisor will usually redo the optimization based on the entire set of VMs and all their disk operations anyway, so trying to micro-optimize on the level of a single VM is wasted effort.)



                Some disks may have restrictions or recommendations on I/O operation sizes:



                # fdisk -l /dev/sdb
                Disk /dev/sdb: 1.8 TiB, 2000398934016 bytes, 3907029168 sectors
                Units: sectors of 1 * 512 = 512 bytes
                Sector size (logical/physical): 512 bytes / 4096 bytes
                I/O size (minimum/optimal): 4096 bytes / 4096 bytes


                For example, this HDD internally uses 4k sector size, although it emulates traditional 512-byte disk sectors. As a result, a minimum I/O size of 4k is specified.







                share|improve this answer














                share|improve this answer



                share|improve this answer








                edited yesterday

























                answered Nov 23 at 9:41









                telcoM

                14.6k11842




                14.6k11842






















                    TsReaper is a new contributor. Be nice, and check out our Code of Conduct.










                    draft saved

                    draft discarded


















                    TsReaper is a new contributor. Be nice, and check out our Code of Conduct.













                    TsReaper is a new contributor. Be nice, and check out our Code of Conduct.












                    TsReaper is a new contributor. Be nice, and check out our Code of Conduct.
















                    Thanks for contributing an answer to Unix & Linux Stack Exchange!


                    • Please be sure to answer the question. Provide details and share your research!

                    But avoid



                    • Asking for help, clarification, or responding to other answers.

                    • Making statements based on opinion; back them up with references or personal experience.


                    To learn more, see our tips on writing great answers.





                    Some of your past answers have not been well-received, and you're in danger of being blocked from answering.


                    Please pay close attention to the following guidance:


                    • Please be sure to answer the question. Provide details and share your research!

                    But avoid



                    • Asking for help, clarification, or responding to other answers.

                    • Making statements based on opinion; back them up with references or personal experience.


                    To learn more, see our tips on writing great answers.




                    draft saved


                    draft discarded














                    StackExchange.ready(
                    function () {
                    StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2funix.stackexchange.com%2fquestions%2f483620%2fdoes-linux-systems-flush-dirty-pages-to-disk-one-block-by-one-block%23new-answer', 'question_page');
                    }
                    );

                    Post as a guest















                    Required, but never shown





















































                    Required, but never shown














                    Required, but never shown












                    Required, but never shown







                    Required, but never shown

































                    Required, but never shown














                    Required, but never shown












                    Required, but never shown







                    Required, but never shown







                    Popular posts from this blog

                    Scott Moir

                    Souastre

                    Morgemoulin