How to fill a device with zeros, without overwriting the bytes that are already zeros?
I have a USB flash drive usb 3, the reading speed is much more than the writing speed.
Let's say that 99% of the flash memory is already full with zeros, and I would like to fill it with zeros until 100%, by overwriting all the memory flash with zeros dd if=/dev/zero of=/dev/FLASH.
This process is going to be long, and it will minimize the life expectancy of the flash drive.
I thought, maybe it would be much quicker to check which areas are non-zero, and overwrite only those non-zertos areas with zeros?
Are there anyways of doing this? If it is interesting, I would need all this for security reasons.
security dd
asked Sep 16 '18 at 15:30
Slok
82
add a comment |
I have a USB flash drive usb 3, the reading speed is much more than the writing speed.
Let's say that 99% of the flash memory is already full with zeros, and I would like to fill it with zeros until 100%, by overwriting all the memory flash with zeros dd if=/dev/zero of=/dev/FLASH.
This process is going to be long, and it will minimize the life expectancy of the flash drive.
I thought, maybe it would be much quicker to check which areas are non-zero, and overwrite only those non-zertos areas with zeros?
Are there anyways of doing this? If it is interesting, I would need all this for security reasons.
security dd
asked Sep 16 '18 at 15:30
Slok
82
Does the device have a file system on it? And files? Mainly, though, if this is a one-time affair, thendd if=/dev/zero of=/dev/FLASHwould be a heck of a lot faster than writing a program to read the drive one block at a time, comparing it to zeros, and rewriting it if it is.
– RonJohn
Sep 16 '18 at 16:55
dd if=/dev/zero of=/dev/FLASHis not secure enough for legal/compliance purposes, if that is a concern readmoar on secure erase.
– mikst
Sep 16 '18 at 17:22
who do you need to be secure from? what sort of budget does the attacker have? writing zeros to a flash drive may only hide the data.
– Jasen
Sep 16 '18 at 20:45
@RonJohn imagine that there is no file system. I want to be 100% that there are only zeros on the flash drive, regardless of whether there is a file system there or not.
– Slok
Sep 17 '18 at 14:51
@Jasen filling the flash drive with zeros I consider sufficient in my circumstances.
– Slok
Sep 17 '18 at 14:51
add a comment |
I have a USB flash drive usb 3, the reading speed is much more than the writing speed.
Let's say that 99% of the flash memory is already full with zeros, and I would like to fill it with zeros until 100%, by overwriting all the memory flash with zeros dd if=/dev/zero of=/dev/FLASH.
This process is going to be long, and it will minimize the life expectancy of the flash drive.
I thought, maybe it would be much quicker to check which areas are non-zero, and overwrite only those non-zertos areas with zeros?
Are there anyways of doing this? If it is interesting, I would need all this for security reasons.
security dd
asked Sep 16 '18 at 15:30
Slok
82
I have a USB flash drive usb 3, the reading speed is much more than the writing speed.
Let's say that 99% of the flash memory is already full with zeros, and I would like to fill it with zeros until 100%, by overwriting all the memory flash with zeros dd if=/dev/zero of=/dev/FLASH.
This process is going to be long, and it will minimize the life expectancy of the flash drive.
I thought, maybe it would be much quicker to check which areas are non-zero, and overwrite only those non-zertos areas with zeros?
Are there anyways of doing this? If it is interesting, I would need all this for security reasons.
security dd
security dd
asked Sep 16 '18 at 15:30
Slok
82
asked Sep 16 '18 at 15:30
Slok
82
edited Sep 16 '18 at 16:04
user88036
asked Sep 16 '18 at 15:30
Slok
82
asked Sep 16 '18 at 15:30
Slok
82
asked Sep 16 '18 at 15:30
Slok
82
82
Does the device have a file system on it? And files? Mainly, though, if this is a one-time affair, thendd if=/dev/zero of=/dev/FLASHwould be a heck of a lot faster than writing a program to read the drive one block at a time, comparing it to zeros, and rewriting it if it is.
– RonJohn
Sep 16 '18 at 16:55
dd if=/dev/zero of=/dev/FLASHis not secure enough for legal/compliance purposes, if that is a concern readmoar on secure erase.
– mikst
Sep 16 '18 at 17:22
who do you need to be secure from? what sort of budget does the attacker have? writing zeros to a flash drive may only hide the data.
– Jasen
Sep 16 '18 at 20:45
@RonJohn imagine that there is no file system. I want to be 100% that there are only zeros on the flash drive, regardless of whether there is a file system there or not.
– Slok
Sep 17 '18 at 14:51
@Jasen filling the flash drive with zeros I consider sufficient in my circumstances.
– Slok
Sep 17 '18 at 14:51
add a comment |
Does the device have a file system on it? And files? Mainly, though, if this is a one-time affair, thendd if=/dev/zero of=/dev/FLASHwould be a heck of a lot faster than writing a program to read the drive one block at a time, comparing it to zeros, and rewriting it if it is.
– RonJohn
Sep 16 '18 at 16:55
dd if=/dev/zero of=/dev/FLASHis not secure enough for legal/compliance purposes, if that is a concern readmoar on secure erase.
– mikst
Sep 16 '18 at 17:22
who do you need to be secure from? what sort of budget does the attacker have? writing zeros to a flash drive may only hide the data.
– Jasen
Sep 16 '18 at 20:45
@RonJohn imagine that there is no file system. I want to be 100% that there are only zeros on the flash drive, regardless of whether there is a file system there or not.
– Slok
Sep 17 '18 at 14:51
@Jasen filling the flash drive with zeros I consider sufficient in my circumstances.
– Slok
Sep 17 '18 at 14:51
Does the device have a file system on it? And files? Mainly, though, if this is a one-time affair, then
dd if=/dev/zero of=/dev/FLASH would be a heck of a lot faster than writing a program to read the drive one block at a time, comparing it to zeros, and rewriting it if it is.– RonJohn
Sep 16 '18 at 16:55
Does the device have a file system on it? And files? Mainly, though, if this is a one-time affair, then
dd if=/dev/zero of=/dev/FLASH would be a heck of a lot faster than writing a program to read the drive one block at a time, comparing it to zeros, and rewriting it if it is.– RonJohn
Sep 16 '18 at 16:55
dd if=/dev/zero of=/dev/FLASH is not secure enough for legal/compliance purposes, if that is a concern readmoar on secure erase.– mikst
Sep 16 '18 at 17:22
dd if=/dev/zero of=/dev/FLASH is not secure enough for legal/compliance purposes, if that is a concern readmoar on secure erase.– mikst
Sep 16 '18 at 17:22
who do you need to be secure from? what sort of budget does the attacker have? writing zeros to a flash drive may only hide the data.
– Jasen
Sep 16 '18 at 20:45
who do you need to be secure from? what sort of budget does the attacker have? writing zeros to a flash drive may only hide the data.
– Jasen
Sep 16 '18 at 20:45
@RonJohn imagine that there is no file system. I want to be 100% that there are only zeros on the flash drive, regardless of whether there is a file system there or not.
– Slok
Sep 17 '18 at 14:51
@RonJohn imagine that there is no file system. I want to be 100% that there are only zeros on the flash drive, regardless of whether there is a file system there or not.
– Slok
Sep 17 '18 at 14:51
@Jasen filling the flash drive with zeros I consider sufficient in my circumstances.
– Slok
Sep 17 '18 at 14:51
@Jasen filling the flash drive with zeros I consider sufficient in my circumstances.
– Slok
Sep 17 '18 at 14:51
add a comment |
1 Answer
1
active
oldest
votes
Security reasons aside, let's do it. We can (ab)use GNU ddrescue.
To detect sectors of zeros --generate-mode is useful.
When
ddrescueis invoked with the--generate-modeoption it operates in "generate mode", which is different from the default "rescue mode". That is, if you use the--generate-modeoption,ddrescuedoes not rescue anything. It only tries to generate amapfilefor later use.
[…]
ddrescuecan in some cases generate an approximatemapfile, frominfileand the (partial) copy inoutfile, that is almost as good as an exactmapfile. It makes this by simply assuming that sectors containing all zeros were not rescued.
[…]
ddrescue --generate-mode infile outfile mapfile
(source)
Let's pretend your device is outfile from previous ddrescue run. We cannot use it as infile (because ddrescue refuses to work when infile and outfile are the same file), we need a dummy one, /dev/zero will do. We should know the physical sector size of your device and use it with -b option. This command may help:
lsblk -o NAME,PHY-SeC /dev/FLASH
Here I assume it's 512.
ddrescue -b 512 --generate-mode /dev/zero /dev/FLASH flash.map
Now flash.map describes every sector either as non-tried (?) or as finished (+), depending on whether it was full of zeros or not. The next step is to fill non-zero sectors with zeros; --fill-mode is perfect for this job:
When
ddrescueis invoked with the--fill-modeoption it operates in "fill mode", which is different from the default "rescue mode". That is, if you use the--fill-modeoption,ddrescuedoes not rescue anything. It only fills with data read frominfilethe blocks ofoutfilewhose status character frommapfilecoincides with one of the type characters specified as argument to the--fill-modeoption.
(source)
We must use the same -b value as with --generate-mode, additionally --force to overwrite the output device. This is the command:
ddrescue -b 512 --force --fill-mode=+ /dev/zero /dev/FLASH flash.map
This time /dev/zero is not just a dummy argument, it's the actual source of data (zeros) written to the device.
After ddrescue finishes, invoke sync. Now /dev/FLASH is filled with zeros.
answered Oct 29 '18 at 23:29
Kamil Maciorowski
1,1441625
add a comment |
Your Answer
StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "106"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});
function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: false,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: null,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});
}
});
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2funix.stackexchange.com%2fquestions%2f469399%2fhow-to-fill-a-device-with-zeros-without-overwriting-the-bytes-that-are-already%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
1 Answer
1
active
oldest
votes
1 Answer
1
active
oldest
votes
active
oldest
votes
active
oldest
votes
Security reasons aside, let's do it. We can (ab)use GNU ddrescue.
To detect sectors of zeros --generate-mode is useful.
When
ddrescueis invoked with the--generate-modeoption it operates in "generate mode", which is different from the default "rescue mode". That is, if you use the--generate-modeoption,ddrescuedoes not rescue anything. It only tries to generate amapfilefor later use.
[…]
ddrescuecan in some cases generate an approximatemapfile, frominfileand the (partial) copy inoutfile, that is almost as good as an exactmapfile. It makes this by simply assuming that sectors containing all zeros were not rescued.
[…]
ddrescue --generate-mode infile outfile mapfile
(source)
Let's pretend your device is outfile from previous ddrescue run. We cannot use it as infile (because ddrescue refuses to work when infile and outfile are the same file), we need a dummy one, /dev/zero will do. We should know the physical sector size of your device and use it with -b option. This command may help:
lsblk -o NAME,PHY-SeC /dev/FLASH
Here I assume it's 512.
ddrescue -b 512 --generate-mode /dev/zero /dev/FLASH flash.map
Now flash.map describes every sector either as non-tried (?) or as finished (+), depending on whether it was full of zeros or not. The next step is to fill non-zero sectors with zeros; --fill-mode is perfect for this job:
When
ddrescueis invoked with the--fill-modeoption it operates in "fill mode", which is different from the default "rescue mode". That is, if you use the--fill-modeoption,ddrescuedoes not rescue anything. It only fills with data read frominfilethe blocks ofoutfilewhose status character frommapfilecoincides with one of the type characters specified as argument to the--fill-modeoption.
(source)
We must use the same -b value as with --generate-mode, additionally --force to overwrite the output device. This is the command:
ddrescue -b 512 --force --fill-mode=+ /dev/zero /dev/FLASH flash.map
This time /dev/zero is not just a dummy argument, it's the actual source of data (zeros) written to the device.
After ddrescue finishes, invoke sync. Now /dev/FLASH is filled with zeros.
answered Oct 29 '18 at 23:29
Kamil Maciorowski
1,1441625
add a comment |
Security reasons aside, let's do it. We can (ab)use GNU ddrescue.
To detect sectors of zeros --generate-mode is useful.
When
ddrescueis invoked with the--generate-modeoption it operates in "generate mode", which is different from the default "rescue mode". That is, if you use the--generate-modeoption,ddrescuedoes not rescue anything. It only tries to generate amapfilefor later use.
[…]
ddrescuecan in some cases generate an approximatemapfile, frominfileand the (partial) copy inoutfile, that is almost as good as an exactmapfile. It makes this by simply assuming that sectors containing all zeros were not rescued.
[…]
ddrescue --generate-mode infile outfile mapfile
(source)
Let's pretend your device is outfile from previous ddrescue run. We cannot use it as infile (because ddrescue refuses to work when infile and outfile are the same file), we need a dummy one, /dev/zero will do. We should know the physical sector size of your device and use it with -b option. This command may help:
lsblk -o NAME,PHY-SeC /dev/FLASH
Here I assume it's 512.
ddrescue -b 512 --generate-mode /dev/zero /dev/FLASH flash.map
Now flash.map describes every sector either as non-tried (?) or as finished (+), depending on whether it was full of zeros or not. The next step is to fill non-zero sectors with zeros; --fill-mode is perfect for this job:
When
ddrescueis invoked with the--fill-modeoption it operates in "fill mode", which is different from the default "rescue mode". That is, if you use the--fill-modeoption,ddrescuedoes not rescue anything. It only fills with data read frominfilethe blocks ofoutfilewhose status character frommapfilecoincides with one of the type characters specified as argument to the--fill-modeoption.
(source)
We must use the same -b value as with --generate-mode, additionally --force to overwrite the output device. This is the command:
ddrescue -b 512 --force --fill-mode=+ /dev/zero /dev/FLASH flash.map
This time /dev/zero is not just a dummy argument, it's the actual source of data (zeros) written to the device.
After ddrescue finishes, invoke sync. Now /dev/FLASH is filled with zeros.
answered Oct 29 '18 at 23:29
Kamil Maciorowski
1,1441625
add a comment |
Security reasons aside, let's do it. We can (ab)use GNU ddrescue.
To detect sectors of zeros --generate-mode is useful.
When
ddrescueis invoked with the--generate-modeoption it operates in "generate mode", which is different from the default "rescue mode". That is, if you use the--generate-modeoption,ddrescuedoes not rescue anything. It only tries to generate amapfilefor later use.
[…]
ddrescuecan in some cases generate an approximatemapfile, frominfileand the (partial) copy inoutfile, that is almost as good as an exactmapfile. It makes this by simply assuming that sectors containing all zeros were not rescued.
[…]
ddrescue --generate-mode infile outfile mapfile
(source)
Let's pretend your device is outfile from previous ddrescue run. We cannot use it as infile (because ddrescue refuses to work when infile and outfile are the same file), we need a dummy one, /dev/zero will do. We should know the physical sector size of your device and use it with -b option. This command may help:
lsblk -o NAME,PHY-SeC /dev/FLASH
Here I assume it's 512.
ddrescue -b 512 --generate-mode /dev/zero /dev/FLASH flash.map
Now flash.map describes every sector either as non-tried (?) or as finished (+), depending on whether it was full of zeros or not. The next step is to fill non-zero sectors with zeros; --fill-mode is perfect for this job:
When
ddrescueis invoked with the--fill-modeoption it operates in "fill mode", which is different from the default "rescue mode". That is, if you use the--fill-modeoption,ddrescuedoes not rescue anything. It only fills with data read frominfilethe blocks ofoutfilewhose status character frommapfilecoincides with one of the type characters specified as argument to the--fill-modeoption.
(source)
We must use the same -b value as with --generate-mode, additionally --force to overwrite the output device. This is the command:
ddrescue -b 512 --force --fill-mode=+ /dev/zero /dev/FLASH flash.map
This time /dev/zero is not just a dummy argument, it's the actual source of data (zeros) written to the device.
After ddrescue finishes, invoke sync. Now /dev/FLASH is filled with zeros.
answered Oct 29 '18 at 23:29
Kamil Maciorowski
1,1441625
Security reasons aside, let's do it. We can (ab)use GNU ddrescue.
To detect sectors of zeros --generate-mode is useful.
When
ddrescueis invoked with the--generate-modeoption it operates in "generate mode", which is different from the default "rescue mode". That is, if you use the--generate-modeoption,ddrescuedoes not rescue anything. It only tries to generate amapfilefor later use.
[…]
ddrescuecan in some cases generate an approximatemapfile, frominfileand the (partial) copy inoutfile, that is almost as good as an exactmapfile. It makes this by simply assuming that sectors containing all zeros were not rescued.
[…]
ddrescue --generate-mode infile outfile mapfile
(source)
Let's pretend your device is outfile from previous ddrescue run. We cannot use it as infile (because ddrescue refuses to work when infile and outfile are the same file), we need a dummy one, /dev/zero will do. We should know the physical sector size of your device and use it with -b option. This command may help:
lsblk -o NAME,PHY-SeC /dev/FLASH
Here I assume it's 512.
ddrescue -b 512 --generate-mode /dev/zero /dev/FLASH flash.map
Now flash.map describes every sector either as non-tried (?) or as finished (+), depending on whether it was full of zeros or not. The next step is to fill non-zero sectors with zeros; --fill-mode is perfect for this job:
When
ddrescueis invoked with the--fill-modeoption it operates in "fill mode", which is different from the default "rescue mode". That is, if you use the--fill-modeoption,ddrescuedoes not rescue anything. It only fills with data read frominfilethe blocks ofoutfilewhose status character frommapfilecoincides with one of the type characters specified as argument to the--fill-modeoption.
(source)
We must use the same -b value as with --generate-mode, additionally --force to overwrite the output device. This is the command:
ddrescue -b 512 --force --fill-mode=+ /dev/zero /dev/FLASH flash.map
This time /dev/zero is not just a dummy argument, it's the actual source of data (zeros) written to the device.
After ddrescue finishes, invoke sync. Now /dev/FLASH is filled with zeros.
answered Oct 29 '18 at 23:29
Kamil Maciorowski
1,1441625
edited Dec 19 '18 at 7:38
answered Oct 29 '18 at 23:29
Kamil Maciorowski
1,1441625
answered Oct 29 '18 at 23:29
Kamil Maciorowski
1,1441625
answered Oct 29 '18 at 23:29
Kamil Maciorowski
1,1441625
1,1441625
add a comment |
add a comment |
Thanks for contributing an answer to Unix & Linux Stack Exchange!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Some of your past answers have not been well-received, and you're in danger of being blocked from answering.
Please pay close attention to the following guidance:
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2funix.stackexchange.com%2fquestions%2f469399%2fhow-to-fill-a-device-with-zeros-without-overwriting-the-bytes-that-are-already%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Does the device have a file system on it? And files? Mainly, though, if this is a one-time affair, then
dd if=/dev/zero of=/dev/FLASHwould be a heck of a lot faster than writing a program to read the drive one block at a time, comparing it to zeros, and rewriting it if it is.– RonJohn
Sep 16 '18 at 16:55
dd if=/dev/zero of=/dev/FLASHis not secure enough for legal/compliance purposes, if that is a concern readmoar on secure erase.– mikst
Sep 16 '18 at 17:22
who do you need to be secure from? what sort of budget does the attacker have? writing zeros to a flash drive may only hide the data.
– Jasen
Sep 16 '18 at 20:45
@RonJohn imagine that there is no file system. I want to be 100% that there are only zeros on the flash drive, regardless of whether there is a file system there or not.
– Slok
Sep 17 '18 at 14:51
@Jasen filling the flash drive with zeros I consider sufficient in my circumstances.
– Slok
Sep 17 '18 at 14:51