Securely removing data from smartphones (tested on WFS with CM7.2)

Securely removing data from smartphones (tested on WFS with CM7.2) - HTC Wildfire S

Due to my strong involvement with IT security, today I made an analysis of how effective several wipe-options are on Android phones. I tested this on an HTC Wildfire S (Marvel) running CWM-5.0.2.8 recovery and an inofficial CyanogenMod 7.2 build from 2012-07-17 as ROM.
For the test I first created a memory dump as well as a NANDroid backup of my entire phone. The memory dump was created using the following commands on an adb shell with the Recovery.
Code:
cat /dev/mtd/mtd4 > /sdcard/pre-wipe/cache.img
cat /dev/mtd/mtd5 > /sdcard/pre-wipe/userdata.img
cat /dev/mtd/mtd6 > /sdcard/pre-wipe/devlog.img
I didn't dump mtd0 to mtd3 since these should contain firmware-related data only so there should be no sensitive information here.
Then I booted the phone into CM and did a factory reset. After this was performed, I went back into recovery and created another set of dumps under "/sdcard/wipe-android/*". Afterwards I booted into CWM and restored the NANDdroid backup to prepare for the next test.
I then used the wiping options of CWM on the phone and created a set of dumps under "/sdcard/wipe-cwm/*". Then I restored the NANDroid backup to prepare for the next test.
I shut down the phone and booted it into HBOOT mode, performing a "Factory Reset". After this, HBOOT will boot the ROM, at which point I pulled the battery to prevent the ROM from writing too much to the wiped partitions.
Then I booted back into CWM, took a build of "mtdutils" and did an erasure of mtd5 (userdata) followed by a dump to "/sdcard/wipe-mtdutils/userdata.img".
Then I loaded all these images onto my computer and did a hex comparison. I went through the "pre-wipe/userdata.img" image and saw that the string "Thank you" appears lots and lots of times in several e-mails and SMS messages and so on. So I went through the other userdata dumps and searched for "Thank you" but nothing was found. Not even in the simple "Factory reset" case performed from within the ROM. I expected the ROM to run "mkfs" getting rid of the file system but not of actual data (like what happens when you "format" a drive on a PC - the operating system won't be able to find data anymore, but it's still there on the device and ready to be recovered with the appropriate tools - at least on magnetic hard drives - solid-state-drives make use of the TRIM/Discard command that newer operating systems implement that will actually erase data from the Flash memory inside the SSD, even though it's not considered "secure" and "ATA secure erase" should be used on such drives when security against remanent data is important - also beware that, unless you run Linux and massively tweak your operating system, TRIM/Discard is not used on encrypted drives) but it appears that the data is actually gone (even though some metadata was still in place).
What's interesting is that the images look different after all "wipe" methods. The wipe from within Android leaves most metadata intact, followed by the wipe from within HBOOT, then the wipe from within CWM and the wipe using "mtdutils", as we expected, actually leaves nothing on the device. All pages are cleared to either 2048 bytes of 0x00 (all zeroes) or 2048 bytes of 0xff (all ones). What's interesting is that different pages get reset to different values, so after an "mtdutils" erase there are NAND pages that are actually fully erased (2048 bytes of 0xff), while others are fully programmed (2048 bytes of 0x00). I don't really understand why this is so, but all pages get cleared to one of these two values so this shouldn't leave traces on the device.
So conclusion? Even the most basic wiping method (Factory reset from within the ROM) seems adequate when selling the phone. Chances are that there might be something recoverable with physical access to the NAND chip or the processor (e. g. JTAG access) as the processor makes use of memory virtualization and has address space that is hidden from the user. In particular, devices with eMMC memory (e. g. devices running on MSM8xxx chipsets) might hold remanent data that's invisible from the processor as a whole, since these chips are "overprovisioned" and make use of internal (integrated into the memory package) wear-levelling and bad block replacement that is transparent to the processor and therefore firmware, much like an SSD does inside a computer. So if you have somthing to hide from the CIA, you better physically destroy the phone. However, an "mtdutils"-wipe will probably make your data unrecoverable for at least 99.99 % of the population and thus should be enough for literally any case. Even a simple factory reset from within your ROM seems to dispose of most sensitive data, so this should also serve at least 98 % of all cases.
Beware that this test was performed on a HTC Wildfire S with the (currently) most recent inofficial CyanogenMod 7.2 build from 2012-07-17 installed. Other ROMs may not be as thorough, so take the information from this post only as a guideline. When in doubt, verify the erasure yourself. It's not too hard.

Related

[NST]Touch-Formatter v2 [Factory restore, reset, update to 1.1 merged]

I am not responsible for any damage your nook suffers.
Officially supported by The Nooter Project for Nook Simple Touch
http://code.google.com/p/nooter/
Touch-Formatter
(Tool to return to stock)
Information:
What it does:
Formats: /data, /cache, /system
Installs 1.1 /system.
Regenerates /data automatically.
Bugs:
CWM may not refresh the screen correctly when booted, move the cursor with the right keys so it refreshes the screen.
If CWM hangs while rebooting, dont worry, force shutdown, and start your nook again, nothing bad happens.
Future updates: (In order of priority).
Update to 1.1.2
Be compatible with NSTG (Nook Simple Touch Glowlight)
Differentiate between the NST and NSTG (Nook Simple Touch Glowlight) so to make only one zip.
Backup /factory + Wipe the complete NST + Recreate the whole NST partition table + Restore /factory
User manual:
Things you will need:
CWM
Thread: http://forum.xda-developers.com/showthread.php?t=1360994
Direct download links:
http://forum.xda-developers.com/attachment.php?attachmentid=806435&d=1323121399
http://forum.xda-developers.com/attachment.php?attachmentid=806434&d=1323121315
Download it and burn it to an sd-card, (windows users use this to burn the image https://launchpad.net/win32-image-writer/+download)
You must have an external microSDCard reader to burn CWM, not the NST.
The button layout of CWM:
Both Buttons on the left: BACK
Upper button on the right: UP
Lower button on the right: DOWN
n button: SELECT
Power button: TOGGLE DISPLAY
Zips:
Download http://nooter.googlecode.com/files/Alpha-FormatTouch-2.zip
Old:
Download http://nooter.googlecode.com/files/Alpha-FormatTouch.zip and copy it on the sd card burnt with CWM
Instructions:
Copy the zip onto the root directory of the sdcard you burned the CWM.(Don't extract them)
Insert the sdcard on your nook, and boot it.
On CWM select install zip from sdcard
Then select choose zip from sdcard
Select Alpha-FormatTouch-2.zip, click yes and wait till the process finishes.
Go back, eject the sd card, and click reboot.
On future updates I'll try: automatically make a backup of /factroy, recreate the whole nook partition table so that people that screw hard can breathe new life into their NST easily.
Index
Automatic Methods:
[NST]MinimalTouch 1.1beta5
[NST]Touch-Formatter
Manual Tutos:
Skip registration (OOBE)
Making the manual process LESS PAINFULL
Setting up adb manually on the nook touch
Setting up root access on NST through adb and installing busybox
Improve battery life(testing)
Backup bookmarks and annotations(testing)
Enable non market app installs
Installing XorZone's B&N button modifier
Change the powered off screen image
Blocking OTA updates
Installing new fonts for your nook (testing)
Installing Gapps (+launcher, etc)
Totally uninstall Gapps (my repack), unrooting, erasing and restoring
Interesting or useful specific apps or hacks for Nook Simple Touch
nook 1.1 update
Thanks to:
ros87 for n2T-Recovery (http://forum.xda-developers.com/showthread.php?t=1289233)
mali100 for the correct command for the /data restoration and for CWM (http://forum.xda-developers.com/showthread.php?t=1360994)
bisbal for trying it out and giving ideas.
meghd00t for pointing out factory.zip is common across more than one NST and researching how to Resize Nook STR Partitions (http://forum.xda-developers.com/showthread.php?t=1225196)
dobbing for the copy of the 1.1 update.

Thanks eded333. Seems Nook touch developers are back on track. Glad to see all the busy posts. Cheer up.

eded333 said:
As some people where having trouble returning to stock after rooting, this is a semi automatic method, easy to follow, that will leave your nook stock (if you havent erased the unique data, flashing Noogie into the NST, which isnt recoverable ¬¬).
Click to expand...
Click to collapse
eded333,
Could you tell where unique data kept (what files)?
Hopefully, it’s small enough and easy to backup / zip
If Touch-Formatter can read the file from SD, it can restore unique data easily, right?

ApokrifX said:
eded333,
Could you tell where unique data kept (what files)?
Hopefully, it’s small enough and easy to backup / zip
If Touch-Formatter can read the file from SD, it can restore unique data easily, right?
Click to expand...
Click to collapse
If i'm not wrong /rom and /factory both hold unique info for every nook, as mac, etc.
If you root your device, the only partitions which are touched are /data and /system, so dont worry for that.
Yes, it should be easy to, for example, to create a Backup.zip which did a backup of those files, partitions, or anything you want and then add to this or another zip a way to restore them from the SD.
Anyway there is allready a tuto for something like that, which creates a full backup of your Nook and it should be the first step before playing with it:
http://forum.xda-developers.com/showthread.php?t=1142983
Edit:
The backup done by CWM dosn't backup /rom and /factory.

So do I have to register again after using this? Or does it stay registered? (I haven't had to wipe my Nook in a while. I'm so proud of myself! xD)

Googie2149 said:
So do I have to register again after using this? Or does it stay registered? (I haven't had to wipe my Nook in a while. I'm so proud of myself! xD)
Click to expand...
Click to collapse
This completely erases /data /cache and /system.
So... yes , you will need to register again, after using this.

eded333 said:
If i'm not wrong /rom and /factory both hold unique info for every nook, as mac, etc.
If you root your device, the only partitions which are touched are /data and /system, so dont worry for that.
Yes, it should be easy to, for example, to create a Backup.zip which did a backup of those files, partitions, or anything you want and then add to this or another zip a way to restore them from the SD.
Anyway there is allready a tuto for something like that, which creates a full backup of your Nook and it should be the first step before playing with it:
http://forum.xda-developers.com/showthread.php?t=1142983
Or you can use the latest CWM: http://forum.xda-developers.com/showthread.php?t=1360994
Click to expand...
Click to collapse
That’s exactly what I want to avoid – to create full 1.8GB backup.
Isn’t it nice to have tiny backup, email to self, just in case?
There is /rom folder, but no /factory one.
/rom “zipped” is 32KB only
Searched both threads you mentioned – cannot find anything related to /factory folder.
Does /rom/devconf backup sufficient?

ApokrifX said:
That’s exactly what I want to avoid – to create full 1.8GB backup.
Isn’t it nice to have tiny backup, email to self, just in case?
There is /rom folder, but no /factory one.
/rom “zipped” is 32KB only
Searched both threads you mentioned – cannot find anything related to /factory folder.
Does /rom/devconf backup sufficient?
Click to expand...
Click to collapse
While your idea with just backing up the unique data (which resides in both the rom partition and the factory one) might seem a good one, what happens when you screw up your NST the way that 99% of the users that asks me for help does?
If you delete/corrupt/overwrite boot, rom, factory or data, then your tiny rom backup won't help you much unless you can get a copy of the other partitions from someone else.
And then there's the problem with alignment of the data partition, which is part of an extended partition.. The first thing people usually kills is the partition table , and simply restoring it from another NST will (in 70% of the cases) not bring back the extended partitions
My vote would be a little yes and mostly no

ros87 said:
While your idea with just backing up the unique data (which resides in both the rom partition and the factory one) might seem a good one, what happens when you screw up your NST the way that 99% of the users that asks me for help does?
If you delete/corrupt/overwrite boot, rom, factory or data, then your tiny rom backup won't help you much unless you can get a copy of the other partitions from someone else.
And then there's the problem with alignment of the data partition, which is part of an extended partition.. The first thing people usually kills is the partition table , and simply restoring it from another NST will (in 70% of the cases) not bring back the extended partitions
My vote would be a little yes and mostly no
Click to expand...
Click to collapse
I think a backup of ROM itself should be a yes. Because if you have that and somehow completely absolutely destroy your partition, you will be able to with a little work and kindness from others eventually completely restore your device, in fact you could create a generic copy of the partitions blank or otherwise then use that to restore a device, have a script take the rom insert it write /boot /system etc for you and you're good to go.
However this shouldn't be used in place of a proper backup.

ros87 said:
While your idea with just backing up the unique data (which resides in both the rom partition and the factory one) might seem a good one, what happens when you screw up your NST the way that 99% of the users that asks me for help does?
If you delete/corrupt/overwrite boot, rom, factory or data, then your tiny rom backup won't help you much unless you can get a copy of the other partitions from someone else.
Click to expand...
Click to collapse
That’s where you Touch-Formatter helps me.
It’ll restore generic copy, my tiny backup makes it “personal” than.
That’s how B&N does it on factory, right?
---------- Post added at 03:43 AM ---------- Previous post was at 03:39 AM ----------
BTW: Where is factory partition?
Code:
#df
/dev: 116512K total, 0K used, 116512K available (block size 4096)
/sqlite_stmt_journals: 4096K total, 0K used, 4096K available (block size 4096)
/rom: 16116K total, 217K used, 15899K available (block size 512)
/system: 285583K total, 196911K used, 88672K available (block size 1024)
/data: 808292K total, 313252K used, 495040K available (block size 4096)
/cache: 237987K total, 8344K used, 229643K available (block size 1024)
/sdcard: 7774208K total, 113824K used, 7660384K available (block size 32768)
/media: 241947K total, 759K used, 241187K available (block size 512)
---------- Post added at 03:51 AM ---------- Previous post was at 03:43 AM ----------
GabrialDestruir said:
...in fact you could create a generic copy of the partitions blank or otherwise then use that to restore a device, have a script take the rom insert it write /boot /system etc for you and you're good to go.
Click to expand...
Click to collapse
Gabrial,
Do you think it’ll be possible to connect via adb and push back /rom partition content to restored generic image.
Providing we replaced uRamdisk and can use adb connect via USB.
Would it be sufficient?

ApokrifX said:
That’s where you Touch-Formatter helps me.
It’ll restore generic copy, my tiny backup makes it “personal” than.
That’s how B&N does it on factory, right?
---------- Post added at 03:43 AM ---------- Previous post was at 03:39 AM ----------
BTW: Where is factory partition?
Code:
#df
/dev: 116512K total, 0K used, 116512K available (block size 4096)
/sqlite_stmt_journals: 4096K total, 0K used, 4096K available (block size 4096)
/rom: 16116K total, 217K used, 15899K available (block size 512)
/system: 285583K total, 196911K used, 88672K available (block size 1024)
/data: 808292K total, 313252K used, 495040K available (block size 4096)
/cache: 237987K total, 8344K used, 229643K available (block size 1024)
/sdcard: 7774208K total, 113824K used, 7660384K available (block size 32768)
/media: 241947K total, 759K used, 241187K available (block size 512)
---------- Post added at 03:51 AM ---------- Previous post was at 03:43 AM ----------
Gabrial,
Do you think it’ll be possible to connect via adb and push back /rom partition content to restored generic image.
Providing we replaced uRamdisk and can use adb connect via USB.
Would it be sufficient?
Click to expand...
Click to collapse
It only gets mounted when running restores, not while the system is in use. But yes assuming your generic image had adb access you could push it back to /rom the issue however is that Touch-Formatter while great for returning devices to stock wouldn't fix partition issues, so if you screw up your partitions you'll need more than just this to fix it.

I will work on (when I have some time) making a blank image with just a generic /boot, with all the partitions correctly done of the NST, but empty.
This image, compressed, shouldnt occupy more than a few megabytes, then make a zip which backups the sensitive data, /rom, /factory and create another zip, which should destroy all the data on the NST, burn this empty image, restore /rom and /factory, then trigger automatically reset/restore to end up with a 100% clean nook, even if you screw it hard.
Is this what you were asking for ApokrifX? Or did I get it wrong?

Is there really unique data on /factory ? I thougt there is only some duplicate data from the rom partition.

eded333 said:
Anyway there is allready a tuto for something like that, which creates a full backup of your Nook and it should be the first step before playing with it:
http://forum.xda-developers.com/showthread.php?t=1142983
Or you can use the latest CWM: http://forum.xda-developers.com/showthread.php?t=1360994
Click to expand...
Click to collapse
Making a normal backup with CWM doesn't include the /rom and /factory partition.

mali100 said:
Making a normal backup with CWM doesn't include the /rom and /factory partition.
Click to expand...
Click to collapse
Mmmm, I thought it did a full rom backup, I'll change the advice on the previous post, thanks.

mali100 said:
Is there really unique data on /factory ? I thougt there is only some duplicate data from the rom partition.
Click to expand...
Click to collapse
Yep, factory contains a copy of the rom data which gets extracted to rom when you do a factory restore.

eded333 said:
I will work on (when I have some time) making a blank image with just a generic /boot, with all the partitions correctly done of the NST, but empty.
This image, compressed, shouldnt occupy more than a few megabytes, then make a zip which backups the sensitive data, /rom, /factory and create another zip, which should destroy all the data on the NST, burn this empty image, restore /rom and /factory, then trigger automatically reset/restore to end up with a 100% clean nook, even if you screw it hard.
Is this what you were asking for ApokrifX? Or did I get it wrong?
Click to expand...
Click to collapse
eded333,
That’s exactly what I meant!
---------- Post added at 04:01 AM ---------- Previous post was at 03:01 AM ----------
ros87 said:
Yep, factory contains a copy of the rom data which gets extracted to rom when you do a factory restore.
Click to expand...
Click to collapse
That’s all?
Anyway, where is it (factory partition)?
I.e. what is # in /dev/block/mmcblk0p#
“fdisk -l” shows nothing...

Factory, should be, if i'm not wrong /dev/block/mmcblk0p3

ApokrifX said:
That’s all?
Anyway, where is it (factory partition)?
Click to expand...
Click to collapse
No that's not all
And it's located where eded said it is.

Guys,
Need a little help here:
http://forum.xda-developers.com/showthread.php?p=22214127#post22214127
Basically, how do we change NST MAC?
Sorry, don’t know where else to ask…

[Q] Full - Full wipe of internal SD card

Hello,
I've done ROM flashes where instructed to perform full wipes through the recovery menu, or while in the OS. But when selling/getting rid of old Android phones (today, specifically speaking of the Note II i317 AT&T) will it make my personal data unrecoverable by methods I've used in the past to recover accidentally deleted photos/documents?
Basically, I don't want to sell my phone and Joe Blow to be able to recover all (or any) of the private pics/sensitive data of said phone (ie steal/sell my identity). How can I go about securely formatting my phone to make that data unrecoverable, yet without bricking the device?

kintamanate said:
Hello,
I've done ROM flashes where instructed to perform full wipes through the recovery menu, or while in the OS. But when selling/getting rid of old Android phones (today, specifically speaking of the Note II i317 AT&T) will it make my personal data unrecoverable by methods I've used in the past to recover accidentally deleted photos/documents?
Basically, I don't want to sell my phone and Joe Blow to be able to recover all (or any) of the private pics/sensitive data of said phone (ie steal/sell my identity). How can I go about securely formatting my phone to make that data unrecoverable, yet without bricking the device?
Click to expand...
Click to collapse
Well, consider a harddrive. It is possible to recover a lot of files after being wiped, and the wipe from the internal recovery doesn't at all (to my knowledge) remove any internal media (files). If of course the recovery doesn't have a added feature to delete partitions i suppose.
And of course, external third part software can recover the files you once deleted, like this: Enter Youtube > search for recover deleted files android. You will find lots of result of how to recover images as a example.
You would need to find a way to unrecoverably delete the files you don't want to share when you sold it, or you could take advantage of the statistics of the fact that quite many people don't know how to recover files.

http://lifehacker.com/5808280/what-should-i-do-with-my-phone-before-i-sell-it
http://www.webroot.com/us/en/home/r...ation/how-to-wipe-your-device-before-donating
http://www.makeuseof.com/tag/how-to-wipe-history-on-android/
Three good sites above..
One good tip was encrypt phone before wiping...

[Q] Function of Nexus 5X partitions, and which to back up

Hi,
Where can I find an explanation of the Nexus 5X partitions, and which of those are changed during use and are a good idea to back up?
Coming from Nexus 7 3G 2012, I see the 5X has quite a few more partitions. I've searched this site and the wider Internet for their purpose but have come up only with a parted listing without explanation.
E.g. what typically goes into "vendor", why do "system" and "vendor" have "... image" counterparts, and what exactly goes into the crucial "EFS" partition?
Following from that, it seems that an unlocked but otherwise unmodified device can be fully restored from the factory image and a data partition backup (apart from perhaps needing to restore EFS in extreme cases), right?
Or are there other partitions that may get modified during normal use and need to be backed up too?

I've come across one of the answers.
It seems the vendor partition contains the platform-specific drivers/binaries that were previously stored in the /system partition: https://plus.google.com/+JeanBaptisteQueru/posts/akHWypRNEn3.

...and according to this the "... image" selections aren't device partitions, but TWRP options to add fastboot flashable image backups of the corresponding partitions.

fvisagie said:
what exactly goes into the crucial "EFS" partition?
Click to expand...
Click to collapse
Continuing the monologue, although I haven't found a definitive source, most authorative-sounding ones like this one and this one claim it contains device-specific IDs, mostly connectivity, such as IMEI/ESN, Wi-Fi and Bluetooth MAC addresses, network unlock information etc.
fvisagie said:
Following from that, it seems that an unlocked but otherwise unmodified device can be fully restored from the factory image and a data partition backup (apart from perhaps needing to restore EFS in extreme cases), right?
Or are there other partitions that may get modified during normal use and need to be backed up too?
Click to expand...
Click to collapse
I recall from a previous experience that to be completely safe, user data/internal storage (/sdcard) needs to be backed up and restored too. Most Android apps that had been run and had created data on /sdcard before the backup will fail to run if restored without their /sdcard content.

[Help] Possible to dump the userdata partition?

So I've just gotten the infamous hardware induced boot loop and I've had my return request honoured by Amazon.
While I still have access to the bootloader (nothing else though) and have time to format the userdata partition, I wondered if there's a way to dump the userdata partition so I can retrieve some data from it later (although it's encrypted, I know the password of course).
If this is possible, how can it be done? And how could I decrypt it if there's a method known? If the method of decryption is not known, I guess I could possibly be able to flash it to my new Pixel when I order it, but I'm not entirely sure where/if there's data included within the raw dump that would specify where the partition should be placed (I've not looked into it and only have 2 days before my device MUST be shipped) maybe as an offset from partitions before it, or maybe a specific block range in the flash memory it would be written to.
If this is far from possible, then I don't mind, the only thing I really want to recover is a few meaningless pictures/videos (they're backed up anyway) and the data for my 2FA app so I can drop it in my new device without needing all the hassle of resetting all the codes my every account I have with 2FA enable...

Lineageos major version updates

When updating a major version - i.e. from Lineageos 16 to Lineageos 17 - it would seem you have to go through steps similar to the initial install (sideloading and what not).
My question is, do you lose everything on the device after performing this? Do I lose all of my installed apps, settings, messages, etc? Or is it just an overwrite of the core operating system?
My concern always is that I'll forget to back something up that I wish I'd backed up. Probably prudent to go ahead and backup stuff before attempting the upgrade. But I'm just wondering how much time should I spend insuring that everything I need is backed up. If the apps, settings, messages, etc are all expected to be there after upgrading, then I'm probably less concerned about spending a large chunk of time insuring everything is backed up.

all your data is stored in the /data partition which is not touched by the lineageos install. lineageos will install into the /system partition. you might edit your boot partition too, depending on your phone, but your data which includes all your installed apps (not the system apps that came with the phone) and your messages, settings, etc, is not touched. your camera photos and some settings are also on your /sdcard which is also untouched as long as you dont do a factory reset or wipe. that erases the /data and /sdcard partition.
if something goes wrong and your phone stops working then you can probably fix it but you want to have a backup in case you do need to wipe. you may or may not be able to recover the data, depends on the problem. always backup.

Develop Bugs

welcome