Ok, these are specifically for MIUI, but they can easily be adapted to any other rom by editing the shell scripts. I'll explain how they work and it should be fairly apparent how one would need to modify for their particular rom/needs. Don't know if anyone will actually use these, but I thought I'd share. I also think it's a really good example of how CWM can be used for more than just flashing stuff.
So I've been running MIUI for a while now and I'm really happy. One little issue that popped up was every Friday when the rom was updated, the OTA update app icon gained a little "1" badge, indicating an update was available. While it's nice to be notified of the update, I occasionally didn't have the time or interest in updating so the little badge remained. I put that app in a folder, but then the folder had the little 1 badge. After a while, I started deleting the update app after install since I could always download and install updates through ROM Manager just as easily. Then I started deleting the update app from the downloaded ROM zip, along with several other apps I didn't want (Facebook, Twitter, the MIUI FileExplorer, etc) before flashing it. But this entailed downloading the Friday update on my phone, transferring it to my desktop, opening the zip in 7 Zip, deleting the unwanted files, uploading it back to my phone and then flashing. Quite a PITA.
Another issue I ran into was that the built-in MIUI backup app failed to backup my SMS/MMS messages. Actually, I don't know if it was failing to back them up, or failing to properly restore them, but at the end of the day, it wouldn't keep my text messages. I know I could avoid this by just installing each update over the current ROM, but I've always believed cleanly installing an OS helps avoid issues. I prefer to wipe everything, install clean and then restore my data.
So, the solution(s) I came up with was to mod the downloaded ROM zip file right on the phone, and manually backup and restore my data. While it took a little while to get set up, I find it actually works considerably faster and smoother than using MIUI's backup/restore app. I now have 3 simple shell scripts that are run through Clockwork Recovery just like installing any zip - one to mod the ROM after downloading, and two to backup and restore my data. Here they are:
MIUI-Mod.zip
This contains a shell script and a zip binary to handle the adding and deleting of files to/from the downloaded rom zip. When run through CWM, it copies zip and the script to /tmp and runs the script. Because the ROM zip is named differently every Friday, I figured the easiest solution was just to rename it manually. So, this script assumes the ROM zip is named "miui.zip" and is located in the root of the SD card. It's a simply matter to download the ROM on Friday, move it to /sdcard and rename it miui.zip. Here's what the script does in 2 commands:
/tmp/zip -d /sdcard/miui.zip /data/app/Twitter.apk /data/app/Facebook.apk /system/app/Email.apk /system/app/FileExplorer.apk /system/app/Gmail.apk /system/app/MIUI.us_Forums_1.1.2_AdSupported.apk /system/app/Superuser.apk /system/app/Updater.apk /system/app/MIUIStats.apk /system/media/lockscreen/* /system/media/wallpaper/*
/tmp/zip /sdcard/miui.zip /system/app/Gmail.apk
Click to expand...
Click to collapse
The first line simply opens the zip and deletes the listed files: Twitter, Facebook, Email, etc. It also removes all the wallpapers for the lock and main screens (I don't use them, so I didn't want them, but it's a good example of the use of wildcards like *).
The second line adds in the Gmail.apk from my current ROM. The latest versions of the Gmail app included in MIUI lock the mail database, which conflicts with a widget I use, so I just replace it with an older version. Every time I run the script, it removes the included Gmail app and adds in my current one. If you didn't want to add anything to the ROM, you could just delete this line entirely.
Data-Backup.zip
This just runs a shell script that creates a tar file in the root of your sdcard containing the listed files and directories. Tar files retain file permissions and owner information, so everything can be restored seamlessly. Here's how it works, it's pretty simple:
tar -c -f sdcard/backup.tar data/app/com.adobe.flashplayer-1.apk data/data/com.adobe.flashplayer data/app/com.speedsoftware.rootexplorer-1.apk data/data/com.speedsoftware.rootexplorer data/app/com.touchtype.swiftkey-1.apk data/data/com.touchtype.swiftkey data/data/com.android.browser data/data/com.android.deskclock data/data/com.android.launcher data/data/com.android.mms data/data/com.android.phone data/data/com.android.providers.contacts data/data/com.android.providers.settings data/data/com.android.providers.telephony data/data/com.android.providers.userdictionary data/data/com.android.thememanager data/data/com.android.vending data/data/com.google.android.gsf data/data/com.miui.camera data/data/com.miui.notes data/data/com.noshufou.android.su data/system data/misc/wifi
Click to expand...
Click to collapse
This creates the tar file and includes Adobe's Flash player and it's data, Root Explorer and it's data, and SwiftKey and it's data (I paired it down to just these three for this example). After that, it backs up a bunch of other data directories which include the current Browser settings, alarms, launcher settings and home screen layouts including widgets, phone logs, the current theme settings, any MIUI notes, current Wifi connections and, yes, SMS and MMS messages.
Obviously, you'd want to add in your specific apps and data directories. Sure, it takes a little work to set this up, but as I said, backing up is very quick and easy afterwards.
Data-Restore.zip
This just runs a shell script that restores all the files from the backup.tar on your SD card:
tar -xf /sdcard/backup.tar
Click to expand...
Click to collapse
Doesn't need much explanation.
So, the process is:
1-Download the Friday update through ROM Manager, move it to the root of my SD card and rename it to miui.zip
2-Reboot into CWM and run Backup-Data.zip to back up my apps and data
3-Run MIUI-Mod.zip to modify the downloaded MIUI rom
4-Wipe everything (cache/data/system)
5-Install the modded MIUI rom
6-Run Data-Restore to restore apps/data
7-Reboot
It's literally about 5 minutes after downloading the ROM and I'm updated and rebooted with a CLEAN os install and all my apps, data and settings intact, my home screen layouts, keyboard, wallpapers, wifi settings and everything retained.
Some notes:
1-Before running these in CWM, you have to make sure you have system, data and sdcard mounted since the scripts will be reading and/or writing to all of these. When you first reboot to CWM, just go to mounts and manually mount each one. The scripts could certainly be set up to mount them automatically, but mount points vary from phone to phone, and it's not difficult to mount through CWM. Also, after flashing a ROM, the ROM install script may unmount some of these, so before you restore your data, you might have to manually remount.
2-If you're adding anything from your current ROM to the zip (like how I add in my current Gmail.apk), make sure you wipe everything *after* running the mod script, otherwise you're deleting everything and then trying to copy file(s) that were just deleted.
3-I would highly recommend using Notepad++ to edit scripts, since it properly saves Unix line breaks.
4-as always, use at your own risk and a full CWM backup is highly recommended before messing with these.
That's pretty much it. Again, it does take a little while to get set up, but once it is, it's really simple. If you install a new app between updates, you just have to edit the backup script, or you can just keep the scripts maintaining a base setup and reinstall various apps through the market each time. This specific process is probably most useful for ROMs that are updated regularly - nightly or weekly, although I'm sure people can find other uses for running scripts through CWM.
Hope someone might find these useful.
This is really cool. I run Miui and would really like to be able to do a fresh install each week, but hate the hassle of resetting up my phone, even using Titanium backup, it's still a time consuming process. So needless to say, I flash the new rom over top of the old version without wiping just to save time.
So, basically I can open up the data restore zip in 7 zip, add in a few of my apps/data such as Tasker and so forth, close it and flash? If so, that will be great.
Thanks for sharing this!!!
You don't need to touch the Data-Restore.zip, all that does is extract the backup.tar file created with the backup script. What you'd do is open the Data-Backup.zip and edit the backup file (it's just a text file). Add in any of your apps and their corresponding data directories and when you flash the Data-Backup.zip in CWM, it'll back up those files/folders.
To find the files you want to backup, use something like Root Explorer and look in /data/app. There you'll find all the 3rd party apps you've installed. Add each one to the tar command in the backup file just like the ones I included. Then go into /data/data and find each app you want to back up's corresponding data folder. Add them to the tar command too.
Save the file, re-zip it and when you flash it through CWM, it'll back up the new files.
Just what i wanted
I also always install everything from clean system so setting up everything is a pain.
Would this work for Internal storage? I have a Motorola RAZR XT910 and running JBX roms.
Google does install all the apps when i sign in into the new rom but then i always had to move each folder to the correct destination manually using some file explorer. was thinking of making something like this but didnt know how to start. Thanks, I will now try and mod these to suit my phone :good:
Related
Today I just tried to install some fonts using Root Explorer and all of a sudden I was stuck in boot loops. So I tried to boot into recovery mode and there was an update.zip on my device (probably from when I was originally rooting) which I think the recovery mode ran. After that, I was in clockwork and I flashed a new font.zip file and that still left me in a boot loop state.
I fixed the boot loop by manually changing permissions on DroidSans.ttf and DroidSans-Bold.ttf to match the other fonts and change owner/group to root:root.
This stopped the boot loop but now every time I boot my phone it force closes all my apps that were previously on sd. I'm not sure why this is happening and it might be other apps that are also not on my sd. It is also not showing all of my installed apps and Launcherpro no longer runs and I am stuck with default htc sense launcher. I would prefer to not wipe the entire phone to fix this.
I have tried a different sd card with the same data and I have also tried formatting the sd card and copying my files back on. Still stuck with force close errors. I am not sure what to do at this point. Any help?
Try fix permissions.
I have tried running the fix_permissions script. Didn't seem to work. Unless there's another method you are referring to. I am unable to launch ROM Manager and it seems like all of my apps are 'new' and never launched before from what they show. Including my alarm settings, those are missing too.
Think I'd restore a backup taken prior to all the font changes. You did make a backup, didn't you?
________________________________
Unrevoked forever
SkyRaider Sense 3.5
Radio 2.15.00.09.01
rigman said:
Think I'd restore a backup taken prior to all the font changes. You did make a backup, didn't you?
________________________________
Unrevoked forever
SkyRaider Sense 3.5
Radio 2.15.00.09.01
Click to expand...
Click to collapse
I do have a backup but unfortunately its a little bit old and I would rather fix this if possible. It feels like some folder is missing permissions. Because even stock Google talk crashes and market crashes when I try to view an app but I can search around if I want.
Another route I can go is to backup my data for apps and then install a new Rom as I have been wanting to. This would allow me to fix my problems but I need to know what folders I have to manually backup for saves.
Well some good news. After messing around with permissions on some files on the sdcard I was able to restore my notification sound back and also able to make GTalk work properly. So it looks like there is hope yet.
Where do apps store settings such as configuration, db, etc?
It looks as if my data/data partition is wiped? I'm not sure what happened but I guess that means all my app saves are gone.
Am I right to assume that?
I used this application NotEnoughSpace made by a xda member and using it to symlink some of my apps from data partition to a new area on emmc seems to work. However when I try to restore back to my data partition, it doesn't work.
So this leads me to believe that the permissions on my data partition are off and I need to change the permissions, owner, and group for the data partition and all files inside it. The command would be:
Code:
chmod 664 data/*
chown root:root data/*
And run this while outside of the main data folder. As long as the right owner/group is root:root.
Can someone confirm?
So I made an update.zip before I went from my 4.1 ROM to a 4.2 ROM. However whenever I try to flash my update.zip file it makes my phone not boot. Unfortunately I was stupid and didnt make any other backups. Is there a way to restore individual apps/data from the update.zip file?
Could have something to do with how titanium created the file.
Does the .zip only contain the tibackup app? Or others as well?
Sent from my Galaxy Nexus using xda premium
It contains a large list of apps. When I created the zip I went through the long list of stuff and only backed up what I wanted (none of the red system stuff).
All in all its not a huge deal. Was mostly wanting my SMS/Call Logs/Wi-Fi
All the other stuff would be nice but can be redone
File and directory structure has changed in 4.2.
Sent from my Nexus 7 using Tapatalk 2
najaboy said:
File and directory structure has changed in 4.2.
Sent from my Nexus 7 using Tapatalk 2
Click to expand...
Click to collapse
Yep I dont think thats the cause though as it also happens if I try going back to my 4.1 ROM (JBSourcery).
I went through my update.zip file and removed a bunch more stuff that I likely didnt need and it worked!!! Obviously I backed something up into the zip that was causing the issue. There was also some system apks/files in there that I am positive I didnt choose for the update.zip... not sure if it auto-adds some or if I just spaced out but so far so good!
EDIT: Well it kind of worked... I got some of the apps but some didnt show at all and it didnt restore my SMS messages
Just to update if anyone else runs into this...
As I said above I got it to work by manually editing the .zip file to remove some stuff that I didnt need but it didnt restore my SMS/Call logs or anything like that (Wi-Fi APs as well).
I then went into the .zip and removed all the SMS/Call Log/Wi-Fi lists and then put them in the TitaniumBackup folder on my phone (overwriting my old ones) so it will allow me to restore. I also tried removing all files from my .zip file except the SMS/Call Log/Wi-Fi stuff and will see if reflashing helps
It would be a really great addition to TiBu to have the ability to restore from your .zip file from within TiBu. I dont think it would be super hard to implement but I'm not a dev so thats just speculation.
I will update again later if my workaround does the trick.
Not sure if you've ever heard of it, but grab the app named "App2zip". It will do just that. You select apps that you want installed back on your ROM after flashing a new one. When finished hit the OK button and it will make a flashable zip of all the selected apps. It also keeps all user & system apps on separate pages so you don't accidentally backup a system app that might cause issues. Another good thing is that it works for 4.2! I used it to restore over 150 apps on Mmuzzy's first build after 4.2 dropped and everything restored properly. Just know it zips only the apps, no data. If you want your app data backed up use TiBu and restore data after you flash your ROM & apps zip.
i was able to restore my apps using the update.zip from tibu, it then restored data after i booted the OS. I had to mount /system before i flashed the update.zip because if i didn't mount it first, it wouldn't restore my apps at all. The only problem is my phone lagged like an SOB after restoring the data. I just use tibu to restore apps and data after i boot the phone, keeps the problems off my phone.
RoyJ said:
Not sure if you've ever heard of it, but grab the app named "App2zip". It will do just that. You select apps that you want installed back on your ROM after flashing a new one. When finished hit the OK button and it will make a flashable zip of all the selected apps. It also keeps all user & system apps on separate pages so you don't accidentally backup a system app that might cause issues. Another good thing is that it works for 4.2! I used it to restore over 150 apps on Mmuzzy's first build after 4.2 dropped and everything restored properly. Just know it zips only the apps, no data. If you want your app data backed up use TiBu and restore data after you flash your ROM & apps zip.
Click to expand...
Click to collapse
Yep I have used App2Zip. TiBu has basically replicated that functionality.
So as is normally the case the problem here was my own stupidity.
I thought that by choosing the option to make an update.zip and then choosing all the programs I wanted it would make a current version backup of the apps I selected. However I think how it works is that it only makes the .zip file out of an existing backup. So I should have run a full TiBu backup before making my .zip. I verified this by going through the xml file with my text messages and verifying that it was old stuff.
So in the end it was my fault for not understanding how the program worked. I still think the ability to restore directly from the update.zip from within TiBu would be a welcome addition though.
Hello guys!
I want to format everything and give a fresh start on my GNexus. I am finding it very laggy and I think that the problem is on using the same app backup for almost an year. I make the full backup in the recovery and after wiping everything and installing a new rom, I restore everything. I believe that there are many apps in my phone that I don't use anymore and are there as trash that needs to be cleaned...
I know about Titanium Backup, but I'm on ART and I never learned how to use the damn program correctly, so there is much complication to just backup some apps. What I really need is a method for backing up JUST the data (ex: the savedata from the games) and not the apk. So that I can wipe everything in the phone (all the partitions), install the rom and fixes and everything and then intall via google play the apps again and put the backed data in its place again.
I don't know if I made myself clear, maybe I'm just complicating, but if you play or played roms on an emulator, you have the rom and its save. I want to backup the save file, delete the rom and the emulator itself. Then install the emulator again (clean install), donwload the roms (clean ones) and put the old save in the place. :good:
Well, thanks in advance :laugh:
Rayaxe said:
I know about Titanium Backup, but I'm on ART and I never learned how to use the damn program correctly,
Click to expand...
Click to collapse
Titanium works on ART now, as is my understanding. It's not complicated to learn. Just select the app and hit backup, and Bingo Bango, you're done.
Then when you restore, you can select just data.
Piece of cake, ace.
yeah, I finnaly made it, my problem was ART, I didn't notice they updated the app ehehe
thanks
Yeah I personally don't recommend tibu. I've tried it myself and had issues related to Android version at the time and I just see this over and over. The other thing is that there are excellent options available without an app. My favorites for backing up just /data/data/<app_data_dir> are:
Code:
#> cp -a /data/data <backup_location>
OR you can use
Code:
#> adb backup -noapk -all
Type adb in terminal on phone or on computer to see more info for this one, that's basically it though. You can use adb to backup directly onto your phone or computer, very useful when you need some space!!
The most important thing though when you're backing up data is to preserve permissions and be able to set new ones for data if the apk owner "id" changes, e.g. app re-installations. Otherwise you'll run into some pretty bad problems.. not so fun. So definitely use a tar archive for file permissions preservation and some fast lzma lzop compression on top of the tar archive. 'busybox tar' usually has all of these options. A lot of roms' busybox and functions get stubbed and made useless, so I would check out Terminal IDE if you want to go this route. 'Definitely want the full gnu options and consistency. Vanir by itself is also good here, and I use it 24/7 for this reason. It really does have a hemi, they're not kidding.
7175 said:
Yeah I personally don't recommend tibu. I've tried it myself and had issues related to Android version at the time and I just see this over and over. The other thing is that there are excellent options available without an app. My favorites for backing up just /data/data/<app_data_dir> are:
Code:
#> cp -a /data/data <backup_location>
OR you can use
Code:
#> adb backup -noapk -all
Type adb in terminal on phone or on computer to see more info for this one, that's basically it though. You can use adb to backup directly onto your phone or computer, very useful when you need some space!!
The most important thing though when you're backing up data is to preserve permissions and be able to set new ones for data if the apk owner "id" changes, e.g. app re-installations. Otherwise you'll run into some pretty bad problems.. not so fun. So definitely use a tar archive for file permissions preservation and some fast lzma lzop compression on top of the tar archive. 'busybox tar' usually has all of these options. A lot of roms' busybox and functions get stubbed and made useless, so I would check out Terminal IDE if you want to go this route. 'Definitely want the full gnu options and consistency. Vanir by itself is also good here, and I use it 24/7 for this reason. It really does have a hemi, they're not kidding.
Click to expand...
Click to collapse
looking for this method, very nice!
thanks, next time I need to backup I will use this xd
Okay I'm really in need of a response from who knows the mechanic of backing up
I've two questions
1. I've noticed that u can actually flash apps with a zip file. So if I put my personal apps into the rom folder after extracting them and recompress, is it possible that my apps will also get flashed along w the rom?
2. I've also noticed when u do a factory reset in the recovery, it only deletes cache and data. If I copy the data folder to my sdcard and paste to the orig directory after factory reset, is it possible that the data will get restored?
I'm not just curious. I think if this is possible, it'll be much dependable
Thnx
Sent from my Galaxy Nexus using xda app-developers app
christianyu said:
Okay I'm really in need of a response from who knows the mechanic of backing up
I've two questions
1. I've noticed that u can actually flash apps with a zip file. So if I put my personal apps into the rom folder after extracting them and recompress, is it possible that my apps will also get flashed along w the rom?
2. I've also noticed when u do a factory reset in the recovery, it only deletes cache and data. If I copy the data folder to my sdcard and paste to the orig directory after factory reset, is it possible that the data will get restored?
I'm not just curious. I think if this is possible, it'll be much dependable
Thnx
Sent from my Galaxy Nexus using xda app-developers app
Click to expand...
Click to collapse
1. Yes, put the app's APK under /data/app in your zip package (although I do think that you need to modify the updater-script a bit to make it actually flash the apps).
2. Depending on how you do it - you MUST keep ALL the files' and folders' permissions untouched (which is nearly impossible since there are just WAY too many of them), or when you simply copy back, the permissions will be messed up and you'll face continuous FCs. Better use apps like Titanium Backup or, if you're brave enough, try restoring only the /data partition from an existing nandroid backup.
Sent from Samsung Galaxy Nexus @ CM11
1. How do I modify the updater script?
2. Is it possible to restore only part of the nandroid backup?
Sent from my Galaxy Nexus using xda app-developers app
Titanium Backup is the best way. The way you are doing it sounds like a lot of unnecessary work.
look for an app called zipme from author jrummy on the play store
I wrote some code a long while back that allows you to pack/encrypt and unpack/unencrypt your own android backups so that you can do your own partial backups and restores.
See http://forum.xda-developers.com/showthread.php?t=1730309
However, for the past 18 months, I have just used BackupPC to back up my android devices. BackupPC is a linux-based backup that allowd for automated backups with all types of options for incrementals & fulls. The backup files are pooled so that you can backup every day while requiring minimal extra space since only new files add to the poo size. It can use several different transport systems for the backup ranging from rsync to rsyncd to tar to ftp etc.
I use rsync (over ssh) since it is fast -- only the deltas need to be transported across the network when files change.
Basically, all you need to do is install ssh + rsync, modify the server side config file and then you get automatic backups at your own pre-determined frequency over wifi whenever your device is connected to the wifi network.
Now BackupPC is probably overkill for just backing up a droid -- since it is typically used to backup whole networks of linux, Mac, Windows, etc. pc's. But since I already use it to backup all my home computers, it was a cinch to modify the config file to also backup my droid devices.
I verified that it worked when my Gnexus broke and I was able to flash the replacement device which restored not just my apps and data but also all my settings.
jsgraphicart said:
Titanium Backup is the best way. The way you are doing it sounds like a lot of unnecessary work.
Click to expand...
Click to collapse
I agree. The play store has a bunch of other ones too but I would go with Titanium.
hsypsufan said:
I agree. The play store has a bunch of other ones too but I would go with Titanium.
Click to expand...
Click to collapse
I am using EaseBackup and I find it very good. Helium is another option but I found it buggy.
I tried
adb backup -all
but the resulting file is 60MB, but should be 100GB.
Edit: Some more information:
I'm running HavocOS, but the apps on this Rom are unstable. They are stable for almost 24h every time I flash most recent GApps, but by now I don't really think it's GApps anymore.
Aside from EVERY app being unstable copying files is not stable either. I just copied 150GB of images and videos from internal as well as "external" SD-card to my computer without any interrupt and that worked without any issues. If I try to copy a file from internal to external SD card within the phone, then this works in one out of three tries for every single file.
So next thing is try another ROM, then maybe an older ROM and then search for Hardware defects, before I want to try next ROM I want to create a backup.
There is A LOT of config files here and alot of them are linked to images all over the place, so I will likely have to switch to the backup a few times because if I just save the parts part by part now I will 100% forget a few important things.
This seems like a job for TWRP backup. Oh and btw, I did a TWRP-backup just when I migrated to this ROM, but it is defect! I tried to create a full backup withing TWRP >10 times today and yesterday and it all failed. Oh yes, moving files withing this device is buggy right now. But I can do it via adb, can't I? How?
andy01q said:
I tried
adb backup -all
but the resulting file is 60MB, but should be 100GB.
Edit: Some more information:
I'm running HavocOS, but the apps on this Rom are unstable. They are stable for almost 24h every time I flash most recent GApps, but by now I don't really think it's GApps anymore.
Aside from EVERY app being unstable copying files is not stable either. I just copied 150GB of images and videos from internal as well as "external" SD-card to my computer without any interrupt and that worked without any issues. If I try to copy a file from internal to external SD card within the phone, then this works in one out of three tries for every single file.
So next thing is try another ROM, then maybe an older ROM and then search for Hardware defects, before I want to try next ROM I want to create a backup.
There is A LOT of config files here and alot of them are linked to images all over the place, so I will likely have to switch to the backup a few times because if I just save the parts part by part now I will 100% forget a few important things.
This seems like a job for TWRP backup. Oh and btw, I did a TWRP-backup just when I migrated to this ROM, but it is defect! I tried to create a full backup withing TWRP >10 times today and yesterday and it all failed. Oh yes, moving files withing this device is buggy right now. But I can do it via adb, can't I? How?
Click to expand...
Click to collapse
try using the migrate app
Which migrate app?
andy01q said:
Which migrate app?
Click to expand...
Click to collapse
The app's name is migrate..search on playstore or xda thread
Got it, thank you.
It's not working though for three reasons:
1st: It's not a full backup. I got what I need for migrating, I want a full backup to return to the current state including filesystem, tree and random files. Ideally A full copy of all filesystems which wouldn't fit onto any of the filesystems even if this was what the app did.
2nd: It's not via ADB. Backing up from inside the phone doesn't work as I already wrote.
3rd: Doesn't recognize my SD card (yes it's mounted as R/W) I don't know why I even checked that as the app is useless for my case anyway.