[Q] Merge wasted EMMC with SD card? - Droid Incredible Q&A, Help & Troubleshooting

My HTC Incredible has 8GiB of internal flash available for use. I assume some other Android devices also have a good amount of internal flash.
I have a 32GiB microSD card, but I'd like to add in that other 8GiB, as it's a substantial amount.
I can put things there from adb, or some file managers, but a lot of apps don't access it and mounting the flash to /sdcard and then swapping back gets old.
So, idea -- would it be possible to "merge" the two by a trick of mounting (sort of like JBoD RAID) and get the entire 40GiB under /sdcard? en.wikipedia.org/wiki/Union_mount
I believe UnionFS and AUFS can do this, and I believe Cyanogenmod already has the latter. Has anyone tried making use of that otherwise nearly-wasted chunk of space?
Thanks.

I modified my vold.fstab file to trick the phone into thinking one was the other, but for what your asking I don't think can be done. Symlinks aren't possible using FAT32 (which is what we have the sdcard and emmc formatted as) so that's out. Overall I can't think of any way to do it. Not sure what you're thinking about there with CM, it most assuredly doesn't do that. Also, there are only a small handful of other devices that have emmc's out there, the majority only have sdcards.

Related

Are apps stored on the ROM partition?

Phones typically have a partition of encrypted memory for the OS and Apps. Most Win7 Phones have 512MB of this ROM. In other phones, the ROM severely limits the number or size of apps. Are apps in Windows 7 Phone stored in the ROM as well?
I didn't think so until I saw some phones have 1GB of ROM. If just the OS + 60MB of pre-installed software is allowed, what's the advantage of 1GB of ROM?
ROM should be Read Only Memory, so I guess that nothing except maybe major system updates will be installed on the ROM.
ROM is a misnomer but nevertheless the ROM partition is what manufacturers refer to for application space.
I don't see the point of having larger ROM partitions unless apps will be stored on the ROM. I'm hoping that if Windows 7 Phone stores apps on the ROM, it can efficiently recognize and install only the necessities onto the ROM and string the rest of the app over the single storage volume. Otherwise, we're looking at an OS with severely limited app storage.
I read in the Microsoft spec that any app pre-installed by the manufacturer needs to be restorable. So they probably have a backup copy of the app in the ROM section in that case.
athompson said:
I read in the Microsoft spec that any app pre-installed by the manufacturer needs to be restorable. So they probably have a backup copy of the app in the ROM section in that case.
Click to expand...
Click to collapse
Yes you are correct, however, as stated in the original post, the limit for pre-installed software is 60MB.
I know that side-loading isn't possible, but I still have to ask: can you remove the sd card and load an app directly onto it with a card reader? I know that removing the card causes a hard reset, but if you pull the battery first, then can you remove the card, mess with it, replace the card and then replace the battery without having the device hard reset? It seems like you could at least access the sd card file system this way (but I'm probably missing something).
Farmer Ted said:
I know that side-loading isn't possible, but I still have to ask: can you remove the sd card and load an app directly onto it with a card reader? I know that removing the card causes a hard reset, but if you pull the battery first, then can you remove the card, mess with it, replace the card and then replace the battery without having the device hard reset? It seems like you could at least access the sd card file system this way (but I'm probably missing something).
Click to expand...
Click to collapse
Surely, if the OS is split between the SD and phone memory then trying to just use the SD in that way would result in corruption?
Also, I imagine that the OS does self consistency checks, in some way, so that when you reinserted the SD card it'd notice that it's been altered?
Of course, that's a guess and I could be completely wrong.
I think I read in another thread that a few people have pulled the sd card, and couldn't read it with a reader. I guess it's encrypted (makes sense, but it still sucks).
Farmer Ted said:
I think I read in another thread that a few people have pulled the sd card, and couldn't read it with a reader. I guess it's encrypted (makes sense, but it still sucks).
Click to expand...
Click to collapse
Da_G mentioned its like a RAID0 with striping between the internal flash and the external flash card. Striping means that the data is divided between the two storage divices, which makes access faster (since you can get them both in parallel). It also means that if one of the memory devices fails, all data is lost. Which comfirms what we've seen in the Microsoft documentation, that if you remove the flash card and replace it with a different one, all your data is lost.
athompson said:
Da_G mentioned its like a RAID0 with striping between the internal flash and the external flash card. Striping means that the data is divided between the two storage divices, which makes access faster (since you can get them both in parallel). It also means that if one of the memory devices fails, all data is lost. Which comfirms what we've seen in the Microsoft documentation, that if you remove the flash card and replace it with a different one, all your data is lost.
Click to expand...
Click to collapse
That makes me think, If it's like a raid0, and your internal memory has 8gb, then you dump in a 16gb card, will it be like raid where only 8gb of the 16gb will be usable (in the raid) ?
DavidinCT said:
That makes me think, If it's like a raid0, and your internal memory has 8gb, then you dump in a 16gb card, will it be like raid where only 8gb of the 16gb will be usable (in the raid) ?
Click to expand...
Click to collapse
That's a good point, it doesn't sound too cool, does it. From what I understand about raid0, the purpose is to divide the data between two different disks, so you can get twice the IO bandwidth (since you can load half the file from each one, get it twice as fast). Raid0 doesn't do any backup, so if either hard drive fails, the whole thing is unrecoverable. Which is why most people don't use raid0.
But since it doesn't do any backup, it means there's no 'wasted' space, which means the full 16gb will likely be usable.
RAID0 is in ", better expression is JBOD (but data is stripped between the drives somehow).
Anyway, it is similar to current WM. There is IMGFS with some reserved space (a lot more actually, so all image updates fit in there without flashing phone, I'd be guessing that about 200MB is left as reserved space). The rest of the storage is as userspace, where are all stuff installed.
DavidinCT,
Your thinking of RAID 1 (mirroring) RAID 0 is striping without parity. So it basically writes/reads across both .. it looks like one volume but writes/reads could be on either storage.. depending on where they were written..
So.. when you remove the card .. or if you alter the card... it beaks the stripe set.. and toasts the storage volume.. if the storage is encrypted that would additionally complicate things..

[Q] Swap internal and external SD

In older versions of Android the external SD would be mounted as /mnt/sdcard but now that is used for the internal memory and the external SD is now /mnt/sdcard/external_sd. This is a bit of a problem as some apps can only save to /mnt/sdcard and it quickly takes up our phone's internal memory. I have offline GPS maps and a few Gameloft games which can only save their data to /mnt/sdcard and as you can imagine free space is very limited.
What I would like is to switch this around. Basically I have a 32GB SD card that I would like to use a storage for some of my apps as it is not being fully utilized. I've searched around and it doesn't seem like there is an easy way to do this. I'm not completely sure but it seems like there are methods that might be specific to different phones which requires modifying a file in /system/etc.
Will this safely work for our phone?
check this topic
http://forum.xda-developers.com/showthread.php?t=1458153&page=3
posts 21 to 25 and 33 & 34..
For Recovery swapping has been tried..
You may get some idea..
Maybe the use of symbolic links could help, but I never tried it on an Android device.
I had read people Mapping the External under a directory for example the Games directory and therefore all games would in theory be on the External. I think this is what you mentioned as symbolic links.
How can you do that. I tryed a million things. Last time i installed literom with added swap internal memory to sd mod. And after i copy something in phone i loose the mobile signal. I guess the card is getting corrupted or something. How can i make it so the sd card wont be unmounted when i plug the phone in pc ?
Might want to consider this:
Root External 2 Internal SD
I've not used this specific one myself but I have used similar apps in the past.
I didn't use that yet.
But what i found untill now
1. Using Directory Bind wich makes a link between actual place of the files and where they should be. Example You put com.xxx.xxx in your SD card and set the progam to tell android when is looking into sdcard/android/data/com.xxx.xxx go for files in sdcard/external_sd/com.xxx.xxx
This method is good for games and files for them
2. Another thing is modifing vold.fstab and swapping the way adroid mounts those partitions. (this option is buggy somehow)

SD card ext and swap partitions: How useful? How large?

I'm using Mimicry 1.4.0 and quite happy with it in general. I've figured out that 4EXT is the way to go, and have ensured that all my internal partitions are ext4 (just now actually, I'm interested to see how much it helps). I have seen various people suggesting repartitioning of the SD card as a way to see a significant performance enhancement as well, and this seems reasonable to me but certain information on the matter seems to be lacking. There's a few posts here and there on the subject that have additional detail, but they're in other device forums or in response to folks running different ROMs..
So:
How helpful is repartitioning?
Should I just create an ext1 partition? Is ext2 additionally helpful? What about a swap partition?
What do the damn things actually do?
How large should I make the partitions? I assume there's a point of diminishing returns.
Since I am a cheap bastard, my cheap SD card is likely quite slow. How do I tell how fast it is? How relevant is that?
Much thanks!
+1
I'm also interested to know if anyone using swap has seen any performance improvements.
When you make for example 128Mb ext4 and 256 Mb swap partition to your sd card for G2 and plugged in it. Stock rom will not use the swap file but stock rom will detect the ext4 part and use it for keeping the cache there.
and for the custom roms like CM series and Sense roms you'll need to make a swap partition and if you like ext4 partition on sd card. (ext4 is not necessary).
Because G2 and desire Z has 368Mb ram and these new custom roms are needing a bit larger space then stock rom. thats why you can use " swap_enable_v2" for additional ram using. ( Remember if your sd card's write speed more then 6 Mb/s ) of you can simply use some scripts like SuperCharger_V6 or some other like this.
I stongrly suggest you to use Swap enabler and SuperCharger_V6 at the same time like me. it increased my desire z's permormance more then %40 and it awesome now.
:good:
Well, I'd say swap partition does not always is a good idea - constant reads and writes of the specific part of the card may degrade it's performance.
That's why I'd suggest trying DataSwap, which can just create plain file on the card.
I'm using it for over 4 months now, and I'm pleased.
On the side note AFAIR /sdcard should be FAT, and not EXT4
Partitioning the sd card or creating a swap file on sd card is not so different. Still mskes alot reads and writes
Sent from my Nokia 3310
If you are using ICS/JB - I put my swap file on the /cache directory since it doesn't seem to ever get filled up. Then I am not using the sd card.

[q] re-partitioning the glide

i was wondering, when we can get ASOP/AKOP or CM9/10 we can re-partition the glide to have only system partition and the extranal sd card can act as the internal one. we all know that the 8GB is not enough, especially with 2gb system and 6gb sd card - i have 2gb in system and 4gb in sd card - but if we can combine it to full partition then it will be more then enough as a basic install space - cos allot of apps can be moved to sd card and leave little room on the system partition.
It's already possible to change the mount points so that your physical sd card is mounted as /sdcard, and the internal storage takes its place at /sdcard/external_sd. It's a quick vold.fstab edit. You can see my post about it here, with a pre-modified file and a backup of the original.
You'll need root and busybox (or some form of file explorer with root permissions) to copy the file over, but it's pretty straightforward. As for repartitioning the internal storage to be used for /data, I've got no clue how to do that, but this is a reasonable enough workaround.
i know about that workaround but still there some side effects to it, like the usb mounting in the PC.
still i think its doable if creating aosp from source with cross with our ics source from samsung..
maybe a more knowledgeable dev can shed some light on the subject.
Changing the actual partitioning isn't something that's commonly done, as it can wreck compatibility with roms (including official updates) that expect things a certain way. The only issue in ICS with the vold.fstab edit is the internal memory isn't mounted on the phone at boot, you have to manually mount it any time you reboot. MTP file transfers work just fine (with your physical sdcard being labeled Phone and the internal labeled Card) as does the legacy USB Mass Storage mode.
If you really want to mess with it, look into partition editing using parted. This can go very very wrong, and may not work at all, so proceed with caution.
i see thanks for the info

[DEV] Fusion Drive for the GS4

Hopefully we can implement a "fusion" drive like newer macs. I know it's possible to make your own with a few unix commands, so I wonder if another dev and I could get it working on the S4. A fusion drive is basically a SSD+Traditional HDD that the system sees as one drive. It automatically moves the lesser used items to the slower drive and keeps the system intensive files on the faster drive. Essentially we could use this technology to make our MicroSD and EMMC appear as one unit on the OS.
If any DEVs are willing to work with me please shoot a PM. No other users at this time please.
Try directory bind. Can setup to mirror items on your ext-sd
Sent from my SGH-M919 using xda premium
dannyben said:
Try directory bind. Can setup to mirror items on your ext-sd
Sent from my SGH-M919 using xda premium
Click to expand...
Click to collapse
This wouldn't mirror anything. It would show the system storage as (mine personally) 64gb+16gb as one unit.
Same thing. Can setup a folder on internal sd called extsd and turns internal sd to 80gb
Sent from my SGH-M919 using xda premium
Personally, I think this is silly and hope to never see it implemented on any ROMs that I enjoy. Just my opinion. Could be good as a flashable package that ppl have the option of installing through Recovery, though. I'll throw the idea out there on the irc chats and let me ppl know you're looking for a partner in crime.
I know internal storage reads faster than an sdcard, would this allow us to basically have 48g internal memory?
Sent from my SGH-M919 using xda premium
inphamous36 said:
I know internal storage reads faster than an sdcard, would this allow us to basically have 48g internal memory?
Click to expand...
Click to collapse
If you're asking "will this magically speed up my slower SD card", think about why that's not the best question.
Why not just write a script that swaps internal and external SDs?
Sent from Spaceball One.
blackknightavalon said:
Why not just write a script that swaps internal and external SDs?
Sent from Spaceball One.
Click to expand...
Click to collapse
Not possible/also not a good idea at all to swap the SDs. The only way you can get a "hybrid" system like what the OP suggests is either the actual fusion of the hardware with special custom firmware which handles the offloading of these tasks of moving the most used sectors of data to the faster medium, OR, a custom file system that is drive agnostic and can address multiple storage devices, consistently test their access speeds to assess performance, and intelligently offload data and traffic to the correct disk. Seeing as we have neither in any Android device that I know of, this feat is practically impossible until somebody compiles a new kernel with a file system plugin that does exactly this. It's a great idea, don't get me wrong, however it's going to be a total pain to accomplish, and secondly, removing the external storage, or god-forbid the SD card fails, and the entire checksum for the larger virtual disk has now failed and you've lost all your data. It's a bad idea when you're talking about mobile devices that you depend on daily like this, great for tablets and stuff, but bad for phones (unless you never mess with it or do anything weird that could compromise the SD card).
Cynagen said:
Not possible/also not a good idea at all to swap the SDs. The only way you can get a "hybrid" system like what the OP suggests is either the actual fusion of the hardware with special custom firmware which handles the offloading of these tasks of moving the most used sectors of data to the faster medium, OR, a custom file system that is drive agnostic and can address multiple storage devices, consistently test their access speeds to assess performance, and intelligently offload data and traffic to the correct disk. Seeing as we have neither in any Android device that I know of, this feat is practically impossible until somebody compiles a new kernel with a file system plugin that does exactly this. It's a great idea, don't get me wrong, however it's going to be a total pain to accomplish, and secondly, removing the external storage, or god-forbid the SD card fails, and the entire checksum for the larger virtual disk has now failed and you've lost all your data. It's a bad idea when you're talking about mobile devices that you depend on daily like this, great for tablets and stuff, but bad for phones (unless you never mess with it or do anything weird that could compromise the SD card).
Click to expand...
Click to collapse
Uhh...the script for both the SGS3 and the SGS3mini only works if there's an SDcard in the phone, so it is very much possible.
Sent from Spaceball One.
blackknightavalon said:
Uhh...the script for both the SGS3 and the SGS3mini only works if there's an SDcard in the phone, so it is very much possible.
Sent from Spaceball One.
Click to expand...
Click to collapse
If there is a script for it, awesome, I wouldn't do it myself though, not with only 8gb usable space, I'll burn through that in no time flat. I'm biding my time for 64gb before I invest in a new S4. Back on the topic of the "hybrid"/"fusion" drive, my statement still stands. It's not possible without a file system driver that supports it, or the hardware+firmware to support it (like those hybrid SSD-HDDs on the market).
Cynagen said:
If there is a script for it, awesome, I wouldn't do it myself though, not with only 8gb usable space, I'll burn through that in no time flat. I'm biding my time for 64gb before I invest in a new S4. Back on the topic of the "hybrid"/"fusion" drive, my statement still stands. It's not possible without a file system driver that supports it, or the hardware+firmware to support it (like those hybrid SSD-HDDs on the market).
Click to expand...
Click to collapse
The reasoning is this: The "virtual SD card" is the default storage space. All you'd have to do is apply the script, switch everything over to download into /sdcard, and leave the external SD card alone. For example, this would perpetually leave 9GB of hard drive space open on my phone while it thinks there's 64GB of space to play with.
As I said, it works perfectly on the SGS3mini.
blackknightavalon said:
The reasoning is this: The "virtual SD card" is the default storage space. All you'd have to do is apply the script, switch everything over to download into /sdcard, and leave the external SD card alone. For example, this would perpetually leave 9GB of hard drive space open on my phone while it thinks there's 64GB of space to play with.
As I said, it works perfectly on the SGS3mini.
Click to expand...
Click to collapse
So the script disables the /external_SD and makes it think that the SD card is actually the internal storage, leaving the internal storage alone? Not just swapping them like it sounded? Not bad, that I can see being beneficial with a fast enough SD card in place, however, only in that scenario.
Cynagen said:
So the script disables the /external_SD and makes it think that the SD card is actually the internal storage, leaving the internal storage alone? Not just swapping them like it sounded? Not bad, that I can see being beneficial with a fast enough SD card in place, however, only in that scenario.
Click to expand...
Click to collapse
Have you ever had an older Android device like a G1 or the original MyTouch? The A2EXT hack worked in a similar manner, but this doesn't involve partitioning anything, and a class 4 card could work.
Sent from Spaceball One.
Sorry Cynagen....what I'm talking about requires no special firmware or hardware. It's a unix/linux script that tells the system to read both drives as one. It's not a solution for world hunger or anything, but rather a cool project I plan to work on. Hopefully some smarter people can make it more usable for mass consumption, but I honestly don't care lol. XDA was founded on crazy ideas not intended for the masses anyways
KillaHurtz said:
Sorry Cynagen....what I'm talking about requires no special firmware or hardware. It's a unix/linux script that tells the system to read both drives as one. It's not a solution for world hunger or anything, but rather a cool project I plan to work on. Hopefully some smarter people can make it more usable for mass consumption, but I honestly don't care lol. XDA was founded on crazy ideas not intended for the masses anyways
Click to expand...
Click to collapse
SKIP TO THE BOTTOM FOR THE MOST BASIC OF SCRIPTS TO ACCOMPLISH YOUR GOAL!
Long-winded, drawn-out-explanation:
It requires a whole new file system for a TRUE hybrid system which intelligent offloads the most used files to the fastest media available. You can do a JBOD RAID to do the same thing, but Android doesn't support RAID, and again, that's a "firmware" type thing where the RAID hardware/software translates your calls for sector 6689544 (doesn't exist in the 8gb usable space) to sector 2593544 of your SD card. There is no script that can accomplish this without maintaining an insanely large journal of where everything is located. I thought about this, but it would literally require a faux drive location, an extra service running on Android to listen for the system requesting files (presenting the faux drive location), and then making the appropriate call to the right drive. Wikipedia "File system"
Here's the basics of what happens when a file is requested by an application on ANY operating system:
1. File request to OS
2. File system opens journal (table/list) of files, finds match starting at sector 6689544.
3. File system opens appropriate drive location to retrieve sector 6689544, locks the sector and any other sectors with data for the file (prevent write and corruption of the data)
4. Read-ahead buffers data from subsequent sectors with continued data stream for the requested file
5. A. Returns either data stream if file is larger than buffer size, repeats steps 4 and 5 until finished; B. If smaller than max buffer size, entirety of the file
6. Releases lock on data sector(s) (to prevent corruption during read by an opposing write operation to the same location)
Now, that's just the basics, if you get to the nitty gritty, when you're addressing multiple volumes as one larger volume, this requires intersecting the request for data either at step 2 with replacing the file system, OR providing the original file system a faux "virtual" disk to address as mentioned on Wikipedia:
Some file systems are "virtual", in that the "files" supplied are computed on request (e.g. procfs (Wikipedia)) or are merely a mapping into a different file system used as a backing store.
Click to expand...
Click to collapse
Either way, there has to be something to intelligently keep track of the location of the files, and with a Hybrid system like you're talking about, the best and fastest way would be a whole new file system. Creating a procfs "virtual" service is also possible, but is likely going to be much more prone to bugs and failures unless maintained very heavily. Porting an existing file system that can do this would be much much more feasible and can be built into the kernel directly. A simple script cannot accomplish this without just blindly mapping the SD card into the internal storage file structure.
Short and simple hack to accomplish what you want:
This is called a "symlink" which can already be accomplished quickly with the following command: "mkdir /storage/sdcard0/ext; ln -s /storage/sdcard1 /storage/sdcard0/ext"
There's the script you're talking about, but it's a dirty hack and yes, it'll work provided the "LN" command is available, but again, it's really dirty (I guess most don't care?)
If "mount" and subsequently "umount" are available commands in the Android terminal, something like this should work to completely remove the external SD mapping: (This is based on the Galaxy S II JB file system mappings:
umount /storage/sdcard1
mkdir /storage/sdcard0/ext
mount /dev/emmc /storage/sdcard0/ext
Now this is just a rough sketch of what you want to do, if you could provide me with a dump of a couple things, I can write up a script for you no problem. I'll need the following:
A listing of /dev: "ls /dev" output
The contents of /etc/fstab: "cat /etc/fstab" output
And the information on the "source folder" for the external SD card: "stat /External_SD" (or whatever folder it is)
hope you maje this
LIL_ROOKIE said:
hope you maje this
Click to expand...
Click to collapse
Cynagen said:
I'll need the following:
A listing of /dev: "ls /dev" output
The contents of /etc/fstab: "cat /etc/fstab" output
And the information on the "source folder" for the external SD card: "stat /External_SD" (or whatever folder it is)
Click to expand...
Click to collapse
I don't have this information, I need it to proceed. Pull this data from an ADB connection as root, and dump it to a text file like so:
"ls /dev > /external_sd/dev.txt" (or whatever /external_sd is for the S4's SD card)
"cat /etc/fstab > /external_sd/fstab.txt"
"stat /external_sd > /external_sd/stat.txt"
Then zip those text files and send them to me so I can dissect.
It's fairly simple to use a loop device, a simple chroot environment, and mdraid (reason for chroot) to test this out. You'd use an image file and create a loop device so it can be file system independent. Problem is kernel support for md_raid and loop devices... But I tested this out. .. results weren't that great... But they weren't awful. I used a 64gb SanDisk class 1 billion. . Or whatever that crap is labeled.
I also used rxdsk/rxadm (compiled for armel, obviously) to make a ramdisk. That was quick. That also requires chroot, custom kernel..
Edit: In terms of garbage collection. It's beneficial to have more free space than none. Is there an allotment of hidden space dedicated to this, would increasing the size of that space be beneficial, I don't know.
Heres a little piece from Micron.. What I'd like is info on the (presumably Samsung) nand and if it has said overallotment.
http://www.micron.com/~/media/Docum... Flash/tn2960_garbage_collection_slc_nand.pdf
Blades said:
It's fairly simple to use a loop device, a simple chroot environment, and mdraid (reason for chroot) to test this out. You'd use an image file and create a loop device so it can be file system independent. Problem is kernel support for md_raid and loop devices... But I tested this out. .. results weren't that great... But they weren't awful. I used a 64gb SanDisk class 1 billion. . Or whatever that crap is labeled.
I also used rxdsk/rxadm (compiled for armel, obviously) to make a ramdisk. That was quick. That also requires chroot, custom kernel..
Edit: In terms of garbage collection. It's beneficial to have more free space than none. Is there an allotment of hidden space dedicated to this, would increasing the size of that space be beneficial, I don't know.
Heres a little piece from Micron.. What I'd like is info on the (presumably Samsung) nand and if it has said overallotment.
http://www.micron.com/~/media/Docum... Flash/tn2960_garbage_collection_slc_nand.pdf
Click to expand...
Click to collapse
They're looking to make the EMMC an accessable folder within the internal NAND file structure is what it seems. What I originally thought was they wanted to make a hybrid drive like they were originally mentioning where we could offload the most accessed sectors to the (presumably faster) NAND and relegate all other requests to the slower SD card. Providing a large contiguous drive seamlessly. They just want the dirty hack to accomplish this, and that's their choice. Honestly, I'd rather have the whole package like what you're (and I was previously) aiming for to have the best of both worlds.

Categories

Resources