In short, what I will be doing here is mounting the /system partition as file system EXT2. Default is EXT4. The reason for doing this is simple. The /system partition is more of an access database where information is read and never written. The goal here is to remove journaling and not only journaling but the redundancy of the code tree of EXT4 by simply killing the EXT4 journal and leaving it mounted and formatted as EXT4.
EXT4 – The file system uses a function called journaling. Journaling, in short, is a file system log. What this provides to you as the user is a quick method of recovery if there is ever a system failure: unclean shutdown, file corruption, etc. If there is ever a problem, the journal is called and the data is restored with no issues. The cost of this journaling feature, however, is CPU time/usage. CPU cycles are used to write the journal to the disk.
EXT2 – This file system does not use journal. No recovery method, etc. So if something is corrupted, there is no way to recover corrupted files/blocks/data.
What does all of this mean? – Simple. What it means is those of us who are rooted, running a stable ROM/Kernel combo, and using a backup method such as TWRP or CWM, can safely use EXT2 for /system without any worry because all we need to do is make a backup of our /system partition via recovery, tuck it away on our internal SD card and save it for a rainy day - restore it if there is every an unclean shutdown (battery pull, reboot caused by an unstable kernel, etc.).
What benefits will you see/feel?
Honestly, I have no idea at this point. I know for a fact that the device will boot faster and read operations on the /system partition will be absolutely insanely fast. As fast as they can possibly be on the Note 3. As far as “end of the day” battery savings, well, this is kind of the point of me doing this and sharing. I am going to find out, and post my results here so other people can decide if they do or don’t want to do this as well. If all goes well, I’ll edit this post tomorrow with some quick instruction on how to convert the file system. It should be a quick and easy process.
Other benefits of using EXT2 instead of EXT4:
EXT2 is backward compatible with EXT4. What this means is you can have a file system formatted to EXT2, and it can be mounted as EXT4 and it will only utilize the newer useful features found in EXT4 that were not originally present when EXT2 was introduced. When mounted as EXT4, EXT2 will not use journaling, but will use EXT4’s superior block allocation and “tagging” feature. EXT4 has the ability to mark unused blocks on the disk so it knows to not look there for data – this saves precious time in read operations. EXT2 does not have this feature… except when it is mounted as EXT4
The biggest advantage, here, is to get rid of that useless EXT4 journaling feature used on our RO /system and save CPU cycles. We use backups.We are a different breed of users, right? We demand performance and push our devices to the limits and somehow manage to maintain a stable and usable system, correct?
YES! So, we’re gonna go ahead and take advantage of this super awesome backward compatibility of EXT2 on the /system partition and get some positive performance index out of it
INSTRUCTIONS - convert /system to EXT2
1. Unzip the .zip folder of your ROM of choice.
2. Find /META-INF/com/google/android/updater script.
3. Open the updater-script file in a text editor.
Any lines where you see "EXT4" and "system" in the same line, you need to change that 4 to a 2. Only do this for lines with system and EXT4 in the same line.
4. Save the changes, then zip the ROM back up.
5. Place ROM on your internal SD card, then boot into recovery - YOU MUST BE USING TWRP FOR THIS
6. Once you get into recovery, you will see your TWRP options (there are 8 tiles in recovery) - select the one that says "Advanced"
7. You will now see 6 tiles. On the left side, select the one that says "Terminal Command"
8. At the top you should see a / symbol - this is the directory you are going to begin your session in. You want it to be /
9. On the bottom right of the screen, hit "Select"
This next part is very important. Do not mess up.
When you get to the terminal screen, type this command:
Code:
mke2fs /dev/block/mmcblk0p23
DO NOT continue until you have double checked that the number you punched in is 23! I cannot stress this enough. 23.... 23..... 23..... again that last part is "...mmcblk0p23"
After you have verified the correct partition number 23, hit the GO button on the keyboard to execute the command. It will run for a couple seconds and finish up.
After it finishes reformatting, flash your ROM of choice (the one where you edited any lines). When you are done, boot up.
You are now running an EXT2 file system for your /system partition. Welcome to the thunder dome
Cool, can't wait to try, thanks for this finding
Sent from my SM-N900T using Tapatalk
Why not just create and mount an ext4 filesystem with no journal? There is a reason they made the ext4 driver able to mount ext2,3 filesystems. Aside from journaling, which can be disabled, ext4 and it's driver are generally better.
edit - Didn't mean to quote.
muqali said:
Why not just create and mount an ext4 filesystem with no journal? There is a reason they made the ext4 driver able to mount ext2,3 filesystems. Aside from journaling, which can be disabled, ext4 and it's driver are generally better.
edit - Didn't mean to quote.
Click to expand...
Click to collapse
The reason is because the op paths in the file system tree (speaking of EXT4) are optimized for the journal. Without the journal the code path is redundant and the journal-optimized block has empty overhead. EXT2's strength is it's simplicity in structure.
Basically what this translates to is EXT4 formatted disk, mounted as EXT4 and with no journal, is not as fast or efficient as EXT2 formatted disk mounted as EXT4.
I am running this set up right now. This thread will be updated soon with instruction on how to convert it.
Ext4 has quicker read times than Ext2 which too me is going to the biggest perofmance increase.
This sounds interesting, but I think I'll wait to see others' experiences before giving it a go myself. Will be keeping an eye on this thread though.
Sent from my SM-N9005 using XDA Premium 4 mobile app
The only reason EXT4 has faster read times is because of what I just explained - ext4 uses a method of marking unused blocks on a disk, so it doesn't scan them looking for something. EXT2 does not have this feature. However, when an EXT2 formatted disk is mounted as EXT4, it does utilize this feature of EXT4.
Do you have any data, your own or third party to back this up? Do you plan on using an ext2 system with the ext4 driver? Or the old ext2 fs driver? As I understand it, the ext4 driver was given the ability to mount 2/3 filesystems for simplicity sake as well as the newer driver being better but I could be mistaken.
edit - The way you explained it in your last post seems to sound like you'll use ext4 driver on an ext2 fs, I was just asking because I wasn't 100% sure.
muqali said:
Do you have any data, your own or third party to back this up? Do you plan on using an ext2 system with the ext4 driver? Or the old ext2 fs driver? As I understand it, the ext4 driver was given the ability to mount 2/3 filesystems for simplicity sake as well as the newer driver being better but I could be mistaken.
edit - The way you explained it in your last post seems to sound like you'll use ext4 driver on an ext2 fs, I was just asking because I wasn't 100% sure.
Click to expand...
Click to collapse
That was the intention, correct. Using EXT2 formatted /system but mount it as EXT4 - for the reasons I outlined. To effectively remove journaling and not only that, but the redundant overhead of journaling in the fs tree which would still remain if I were to, say, run this command...
Code:
tune2fs -O ^has_journal /dev/block/mmcblk0p23
EDIT** and so far, yes the difference has been noticeable in boot up time - which is a clear indicator of a performance increase. It loads the /system much faster during boot. Battery life has been slightly better as well.
Do you have a video or benchmark showing the differnce?
LancerV said:
Do you have a video or benchmark showing the differnce?
Click to expand...
Click to collapse
A benchmark? People actually use those? lol.
My own device before doing this mod booted in about 21 seconds from the off position. After doing this it was 15 seconds. I full booted it 3 times for each EXT4 and EXT2... it was consistently the same.
You don't need a benchmark to understand there is an obvious performance increase there. Battery life has also been a little better today. My usage on a daily basis is consistently the same with no changing variables. 15 hours off the charger and 75% battery with 1 hour and 30 minutes screen on time is what I typically see. Today I was 15 hours off the charger, 81% battery with 1 hour and 30 minutes screen on time.
CPU cycles have been reduced, obviously.
If you wanna give it a go... You won't regret it. Post your results here so we can compare :highfive:
Nothing wrong with storage benchmarks, hell the best Windows/Linux/Unix storage benchmark is from 2006 and works perfectly. It's pretty much a staple when it comes to setting up Fibre, and 10Gbe SAN's to make sure the throughput you should be getting you are getting and to test various raids in their IO performance along with read/write performance.
So to say lol people still use benchmarks is pretty naive.
LancerV said:
Nothing wrong with storage benchmarks, hell the best Windows/Linux/Unix storage benchmark is from 2006 and works perfectly. It's pretty much a staple when it comes to setting up Fibre, and 10Gbe SAN's to make sure the throughput you should be getting you are getting and to test various raids in their IO performance along with read/write performance.
So to say lol people still use benchmarks is pretty naive.
Click to expand...
Click to collapse
I have not seen one benchmark available on the playstore that is even remotely relevant. What a joke it is to see a tech article running a quadrant test comparing two devices using a benchmark... so, if you were talking about one of those - quadrant, antutu, etc... none of these are going to give me an accurate representation of what kind of performance increase or decrease I am seeing for /system because they don't test those partitions. There are more accurate ways to run a performance test on a disk.... but it's quicker for me to just go ahead and reboot the device since, well, I know what I am doing and happen to be doing it on a portion of the disk where a majority of the work by the CPU is coming from accessing this particular partition of the device.
During boot, there is one task being executed and one task only... booting! Any most everything being loaded is coming from where? /system
Naive? I spent time on Google's team as a software engineer a couple years back... I am far from naive. You are just not looking at this from a practical or logical point of view.
I'll give you an analogy before I go to bed so you can better understand:
If your car runs out of gas, and stops running, and somebody gives you a ride home, then somebody else calls you and says "Hey, I just saw your car on the side of the road, saw your post on G+ about running out of gas. Well, I happened to have a gas can with me and put a gallon of gas in it for you. Go pick it up!"
Now you get a lift back to your car, put the key in, and it starts right up. Do you then turn it off and walk to a gas station, get a gas can, walk back, put more gas in it, and drive away? No, you don't. Because you have already obtained a positive result. There is no need for "testing" at that point. You have clearly established a positive result.
Anyways, benchmarks.... can be useful sometimes. But you are mistaken if you think:
A) They will give me anymore information than I know at this point regarding the test method...
B) I am going to be allocating more time and resources to run a "relevant" test on my /system partition
C) I have not already obtained a clear result
Not trying to sound arrogant... but I've been doing this a while. I'll pass on the benchmarking this time around.
cun7 said:
I have not seen one benchmark available on the playstore that is even remotely relevant. What a joke it is to see a tech article running a quadrant test comparing two devices using a benchmark... so, if you were talking about one of those - quadrant, antutu, etc... none of these are going to give me an accurate representation of what kind of performance increase or decrease I am seeing for /system because they don't test those partitions. There are more accurate ways to run a performance test on a disk.... but it's quicker for me to just go ahead and reboot the device since, well, I know what I am doing and happen to be doing it on a portion of the disk where a majority of the work by the CPU is coming from accessing this particular partition of the device.
During boot, there is one task being executed and one task only... booting! Any most everything being loaded is coming from where? /system
Naive? I spent time on Google's team as a software engineer a couple years back... I am far from naive. You are just not looking at this from a practical or logical point of view.
I'll give you an analogy before I go to bed so you can better understand:
If your car runs out of gas, and stops running, and somebody gives you a ride home, then somebody else calls you and says "Hey, I just saw your car on the side of the road, saw your post on G+ about running out of gas. Well, I happened to have a gas can with me and put a gallon of gas in it for you. Go pick it up!"
Now you get a lift back to your car, put the key in, and it starts right up. Do you then turn it off and walk to a gas station, get a gas can, walk back, put more gas in it, and drive away? No, you don't. Because you have already obtained a positive result. There is no need for "testing" at that point. You have clearly established a positive result.
Anyways, benchmarks.... can be useful sometimes. But you are mistaken if you think:
A) They will give me anymore information than I know at this point regarding the test method...
B) Allocating the time and resources to run a relevant test on my /system partition
C) I have not already obtained a clear result
Not trying to sound arrogant... but I've been doing this a while. I'll pass on the benchmarking this time around.
Click to expand...
Click to collapse
Right on. Great work OP. While EXT2 isnt for everyone. Experienced users can gsin value from using it. Nice Work!!!
Sent from my Samsung Galaxy Note 3 using JellyBombed Tapatalk.
My phone takes about 15secs to boot up not sure Im going to really notice a 1-2sec diffrence, a better analogy would be taking a car that does 0-60 in 6secs and a pos looking car and one that does 0-60 in 6.2 secs and be looking like a sports car. The differnce would be so minuscial that if you asked people which one accelerated faster they would naturally pick the one that looks like a sports car because thats how peoples minds are trained to think sportty looking car = quicker.
Its kinda a mental placebo effect, hence why I asking if you had a boot up video seems like that would be proof enough.
Also ran a top command and suprise system is using 1% and its the top command so I dont see where you are coming up with the extreme cpu cycles you say journaling is causing. Not to mention the fact /system is pretty much only called upon boot. Most other times its going to be pulling r/w cycles from /data or /sdcard.
LancerV said:
My phone takes about 15secs to boot up not sure Im going to really notice a 1-2sec diffrence, a better analogy would be taking a car that does 0-60 in 6secs and a pos looking car and one that does 0-60 in 6.2 secs and be looking like a sports car. The differnce would be so minuscial that if you asked people which one accelerated faster they would naturally pick the one that looks like a sports car because thats how peoples minds are trained to think sportty looking car = quicker.
Its kinda a mental placebo effect, hence why I asking if you had a boot up video seems like that would be proof enough.
Also ran a top command and suprise system is using 1% and its the top command so I dont see where you are coming up with the extreme cpu cycles you say journaling is causing. Not to mention the fact /system is pretty much only called upon boot. Most other times its going to be pulling r/w cycles from /data or /sdcard.
Click to expand...
Click to collapse
The placebo effect you are speaking of are typically people that don't know what they are doing, and are simply just excited about something.
An example of this would be the 9 out of every 10 posts I see in a ROM thread over the years talking about how much smoother their device is after flashing "xxxxx ROM"... a stop watch is a pretty reliable method for checking boot times, which is what I used for this particular scenario because I knew it would be relevant. Also the fact that my usage is pretty much exactly the same every day is a pretty solid test attribute. Any change in battery life at 15 hours... even if it is 1%... I notice it.
Also, my own personal ROM that I use is entirely odexed. So, nothing is being drawn from /data/dalvik-cache that is a /system/framework or /system/app function. It is noticeably more efficient because the 280+ .odex files are being read from the same location... which is now ext2, and not ext4.
Is this kind of starting to make sense now or am I losing you more?
I am not aware of all the file systems on linux. What happens to the partition if you restore a nandroid backup of the system partition that was converted to ext2 but the backup was ext4? And/or if backup was ext2? Just curious.
Sent from my SM-N900T using Tapatalk
Compusmurf said:
I am not aware of all the file systems on linux. What happens to the partition if you restore a nandroid backup of the system partition that was converted to ext2 but the backup was ext4? And/or if backup was ext2? Just curious.
Sent from my SM-N900T using Tapatalk
Click to expand...
Click to collapse
It will write the data to the disk as it should. You can do this without having any issues.
I've never tried restoring an EXT2 backup to an EXT4 formatted disk... So I have no answer for you there. It all kind of depends on exactly how TWRP's backups are created... I think they use .tars ... I'm not sure
Would this by any chance work on an AOSP/CM/AOKP ROM? I've tried using it with CM and LiquidSmooth but it stops every time after "erasing system" with "script aborted (no error message)".
Nevermind, I found out the issue. I went back into the updater-script and removed this line: "format("ext4", "EMMC", "/dev/block/platform/msm_sdcc.1/by-name/system", "0", "/system");". It flashed perfectly after that (Liquid Smooth AOSP ROM).
Edit: Had an issue with the ROM not booting at first, it hung on the splash screen. What I did was reboot into TWRP, wiped cache and dalvik cache, went into terminal (in advanced) and typed this command: "setenforce 0". After that, type "getenforce" and make sure SELinux is set to Permissive. I'm not sure if the SELinux command helped at all but I did it just in case.
Hypercore said:
Would this by any chance work on an AOSP/CM/AOKP ROM? I've tried using it with CM and LiquidSmooth but it stops every time after "erasing system" with "script aborted (no error message)".
Nevermind, I found out the issue. I went back into the updater-script and removed this line: "format("ext4", "EMMC", "/dev/block/platform/msm_sdcc.1/by-name/system", "0", "/system");". It flashed perfectly after that (Liquid Smooth AOSP ROM).
Edit: Had an issue with the ROM not booting at first, it hung on the splash screen. What I did was reboot into TWRP, wiped cache and dalvik cache, went into terminal (in advanced) and typed this command: "setenforce 0". After that, type "getenforce" and make sure SELinux is set to Permissive. I'm not sure if the SELinux command helped at all but I did it just in case.
Click to expand...
Click to collapse
Code:
"format("ext4", "EMMC", "/dev/block/platform/msm_sdcc.1/by-name/system", "0", "/system");"
Change that 4 to a 2...
Code:
"format("[COLOR="Red"]ext2[/COLOR]", "EMMC", "/dev/block/platform/msm_sdcc.1/by-name/system", "0", "/system");"
Related
[size=+2]This version is now deprecated in favor of the 2.0 APK version. Please see this link: http://forum.xda-developers.com/showthread.php?t=784691[/size]
This version is not recommended for use any longer.
Details about what this fix does:
Creates a VIRTUAL EXT2 filesystem inside the stock RFS filesystem on the internal SD card, with a 4KB block size. This means that this lag fix creates a buffer between the real filesystem and the android system. This buffer should reduce the amount of disk I/O required for all operations by utilizing EXT2 buffering, as well as not writing file access times to disk, etc.
Folders that are currently supported:
/data/data
/data/system
/data/dalvik-cache
/data/app
/data/app-private as a symlink to /data/app/app-private
/dbdata/databases is not supported. It appears to be ROM backed, and can cause problems if overwritten.
Benefits over version 1
1.5GB of application data available, with no data loss.
e2fsck of the EXT2 partition on each boot.
Correct busybox version included! YES!
/app and /app-private directory included in the fix for faster application installs.
/dbdata/databases included in the fix, expected to give a big performance boost for apps that use it.
Mounts instead of symlinks for extra performance as OS does less work (about 100 or so more quadrant).
Benefits over other lag fixes
Open source, with full comments and ease of editing.
Works on any and all firmware versions, including any yet unreleased froyo versions (that don't change file structure).
Credits
Big thanks to mimocan for putting us all on the right track in how to sort out lag problems!
Big thanks to ykk_five for showing us all how well loopback filesystem mounting works!
Big thanks to cyanogen mod for e2fsck
Requirements for One Click Lag Fix 2.0
Rooted phone - http://www.addictivetips.com/mobile...t-samsung-galaxy-s-i9000-with-a-single-click/
Windows computer with SGS Drivers (Samsung Kies), or the ability to read through the batch file and run it yourself.
(Beta Release) The ability to reflash your device if something goes wrong.
No other lag fix installed. If you installed One Click Lag Fix 1.0, then use the uninstall function which came with that lag fix before running this lag fix. (Untested but assumed to be working, please help out here.)
1.5GB of freespace on Internal SD Card for swap files while the fix is working (/sdcard).
"Internal phone storage" in Settings->SD Card must read greater than 500mb (0.5GB) of free space.
How to run One Click Lag Fix 2.0
Place your phone into USB debugging mode: Settings->Applications->Development
Attach your device to your computer. Do not mount the drives.
Download the attached ZIP file.
Unzip to a folder of your choice.
Double click "lagfixme.bat".
Don't double click "unlagfixme.bat".
Wait for it to complete.
You will need your phone to be unlocked when it runs the script, so that you can accept the permissions request that will appear on your device.
How to remove One Click Lag Fix 2.0
Double click "unlagfixme.bat"
Wait for it to complete.
Known Issues For All Versions
Some custom firmwares use up all available space in /system. This fix requires some libraries to be placed in /system/lib. These libraries are used to create the filesystem properly, and to check it for errors on every boot.
If your firmware does not have the available space (around 1mb) in /system, do not use this fix! Your phone will not boot and will have to be restored from backup / reflashed.
Current known firmwares with this issue: None yet. Please provide the firmware version+mods if you encounter this issue. It will show up as an out of space error in the log, under 'Copying libraries'.
Known Issues 2-1, 2-2
Paid apps from the market have issues.
Google maps and other pre-installed ROM-backed applications have issues.
2-3 Changes from 2-2
/dbdata has been removed. This fixes maps issues.
/app-private is now a symlink to /app/app-private. This fixes paid apps issues.
Alternate installation methods for similar fixes
Tayutama has made an update.zip version that is easy to install - http://forum.xda-developers.com/showpost.php?p=7632258&postcount=208
Chainfire has a .NET version of this fix with some nice features - http://forum.xda-developers.com/showthread.php?t=751513
Frequently Asked Questions
Q: My apps are force closing while this fix is running, and I can't use my phone!
A: By design. The script has backed up/copied your apps over to the internal SD card (remember the requirement about 1.5GB of free space on the SD card?). It is now overwriting them with a 1.5GB file. As the file overwrites an app that is trying to do something, it will probably force close. This is normal.
Closing all running apps, and removing widgets before running this fix can make the process much smoother, though.
Q: The script can't transfer files to my phone / The script can't run / Help help I'm dieing!
A: Read the first post again.
Q: My paid apps from the market don't show up.
A: I will hopefully have a fix for this sooner or later. Hold tight! It's in the known issues. I don't have access to paid apps, so I can't test this.
Q: Does this need busybox?
A: No, busybox is included.
Q: I only have 200mb of free space now! What gives?
A: The lag fix has made a 1.5GB file, and is storing all of the data inside there. The side effect is that the free space meter is now incorrect. Sorry, this can't be helped.
You can check real free space by using ADB like this:
Code:
adb shell
su
busybox df -h
Q: When I use a backup tool, the backup is now 1.5GB big! It's taking forever!
A: The backup tool isn't designed to work with this fix. It will work, it just won't work well. Hopefully this fix will be short lived, and either Samsung will give us a new update, or someone will give us a good custom firmware that can natively mount what we need, where we need it. Or someone might come out with a better backup manager. Until then, we suffer.
Q: Will a reflash wipe this fix?
A: Yes, a reflash will wipe everything this fix did.
Q: Can this brick my phone?
A: If you know how to get to the download mode from power off (hint: volumedown+home+power), then almost nothing short of throwing your phone off a tall building can actually brick it. If you can't do this though, or don't know someone who can, then you're better off waiting for samsung to release a fix. Anything that moves files around on your device has the potential to break things, and this fix has no QA department.
Q: Why is /dbdata not included in your fix, but other people have included it?
A: Other people have included it in the same way my 2-2 fix includes it. However, /dbdata is ROM backed. This means that the real files are on ROM, and only the changes appear in the /dbdata folder. When copying or moving files from this folder, you would need to specify each folder by exact name to ensure that it was copied across, and each firmware can have their own names. (This is because RFS wildcard will not catch an unused ROM backed file.) In some cases, you can get lucky and have this work perfectly because you have already used all the files in /dbdata. There is no fail safe method to do this though, and /dbdata does not make a big difference to performance. (It is already on NAND flash.) If you want to try your luck, v2-2 is still available.
Q: Why does this lag fix work? Is it slowly destroying my phone?
A: Let's say an application counts from 1 to 10, and writes the value each time to disk.
Stock:
1 -> App tells RFS to write 1 to disk -> RFS writes 1 to disk -> RFS writes journal saying to changed the value on the disk.
2 -> App tells RFS to write 2 to disk -> RFS writes 2 to disk -> RFS writes journal saying to changed the value on the disk.
..
9 -> App tells RFS to write 9 to disk -> RFS writes 9 to disk -> RFS writes journal saying to changed the value on the disk.
10 -> App tells RFS to write 10 to disk -> RFS writes 10 to disk -> RFS writes journal saying it changed the value on the disk.
Total physical disk writes: 20. Speed: SLOW! Wear and tear on disk: HIGH!
Lag Fix:
1 -> App tells EXT2 to write 1 to disk -> EXT2 stores 1 in RAM.
2 -> App tells EXT2 to write 2 to disk -> EXT2 stores 2 in RAM.
..
9 -> App tells EXT2 to write 9 to disk -> EXT2 stores 9 in RAM.
10 -> App tells EXT2 to write 10 to disk -> EXT2 stores 10 in RAM.
..
EXT2 tells RFS to write 10 to disk -> RFS writes 10 to disk -> RFS writes journal saying it changed the value on the disk.
Total physical disk writes: 2. Speed: FAST! Wear and tear on disk: LOW!
This isn't exactly what is happening, but it gives you the general idea.
Q: Can this mod work on other Android devices? Would we see a performance boost on them as well? If not why is it limited to the Galaxy S?
A: SGS has very very good hardware, but it has some parts of it's hardware poorly implemented. The filesystem that samsung chose to use is custom-built using FAT32 as a base, RFS. It has a lot of the problems that FAT32 has, and should have been left back in the 90s, or even the 80s.
One of the big issues with it is how it handles multiple requests - it blocks. It blocks everything. When your mail app wants to read the mail you just tried to view, but your twitter app is busy writing a new tweat it just received, your mail app is forced to wait.
This is bad, but it could be worse! And it is... your twitter app didn't just get one tweat, it got 50 tweaks. It is busy writing the tweats one by one to the filesystem. This would be fine, since all modern filesystems will buffer writes, so instead of writing each one at a time, they will batch them together and write it as a big chunk. Uh oh - RFS does no buffering at all! After each write, it will also write an update to the grafted-on journal system. Guess what happened to your mail you were trying to view while all this happened? It 'lagged' and you got a black screen for half a second, before the mail popped into view.
Luckily the hardware on the device is so good that you usually don't even notice the problem until you have a lot of apps running, all writing their updates when you unlock the phone.
This is mostly speculation based on experiments done on RFS -- RFS is closed source, and we have no idea if the problems are just badly set settings (such as a block size that is too small), coding bugs in the implementation, or if RFS is just really that badly designed.
This fix just grafts a buffer on top of the RFS filesystem, using a very very simple and fast filesystem, EXT2. It fixes most of the issues by writing to RFS as seldom as possible.
So no, this fix won't fix other devices, since they're already running quite close to maximum speed. The SGS at stock is running nowhere near maximum speed, and this lag fix takes it a bit closer. You could probably speed up other devices by tweaking the filesystem settings to give them a big buffer or similar, but it isn't really needed. (I haven't actually tried to put an EXT2 onto any other Android phone, as I don't have any other Android phone, so this is just speculation.)
Q: My phone is fast now!
A: Yeah.
Oh, awesome. I managed to post this in the wrong forum. Doh!
Could a moderator please move this to Android Dev sub forum?
haha looks awesome dude but you are correct, wrong section im gonna try it now, will report in 5 mins!
A few people in the other thread said they had the first lag fix working on the captivate. Anything that might change that with this release? I came to download the old one, and here a new one is
abra-cadabra...
*POOF*
Done!
E_man5112 said:
A few people in the other thread said they had the first lag fix working on the captivate. Anything that might change that with this release? I came to download the old one, and here a new one is
Click to expand...
Click to collapse
The old one is still here. Nothing has changed that should stop it working on the captivate though, but it is completely untested. Use the 1.0 (which has had a lot of testing) until this one has been put through the paces.
sirphunkee said:
abra-cadabra...
*POOF*
Done!
Click to expand...
Click to collapse
Thanks.
RyanZA said:
The old one is still here. Nothing has changed that should stop it working on the captivate though, but it is completely untested. Use the 1.0 (which has had a lot of testing) until this one has been put through the paces.
Click to expand...
Click to collapse
E_man5112 said:
A few people in the other thread said they had the first lag fix working on the captivate. Anything that might change that with this release? I came to download the old one, and here a new one is
Click to expand...
Click to collapse
I'm running the manual ext2 fix with Q scores 2000+. I'll try 2.0 Util tonight and post feedback....
Someone got some benchmarks?
Maybe for xxjp3? I only reach 1900 in quadrant.. /:
Sent from my GT-I9000 using Tapatalk
gonna test report in 5minutes
RyanZA said:
The old one is still here. Nothing has changed that should stop it working on the captivate though, but it is completely untested. Use the 1.0 (which has had a lot of testing) until this one has been put through the paces.
Click to expand...
Click to collapse
I'm in the process of flashing back to stock on my Captivate so I can give this a legit test. I'll report back here and let you know how it goes. Assuming my Captivate doesn't melt I'll update my OP from the thread about the original fix in the Captivate section!
About what lag is this?
Sent from my HTC Desire using XDA App
Dominik06 said:
Someone got some benchmarks?
Maybe for xxjp3? I only reach 1900 in quadrant.. /:
Sent from my GT-I9000 using Tapatalk
Click to expand...
Click to collapse
Benchmarks are roughly the same as 1.0. You can expect about 100 points more on average though, at least on my phone.
I just removed the V1 version and finished the V2.
It finally finished and rebooted the phone. now it tells me my internal memory storage is full. None of my Widgets will load.
Looking i see only 14MB free in internal storage.
thoughts?
Zilch25 said:
I'm in the process of flashing back to stock on my Captivate so I can give this a legit test. I'll report back here and let you know how it goes. Assuming my Captivate doesn't melt I'll update my OP from the thread about the original fix in the Captivate section!
Click to expand...
Click to collapse
Just as long as you point out that this fix isn't heavily tested yet!
RyanZA said:
Just as long as you point out that this fix isn't heavily tested yet!
Click to expand...
Click to collapse
Oh I know, I'm going to emphasize that heavily, lord knows I don't want to get stuck on tech support all night
clubtech said:
I just removed the V1 version and finished the V2.
It finally finished and rebooted the phone. now it tells me my internal memory storage is full. None of my Widgets will load.
Looking i see only 14MB free in internal storage.
thoughts?
Click to expand...
Click to collapse
Thoughts? That's bad.
You should be seeing roughly 215mb free. Did you see any errors in the log at all?
Everything is working fine for me. First reboot after lagfix had some problems with downloading apps from market, but after a second reboot it got fixed.
"Biggest" problem now, is just the phone stating that my internal phone storage is too low, with an icon on notification bar that I cannot remove. Already deleted some .bak from /data folder, any more tips what I can delete to get rid from this message?
Regarding the fix. apps are indeed much more snappier, no lag on the system when installing apps from market, and overall if the phone would continue like this for the next 48 hours, lag fix form me is solved. I had some problems after some 24h, with some lag, even with the previous version of the lag fix.
clubtech said:
I just removed the V1 version and finished the V2.
It finally finished and rebooted the phone. now it tells me my internal memory storage is full. None of my Widgets will load.
Looking i see only 14MB free in internal storage.
thoughts?
Click to expand...
Click to collapse
Looking through my old V1 remove script, I believe I was leaving behind the .bak files! Nastyyy... I'll update V2 to remove those if they're there on install. Will hopefully clear up any problems.
Since I installed JPK on a test phone after claims that the RFS related lag has been fixed.. well.. I'd like to share my impressions:
- I have installed approx 100 apps after the flash (flashed / wiped / factory reset btw)
A this point, the phone while slightly slower than one with Voodoo is pretty fast
A few days later, it's still good enough.
- 5 days later, lag starts to be felt, and it's stronger every next day until it becomes annoying to use
That's exactly the same experience as I had on various Eclair roms without any lag-fix. I also had this on JP3 (early Froyo).
So yeah. I looked into the /init binary from samsung and it's supposed to make some file system checks from time to time when you restart the phone, but it does not appear to really carry on with that.
So I ran the checks myself:
- you need a rooted phone, and adb or a terminal
- find all RFS partitions:
$ su
# mount|grep -i rfs
- kill all processes, go flight mode and remount them read-only
$ su (if you're not root.. not going to repeat it again for subsequent commands)
# kill -9 <pid> of anything that use the patrition
# mount -oremount,ro /dev/block/....
- check the RFS filesystem and correct errors
# /system/bin/fsck_msdos -p -f /dev/block/....
Surprise, a million of RFS errors fixed such as:
/path/to/file starts with free cluster FIXED
Cluster XXX continues with cluster XXX in FAT 0 but is marked free in FAT 1 FIXED
Detected cluster chain loop head XXXX for p XXXX FIXED
FSNext block XXX is correct NumClusters XXX FIXED (weird one)
No LOST.DIR FIXED
Lost cluster chain at cluster XXX YY clusters lost FIXED (prolly lost files/data here!)
Repeat for each partition
reboot the phone at the end
SURPRISE! It doesn't lag much anymore.
Use the phone an hour, do the check again and you will see it's already full of errors.
Conclusion:
RFS is bugged (we knew that didn't we) but it looks like it's fixable, if ever Samsung figures out what is corrupting the file system exactly (it's closed source so we can't really find out easily)
It might be possible to figure it out by looking at VFAT sources too.
I would be very interested to see if that fixes the lag for everyone or if i'm an isolated case and all my RFS partitions are on bad hardware or if it's really software corruption as I am guessing
Small disclaimer:
im not responsible for any data loss etc. no warranties etc. fixing file system even readonly might cause data loss due to the bugs in RFS. you've been warned lol.
My God,
I think a lot of eyes will be on the BOUNTY
Could someone just create an app that does this?
bilboa1 said:
Since I installed JPK on a test phone after claims that the RFS related lag has been fixed.. well.. I'd like to share my impressions:
- I have installed approx 100 apps after the flash (flashed / wiped / factory reset btw)
A this point, the phone while slightly slower than one with Voodoo is pretty fast
A few days later, it's still good enough.
- 5 days later, lag starts to be felt, and it's stronger every next day until it becomes annoying to use
That's exactly the same experience as I had on various Eclair roms without any lag-fix. I also had this on JP3 (early Froyo).
So yeah. I looked into the /init binary from samsung and it's supposed to make some file system checks from time to time when you restart the phone, but it does not appear to really carry on with that.
So I ran the checks myself:
- you need a rooted phone, and adb or a terminal
- find all RFS partitions:
$ su
# mount|grep -i rfs
- kill all processes, go flight mode and remount them read-only
$ su (if you're not root.. not going to repeat it again for subsequent commands)
# kill -9 <pid> of anything that use the patrition
# mount -oremount,ro /dev/block/....
- check the RFS filesystem and correct errors
# /system/bin/fsck_msdos -p -f /dev/block/....
Surprise, a million of RFS errors fixed such as:
/path/to/file starts with free cluster FIXED
Cluster XXX continues with cluster XXX in FAT 0 but is marked free in FAT 1 FIXED
Detected cluster chain loop head XXXX for p XXXX FIXED
FSNext block XXX is correct NumClusters XXX FIXED (weird one)
No LOST.DIR FIXED
Lost cluster chain at cluster XXX YY clusters lost FIXED (prolly lost files/data here!)
Repeat for each partition
reboot the phone at the end
SURPRISE! It doesn't lag much anymore.
Use the phone an hour, do the check again and you will see it's already full of errors.
Conclusion:
RFS is bugged (we knew that didn't we) but it looks like it's fixable, if ever Samsung figures out what is corrupting the file system exactly (it's closed source so we can't really find out easily)
It might be possible to figure it out by looking at VFAT sources too.
I would be very interested to see if that fixes the lag for everyone or if i'm an isolated case and all my RFS partitions are on bad hardware or if it's really software corruption as I am guessing
Small disclaimer:
im not responsible for any data loss etc. no warranties etc. fixing file system even readonly might cause data loss due to the bugs in RFS. you've been warned lol.
Click to expand...
Click to collapse
VOLD is what does the RFS check at boot. It seems to run, although maybe it fails? It definitely does run though, you can check it running using 'ps' on phone boot. Maybe it's only cleaning up /sdcard though.
At any rate, running the disk check does help with speed, but it doesn't help that much. It's still slow. I think if you want to stick with RFS, you need to do a defrag as well as the filesystem check. After some use, RFS must be very very fragemented on most people's phones.
Even in perfect condition though, RFS still has some very nasty properties such as locking the entire disk when a write occurs, not doing buffering, etc etc.
INeedYourHelp said:
Could someone just create an app that does this?
Click to expand...
Click to collapse
An app can't do this, since the app would have to be running off RFS and would crash/have to be killed to perform the FS checks. It could be done on boot using some trickery and the playlogos / replace binary trick. Or it can be done by replacing the init script with some kernel hackery. But at that point, you might as well just use a real filesystem.
I guess a PC .bat file could be made that uses adb to do this for you and then reboot the phone... Doesn't seem worth the trouble though!
RFS is journelled. You sure the filesystem doesn't do the journaling properly?
Nice job figuring this out. Is someone forwarding all these findings to that Samsung dev?
It's not about RFS vs the other filesystems. I'm well aware of the performance of RFS. But it's decent enough when it's working properly. It's not nearly as fast as ext but fast enough that you don't get annoyed.
Thus fixing RFS would make the life of many people who aren't technically inclined better. If the RFS do get all corrupted everywhere, and Samsung figure that out and fixes it, it means a good thing for most people.
The rest of us will end up on ext anyways
And yeah I think the fscheck at boot fails, it must fail actually, i don't see how else it would happen.
bilboa1 said:
It's not about RFS vs the other filesystems. I'm well aware of the performance of RFS. But it's decent enough when it's working properly. It's not nearly as fast as ext but fast enough that you don't get annoyed.
Thus fixing RFS would make the life of many people who aren't technically inclined better. If the RFS do get all corrupted everywhere, and Samsung figure that out and fixes it, it means a good thing for most people.
The rest of us will end up on ext anyways
And yeah I think the fscheck at boot fails, it must fail actually, i don't see how else it would happen.
Click to expand...
Click to collapse
What firmware did you test on, btw? I've noticed that JPK does seem to take longer on the FS checks, so maybe they have it fixed already (doubt it though)?
RyanZA said:
What firmware did you test on, btw? I've noticed that JPK does seem to take longer on the FS checks, so maybe they have it fixed already (doubt it though)?
Click to expand...
Click to collapse
on JPK actually
im going to flash JM8 to see if its the same in fact, but i expect so
RyanZA said:
VOLD is what does the RFS check at boot. It seems to run, although maybe it fails? It definitely does run though, you can check it running using 'ps' on phone boot. Maybe it's only cleaning up /sdcard though.
Click to expand...
Click to collapse
I think VOLD only does the checks for the Internal SD and External SD, not the RFS partitions.
Interesting find though, about the FS errors.
I’ve been wondering why the lag appears after a couple of days. I suspected corruption in one way or another myself, as it stays after a reboot it could not have been RAM and there are no signs of running out of space. Thanks for your research and I hope it will lead to new fixes. Sad but true my fix is to reflash my phone almost weekly.
I run Doc's JPK super slim ROM which is really nice but still it lags, even with OCLF installed.
Yesterday I take my phone out of my pocket to take a quick photo. My phone wakes up and I sweep the glass. Halfway through the sweeping the animation stops. I wait patiently for a second or three and there my home screen is. (No widgets, no animations, just a single home screen with 12 icons on it of the applications I actually use). I click on the camera icon and I wait another 5 seconds for the camera to realize it is not supposed to sleep on duty. I make a photo, the actual moment is already gone by now but hey I have the thing in my hand. It just takes another 5 seconds to store the photo.
It is like being in a hurry with a toddler with you. You want to go quicker but you can’t get angry cause the little thing just can’t go faster. It has to stop and wonder about life every once in a while.
I like my phone and I am sure it will grow up.
Very interesting findings! Sure hope Samsung sees this or this is forwarded to some Samsung techs.
Maybe move this to the development forum then it might get more traffic from people that can actually help.
borchgrevink said:
Very interesting findings! Sure hope Samsung sees this or this is forwarded to some Samsung techs.
Click to expand...
Click to collapse
do you really think Samsung is reading every thread here on XDA?
Sent from my GT-I9000 using XDA App
matty___ said:
do you really think Samsung is reading every thread here on XDA?
Sent from my GT-I9000 using XDA App
Click to expand...
Click to collapse
No need to be uppity my friend ;-)
Some further info on possible RFS fixes here:
http://forum.xda-developers.com/showpost.php?p=8445244&postcount=143
With the sync issue fixed, and corruption fixed with checks, RFS might just work 'okay'! It won't be as fast as EXT, but it should still work fine if we can sort out all the bugs! And we have a much better chance of getting Samsung to include these fixes in a new firmware too.
If we could get hold of Samsung somehow, and manage to convince them that they should
1) disable the always sync behaviour
2) do full filesystem checks at boot up (or provide a utility to do the checks)
then RFS should be A LOT more usable!
Could someone perhaps turn this into a .bat-file?
RyanZA said:
Some further info on possible RFS fixes here:
http://forum.xda-developers.com/showpost.php?p=8445244&postcount=143
With the sync issue fixed, and corruption fixed with checks, RFS might just work 'okay'! It won't be as fast as EXT, but it should still work fine if we can sort out all the bugs! And we have a much better chance of getting Samsung to include these fixes in a new firmware too.
If we could get hold of Samsung somehow, and manage to convince them that they should
1) disable the always sync behaviour
2) do full filesystem checks at boot up (or provide a utility to do the checks)
then RFS should be A LOT more usable!
Click to expand...
Click to collapse
That quite right, I noticed the sync issue also.
As for the corruption, 2 things should be fixed:
- fsck from init should actually fsck the partitions properly on boot (we can sort of fix that by calling it in a script again)
- corruption should in theory not even happen so Samsung would have to work on RFS for that one
I ran JM8 for 2 days now and my rfs partitions are full of errors, like in JPK, just for confirmation if there needed to be one.
In fact it's still running fsck on /system as I'm writting and issues are filling the terminal .. its been running for 30s already lol
woeds said:
Could someone perhaps turn this into a .bat-file?
Click to expand...
Click to collapse
I haven't tried that but it might be easier to enable adb over usb (in development settings) then .. make sure you are rooted, type that:
> adb shell
$ su
#
<prompt on the phone for root, click allow>
reboot into recovery
> adb shell
$ su
# mount -oremount,ro /dev/block/stl9
# mount -oremount,ro /dev/block/stl10
# mount -oremount,ro /dev/block/stl11
<for safety i'm not including stl3 it's the EFS)
if there's any error due to "filesystem busy" them stop there, it means it doesn't work
otherwise:
#/system/bin/fsck_msdos -p -f /dev/block/stl9
#/system/bin/fsck_msdos -p -f /dev/block/stl10
#/system/bin/fsck_msdos -p -f /dev/block/stl11
# reboot
[ROM]JPM/JP6 "GoodBye RFS" ROM 0.2 (pre rooted, pre lagfixed, extremely experimental)
This is an ultimately experimental ROM. It might break your phone, cause headaches and data loss. This is only a showcase on how a ROM (mostly) without RFS(*) would work.
(*): this is only marketing bull****. The /efs partition still uses rfs in this rom.
This ROM contains knowledge from ChainFire, newmail, RyanZA, supercurio, ykk_five, ziggy (and me). Many thanks to them.
For the people who only care about scores and such:
Quadrant: 2006
Linpack: 14.501
Result of mount command:
Code:
/dev/block/stl9 on /system type [B]ext2 [/B](ro,noatime,nodiratime,errors=continue)
/dev/block/loop2 on /data type [B]ext2 [/B](rw,noatime,nodiratime,errors=continue)
/dev/block/loop1 on /dbdata type [B]ext2 [/B](rw,noatime,nodiratime,errors=continue)
/dev/block/stl11 on /cache type [B]ext2 [/B](rw,noatime,nodiratime,errors=continue)
For the people who actually care:
What is this?
This is a highly expermiental all-in-one lagfixed and rooted ROM based on JPM and JP6.
How to install?
You don't want to install this. Seriously... But you can find the DL link somewhere in this post if you're still curious. Simply open up Odin, use the contained pit, CSC, MODEM and PDA files, check re-partition, and wait.
It hangs after flashing at the initial boot screen (this was for version 0.1. Version 0.2 should boot up faster)
Yep, because it generates a 1.5GB large file, this takes around 3-4 minutes. It might also crash (shutdown, or switch to the battery loading screen) during this time, in case you have to reboot and try again. (or switch to a rom that's not experimental)
Hey, do I have to wait every boot? (this was for version 0.1. Version 0.2 should boot up faster)
Nope, only the first and second boot, and after each factory reset. (between the first and second boot there is a factory reset because of the CSC being applied by the recovery module)
Is this rom safe?
No. Moving near a melted nuclear reactor is safer than this.
Hey, this doesn't work / All my data is wiped / etc.
You've been warned.
For the technical people:
How?
Factoryfs.rfs was converted to ext2 on the phone prior flashing, and during flashing you actually flash "/system" as ext2 instead of rfs. Dbdata is empty, and will be replaced on the first boot, and cache.rfs was also converted to ext2 (so the csc files can be loaded on first boot), but will be actually reformatted on first boot. A pre-init script takes care of the cleaning up, and creating of ext2 partitions. The /data and /dbdata partitions are actually "dual-ext2'd" so they reside in a loop device which resides in an ext2 partition. The rationale behind this is to delay the fsync's which is the major cause of the lag. I also tried to do an "only ext2 without loop device" version, but that was useless (I've even got quadrant scores below 1k(!) with it). /cache is only single ext2'd however for recovery functions to work.
Everything is done by the /sbin/fs-patch.sh script. Feel free to check it. The script logs into the /res directory, you can check whether there were any problems during mount there (if you could actually boot of course...)
Why ext2?
The ext4 module compiled from the official sources do compile, do load, mkfs.ext4 does create the partition, but when mounting the phone drops a kernel panic. Until I can find an ext4 (or ext3) module that actually works there will be ext2 used, just to demonstrate that a no-rfs solution is actually working.
Why JPM/JP6?
You can find the rationale on the multinational ROM topic. This rom is actually the lagfixed version of that rom without the extras.
DL link:
0.1: http://android.sztupy.hu/dl/SztupY_JPM_JP6_NORFS_ROM_0.1.zip (on my server. Might be slow)
0.2: http://android.sztupy.hu/dl/SztupY_JPM_JP6_NORFS_ROM_0.2.zip (on my server. Might be slow)
Differences between 0.1 and 0.2:
0.1 uses rfs on the device + ext2 on a loop
0.2 uses ext2 on both the device and on the loop. 0.2 also has a much-much faster loop file creator utility (that creates the 1.8GB file in a second), so booting times should be normal again.
0.2 has higher quadrant scores (breaks the 2k barrier), but 0.1 felt more responsive (at least for me). I don't know anything about battery usage and such.
0.2 has problems with apps from the market, that install themselves on the sdcard. Hotfix
Links:
- scripts from initramfs
Appreciate you can post more comments about the problems and performance of this experimental ROM
Thanks for the great work!
Sent from my GT-I9000 using XDA App
this is exciting
Just threw this on my Captivate(since firmwares are semi-compatible), waiting for the ext2 partition to get created, and will report back. This is pretty awesome though, good work!
Thank you, looks very-very-very cool! Flashing now.
Great work.
Sent from my GT-I9000 using XDA App
Excellent work! Can you make a version that makes the /data partition native ext2 instead of a loop device? Thanks!
Woah, great stuff, though I'm not gonna try it, sounds to dangerous. I've been using your other ROM and lovin' it, thanks!
Hey sztupy, good work man... cant wait to see the first reports...
Linpack: 14035
What?!?
anyone has any success loading this rom? i was being curious, i tried flashing on my phone, but kept restarting and looping at the initial Galaxy S GT-i9000 screen.
by the way the linpack score, is it 14035 or its just missing a decimal point? lol!
giantdroid said:
anyone has any success loading this rom? i was being curious, i tried flashing on my phone, but kept restarting and looping at the initial Galaxy S GT-i9000 screen.
by the way the linpack score, is it 14035 or its just missing a decimal point? lol!
Click to expand...
Click to collapse
Try to reflash it using Odin. That's what I did and it's running now
I'm getting 14.211 mflops in 5.9 secs
What languages are in swype ?
Does it contain romanian ?
how is the stability so far, for you that have tested it? and screw the scores. How is the feeling? any lagg? anything?
Nice one! Has anybody had any issues with /system being EXT yet? I'm guessing not, but this is going straight on my phone anyway!
EDIT: noatime,nodiratime performs a lot better than relatime btw
Why is /data on loopback on top of RFS? Seems silly!
You should have done /data as a loopback on top of /dbdata EXT2 for extreme speed! Do it!
Just installed this ROM/Kernel...
Setup and Flashing went without any hitch. The first boot was longer than usual, but as it was forewarned..so no surprises there.
I will give it a thorough test today and post the issues here, if any.
Thank you for this amazing work.
lealdud1 said:
how is the stability so far, for you that have tested it? and screw the scores. How is the feeling? any lagg? anything?
Click to expand...
Click to collapse
No Lag at at all. This is by far the ZIPPIEST ROM I have tried...(without any lag-fix installed)
I agree, screw the scores...it's the feel and the response that matters and this ROM is just great at it.
Haven't used it much but no FC or any other errors...so far, so good.
RyanZA said:
Nice one! Has anybody had any issues with /system being EXT yet? I'm guessing not, but this is going straight on my phone anyway!
EDIT: noatime,nodiratime performs a lot better than relatime btw
Why is /data on loopback on top of RFS? Seems silly!
You should have done /data as a loopback on top of /dbdata EXT2 for extreme speed! Do it!
Click to expand...
Click to collapse
I'll definatly give this rom a try when this ^ has been done and when a few more ppl have tested it without to many issues.
Thanks in advance for all your work on this rom guys, I love your other rom sztupy.
Sounds very exciting! Will probably wait until you manage to get /data as ext2 to give it a try. Still, well done with this!
will try on saturday...just to see if any further "improvement" will crop up by then
thank you sztupy for putting this out here!
**** EXT4 for DATA (SD) install on second post *****
I've been slowly working on our Kaiser Kernel hoping to get extra life from it. Here are the changes I made so far.
1. Updated to the latest initrd and made a few changed in the init file
2. Changed the default I/O scheduler to noop.
3. Added block data integrity support and v4 support
4. Updated the latest yaffs2 filesupport up to changes dated 1/14/2011
5. Added PM module for Qualcomm 7500 chip
Will try to add BFQ I/O scheduler and see how the performance is compared to noop.
1/29/11 ** UPDATED KERNEL for better YAFFS2 best suited for Kaiser Samsung Memory chip **
Here is the nbh. This is type2 screen and keyboard1.
Will release my latest changes and update to git (need help on this one) when stable.
Please reinstall ROM and wipe data after flashing the kernel. The changes in yaffs2 code might cause issues. You can restore your data from BACKUP after installing the ROM.
************ TO FORCE CHECKPOINT IN DATA PARTITION PLEASE DO THIS IN TERMINAL *******************
su
echo 6 > /sys/module/yaffs/parameters/yaffs_auto_checkpoint
sync
This will change auto checkpointing to 2 and create a checkpoint.
This is just temporary, the checkpoint is deleted in reboot so the commands have to be done before every reboot.
One solution is to use Gscript.apk and type the commands in Gscripts. Execute via Gscript before reboot
**** To verify if you have checkpoint in Data Partition Please do in Terminal ******
To verify if you have yaffs2 checkpoint in the data partition Type the following in terminal
su
cat /proc/yaffs
Scroll down to Device 2 "userdata"
scroll down and look for "blocks_in_checkpt" should be 1 or 2 (not 0, if it is zero then you have no checkpoint in data partition)
Update 2/7/2011
******************EXT4 FILESYSTEM FOR SD INSTALL***************
Because of bad blocks on my Kaiser, I have to give all NAND partition to install on system while installing data on SD with EXT4 filesystem. Here are the requirements:
1. An SD card with at least class 4
2. A way to partition and format your SD with FAT/ext2/Ext4 (ext2 should be as small as possible), I use Ubuntu for this.
3. Kernel with ext4 support (this will format your whole NAND partition to use system and use EXT4 on SD for Data)
Instructions:
1. FORMAT SD (I use Ubuntu DiskUtil) with FAT/ext2/EXT4, on my 8GB SD I use 7gb for FAT/8mb for ext2/1gb for EXT4 but it's your call. The most important part is that you only give the second partition (ext2) the least amount of size.
2. Download the KAISIMG.zip, extract and you will get a file KAISIMG.NBH
3. use Atools to edit accordingly with your Kaiser's screen type and keyboard (NBH is using screen 2 and keyboard 1)
4. Flash the KAISIMG.NBH
HERE IS WHERE IT GETS TRICKY!!! PLEASE FOLLOW THESE INSTRUCTIONS CAREFULLY!!!!
a. PRESS CENTER D PAD to enter ANDROID Installer
b. Choose (*) Sys on NAND
Choose (*) Data on SDCard Partition
c. Scroll Down to Install System and choose it
d. This will wipe your Existing system! Are you sure CHOOSE (YES)
e. Would you like to clear data too (Suggested) CHOOSE (NO) ***DO NOT CLEAR DATA OR THIS WILL CHANGE YOUR THIRD PARTITION TO EXT2 and will not work.
That's it. After installation reboot and you have an optimized ext4 on your SD for data. The mount options for ext4 in this kernel are:
data=writeback, barrier=0, nobh, noauto_da_alloc
Will use barrier=1 for better data security on next release, just enjoying the speed of barrier=0 for now
clemsyn said:
Will try to update our 2.6.25 kernel too if I get some space for my hard drive.
Click to expand...
Click to collapse
Yes, Clemsyn If it's not a big problem please update 2.6.25 too. Thank you.
updates kernel 2.6.25
clemsyn said:
Will try to update our 2.6.25 kernel too if I get some space for my hard drive.
Click to expand...
Click to collapse
This would be nice. Thank you
mt1976 said:
This would be nice. Thank you
Click to expand...
Click to collapse
It would take awhile though since I have to remove my 2.6.32 kernel to put 2.6.25 since I'm limited in space. Donations for a laptop HD would be appreciated
UPDATE:
1. I think one of the reasons of data failure is that YAFFS refuses to write a checkpoint for our data partition. It writes it in the system partition with no issues. The new yaffs2 (updated 1-27-11 which I will update shortly) allows forcing checkpoint so I'll try that one and see if I can create a checkpoint on our data partition.
Finally got a checkpoint in my userdata partition...instructions posted on first post, no update on yaffs needed
clemsyn said:
It would take awhile though since I have to remove my 2.6.32 kernel to put 2.6.25 since I'm limited in space. Donations for a laptop HD would be appreciated
UPDATE:
1. I think one of the reasons of data failure is that YAFFS refuses to write a checkpoint for our data partition. It writes it in the system partition with no issues. The new yaffs2 (updated 1-27-11 which I will update shortly) allows forcing checkpoint so I'll try that one and see if I can create a checkpoint on our data partition.
Click to expand...
Click to collapse
I have a 20 GB Sata laptop drive I could get to you. Used to be in a 360 and then broke the enclosure open for a hdd when the one in my laptop died. Drive works just fine, just small lol
aceoyame said:
I have a 20 GB Sata laptop drive I could get to you. Used to be in a 360 and then broke the enclosure open for a hdd when the one in my laptop died. Drive works just fine, just small lol
Click to expand...
Click to collapse
Sorry about your phone. I was hoping you could test out my new kernel. ANyways, I think yaffs2 was having issues creating a checkpoint on the data partition since yaffs was only creating a checkpoint on proper unmount (which was not done in our kaiser). The latest yaffs can make us force a checkpoint by simply typing the command in the first post and typing "Sync" on terminal automatically creates a checkpoint in data partition. I'm currently testing it now and seeing how it goes.
BTW, I'll take that donation re:HD since I can use it as an external HD and put the 2.6.25 kernel there and improve it (that is if wife permits).
clemsyn said:
Sorry about your phone. I was hoping you could test out my new kernel. ANyways, I think yaffs2 was having issues creating a checkpoint on the data partition since yaffs was only creating a checkpoint on proper unmount (which was not done in our kaiser). The latest yaffs can make us force a checkpoint by simply typing the command in the first post and typing "Sync" on terminal automatically creates a checkpoint in data partition. I'm currently testing it now and seeing how it goes.
BTW, I'll take that donation re:HD since I can use it as an external HD and put the 2.6.25 kernel there and improve it (that is if wife permits).
Click to expand...
Click to collapse
lol she's really not that strict, she just thinks my development for our devices is a waste of time. Just PM me where to send it and ill get it out to you.
aceoyame said:
lol she's really not that strict, she just thinks my development for our devices is a waste of time. Just PM me where to send it and ill get it out to you.
Click to expand...
Click to collapse
Agreed, that's what my wife thinks too I'll PM you my location
BTW, please make sure that the location of yaffs_auto_checkpoint is in /sys/module/yaffs/parameters (some ROM might have it on modules).
UPDATE:
Checkpointing is working great for me so far. Command on first post has to be executed in every reboot to create a checkpoint so typing the commands in first post before rebooting will create a checkpoint and save data partition (me thinks).
You can install Gscript and run the script before reboot
To verify if you have yaffs2 checkpoint in the data partition Type the following in terminal
su
cat /proc/yaffs
Scroll down to Device 2 "userdata"
blocks_in_checkpt should be 1 or 2 (not 0, if it is zero then you have no checkpoint in data partition)
Can anyone help adding this to the shutdown script?
I've done 5 restarts, 2 battery pulls and 2 resets, made sure data partition was checkpointed before doing so....So far data partition has been OK with no FC's or errors, no lost apps with 18mb's left in data partition The thing that surprises me is that Phone Storage size hardly changes in size in every reboots so it's good news I have a feeling we finally nailed down this data corruption issue, but only time can tell since I will shift to prolong use before doing any restarts...
I have sent a PM to scooter hoping he can help me with a shutdown script so we won't need a Gscript. So far, Gscript is doing it for me. Upong reboot, I run the script and make sure I have a checkpoint in data partiion
clemsyn said:
sent a PM to scooter hoping he can help me with a shutdown script so we won't need a Gscript. So far, Gscript is doing it for me. Upong reboot, I run the script and make sure I have a checkpoint in data partiion
Click to expand...
Click to collapse
I've got it working in a script, i have it in a script which is run on startup and then again on a successful shutdown. I'll upload a couple of updates in a bit
EDIT: I have attached 2 updates to the post, one is for cyanogenMod based builds and the other is for all other builds such as fresh froyo etc.
Clemsyn, i have the latest yaffs source compiled into the kernel, i will commit it to git in a bit. With the latest yaffs a checkpoint is created every time sync is executed which makes scripts easier!
scooter1556 said:
I've got it working in a script, i have it in a script which is run on startup and then again on a successful shutdown. I'll upload a couple of updates in a bit
EDIT: I have attached 2 updates to the post, one is for cyanogenMod based builds and the other is for all other builds such as fresh froyo etc.
Clemsyn, i have the latest yaffs source compiled into the kernel, i will commit it to git in a bit. With the latest yaffs a checkpoint is created every time sync is executed which makes scripts easier!
Click to expand...
Click to collapse
I downloaded the CM update, and it didn't enable the checkpoint, still had to enable it manually through terminal.
Testing the kernel right now however, and so far liking it.
scooter1556 said:
I've got it working in a script, i have it in a script which is run on startup and then again on a successful shutdown. I'll upload a couple of updates in a bit
EDIT: I have attached 2 updates to the post, one is for cyanogenMod based builds and the other is for all other builds such as fresh froyo etc.
Clemsyn, i have the latest yaffs source compiled into the kernel, i will commit it to git in a bit. With the latest yaffs a checkpoint is created every time sync is executed which makes scripts easier!
Click to expand...
Click to collapse
Thanks scooter, I'll try it out. I got stuck on a fake vsync and can't get out of it that I was unable to create a checkpoint and upon reboot without a checkpoint, everything was messed up. This is perfect, getting a checkpoint on startup and another in shutdown Thanks a lot.
Krazy-Killa said:
I downloaded the CM update, and it didn't enable the checkpoint, still had to enable it manually through terminal.
Testing the kernel right now however, and so far liking it.
Click to expand...
Click to collapse
That's weird! So it didn't make a checkpoint on startup? I'm running the same script and init.rc and it works fine. I did push the file with adb though, did you fix permissions after updating?
scooter1556 said:
That's weird! So it didn't make a checkpoint on startup? I'm running the same script and init.rc and it works fine. I did push the file with adb though, did you fix permissions after updating?
Click to expand...
Click to collapse
Ooooooh.... Probably not, lol. I'll do that, and edit this post with my results.
*EDIT* Ok, I feel stupid. I still had an old androidupdate.tgz file in my sdcard andboot folder, and the boot menu used it instead of the correct .tar file... Hm, that explained why my Market stopped working, lmao.
Krazy-Killa said:
Ooooooh.... Probably not, lol. I'll do that, and edit this post with my results.
*EDIT* Ok, I feel stupid. I still had an old androidupdate.tgz file in my sdcard andboot folder, and the boot menu used it instead of the correct .tar file... Hm, that explained why my Market stopped working, lmao.
Click to expand...
Click to collapse
lol, rookie error
OK, it works on boot up since I get a checkpoint 1 every boot not sure about shutdown but if scooter says it works, I bet you it works in shutdown too
Scoot, I'm going to try to do a git diff and post it here so all the changes I've made in the config will be there.
Even with that little debacle, I had 0 corruption of data, or missing apps in my /data partition, and had to reboot 4-5 times, each time I used GScript to execute the command manually before shutdown.
And camera still works. ^.^
Krazy-Killa said:
Even with that little debacle, I had 0 corruption of data, or missing apps in my /data partition, and had to reboot 4-5 times, each time I used GScript to execute the command manually before shutdown.
And camera still works. ^.^
Click to expand...
Click to collapse
That's great news and if my memory serves me right, you easily have data corruption in shutdowns and reboot. IMO, scooter's script is the best option we have for this issue and you don't have to remember to use gscript (especially from an old person like me with slight dementia Anyways, I'll continue on with my testing with this script. If it is still working after a few days, I'll try to add Namespace support, CIFFS, PPOE in our kernel.
BTW, here is my git diff and .config file. Please put it up in GIT. Thanks.
Hello everyone,
I've compiled a kernel which separates the NAND memory into 3 partitions (well more then that but anyway). These partitions are system, data and cache like a native Android phone (which also has recovery but we don't need that). Previously cache was linked to a directory on the data partition which was constantly becoming corrupt causing errors on start-up. Often clearing dalvik-cache would allow Android to boot but data was lost. There's nothing to say that this will solve the problem but i'm giving it a go so thought I would post it up for you to try also.
The cache partition is 20mb which means the data partition is only 90mb or so. If you like a million apps on your phone either put some on your SD card or don't use it
I haven't completely incorporated this partition into the initrd which means that currently it will only mount if data is also on NAND but this can be changed easily enough, but for testing purposes I want everything on the NAND chip anyway.
Downloads from market on all versions of Android, unless the init.rc directory is changed in Gingerbread, all go to the cache partition and are deleted on restart. What this means is that you can't download and install an app larger then 20mb (which isn't recommended anyway with the small amount of data space available). This is a drawback but for those that use data on SD it should be possible in the future to utilise more of the NAND memory for cache so you can install whatever you want. I don't know of many apps larger then 20mb anyway though.
I have included a modified version of ATools in the zip. The standard version will overwrite the partition values and the kernel won't work properly so if you need to modify the kernel use the version included. It is also important that you DO NOT change the system or data partition sizes in ATools as this hasn't been modified to account for the cache partition so will mess it all up.
I have also included an androidupdate.tar modules update for the kernel in case your wifi doesn't work after flashing the kernel.
I think that's it for now, let me know how it goes!
EDIT::
One last thing, you will need to wipe your DATA partition to use this kernel because some of your data may be positioned where the cache partition is now, so if you load this kernel and android is heavily corrupt you know what you have to do!
To check the kernel is working correctly you can type "df -h" into a terminal and it should show all of the current partitions including one called cache on mtdblock4.
Should now work with polaris, kaiser and vogue. There is a VOGUEIMG.NBH included for those who have trouble with ATools. It is completely untested on Vogue so sorry if it doesn't work.
EDIT
I have re-uploaded a modified kernel because there was some issues with data with the last one. If you have already applied the last one flash the new one and apply the module update again
testing will leave feedback
Ok, using your Scoot CyanogenMod 6.1.2 (RLS6) [16.03.2011], overclock via atools+roguetools to 529, gpu oc, battery units tweaked to suit my battery.
No SIM card yet, don't have one to put there, sync with google to get contacts.
Installed GoLauncher. Enabled JIT and Keep Launcher in memory.
Once in a while, Android is killing all apps, launcher included... strange...
Nevermind that, it was Cyanogen and the LongPress BackKey. LongPress on our Kaisers during lag can become short press
First couple of days of heavy use I've had in ages with no corruption. First day of my new job today so it's been on Google Navigation for 3 hours and used extensively during the day. Had to do a hard reset at one point and everything was fine. had a few soft reset's too. Don't get me wrong, i'd be surprised if this is a fix, but it's been a while since I've used my Kaiser this much and not had to wipe my data partition!
Update: CM is a bit slow, trying:
VaniljEclair RLS11 - A fast & stable CM 5.0.8 for Vogue/Kaiser/Polaris [2010-08-19]
And everyting seems good. I've pushed it a little, several normal reboots.
Had to reflash system (didn't reflash data). No errors, FC's, whatevar. Everything works, yet didn't try to make/receive calls. Its 1am here...
Now this particular version, has a god damn bug that is get the best of me. The Power Button and the red button won't make the phone go to standby. If i long press, i get the normal reboot/power off/airplane etc menu, on both keys. But single press is ignore... Any idea ?
scooter1556 , i have a question. It is not the purpose of the thread or kernel, but how difficult would it be to activate scalling of the cpu ?
Right now, the two 3 biggest issues with Android on the Kaiser are, IMHO, Data corruption, Battery life, performance.
Please share your thoughts
Using this kernel with Valentine 1.02. GPU overclocked and CPU @ 520mhz. Super fast and for the first time... no data corruption! Verry happy so far!
daedric said:
scooter1556 , i have a question. It is not the purpose of the thread or kernel, but how difficult would it be to activate scalling of the cpu ?
Right now, the two 3 biggest issues with Android on the Kaiser are, IMHO, Data corruption, Battery life, performance.
Please share your thoughts
Click to expand...
Click to collapse
Well in theory not too hard, there is a feature built into the linux kernel called cpufreq which is currently disabled in our kernel because our cpu and chipset code doesn't support it but krazy-killa has been working on getting it working in his kernels with a little success as far as i'm aware but i'm not that sure. I haven't personally looked into it but if I get some spare time I definitely will. I've started a new job now which i'm sure will keep me quite busy but I still intend to so some work on the Kaiser when I get time
Best of luck on your new job! I wish i had enough knowledge to continue the kernel development, but i'm quite a ignorant
Still, thank you, till now it's one of the best kernels i've used.
Hi Scoot!
I love your kernel. No data corruption so far (two days). did a few stylus resets and stuff.. Magical =D. Now IMO we only need better sleeping and autofocus to call kaiser a fully android device! Please Keep up the awesome job, scoot! thanx, danke, sposeeba, bedankt, aciu, dzenkuja!
Nice job scooter, I've been using Scoot CyanogenMod 7.0.2 RLS2 [28/04/11] @ 500MHz, and it seems fine so far.
I tried Fat Free Froyo before CyanogenMod, which also worked great for the few hours it was running. I'm not sure which one to stick with though.
Keep up the good work. I wish I could help with the development, but I have yet to learn programming (been ten years now ).
This kernel looks pretty stable, but just a few quick questions:
1) What filesystem is used for the cache? If YAFFS, could it get corrupted? If Yes, what would happend then?
2) Is it really needed to have 20MB for the cache? ATM it seems to be used only by a like 1-2MB on the cache partition, would not it be enough to have for example 10MB for the cache leaving more space for apps?
3) What is the difference using apps in the system and data partitions? If I would run off the space on the data partition, can I put some apps in the system apps folder to save the space without wandering about how it works?
Thanx for your great work, finally it seems to be fully usable for me and if you and Krazy-Killa would be able to work together to manage the power consumption, then it would be amazing, because I am going to get new phone (running some new extensive business and I can not afford to be not reachable when something happends on Kaiser) and give this lovely piece of HW to my son... And I really do not want him to have a WM machine But Android needs to be stable for him to use Well... he will get it for his 10th birthday at 6.9. so we still have some time to tweak it a bit more
if this works then it would be magical, your CM builds were so fast and awesome and if there's no data loss then... <3
Made 2 mistakes when building kernel (selected tilt instead of normal and forgot to OC to 480MHz), and running kingshui's 12/15/2010 build 2.2.1. seems ROCK solid and fast! I applaud your use of ramzswap for the extra 20mb, bravo!
I'm just hoping I can go back and rebuild the kernel with the correct settings without messing up the system/data/cache partitioning.
PoXFreak said:
Made 2 mistakes when building kernel (selected tilt instead of normal and forgot to OC to 480MHz), and running kingshui's 12/15/2010 build 2.2.1. seems ROCK solid and fast! I applaud your use of ramzswap for the extra 20mb, bravo!
I'm just hoping I can go back and rebuild the kernel with the correct settings without messing up the system/data/cache partitioning.
Click to expand...
Click to collapse
They aren't mistakes, those are the settings I need for my Kaiser. I included a modified version of ATools for you all to modify the kernel for your devices as I stated in the first post
a.s.j said:
1) What filesystem is used for the cache? If YAFFS, could it get corrupted? If Yes, what would happend then?
2) Is it really needed to have 20MB for the cache? ATM it seems to be used only by a like 1-2MB on the cache partition, would not it be enough to have for example 10MB for the cache leaving more space for apps?
3) What is the difference using apps in the system and data partitions? If I would run off the space on the data partition, can I put some apps in the system apps folder to save the space without wandering about how it works?
Click to expand...
Click to collapse
1) The cache partition is still using Yaffs2 because it is the easiest filesystem to configure at the minute, but the idea of making it a separate partition like on a native android device is that it can be changed to another filesystem in the initrd relatively easily. If it gets corrupted it will do what it used to I guess, but I haven't had any corruption as of yet so it's difficult to say. Hopefully if the cache partition get's corrupted it shouldn't affect data so you should just be able to wipe and format dalvik-cache in the install menu.
2) I made it 20mb because most android devices have 30mb or bigger and this is mainly because market app downloaded get downloaded here before being installed, so if you installed google maps for instance it would use up 6mb, plus the standard 1-2mb normally being used, but if you only had a 10mb cache for instance, you would only be able to install apps less then 8mb which is a little limiting really. I don't really care much about installing lots of apps in data or on the phone at all for that matter so having 95mb for data is more then enough for me.
3) You can put apps in the system partition in /system/app. The system partition is read only so all data for the app still goes to the data partition. You can still save a little data partition space though if you need it. Most installs of android are 80-95mb which should leave you 5mb or so to play with. Some all language builds pretty much use up all of the partition though so you need to look and see what space you have to play with before putting apps on the system partition. If you overload it you will probably end up corrupting it and having to reinstall. But to answer your last question, the system partition works in the same way as the data partition apart fro the fact that it is read-only and data is read/write.
I've modified my Kernel to use the NAND mtdblock3 (/data for everyone else) as the cache partition, since my NAND is pretty much shot to begin with, and have system and data on SD Card. So far it's made downloading Market Apps twice as fast, loading webpages pretty fast, and basically anything else that the cache is used for.
scooter1556 said:
They aren't mistakes, those are the settings I need for my Kaiser. I included a modified version of ATools for you all to modify the kernel for your devices as I stated in the first post
Click to expand...
Click to collapse
Scoot:
I understand that the settings built into the .nbh are for your device. What I was meaning was I needed to modify the CPU clock up to 480MHz, and for some odd reason my device swaps the "@" and "!" buttons if I set it to "tilt", among other buttons being set wrong.
Also, I am used to having the left and right softkeys set as "vol up" and "vol down" respectively. Is this something I need to make an androidupdate for, or do I have to write a new NBH?
PoXFreak said:
Scoot:
I understand that the settings built into the .nbh are for your device. What I was meaning was I needed to modify the CPU clock up to 480MHz, and for some odd reason my device swaps the "@" and "!" buttons if I set it to "tilt", among other buttons being set wrong.
Also, I am used to having the left and right softkeys set as "vol up" and "vol down" respectively. Is this something I need to make an androidupdate for, or do I have to write a new NBH?
Click to expand...
Click to collapse
Can you not do all of this using the ATools i provided in the attachment in the first post? You should be able to load the nbh into ATools and then set the CPU speed, change the keyboard type and remap the buttons and then save it before flashing. Just don't change the partition sizes as this will mess up the cache partition I added until I make it more permanent.