Related
When I first started on this project, this kitchen existed of 5 script each containing few lines of code, only able to handle one LUA script at a time. I had just helped out on the first ever manila 2.0 VGA port and I could see the use of a kitchen that did most of the manual labour automatically. At that time my understanding of what needed to be done to successfully edit script was limited. As I tried to add more features to the script I learned more about LUA, the handling and decompiling of the scripts. I chose to make the kitchen do as much for you on first run. The kitchen got more mature after I met hilaireg who knew a lot more about batch scripting than I did and who's perfectionist style made the kitchen what it is today.
After we teamed up we started a little manila group and got into contact with co0kiemonster. He's the one who created\updated most of the tools on which the script relies, and so not only optimizing manila handling but also pushing it's boundaries. We put in the applications that best handle manila files and connected these via the script. Since our last update co0kie kept developing the tools and they became a lot better. A batch script isn't really the way to handle this but until we get a real manila GUI, we decided to give the script one last update. Since this will likely be the last big project hilaireg will be presenting here, this last version is dedicated to him.
Workstation/System Requirements:
The Manila Kitchen environment can run on a Windows XP, Windows Vista, and Windows 7 platform providing the following requirements are met:
Microsoft .NET 3.5 SP1 (M9Editor)
Elevated priviledges
Windows XP/Vista/Seven
Kitchen Contents:
ROOT
MANILATOOL.CMD by hilaireg & 12aon
Manila Kitchen Guide
Revision History
_SOURCE folder
_TOOLS folder
Shortcuts to tools
TOOLS
LUATOOL.EXE by Co0kiemonster
M9EXT.EXE by Co0kiemonster
MANILANAMES.EXE by Co0kiemonster
M9EDITOR.EXE by 6Fg8
CFC_GUI.EXE by Chainfire
NOTEPAD2.EXE by Florian Balmer, Compiled by Co0kiemonster
MANILAHASH.EXE by NisseDILLIGAF
XVI32.EXE by Christiaan Maas
RAPICOPY.EXE, RAPIPROC.EXE, RAPISTART.EXE by Scott Seliman
GREP.EXE by Tim Charron
Current Version (Attached Files)
Manila Kitchen 5.05.06, 4.7 MB (06/05/2010)
Previous Versions
Manila Kitchen 5.04.23, 4.7 MB (23/04/2010)
Manila Kitchen 5.03.04, 4.1 MB (04/03/2010)
Manila Kitchen 4.10.16, 9.3 MB (16/10/2009)
Manila Kitchen 4.10.02, 7.7 MB (02/10/2009)
Credits:
Special thanks to the following folks for sharing their knowledge and expertise. If I missed someone, it's purely accidental – send me a note and I will add your name to the list.
6fg8
chainfire
co0kiemonster
sztupy
Overview
The Manila Kitchen is an automated environment that includes tools (utilities) to perform decompilation, editing, and recompilation of HTC Manila OEM files.
Instructions are passed to the MANILATOOL command script which in turn, initiates various executables to perform the necessary actions required on the HTC Manila OEM files.
Some of the tools used by MANILATOOL were written by 6fg8, chainfire, co0kiemonster, and sztupy. As such, some of these tools are not compatible with excessively long path names or paths that contain spaces or brackets. To avoid issues, place the Manila Kitchen in a relatively short path that does not contain spaces or brackets such as: C:\XDA\Manila_Kitchen
The Manila Kitchen provides the following benefits:
Consolidated Tools
All of the tools required to perform Manila related activities have been incorporated. These include decompilation, comparison, recompilation, .CAB creation, direct deployment to mobile devices, updating Mode9 Editor reference index, and backing up. The executables are initiated with the appropriate command parameters and in the correct sequence by MANILATOOL thus eliminating the "guess work" typically required.
Improved Resiliency
The activities required to manage HTC Manila OEM files are complex in nature. In some cases, a Manila file may fail to be decompiled or cause one of the initiated executables to generate and error as a result of an unexpected condition. The MANILATOOL command script will attempt to recover from such errors when possible so as to complete the command that was initiated.
Command Line Help and Version Information
Wondering what command parameters are available in the MANILATOOL command script? Simply append -HLP to the end of the MANILATOOL command script to display a summary of parameters available. To determine the version of the MANILATOOL command script in use, append -VER to the end of the MANILATOOL command script.
Enhanced File Recovery
Accidentally changed a file and need the previous version? The MANILATOOL command script performs a backup before any critical operation making it easier to retrieve a previous version of a file.
Enhanced Script Diagnostics
Although a significant of amount effort has gone into the development of the Manila Kitchen, it is possible that a script logic error may occur. To assist in troubleshooting such errors, the MANILATOOL command script includes a comprehensive logging debugger.
Topic Index
Preparing the Kitchen Environment ....... 3
Initializing the KitchenDecompiling Manila OEM Files ............ 4
LUA Editing
Mode9 Editing
General Editing
Updating 'm9editor.names.txt'
Updating the RepositoryWorking with Manila Files ............... 5
Initializing the KitchenRecompiling Manila OEM Files ............ 6
RecompilationTesting the Manila OEM File Changes ..... 7
Installer .CAB Method
Direct to Device MethodReference Web Links ..................... 8
Frequently Asked Questions (FAQ) ........ 9
Preparing the Kitchen Environment
The Manila Kitchen environment must be initialized prior to initial use. During the initialization phase, all MANILATOOL command script parameters are enabled and the contents of following folders, sub-folders, and files are removed and recreated:
_backup
_queue
Decompiled
Release
Repository
Workspace
The following configuration files will be reset to their initial settings:
m9editor.cfg .\_TOOLS\M9Editor
notepad2.ini .\_TOOLS\Notepad2
xvi32.ini .\_TOOLS\XVI32
The following applications shortcuts will be reset to their initial settings:
CFC_GUI.lnk
M9Editor.lnk
manilaHASH.lnk
Notepad2.lnk
HexEdit32.lnk
The following Manila Mode9 reference index will be reset to their initial settings:
m9editor.names.orig.txt .\_TOOLS
mnf_m9paths.orig.txt .\_TOOLS
m9editor.names.orig.txt .\_TOOLS\M9Editor
Initializing The Kitchen
The following procedure will initiate the Manila Kitchen initialization process. All HTC Manila OEM files found in the .\_SOURCE folder will be copied to the .\_queue folder.
Currently, there are two versions of Manila reference index names; one for pre-Manila 2.5 and one for post-Manila 2.5. It is possible to set the initial Manila reference version during the kitchen initialization procedure. For example, if the .\_SOURCE folder contains a specific version of HTC Manila OEM files, specify the Manila version as part of the parameter. If the kitchen will contain a mix of pre-Manila 2.5 and post-Manila 2.5 files, an incomplete set of Manila files, or an unknown Manila file, specify 0.0 as the kitchen version.
The following procedure will initiate the Manila Kitchen initialization process.
► To initiate a Manila Kitchen reset and specify Manila version 1.3
Place a copy of the original (untouched) HTC Manila OEM files or packages in the .\_SOURCE folder.
Double-click (launch) the Command Prompt shortcut found in the root of the kichen folder.
At the prompt, type the following command: manilatool.cmd -oem:1.3
Press ENTER.
► To initiate a Manila Kitchen reset and specify Manila version 2.5
At the prompt, type the following command: manilatool.cmd -oem:2.5
Press ENTER.
► To initiate a Manila Kitchen reset and specify a mix, small set, or unknown Manila files
At the prompt, type the following command: manilatool.cmd -oem:0.0
Press ENTER.
Decompiling Manila OEM Files
HTC Manila OEM files must be decompiled before changes can be made to them. The decompilation of HTC Manila OEM files occurs on files that are found in the .\_queue folder. Use the Manila Kitchen initialization process to copy the HTC Manila OEM files from the .\_SOURCE folder to the .\_queue folder before attempting to decompile files.
As files are processed, they are removed from the .\_queue folder and placed in a .\Processed folder. Once completed, decompilation mode will be disabled so as to prevent accidental loss of work to decompiled Manila files. The Manila Kitchen must be initialized in order to enable decompilation mode.
Decompilation
The decompilation process passes each manila file found in the .\_queue folder to the LUA processor (luatool.exe) tool. During the decompilation process, the LUA processor (luatool.exe) tool validates the decompilation and produces additional output files if the decompilation was not successful.
Additionally, the decompilation process will extract a copy of embedded Manila (Mode 9) scripts, initiate the LUA processor (luatool.exe) tool to decompile the extracted script, and validate that the decompilation occurred successfully.
Upon completion, a decompiled version of the Manila file will be found in one of the following folders - which are determined by the results of the decompilation process:
Completed: .\Workspace\_lua (successfully decompiled)
Incomplete: .\Workspace\_lua (partially decompiled)
Error: .\Workspace\_lua (failed decompilation)
InternetPortal: .\Workspace (XML files)
Manilapages: .\Workspace (XML files)
Mode9: .\Workspace (Mode9 files)
PNG: .\Workspace (Image files)
QTC: .\Workspace (Image files; in Graphics folder)
SQLite: .\Workspace (Database files)
TTF: .\Workspace (True Type Font files; in Graphics folder)
XML: .\Workspace (XML files)
Unknown: .\Workspace (Unknown files)
The following procedure will initiate the Manila Kitchen decompilation process.
► To initiate HTC Manila OEM file decompilation
Ensure desired HTC Manila OEM files are present in the .\_queue folder once the kitchen initialization has completed.
Double-click (launch) the Command Prompt shortcut found in the root of the kichen folder.
At the prompt, type the following command: manilatool.cmd -dec
Press ENTER.
Working with Manila Files
Once the HTC Manila OEM files have been decompiled, changes may be made as required. The Manila Kitchen provides a single work area for Manila file editing; this is:
Workspace
Workspace
This workspace contains all of the decompiled Mode9 files, graphics, and other Non-LUA files that are typically manipulated using the M9Editor, Notepad2, and/or CFC_GUI.
Workspace\_lua
This area of the workspace contains all of the decompiled LUA files. Use a standard UTF-8 compatible editor such as Notepad2 to make changes to the files.
LUA Editing
Files that have failed to decompile during the initial decompilation process typically need to be manually edited so as to address the cause of the failure. Editable *.lua Manila files reside in the .\Workspace\_lua folder and may be change with a text editor such as Notepad2.
Once the Manila file has been fixed, initiate the the LUA Comparison (luatool.exe) tool to validate that the changes made to the file are correct.
The LUA Comparison process will compare the changes made to the decompiled Manila file against the original Manila file. If successful, the Manila file will be moved from the .\Incomplete or .\Error folder to the
.\Completed folder.
The following procedure will initiate the comparison process.
► To initiate Manila file comparison
At the prompt, type the following command: manilatool.cmd -cmp
Press ENTER.
Mode9 Editing (Mode 9, Graphics, PNG)
M9Editor is a tool for editing Manila files; nearly all aspects of a Mode9 LUA file can be viewed, changed, and saved. Mode 9 LUA Manila files are stored in the .\Workspace\Mode 9 folder and are changed with from within the M9Editor.
Additionally, the M9Editor tool provides graphics import/export capabilities as well as Chainfire File Compression (CFC) support. Lastly, the M9Editor tool includes a folder (directory) viewer which displays information specific to Manila files.
Although it is possible to decompile Manila files with the M9Editor tool, the current version (3.0.03) utilizes a deprecated version of the LUA Decompiler (luadec.exe) which is not supported by this Manila Kitchen.
► To initiate the M9Editor tool
Navigate to the Manila Kitchen folder.
Double-click the M9Editor shortcut.
General Editing (InternetPortal, Manilapages, QTC, PNG, SQLite, TTF, XML)
Some of the Manila files found in the .\Workspace folder may also be changed by other tools such as Notepad2 and CFC_GUI.
► To use an other tool to edit certain Manila files
Launch the desired tool (ex: Notepad2, CFC_GUI, etc.)
Select the Open file option from within the tool.
Navigate to the .\Workspace folder.
Select the Manila file to change.
Updating 'm9editor.names.txt'
From time to time, HTC adds new Manila files or updates the names of existing ones. To update the M9Editor Reference Index file, initiate the Manila name scan process using the MANILATOOL command script. The Manila Name Finder (mnf.exe) tool will iterate through the HTC Manila OEM files found in the .\_SOURCE folder and obtain the new names.
Updating the m9editor.names.txt reference index file ensures that the M9Editor tool is able to display the friendly Manila file name making it easier to work with Manila files - instead of the compiled Hash name used by HTC. Currently, there are two versions of Manila reference index names; one for pre-Manila 2.5 and one for post-Manila 2.5.
The Manila name scan process supports the following Manila versions: 2.5, 2.2, 2.1, 2.0, 1.3, 1.2, 1.1, and 1.0. Specifying a Manila version other than 2.5 will initiate the Manila Name Finder (mnf.exe) tool in pre-Manila 2.5 scan mode. The MANILATOOL command script determines which scan mode to use by first querying the 'kitchen-ver' file - if the value is 0.0; the mode specified in the command parameter is utilized.
Use one of the following procedures to initiate the appropriate M9Editor reference index update process.
► To initiate a pre-Manila 2.5 reference index update
At the prompt, type the following command: manilatool.cmd -mnf:2.1
Press ENTER.
► To initiate a Manila 2.5 reference index update
At the prompt, type the following command: manilatool.cmd -mnf:2.5
Press ENTER.
Updating the Repository
Updating the .\Repository folder ensures that a recent backup copy is available in the event that a change to *.lua must be reversed. During the repository update, decompiled *.lua files found in the .\Workspace\_lua folder are copied to the .\Repository\_lua and a hierarchy (by unhashed names) is created in the .\Repository\windows folder.
The .\Repository\windows folder hierachy displays shortcut links to the Manila files that correspond to a Manila feature. For example, navigate the hierarchy and double-click the shortcut link that corresponds to the file that requires modification - the appropriate tool will be launched.
The following procedure will initiate the repository update process.
► To initiate a repository update
At the prompt, type the following command: manilatool.cmd -lib
Press ENTER.
Frequently Asked Questions (FAQ)
What types of files can be placed in the '.\_SOURCE' folder?
Any type of file may be placed in the .\_SOURCE folder. The MANILATOOL command script will only copy files that contain the word "manila" in them to the .\_queue folder.
I accidentally specified the wrong version of Manila files during the kitchen initialization; is there a way to change it without initializing the kitchen again?
The default Manila file version is stored in the kitchen-ver file. Use a standard text editor such as Notepad2 to edit file. The file must contain a three characters equivalent to the version - valid versions are: 2.5, 2.2, 2.1, 2.0, 1.3, 1.2, 1.1, 1.0, and 0.0 (Unknown/Custom)
What is the '._BACKUP' folder used for?
Before most operations occur, a backup of files that will be changed or removed from/in a given folder is performed. This ensures that previous functional files remain available in the event that a "rollback" is required.
What is the difference between the "Incomplete", "Error", and "Completed" (successful) LUA folders?
Manila files that are successfully decompiled are will appear in the "completed" folder - 100% comparison result. Manila files that are decompiled but do not pass the comparison validation (less than 100%) are placed in the "incomplete" folder. Manila files that report a 0% decompilation result are placed in the "error" folder.
What is the difference between the files in the '.\Release\_lua', '.\Release\_error', '.\Release\CAB', and '.\Release\DEVICE' folders?
The .\Release_error folder will only appear when unexpected errors were encountered during Manila LUA file recompilation. The folder will contain logs detailing the problem that was encountered with a Manila LUA file. The folder is removed and recreated at each recompilation.
The .\Release\_lua folder contains the recompiled versions of the Manila LUA files that are found in the .\Workspace\_lua folder. These are the files can be distributed directly to a device or via CAB file.
The .\Release\CAB folder contains the recompiled versions of the files found in the .\Release\_lua folder. These are the files can be installed via ActiveSync or a Storage Card.
The .\Release\DEVICE folder contains the recompiled versions of the files found in the .\Release\_lua folder. These are the files can be installed via ActiveSync or a Storage Card.
What is the difference between the files in the '.\Processed', '.\Workspace', and '.\Repository' folders?
The .\Processed folder contains the HTC OEM Manila files that were copied to the .\_queue during the Manila Kitchen initialization process. During the decompilation process, the files are moved from the .\_queue folder to the appropriate folder - which is determined by the LUA Comparison tool results.
The .\Workspace\_lua folder contains decompiled versions of the HTC OEM Manila files - these are the files that are typically modified and recompiled. The remainder of the folders in the .\Workspace folder contain various non-LUA files.
The .\Repository\_lua folder contains a duplicate copy of the decompiled Manila files. The repository serves a form of backup.
What are the extra .TXT files that appear in the partially decompiled LUA script folders?
The *.error.txt file contains the STDOUT error that was generated by the LUA Decompiler. When this file appears, there is a logical error in there script, which makes it impossible to run the script. The file will usually contain the approximate (line number) location where the problem was encountered.
The *.log.txt file contains the compared disassembly output from the orignal Manila file and the decompiled LUA version - which appears in the folder. When this file appears, it is an indication that the script will function but it does not match the original Manila file.
The *.dis.txt file contains the raw (full) disassembly of the original Manila file and may be used to resolve the decompilation issue.
The *.status.txt file contains the decompilation result - 0% to 100%.
Visit the following website to obtain more information in troubleshooting a failed decompilation:
http://winmo.sztupy.hu
Is the disassembly output always correct?
Yes. Even though the some of the events may be not be displayed in the disassembly output, the correct sequence of events will be provided.
The M9Editor tool provides a built-in decompiler, why not just use that?
The internal LUA Decompiler (luadec.exe) provided with the M9Editor tool is an earlier version. This internal version does not provide the extra files required to resolved decompilation errors.
The M9Editor should be used when editing Mode9 files and attach extracted embedded LUA script back to their original Mode9 files.
There are strange errors such as '.\documents and settings\<profilename>\eee' errors, what is the solution? (Only in older versions)
One of the Manila Kitchen tools is in an unknown state or temporary files are locked and cannot be freed. Note the path location of the .\eee folder, restart the workstation, and manually delete the folder, sub-folders, and files.
An error message is displayed when attempting to delete a file or folder, what is the solution?
A running process is preventing the file or folder from being removed. Restart the workstation and manually delete the file or folder.
Can Error Reporting be disabled so as to not have to click the Close button every time the LUA Decompiler and/or LUA Comparison tool generate an application error?(Only in older versions)
As previously noted, some Manila files cause the LUA Decompiler (luadec.exe) and/or LUA Comparison (compare.exe) tool to generate an application error. Visit the following website(s) for more information on how to suppress the Error Reporting message box.
Windows XP
http://www.windowsnetworking.com/articles_tutorials/Disable-Error-Reporting-Windows-XP-Server-2003.html
Windows Vista/Seven
http://thehiddenguide.com/how-to-disable-error-reporting-in-windows-vista
Recompiling Manila OEM Files
Once editing is complete, the Manila files must be recompiled before they may be used on a mobile device.
Should an error be encountered during recompilation, the MANILATOOL command script will create a Manila LUA error log file in the .\Release\_error folder and continue recompiling Manila LUA files. In the event of a "hard" recompilation error, the contents of .\Release\_lua folder will be removed and the previous successful recompilation will be restored.
Recompilation
The recompilation process occurs on files that found in the .\Workspace\_lua\Completed folder. Once completed, the files are ready for deployment to a mobile device that is directly connected to a workstation or via an installer .CAB file.
The following procedure will initiate the recompilation process.
► To initiate Manila file recompilation
At the prompt, type the following command: manilatool.cmd -rec
Press ENTER.
Testing the Manila OEM File Changes
The Manila Kitchen environment provides additional tools for testing a final work product. Manila files can be distributed to mobile devices using one of the following methods:
Installed .CAB
Direct to Device
The quickest method of distributing recompiled Manila files is via an installer .CAB file. The direct to device method of distributing recompiled Manila files is via ActiveSync over a USB/Serial connection - this method is
quickest during development activities.
Installer .CAB Method
The MANILATOOL command script provides a parameter to automatically create a redistributable .CAB file. The installer .CAB can be copied to the mobile device and launched using the device File Explorer. Alternatively, the installer .CAB file may be installed via ActiveSync. Should an installer .CAB fail to install on a device, it is likely that policy restrictions are in effect.
The following procedure will initiate the .CAB creation process.
► To initiate the CAB Wizard tool
At the prompt, type the following command: manilatool.cmd -cab
Press ENTER.
Direct to Device Method
The MANILATOOL command script includes a parameter to automatically stop the mobile device Manila executable (manila.exe), transfer a copy of the Manila files, and restart the mobile device Manila executable.
The copy operation may fail on a device where policy restrictions are in effect. Additionally, the Manila executable may not restart in such cases.
The following procedure will initiate the comparison process.
► To initiate RAPI Copy tool
At the prompt, type the following command: manilatool.cmd -dep
Press ENTER.
Reference Web Links
Manila 3D Kitchen
http://winmo.sztupy.hu/manilakitchen.html
http://forum.xda-developers.com/showthread.php?t=487331
Manila Tutorial
http://forum.xda-developers.com/showthread.php?t=399212
LUA Decompiler
http://forum.xda-developers.com/showthread.php?t=568281
LUA 5.1 Decompiler
http://winmo.sztupy.hu/luadec.html
http://forum.xda-developers.com/member.php?u=1433290
http://forum.xda-developers.com/showthread.php?t=479910
LUA Decompiling Tutorial
http://winmo.sztupy.hu
http://winmo.sztupy.hu/manilakitchen/rhodium2_manila_wvga_src.zip
TF3D Manila Mode9 Editor
http://forum.xda-developers.com/showthread.php?t=464984
MNF - Manila Name Finder
http://forum.xda-developers.com/showthread.php?t=546820
CFC - The Manila/TF3D Image Editor
http://forum.xda-developers.com/showthread.php?t=437777
TouchFLO/Manila/SenseUI
http://forum.xda-developers.com/group.php?groupid=131
Co0kie's Beta Testing and Development
http://forum.xda-developers.com/group.php?groupid=192
Max Sense Testers
http://forum.xda-developers.com/group.php?groupid=185
GT7 Sense 2.1 Theme Beta Testers
http://forum.xda-developers.com/group.php?groupid=202
Leaked Full EXT/Manila 2.5 Packages-latest official sprint tp2
http://forum.xda-developers.com/showthread.php?t=642817
Enabling the Logging Debugger
Although a significant of amount effort has gone into the development of the Manila Kitchen, it is possible that a script logic error may occur. To assist in troubleshooting such errors, the MANILATOOL command script includes a comprehensive logging debugger.
The logging debugger can provide basic, expanded, or verbose (full) logging. To enable the logging debugger, append -DEB:[1-5] to the end of the MANILATOOL command script. For example, to set the logging debugger level to 1, type the following command at the Command Prompt:
manilatool.cmd -hlp -deb:1
Processing activities are displayed in the Command Prompt window and are recorded in the ManilaTool_Log_#.##.##.txt file. The log file will include the version of MANILATOOL command script and the command parameters that was requested. The following logging debugger levels are available:
Level 0: Default. The MANILATOOL command script typically operates at this level.
Level 1: Provides slightly more details on the script processing activities. Specifiy this logging level to view the results of actions performed on a Manila file.
Level 2: Provides comprehensive details on the processing activities performed on a Manila file. Specify this logging level when trying to determine why a Manila file may have failed to be processed. Choosing this logging level on a large number of Manila files will increase the time it takes to complete Manila file processing.
Level 3: Developer verbose level 1. Specify this logging level to view the script routine names that are invoked during operation as well as the variables that are dynamically set. Choosing this logging level on a large number of Manila files will greatly increase the time it takes to complete Manila file processing.
Level 4: Developer verbose level 2. Specify this logging level to view the script routine names that are invoked during operation as well as the dynamic and common variables that are set. Choosing this logging level on a large number of Manila files will drastically increase the time it takes to complete Manila file processing.
Level 5: Developer verbose level 3. Specify this logging level to view the script routine names that are invoked during operation, display all script variables. Additionally, this logging level will create a .\_debug folder which will contain a copy of the files that are normally removed during a script routine operation. Choosing this logging level on a large number of Manila files will significantly increase the time it takes to complete Manila file processing.
The following log files are generated during MANILATOOL command script processing:
ManilaTool_Log_#.##.##.txt
ManilaTool_FileCopyLog_#.##.##.txt
ManilaTool_LUAFilesLog_#.##.##.txt
ManilaTool_MNFLog_#.##.##.txt
ManilaTool_Mode9Log_#.##.##.txt
ManilaTool_NonLUAFilesLog_#.##.##.txt
Very nice work as always 12
Very nice work on this, I am around now, So I can start to help out more, haha
Forgive me for being an utter dumbass.
I've got your kitchen and was trying it out. However, what files do I need to drop into folder 01_manila? I've tried dropping just the manila file into it, the extracted *.luac scripts but I keep getting errors saying no files found.
Kindly point me in the right direction?
Thanks.
KF
kinnyfaifai said:
Forgive me for being an utter dumbass.
I've got your kitchen and was trying it out. However, what files do I need to drop into folder 01_manila? I've tried dropping just the manila file into it, the extracted *.luac scripts but I keep getting errors saying no files found.
Kindly point me in the right direction?
Thanks.
KF
Click to expand...
Click to collapse
Yes sure, That's the reason I included the m9editor as well, for it's abilty todifferentiate between different kinds of manila files. Copy only the Lua files (unchanged) to the 01_manila folder. There is a checkbox in the m9editor and if you will check it, it allows copying of multiple files. After that you are good to go and you can run the 01_Decompile.bat, 12
12aon said:
That's whats up!
@MRFERRARI23
What I think that going wrong with you is that your files are not called xxxxxxxx_manila make sure they are name that way or the cab creation script won't recognize them.
One other thing I noticed is that the path to the kitchen contains names with spaces like "mr vizziato". This can throw the kitchen off. To test this put the kitchen directly in you C:\ folder or in any rate not in the folder with spaces in in their names, and check back with me, 12
Click to expand...
Click to collapse
the 2 files I have in there now just look like this 18c01b6d_manila and 72ac571f_manila so I think they are in there right
as for running the program im doing so right from my desktop screen cause when I try to run it from c drive an I go to name the cab It doesnt give me that black pop screen it goes to this white page with info that doesnt allow me to do anything!
EDIT: how do I eliminate that space between mr vizziato?? I cleared the space by going to the start menu an going to my name but idk still nothing
12aon, Please, help me. I can`t decompilled file from manila 2.1 "53cc1e4f_manila",
if you can do this - help me please.
View attachment 53cc1e4f_manila.zip
Hey man, I am currently on a trip in the USA and I do not have the means to help you decompile at the moment. If you want it decompiled right away you can check out stupy's threads and tutorials (I have links in my first post) but I warn you this is kinda hard, good luck, 12
12aon said:
Hey man, I am currently on a trip in the USA and I do not have the means to help you decompile at the moment. If you want it decompiled right away you can check out stupy's threads and tutorials (I have links in my first post) but I warn you this is kinda hard, good luck, 12
Click to expand...
Click to collapse
Ok, I will wait )))), and when you will be able to do this?
Thanks, really useful that you did!
Man, this is seriously awesome
Extremely helpful tool, thank you very much
Java SDK can be installed on windows 8,
Unfortunately The Java.exe is not automatically added to path's, this is how to do it
How to get it work?!
Go to Control panel --> System and security --> System --> Advanced settings --> Environment Variables --> Path
And add your java to the path
;C:\Program Files\Java\jdk1.7.0\bin
(As Windows 8 DP, does not add java to its path you need to add it manually)
And now that should make you able to use your Java apps to work properly
Setting up a development environment is really not that hard. I am using Windows 7 with a 64-bit processor, although the procedure is not terribly different across sytems, just make sure you choose the right download for your architecture.
The entire process can be broken down into four major steps. Let's get started.
1) Get Java
You can get the Java Standard Edition Software Development Kit through this link: http://www.oracle.com/technetwork/java/javase/downloads/
At the time of writing this post, the latest version available is 7u3. Click the download button under where it says 'JDK' and you will be taken to a list of installers. Make sure you get an installer from the group who's header is "Java SE Development Kit 7u3"; the other one is just sample code.
Make doubly sure that you choose the right architecture (x86 vs x64) as the wrong one will give you an error message and you will have to go uninstall everything and start over.
After the download has finished, run the installer.
Note: It may give the option to install JavaFX and the Ask Toolbar. While they won't hinder your development in any way, there won't be any need for them within the scope of this guide so you might find it worthwhile to not install them.
2) Get Eclipse
Once you've installed the Java DK, go get a copy of the Eclipse IDE here:
http://www.eclipse.org/downloads/ You can see that the IDE comes in many different flavors. Eclipse Classic will work fine.
Extract the folder and place the 'eclipse' subdirectory in a safe place. Try running eclipse by traversing this directory and running eclipse.exe. If you see the Eclipse splashscreen, everything is OK. If you see an error message then something may have gone wrong when you installed Java. If it complains about missing or corrupted DLLs, you probably have installed the wrong version of Java for your system. If it complains about not being able to find Java (for example, stuff about javaw.exe) you may need to manually add Java's location to your Path environmental variable. There is lots of documentation on how to do throughout the internet and most can probably explain it better than me. (^; Another method to fix the problem is to go find out where all the Java binaries are (Normally something like "C:\Program Files\Java\jdk~~~~~~~\bin"), copy the directory path, and add
Code:
-vm
<FOLDERPATH>\javaw.exe
to the end of the 'eclipse.ini' file that resides alongside 'eclipse.exe'
Note: You may notice that the version of Eclipse you downloaded isn't installed, but instead just runs out of a folder. You may want to consider creating a binaries folder in your home directory to keep all of software of this style in one place.
3) Get Android Plugins for Eclipse
Now that you have Eclipse up an running, you need to set it up for Android development. Start Eclipse up and click the 'Help' menu. Click 'Install New Software...'
In the window that comes up, you should see a combo box labelled 'Work with:' . Paste in:
Code:
https://dl-ssl.google.com/android/eclipse/
then hit enter.
Check 'Developer Tools' and click next. Let Eclipse do it's thing then click 'Finish'.
4) Get the API
Now that you have Eclipse's Android integration plugins, you need to grab an Android API. In Eclipse, under the 'Window' menu, click 'Android SDK Manager.'
The window that opens will automatically go to the internet and grab all the API packages available to download. The Nook Touch currently runs Android 2.1, so find that package set and check it. Click the 'Install <x> Packages' button and let it run.
You are finished! Now to create an Android program. In Eclipse's main menu, navigate File > New > Other... and select 'Android Project' from the 'Android' folder in the window. If you want to modify the source of an existing program, choose the appropriate radio button and browse to the parent directory of the 'src' folder. After you have created your project, you are going to want to make sure it has the right dependencies. On the lefthand pane ('Package Explorer') right click your project folder (The top one) and hit 'Properties.' Under the 'Android' screen, make sure 'Android 2.1' is checked and not another version. Click OK and you should be set.
To generate an APK, under 'File' click 'Export.' In 'Android' click 'Export Android Application.' Choose the right project, click next. If this is the first time you've exported an Android application, you will need to create a keystore. Select the appropriate radio button, navigate to a safe location, and give it your keystore a password. Click 'next' and add all the information you see fit. You will be able to reuse this keystore in future programs. After you've created a keystore, proceed through the export wizard, give your APK a filename, click finish and the file will be generated.
The rest is up to you. (^:
Thanks klausef!
For a non-dev (OK, I used to program short apps in BASIC on my commodore 64 when I was 8 years old), how much can I hope to achieve by setting this up? Do I need to understand a programming language? If so, which? Do you have any suggestions for a noob like me?
Haha I am no Java expert myself but I've been hacking around with the NoRefresh thing and other apps people have coded. You can do more than you think just by rummaging around online documentation.
I've edited my imageres.dll file and have it all nice nice, but when I try to paste it into system 32 (tried safe mode) It will not let me overwrite that file.
I've tried Safe mode, Taking Control, Granting full control permissions of file, system32 folder, windows folder, and C folder (both taking control, and granting full permissions to: User (me) and Administrator (also me) Yes, i'm on windows 7 64bit but there is no Win 7 Forum here, however the imageres.dll is mentioned here, and I'm assuming the file is more-or-less the same. Besides taking ownership in C for Win 8 being bad? I did this to both System32 file and Syswow64 file- its now apparent to me that both are in USE and thats why I can't overwrite/modify them =/ So how do I use my modded one? (startup sound etc.)
Following this:
Code:
How to Take Ownership in Windows 7
1. Locate the file or folder on which you want to take ownership in windows explorer
2. Right click on file or folder and select “Properties” from Context Menu
3. Click on Security tab
4. Click on “Advance”
5. Now click on Owner tab in Advance Security Settings for User windows
6. Click on Edit Button and select user from given Change Owner to list if user or group is not in given list then click on other users or groups. Enter name of user/group and click ok.
7. Now select User/group and click apply and ok. (Check “Replace owner on sub containers and objects” if you have files and folder within selected folder)
8. Click ok when Windows Security Prompt is displayed
9. Now Owner name must have changed.
10. Now click Ok to exist from Properties windows
Once you have taken the ownership of file or folder next part comes is Granting Permissions to that file/folder or object.
How to Grant Permissions in Windows 7
1. Locate the file or folder on which you want to take ownership in windows explorer
2. Right click on file or folder and select “Properties” from Context Menu
3. Click on Edit button in Properties windows Click ok to confirm UAC elevation request.
4. Select user/group from permission windows or click add to add other user or group.
5. Now under Permission section check the rights which you want to grant i.e check “Full Control” under the “Allow” column to assign full access rights control permissions to Administrators group.
6. Click Ok for changes to take effect and click ok final ok to exit from Properties window.
Now you can access files of folder in windows 7 with full permissions and take full control.
also tried cmd prompt
Code:
takeown /F "C:\Windows\System32\imageres.dll"
with Sucess message (probably the same as my right click takeown reg file context thing)
I'm the only person/login/user on my PC. Ugh this is a pain in the butt. No matter what I've googled/tried hasn't worked. I don't have a linux CD/USB boot *(dunno if I do? but don't think I do) to repair/replace file, I may have Win 7 repair on my CD, maybe not.
EDIT: replacing the startup sound in imageres.dll in System32 and SysWOW64 folders still leaves me with default windows startup sound =( Only other file was imageres.dll in winsxs folder- copied, edited the copy, deleted the original with Fileassassin (it was locked, wouldn't edit, blah blah) STILL uses old windows start up sound, WHyyyyyy? theres MUI files with info for imageres.dll but IDK what's left where this sound exists =/ or where its coming from.
there used to be this program made for replacing system files
i don't remember its name, though
http://www.askvg.com/right-click-re...utility-to-replace-system-files-in-windows-7/
No manual methods worked for me, but just came across this and it worked, WOOT!
http://winaero.com/download.php?view.10
Starting with Windows 8, the administrator is no longer the same as root. Microsoft has very, VERY severely limited the administrator's abilities with many aspects of the system. This was done in an ill-conceived notion that they could curb piracy of Windows Store apps (like there's anything worth stealing in there, including games). People have already found a way to pirate Windows Store apps, so it was all just a way to irritate admins and hobbyists in the end.
@dragon_76: Care to explain what you mean by that? For one thing, Administrator on NT has never been "the same" as root (on POSIX); there's always been the local SYSTEM SID (which does some of the things which root would handle, such as being the process that drivers and initial user-space processes run as) and while I'm not entirely sure when the TrustedInstaller SID was added (may have been there since NT3.1 for all I know), it's been a part of Windows since well before Win8. For another thing, unlike root on a POSIX system, Administrator does not (and never has) automatically have access to everything; it's a SID like any other, and ACLs can be put in place to control its access.
Now, two powers that Administrator (and members of the Administrators group) do have are as follows:
1) Take ownership of any securable object (this allows completely overwriting the ACLs).
2) Impersonate any other SID (although some, like TrustedInstaller, are possibly trickier to impersonate in Win8 than they used to be; I need to investigate that).
If what you're complaining about is the fact that \Program Files\WindowsApps\ is now owned by TrustedInstaller and harder to mess with than you'd like, um... sorry? It's not *that* hard to take ownership (or set it back) from TrustedInstaller; you can do it as Admin, in fact.
*Ahem*
Back to *actually* being on-topic, older versions of Windows* had a background service that would undo any modifications to system files by replacing them with un-tampered copies from a backup location. Starting with Vista, such a service supposedly no longer exists, but it would be easy to implement something like it using VSS. So, you should verify that the file isn't being reverted. The simplest check is the modify timestamp and (if it's different on your file than on the stock one) the size. Somewhat more advanced is to use a hash function, such as MD5 or something from the SHA family, on the on-disc file and see whether it matches the version you put there.
Beyond that, you can easily replace system files if you do it while the OS isn't booted. Linux can do it, although that has risks. Better is to use WinPE (the Pre-installation Environment that bootable Windows disks and recovery tools load) and replace it from there.
Incidentally, you can use the command line tool icacls (which may even be present in WinPE by default) to both change ownership and set permissions on files and folders.
* Pre-Vista, which was the first version to switch to using TrustedInstaller to protect system files rather than letting Admin write to them by default. That's the last major change I'm aware of in the powers of the Administrator account on Windows, incidentally...
Current Version: Extraction Tool v5.92, Analytics Tool v5.60, iOS Extraction Tool v0.2, Fit Sync Tool v0.2 (Updated 6/20/2015)
Here is a tool I wrote for Xiaomi Mi band which provides following functionality:
1. Exports data from Xiaomi Mi band on both rooted and non-rooted Android phones and jailbroken (and possibly non-jailbroken) iPhones under Windows and Linux, generating file in CSV format which can be reviewed in Excel.
Bonoboo maintains a script to perform extraction directly on your smartphone (see link in post 3).
2. Extracts detailed (per minute) walking and running information from Mi Band database (thanks to the input provided by mritsch and Osid).
3. Produces files that can be uploaded to fitnesssyncer.com/ to syncronize data with various fitness services: FitBit/HealthVault/RunKeeper/JawBone and soon Google Fit! See 3rd post for details on data mapping for fitnesssyncer.
4. Provides multiple analytics reports to review extracted data. These reports can be shared on cloud drives to be accessed from anywhere (see post 3 for details)
5. Supports localized UI (currently supported languages are English, Russian, French, Spanish and Italian)
6. Support for synchronization data with Google Fit Service - see post #2 for details
How to use:
Preparation steps:
1. If you plan to use multiple packages, unpack them all to same directory.
1a. If you want to extract data from smartphone, download miband_extract package. To extract data from iOS devices, see to "To extract data from iOS devices" step.
1b. If you want to display local charts of your extracted data on your computer, download miband_analytics package.
1c. If you want to synchronize your extracted data with Google Fit, download miband_fit package.
2. Make sure you have USB drivers for your device properly installed and that your device is accessible by ADB when you connect it through USB
3. If you use it on windows, extract package has all binaries included, for Linux/OSX see comments below.
Checking configuration settings:
1. Review SDPath parameter value in run.bat/run.sh. The program will copy files from Mi app location to folder specified in SDPath before pulling them to desktop. In most cases default value (/sdcard) shoud work fine, however if your phone does not have this directory, find the path where your Internal/External SD is mounted and put that path string into SDPath value. Second most common value might be /storage/sdcard0
2. Review config.js and make any changes to your liking (set Goals for sleep hours and daily steps, force override UI language to specific value)
3. If you do not want main report being open every time you run extract, change OpenHTML=Y in run.bat/run.sh to OpenHTML=N
4. If your device is not rooted or have any issues with first (root) method that application uses and prefer to skip straight to the second (backup) method, set ForceBackupMode value to Y in run.bat.
5. If you want to generate Raw data extract in addition to summary data, set ExtractRaw parameter to Y, update Height parameter to your height in centimeters and update Weight parameter to your weight in kilograms. Please note that data extracted from raw blobs may be slightly different from total numbers in summary data.
If you are planning to use ADB over Wifi:
0. I would not be able to provide much support for ADB over Wifi - so many things could go wrong. Check you firewall settings, check TCPIP ports on both ends, but you are basically on your own.
1. If you are planning to use ADB over WiFi, edit run.sh and set up IPAddr value to IP address of your phone, if you use USB cable, leave IPAddr value blank. If you using non-default port, you may need to change TCPPort value.
2.You may also think of a great idea of running syncronization automatically and unattended using ADB over Wifi - at least I liked that idea initially. I gave that idea more thought and as of now I strongly recommend not to do it - having ADB running over Wifi is a security risk, if you have to use it over Wifi, enable it manually, run the sync and disable ADB over Wifi right away.
For Linux/OSX users:
1. You would need to to manually install android-sdk for (adb binary), sqlite3 version 3.8.3 or higher and openssl with zlib support (version 1.0.2 or higher with zlib support is recommended) to uncompress zlib data. Please note that versions of sqlite3 and openssl that are preinstalled on your machine might be too old to be used with this package, so you might need to obtain newer versions. For example, I was told that OSX has sqlite3 version 3.7.13 preinstalled and it is incompatible with some of functions used in script, so you would need version 3.8.3 at least. If you cannot get openssl with zlib support, you can use similar functionality from python instead - you will need to comment out a line with openssl zlib in run.sh and uncomment next line instead which invokes python script.
2. You'll need to grant execute permissions to run.sh by using chmod +x run.sh and you will need to execute run.sh instead of run.bat in steps listed below. You'll also need to make configuration changes in run.sh instead of run.bat
3. Check that your sqlite3 is properly configured for your time zone. Run following command and see if it returns correct timestamp:
bin\sqlite3 dbfile "select datetime('now','localtime');"
Running application:
1. Connect phone through USB and make sure USB debugging setting is enabled on your phone. Unlock phone screen.
2. Execute run.bat - if your phone is rooted, the data would be pulled automatically. If your phone is not rooted you would see backup screen and you need to press "Back up my data" button in the bottom left corner.
3. Data from your mi band will be saved to extract.csv file and extract.js. After extraction is complete, if OpenHTML is set to Y, mi_data.html will be opened automatically to show charts for your Mi usage.
4. HTML reports are using Google Charts framework and Google TOS does not allow storing their scripts offline along with the application, therefore you will need to have working internet access for reports to work. Your data is not being sent to Google, the internet connection is only used to download latest version of Google Charts javascripts.
Troubleshooting connectivity issues:
1. If you phone is rooted:
Review that your phone has /sdcard directory and it is writable. After you run extract, check if you have origin_db file in that directory. If you do not, ADB may have issues writing files to that directory. Check configuration settings section, item 1
2. If your phone is not rooted or if you use ForceBackupMode=Y:
Check if you get full backup screen, which looks like this:
{
"lightbox_close": "Close",
"lightbox_next": "Next",
"lightbox_previous": "Previous",
"lightbox_error": "The requested content cannot be loaded. Please try again later.",
"lightbox_start_slideshow": "Start slideshow",
"lightbox_stop_slideshow": "Stop slideshow",
"lightbox_full_screen": "Full screen",
"lightbox_thumbnails": "Thumbnails",
"lightbox_download": "Download",
"lightbox_share": "Share",
"lightbox_zoom": "Zoom",
"lightbox_new_window": "New window",
"lightbox_toggle_sidebar": "Toggle sidebar"
}
If you do not get this screen, then Android backup functionality on your device is broken. If your phone is rooted, consider setting ForceBackupMode=N and using rooted method; if your phone is not rooted, try to run Helium Backup https://play.google.com/store/apps/details?id=com.koushikdutta.backup and see if it can backup MI app to sdcard.
To extract data from iOS devices:
Thanks to joshmosh for detailed instructions.
This is reported to work on jailbroken iPhone, but may work on regular iPhones as well - if you try it on non-jailbroken iPhone please let me know if it worked for you.
1. Get a copy of itools. Google it to find a download location (cnet might have it)
2. Plug in your iPhone to your PC
3. Open itools and click on Applications in top row
4. On the left you will find name of your iPhone and below "Applications". Click on Applications
5. Locate your application Mi Fit and click it. On the right you will find a button labelled "browse". Click it.
6. A window will open. Doubleclick on "Documents" then click on user.sqlite, then on "Export" above.
7. A window will open asking you where to export this file. Export it to a db subdirectory of miband extractor main directory and click OK. You should have a file db\user.sqlite now.
8. Once file is extracted to db\ directory, run runIOS.bat from main miband extractor directory. It should generate extract.js, extract.csv and minmaxtime.csv files and will open local charts in your browser if you have analytics package installed.
Screenshots:
Report descriptions:
mi_data.html:
This report shows overview of the data being extracted. It consists of 3 elements:
a. The chart on the left shows sleep data over time. Total bar height represents overall sleep time and colored segments represent light and deep sleep parts of it.
b. The chart on the right shows daily activity metrics over time. It shows total distance, number of steps and calories burned.
c. The chart on the bottom shows all data inside text table. You can sort the data by any column by clicking on the title of that column. If you select any specific row in the table, same day will be hightlighted in two other charts.
Both graphic charts also show target goals in green line, so you could see how well your results are matching target values. You can dynamically change target values by pressing + and - buttons. The table below has arrows next to Total Sleep and Total Steps columns which would be green for days when goal was met and red for days when goal was not achieved.
mi_data_byDoW.html:
This report allows you to see the scatter of your activities grouped by the day of the week.
This would be a good chart to see if you are performing better on any specific day of the week.
There are 4 charts on that report:
a. The left chart shows various metrics in scatter chart. You can select a specific metric from the dropdown located above the chart. You can also see a daily goal trendline in green and an approximation line in black. If the approximation line is not flat and you have statistics over many weeks, this might mean that your performance is consistently different depending on the day of the week.
b. The right chart shows line chart data for the metric selected in the left chart. It has time zoom controls at the top to only show the data over last day, last 5 days, 1 week, 1 month, 3 months, 6 months and 1 year. You can also select time interval manually by dragging time sliders at the bottom of the chart.
c. The chart at the bottom is a calendar view for your activity-related metrics. It shows a calendar and each days recorded activity is colored based on whether the value for that activity is above or below the goal. The days that have metric value below daily goal will be colored red, the days with metric value above the goal will be colored blue. The higher the deviation from the goal, the more intense color would be. In other words, the days where results are significantly above the goal would be colored in saturated blue and the days where results are significantly below the goal would be colored in saturated red.
This chart could help searching for more complicated patterns in your performance, patterns that are not just days of the week.
You can also change goal values dynamically by using + and - buttons at the top and see green trendline moving on the left chart and days changing colors on the bottom chart.
d. The last chart shows a timeline of your sleep on a daily level. This chart will only be visible if you select "Sleep Timeline" metric from the dropbox. This chart shows the time you went to sleep and the time you woke up. It shows this information for each day the history data is available and weekends are colored in a different color. This chart could help you to see how consistent your sleep/wake hours across different days and also if you have same sleeping patterns on weekends.
Google Fit Syncronization:
Limitations:
1. Daily summary statistics is synced, so charts by Week and Month will show correct data; chart by day will show correct totals, but not correct hourly breakdown.
Prerequisites:
1. You need to have at least version 5.71 of miband_extract package.
2. You will need to have Internet access and your browser should be configured to enable pop-ups at least from a specific domain below.
3. You need to open following URL: Get Fit Token and allow it to access your Google Fit data. Latest version of Chrome worked for me, while IE 11 had issues with popups.
Installation:
1. Download latest version of meband_fit package and unpack it to the directory where you have miband_extract v5.71 or above.
2. If you upgrade from previous version of fit sync package, you will need to clean up old data populated by old version. To do so run following command:
fit clean
Running Sync:
1. Synchronize your data from smartphone to desktop.
2. Obtain a new OATH key for syncing data with Fit - open Get Fit Token and copy resulting string to your clipboard.
3. Run fit.bat and when asked for new token paste the string from previous step and press enter. If fit.bat did not ask you for token and failed with Authentication error, just run it again.
Troubleshooting:
1. The process is logging into fit\*.log files. If you have errors, review these logs first.
2. The token from step 2 is valid for 1 hour - you will need to refresh the token after that. After 1 hour subsequent fit syncs will start failing until you obtain a new token.
3. Current token is cached in GKey environment variable. Once batch determines that old token is expired, it would clean the variable and will ask you for a new key on a subsequent run.
Related links:
Latest version of Bonoboo port for smartphone extraction.
Known Issues:
1. Vertical axis is not always in sync for two charts in mi_data_byDoW.html
2. Timeline axis is only shown at the bottom of timeline
Version History:
Code:
[B]v5.92 06/20/2015[/B]
Added support for Mi Scale - extracting weight history into weight.csv
[B]v5.90 03/14/2015[/B]
Walking and Running statistics with minute granularity are now extracted from Raw data thanks to the input provided by mritsch and Osid.
[B]v5.80 03/07/2015[/B]
Google Fit sync package 0.3:
Fixed error in 1324438.json file which prevented Fit Sync for some users.
Added FitTools.bat which allows pulling various information from Google Fit account.
[B]v5.80 03/01/2015[/B]
Extraction package:
Added experimental support for raw data extraction. To enable raw data extraction, set ExtractRaw=Y in run.bat
Analytics package:
German localization added to analytics package (thanks to joshmosh)
default app_locale.js file is added to analytics package to support data extracted from iOS devices.
Data Extraction Tool v0.1 for iOS is added along with instructions on how to manually extract Mi Band data from iPhone
[B]v5.71 02/16/2015[/B]
Extraction package:
Extra fields added to extract.csv to support for Google Fit sync v0.2.
extract.json is not created anymore, this data is now calculated on the fly from extract.csv
Google Fit sync package 0.2:
Added sync for activity durations (walk duration, run duration, sleep duration)
Added sync for distance walked. Since distance metric is considered location, Token application will request new permission:
https://www.googleapis.com/auth/fitness.location.write
Fixed activity timestamps - Google Fit expects time to be in UTC time zone.
[B]v5.70 02/16/2015[/B]
Extraction package:
Initial support for Google Fit sync.
Google Fit package 0.1:
Initial release
[B]v5.50 02/05/2015[/B]
Extraction package:
Added new columns to CSV file for better FitnessSyncer compatibility.
Added support for time correction. See db\offset.sql for sample entries.
Updated run.sh from [URL="http://forum.xda-developers.com/member.php?u=6091617"]jlaunay[/URL] to support new run.bat parameters and localization.
Analytics package:
Changed mi_data_byDoW.html to use AnnotationChart for right graph - support for time interval zooming.
Added new metric to mi_data_byDoW.html called "SleepTimeline" to show Sleep Timeline chart.
Various small fixes.
[B]v5.35 02/03/2015[/B]
Extraction package:
Fixed logging issue - adb would sometimes lock the log files and subsequent extraction invocations would fail completely on locked log files.
Added support for ADB over Wifi - see instructions in the first post.
Analytics package:
Fixed sorting order on main table
Second fix for incorrect sleep/wake hours calculation
Updated colors across all reports to be consistent
Converted total/deep/light sleep from minutes to hours and minutes in the main table. Show percentages of deep/light sleep compared to total sleep.
Add fall asleep and wake up times as selecrable metrics to mi_data_byDoW.html
mi_data_byDoW.html absorbed all functionality from mi_data_Sleep.html - the sleep calendar functionality and sleep timeline is available when you select Total Sleep metric. You will need to scroll down to see sleep timeline chart.
[B]v5.3 02/02/2015[/B]
Extraction package:
Add new column with number of walk time + run time in seconds (as per request)
Added new parameter ForceBackupMode to skip attempt at rooted extraction and go straight for backup mode
[B]v5.2 02/1/2015[/B]
Splitted program into data extract package and analytics packages. Data extract should be pretty stable now and reports could be updated independently.
Spanish translation (thanks to Viriatox)
[B]v5.1 01/31/2015 [/B]
Fix for incorrect sleep/wake hours calculation
[B]v5.0 01/31/2015[/B]
Localization setting is pulled from Mi database - no need to set it manually anymore (in most cases)
Two completely new reports - Sleep Report and Statistics by day of the week.
Fixes to unix shell script encoding.
[B]v4.0 01/29/2015[/B]
Month number calculation fixed.
Removed extra rows with zeroes that we incorrectly extracted
Localization moved to separate file, all other components are unified for all languages
Scripts for rooted and non-rooted phones combined into one.
Daily goals and data grouping added on main report.
Unix shell script for data extraction (thanks to jlaunay)
French translation (thanks to jlaunay)
[B]v3.0 01/26/2015[/B]
New script to support data extraction for non-rooted phones
Reverse sort order in table part of report, so newest dates are on top.
Week number and day of the week added to report.
[B]v2.0 01/25/2015 [/B]
All needed Windows binaries included in the package.
Batch file to automatically run all binaries to extract the data.
First report showing overall statistics.
[B]v1 01/24/2015[/B]
Initial release.
Syncronizing with fitnesssyncer.com
FitnessSyncer FAQ: https://www.fitnesssyncer.com/frequently-asked-questions
How to set up import and export.
1. Open account at fitnesssyncer.com/
2. On Dashboard, go to Sources and Destinations and select Add New Source
3. Select Type: Activity, source Amazon S3 (CSB), Dropbox (CSV) or OneDrive (CSV) depending on where your files are stored. Enter path and file name in Path field. Select highlighted columns mappings:
Activity:
Include Date (US) - Column 18
Include Activity - Column 19
Include Distance in Meters - Column 8
Include Duration in Seconds - Column 17
Include Calories Burned - Column 10
Include Total Steps - Column 9
4. Click on Save and authenticate into your cloud storage account.
5. Back on Sources and Destinations account, click on Sync Now link next to your newly added connection to confirm that it loads without errors. Click on Dashboard button and confirm that you see some data in charts there.
6. Go back to Sources and Destinations and select Add New Source again
7. This time, select Type: Sleep, same Source entry, same Path value and set up highlighted column mappings:
Sleep:
Include Date (US) - Column 18
Include Bed Time Hour - Column 20
Include Bed Time minute - Column 21
Include Awake Hour - Column 22
Include Awake Minute - Column 23
Include Sleeping Minutes - Column 2
8. Click Save, go to Sources and Destinations and click Sync Now on Sleep source now to see that it imports without issues.
9. Click on Stream option and see that you have both Daily Activity and Sleep data loaded.
10. Go back to Sources and Destinations and select Add New Destination Task
11. Select SyncType Activity and select Destination from the dropdown list
12. Repeat same steps for Sleep syncronization
Accessing your charts directly from Dropbox:
If you have a Public folder on your dropbox, you can copy all the .js and .html files there and get the shared url by selecting one of htmk files and selecting "Copy public link..." button. It would give you a URL that you can use from anywhere. You need to have Dropbox Pro account or free account created before October 4, 2012 for Public folder functionality to work on your dropbox.
Accessing your charts directly from Google Drive:
1. It is recommended that you create new folder in your Drive for this purpose.
2. Select the folder and Choose Share... option
3. On Share with others page dialog, click on Advanced button in bottom right corner.
4. On Sharing Settings dialog, click on Change... link at the top line of Who has access list.
5. On Link sharing dialog, select On - Public on the web option and click Save button.
6. Copy the URL that is shown on Sharing Settings page and click Done to close it
7. The URL that you copied will look something like the sample below. Copy just the alphanumeric sequence that is located between ?id= and &usp=sharing
Code:
https://drive.google.com/folderview?id=[B][COLOR="Red"]123-abcdcdefghijklmnopqrstUV[/COLOR][/B]&usp=sharing
8. Put www.googledrive.com/host/ before this string and /mi_data.html (or any other report) after it, you should get something like www.googledrive.com/host/123-abcdcdefghijklmnopqrstUV/mi_data.html - this should be your sharing link
Here is Google support page on sharing files/directories: https://support.google.com/drive/answer/2881970?hl=en
Using Helium backup instead of Android backup:
Helium produces .ab files that are ALMOST correct tar file, however besides extra characters on file header level they also have some differences in header entries for each file. I've looked through different methods and utilities and the best option so far is to use Helium_ab2tar.zip package (credit goes to xaos.cz for writing it and to johnny1176 for discovering it). It used to be available on xaos.ic.cz/Helium_ab2tar.zip, but that site is down now. You can still google for Helium_ab2tar.zip to find other locations that have copy of this file.
Plans for future versions:
Code:
0. Get more sleep :)
1. Titles for each chart
2. Implement dashboard comparing current week with previous statistical data. (For same days of week)
3. Use timeline component for right chart in data_byDow.htm
4. Show percentage of good days based on selected goals. Show min/max values for each metric.
5. Modify scatter chart to allow using any other series for X axis (Search/display correlations between sleep time and total number of steps).
6. Make a help page showing all the charts and how to use them.
7. Add ratio of deep/light sleep as part of selectable charts.
8. Merge all reports into single multi-tab document.
9. Parse detailed daily binary data - this is going to be a big one.
How to translate application to your language:
To add support for a new language:
For HTML pages
1. Open locale.js in editor supporting UTF-8 encoding.
2. Create a new section starting with
if (lang == "ru_RU") { }
and replace ru_RU with a locale string for your language.
3. Copy all strings from english version to inside { } brackets and translate the text.
4. Save file in UTF-8 encoding.
5. Add reference to your new language in comments section of config.js
For unix shell script:
For all subsequent entries, replace XX with 2-character locale name of your language
1. Create new po file using following command:
bash --dump-po-strings run.sh > po/XX.po
2. Edit newly generated file and translate msgstr entries using msgid entries as references
3. Save the file and generate new directory for the compiled langage file (.mo) using following command:
mkdir -p i18n/XX/LC_MESSAGES/
4. Compile .po file to .mo file by running following command:
msgfmt -o i18n/XX/LC_MESSAGES/mibandextract.mo po/XX.po
If you do not want to use the i18n folder and want the langage file to be system wide, you just have to comment (adding a #) the line "export TEXTDOMAINDIR=./i18n/" in the script.
Then you will have to move your po file to /usr/share/locale/XX/LC_MESSAGES/ (where XX is your langage).
Changelog:
v5.2 released on 2/1/2015.
v5.3 released on 2/2/2015.
v5.32 released on 2/3/2015
v5.7 released on 2/16/2015
v5.71 + fit v0.2 released on 2/21/2015
v5.8 + fit v0.3 released on 3/7/2015
v5.9 released on 3/14/2015
v5.92 released on 6/20/2015
Reserved
Hi.
Working perfect. Nexus 5 non-rooted.
Thank you very much for your work :good:
Grretings from Spain.
Hi,
It does not work for me at all ... All the time the same thing.
C:\Users\plmaciejewk\Desktop\miband_extract_v53>run
The process cannot access the file because it is being used by another process.
The process cannot access the file because it is being used by another process.
The process cannot access the file because it is being used by another process.
The process cannot access the file because it is being used by another process.
The process cannot access the file because it is being used by another process.
Press "Backup My Data" button on device...
The process cannot access the file because it is being used by another process.
The process cannot access the file because it is being used by another process.
bin\tail: mi.ab: No such file or directory
The process cannot access the file because it is being used by another process.
The process cannot access the file because it is being used by another process.
The process cannot access the file because it is being used by another process.
Could Not Find C:\Users\plmaciejewk\Desktop\miband_extract_v53\mi.ab
Could Not Find C:\Users\plmaciejewk\Desktop\miband_extract_v53\mi.tar
The process cannot access the file because it is being used by another process.
The process cannot access the file because it is being used by another process.
Extraction failed
The process cannot access the file because it is being used by another process.
The process cannot access the file because it is being used by another process.
The process cannot access the file because it is being used by another process.
Can I do anything with this?
I face the same issue as @macia10. I have rooted Nexus 5.
Dell112 said:
I face the same issue as @macia10. I have rooted Nexus 5.
Click to expand...
Click to collapse
I found a temp workaround. Edit rub.bat and remove all >> log from end of the lined. This causes the issue.
macia10 said:
Hi,
It does not work for me at all ... All the time the same thing.
C:\Users\plmaciejewk\Desktop\miband_extract_v53>run
The process cannot access the file because it is being used by another process.
The process cannot access the file because it is being used by another process.
Can I do anything with this?
Click to expand...
Click to collapse
This might be caused by adb process still running in memory and locking log file. I've modified script not to redirect output from adb commands into log file and also added few commands to kill previous adb service once extraction is complete. Could you please download v5.33 and see if it works?
xmxm said:
This might be caused by adb process still running in memory and locking log file. I've modified script not to redirect output from adb commands into log file and also added few commands to kill previous adb service once extraction is complete. Could you please download v5.33 and see if it works?
Click to expand...
Click to collapse
works perfectly
Thanx!!
Thanks, very useful!!
FYI, I needed to make some modifications to make it work in my case:
- In 'run.sh', I had to modify the SDPath variable to the correct path (in my version of Cyanogenmod):
SDPath=/storage/sdcard0
- run.sh log indicated a "permission denied" error. It was caused because adb did not have root privileges in my android. To correct it:
In Android: Settings > Administrative access > Select "Applications and ADB"
- sqlite3 in OSX is an old version, so run.sh complains with:
Error: unknown command or invalid arguments: "once". Enter ".help" for help
Error: unknown command or invalid arguments: "print". Enter ".help" for help
To solve it, install a new version of sqlite3. For example using brew:
> brew install sqlite3
and replace sqlite3 in run.sh for the brew version: /usr/local/Cellar/sqlite/3.8.6/bin/sqlite3
- Files from: miband_analytics need to be in the same folder than miband_extract
With these modifications it is working properly in my mac.
Keep up the good work!
Hi.
I'm extracting data with Bonoboo Mi Band Export Stats, which exports to sd card the files:
app_locale.js; config.js; extract.js; locale.js; Mi_Band_Table.csv; Mi_Band_Table.html and Mi_Band_Graphs.html.
In order to used that exported data with your analytics tools, which files should i move to your folder which contains multiple files like mi_data_byDoW.html and mi_data.html ?
Thanks
herpez said:
Hi.
I'm extracting data with Bonoboo Mi Band Export Stats, which exports to sd card the files:
app_locale.js; config.js; extract.js; locale.js; Mi_Band_Table.csv; Mi_Band_Table.html and Mi_Band_Graphs.html.
In order to used that exported data with your analytics tools, which files should i move to your folder which contains multiple files like mi_data_byDoW.html and mi_data.html ?
Thanks
Click to expand...
Click to collapse
You'll just need at app_locale.js and extract.js. The folder with analytic tools already has config.js and locale.js and CSV file is not required for analytic tools, you only need it if you want to see your data in Excel.
SynerG said:
Thanks, very useful!!
FYI, I needed to make some modifications to make it work in my case:
- In 'run.sh', I had to modify the SDPath variable to the correct path (in my version of Cyanogenmod):
SDPath=/storage/sdcard0
- run.sh log indicated a "permission denied" error. It was caused because adb did not have root privileges in my android. To correct it:
In Android: Settings > Administrative access > Select "Applications and ADB"
- sqlite3 in OSX is an old version, so run.sh complains with:
Error: unknown command or invalid arguments: "once". Enter ".help" for help
Error: unknown command or invalid arguments: "print". Enter ".help" for help
To solve it, install a new version of sqlite3. For example using brew:
> brew install sqlite3
and replace sqlite3 in run.sh for the brew version: /usr/local/Cellar/sqlite/3.8.6/bin/sqlite3
- Files from: miband_analytics need to be in the same folder than miband_extract
With these modifications it is working properly in my mac.
Keep up the good work!
Click to expand...
Click to collapse
Thank you for your feedback.
Regarding SDPath, I'll put a a more clear explanation that SDPath may need to be changed depending on device.
For ADB privileges, the way you had to enable it might be specific to your Android version (I've granted permissions in a different way on my phone), but generally speaking the application should work through Backup logic even when adb has no root permissions.
For sqlite3 issue, could you please tell me which version of sqlite3 OSX has by default, so can add a note in instructions about version compatibility?
Regarding same folder location, I think there is already a mention about the need to have both packages exptracted to the same folder.
xmxm said:
Thank you for your feedback.
For ADB privileges, the way you had to enable it might be specific to your Android version (I've granted permissions in a different way on my phone), but generally speaking the application should work through Backup logic even when adb has no root permissions.
Click to expand...
Click to collapse
I got a message in my Android asking for permission to do the backup, but at the same time I saw the "permission denied" message in the console. Maybe I got both because the script tries to get data from two different mechanisms. Even if I clicked yes to do the backup, the script was not able to get the data until adb was granted permissions.
For sqlite3 issue, could you please tell me which version of sqlite3 OSX has by default, so can add a note in instructions about version compatibility?
Click to expand...
Click to collapse
My Mac with OSX 10.9 (Mavericks) had installed sqlite3 version 3.7.13. I think that the required commands were added in 3.7.15, but I could be wrong. Brew has installed 3.8.6 and is working properly.
SynerG said:
I got a message in my Android asking for permission to do the backup, but at the same time I saw the "permission denied" message in the console. Maybe I got both because the script tries to get data from two different mechanisms. Even if I clicked yes to do the backup, the script was not able to get the data until adb was granted permissions.
Click to expand...
Click to collapse
Windows version supports ForceBackupMode=Y setting to skip root-specific method and go straight for backup method, but I guess that run.sh is outdated by now - I'll need to find a way to copy newest logic changes to unix script. I still think hope that giving ADB root is not mandatory for extract to work.
My Mac with OSX 10.9 (Mavericks) had installed sqlite3 version 3.7.13. I think that the required commands were added in 3.7.15, but I could be wrong. Brew has installed 3.8.6 and is working properly.
Click to expand...
Click to collapse
Thank you for this information - I'll put version numbers in the header.
xmxm said:
Windows version supports ForceBackupMode=Y setting to skip root-specific method and go straight for backup method, but I guess that run.sh is outdated by now - I'll need to find a way to copy newest logic changes to unix script.
Click to expand...
Click to collapse
You will find attached a zip with a new version of run.sh.
run.sh now support ForceBackupMode and internalization.
Thats why I add 2 folders (po and i18n).
If you want to add support for a new langage, let's say spanish for example, you will first have to create a new po file
Code:
bash --dump-po-strings run.sh > po/es.po
Then edit the file newly created with any text editor and translate the "msgid" with the "msgstr"
Once done, you have to create a new directory for the compiled langage file (.mo)
Code:
mkdir mkdir -p i18n/es/LC_MESSAGES/
Then compile the file with
Code:
msgfmt -o i18n/es/LC_MESSAGES/mibandextract.mo po/es.po
For another langage replace "es" with the one you want.
If you do not want to use the i18n folder and want the langage file to be system wide, you just have to comment (adding a #) the line "export TEXTDOMAINDIR=./i18n/" in the script.
Then you will have to move your po file to /usr/share/locale/XX/LC_MESSAGES/ (where XX is your langage).
Version updated to 5.5 with better support for FitnessSyncer and some chart updates
Good Work
xmxm said:
Version updated to 5.5 with better support for FitnessSyncer and some chart updates
Click to expand...
Click to collapse
Thanks for your work!!!