Develop Bugs

Downloads and Stuff

I know, I know, I know. I'm not paying, so I shouldn't complain.
However, I think we can agree that rapidshare is the opposite of efficient for ROM distribution.
Since I'm no coder myself, just an end user (with an engineering degree in telecoms), for me to set up a good solution myself doesn't exactly strike home eighter.
However, we have some VERY capable coders here. And we have a lot of end users with some bandwidth to share. Is there a chance we can set up an rss-based downloader (using bittorrent to distribute roms), and then simply tell regular downloaders to get the torrent file? (And thus save this site bandwidth and storage, and end users gray hairs over downloads that are SLOOOOOW). Setting up a Diamond "rom-farm" spread over the internet would solve a lot of problems. I know _I_ can prolly share half of my 8mbit (on average), and I think there are others out there who can share aswell, giving this group a good potential (and if we can get other groups to cooperate we can prolly increase bandwidth and spread over the internet).
Bittorrent is ideally suited to this work, especially if we have a few "always on" seeding-servers.
//Svein

This is a very good idea.
I'm on a 100MBit, so that won't be a problem.
I also have a spare dual opteron server lying around.
I've seen that tom (who has a Latvian hosting) also has some good bandwidth.

adwinp said:
This is a very good idea.
I'm on a 100MBit, so that won't be a problem.
I also have a spare dualopteron server lying around.
I've seen that tom (who has a Latvian hosting) also has some good bandwidth.
Click to expand...
Click to collapse
unlimited

What we basically need:
1. Someone with a *nix box on a low-latency (bandwidth isn't an issue, latency is) link, willing to run "our" tracker, and set up the needed accounts for uploads, and set up the tracker to rss-feed the seeder servers.
2. Several people willing to run seeder servers. These need bandwidth. The seeder servers should run in rss-fed full automatic mode.
From what I've seen (between the lines) in the posts on this forum, getting people willing to give bandwidth shouldn't be a problem. What we need is a decent tracker software (I'm open to any suggestions), and some template for setting up the seeder servers.
Am I missing anything? Who volunteers to make the tracker? (I can volunteer to run it, and to make sure latency is ... acceptable on the link. I'm a pedantic millisecond-centric ex netadmin. Ex because of health.)
//Svein

this is a great idea. If i can help in any way I will.

The dual opteron box I was talking about is running gentoo linux.
I will have to reconfig it though.
I can take care of most of the setup during the week-end (if I stay home don't go on the potential trip I planned).
Let's take care of all the suggestions via this thread / pm.

What a great idea

sskogen said:
What we basically need:
Am I missing anything? Who volunteers to make the tracker? (I can volunteer to run it, and to make sure latency is ... acceptable on the link. I'm a pedantic millisecond-centric ex netadmin. Ex because of health.)
//Svein
Click to expand...
Click to collapse
I was a net admin for 4 years so I know how you feel.
^.^
Try testing adwinp.org and see if the latency suits your needs (it should; we are on a 10GBit backbone).
^.^

Well this seems promising! While this whole thing gets in place, as a sort of alpha tryout, how about using public trackers such as thepiratebay for example? And have a wiki page with up to date torrent links.

wow great idea =)
only suggestion here is maybe change the title of this thread xD

I uploaded torrents for the radios and Elite ROMs.
Links are available on the Diamond Radio wiki: http://wiki.xda-developers.com/index.php?pagename=Diamond_ExtractedRadioRoms
and the Diamond Elite ROMs wiki: http://wiki.xda-developers.com/index.php?pagename=Diamond_Elite_ROMs
It should work.. normally... and be faster than rapidshare once we get seeders.
Time to flash Elite 3.0 now
If you already have the files, don't download them again! Just copy them to the torrent download directory, like that your torrent will be 100% complete and you will be able to seed and help others download faster.

Have added the 5 torrents to my local box.
//Svein

Torrent sharing is great. I'll help!

Offer to Cookers

All this rapid share, yadda yadda is really getting old quick. If at least a few ROM Cookers are interested. I'll setup a space on my server to allow uploads that point back to an HTTP download section. You would be responsible for uploading the file, then posting a link here on XDA that would be a direct download link. The uploads would be handled by a passworded FTP account. I have about 400gig of download to spare per month, which should be more than enough for a bunch of Roms. Either reply here or send me a PM and we can talk.
I can also set it up, so that people can go to a root folder and find all the Roms and just pick the ones they want. Heck I can even store older Roms on the server so if someone wants an old one it will be up there.
1st point, there will be no warez posted.
2nd point, you must be a recognized cooker.
Well Cookers, drop a line and let me know.
It would end up being something like this, with each cooker having his own folder.
http://captfiero.com/videos

This has been done, before, twice.
And both times, the file disappeared, or the person did, within 4 days.

Well, not Jannaa's!!!! I use it aplace4u.biz and it's AMAZING!!

veyka said:
This has been done, before, twice.
And both times, the file disappeared, or the person did, within 4 days.
Click to expand...
Click to collapse
The only reason I could see a person leaving is if Microsoft sent a letter saying they did not like people cooking the roms and throwing a hissy fit. I have went ahead and created the folder on my remote server and am uploading stuff right now. I have not made the ftp account for uploading, however I should be able to allow open access to what I have on there in about 2hrs.

Capt Fiero said:
The only reason I could see a person leaving is if Microsoft sent a letter saying they did not like people cooking the roms and throwing a hissy fit. I have went ahead and created the folder on my remote server and am uploading stuff right now. I have not made the ftp account for uploading, however I should be able to allow open access to what I have on there in about 2hrs.
Click to expand...
Click to collapse
I hate rapidshare as much as the next non premium person, so i hope this gets off the ground. people don't even use aspupload, or whatever its called..

Ok the Basic is up, http://www.captfiero.com/romsYou will have to mouse over the file name if it is too long to show up, and then just look at your tool bar to see the entire name.
If anyone see's a file that should not be there. (copyrighted) Please let me know and I will remove it.
Also if a file is something "YOU" made or prefer me not to have up, let me know and I will remove it.
I will not make the upload address public so if you want files uploaded, you can send me a PM and I will upload it, or if you are an established cooker, I will give you direct upload settings.

hmm
I am sure my chef would love this feature, his connection sucks back home though, so if this is genuine I will start mirroring his stuff here too ... I guess the more mirrors the better, I been mirroring to 4shared and mediafire all this time... seems to work well
Just FYI tho, CRCinAU has at least around 100GB of traffic a day (may be he can correct me on this) ...

zocster said:
I am sure my chef would love this feature, his connection sucks back home though, so if this is genuine I will start mirroring his stuff here too ... I guess the more mirrors the better, I been mirroring to 4shared and mediafire all this time... seems to work well
Just FYI tho, CRCinAU has at least around 100GB of traffic a day (may be he can correct me on this) ...
Click to expand...
Click to collapse
He said something like that somewhere.. If you think, most rom topics get 1k+ hits per day.. and say 100 download.. a 50 meg file, that's 5gig from nowhere..
And when a new rom is out..

Well I have 1200gig total traffic allowed, I use about 400gig a month for Videos and I want to keep about 400gig aside for spikes. That leaves me about 400gig to play with. I can't mirror the entire PPC world, but the latest roms and a few add ons would be fine.

heh - where abouts is this mirror located?
At the moment, my mirroring system is handling the load quite well - I have a total of 5 mirrors around the globe (1 x EU, 3 x US, 1 x AU) and the amount of traffic is much lower than it was a month ago on the one AU server because of the way things are spread out now.
The mirroring scripts will auto-detect where the user is and send them to the nearest server (or if we don't have a server near you, we randomly pick one to share the load).
You'll notice this is why I don't have direct links to anything anymore - it's all to my download script.
Even DeviceUpdate uses this script to custom-generate it's index based on where you are
That being said, as of my typing this, since 28th August, I've served:
Release 12.11 ROMs: 270 @ 40Mb each (10.8Gb)
Release 11.2 ROMs: 38 @ 41Mb each (1.55Gb)
Device Update Sessions: 1,235 (unknown download amount)
That's pretty good for less than 4 days
Some more mirrors would be nice - although mirrors in the US are probably not as helpful as the rest of the world... There's ~1Gb of content to be mirrored and count on ~100Gb per month of traffic in a lazy month.

FYI: Developers thread options: http://forum.xda-developers.com/showthread.php?t=388819

To my knowledge it is a US based server. I am personally in Canada however it seems that US hosts are far cheaper. I'll send ya a PM with the details.

Free File Mirror Hosting!

I was just wondering would anyone be interested in free file hosting for your ROM/s. What i ask in return is...well...donations...i may ask for a favour in return at a later date or something?
I'm giving chefs and application developers this as a way of saying thanks for all their hard work. That, plus im sick of the slow transfers of dodgy file share/hosting sites like rapid$hare and [email protected]
What you get:
-Free FTP account so you can upload your files with UNLIMITED storage/data transfers.
What you need to do:
Visit www.intertubes.info and read the front cover and shoot me an e-mail or PM on here and let me know why you deserve an account, and it will all be set up for you and i'll e-mail/PM you back with how to get started
At this stage i'll start with a limited number of seats.

lasterato said:
I was just wondering would anyone be interested in free file hosting for your ROM/s. What i ask in return is...well...donations...i may ask for a favour in return at a later date or something?
I'm giving chefs and application developers this as a way of saying thanks for all their hard work. That, plus im sick of the slow transfers of dodgy file share/hosting sites like rapid$hare and [email protected]
What you get:
-Free FTP account so you can upload your files with UNLIMITED storage/data transfers.
What you need to do:
Visit www.intertubes.info and read the front cover and shoot me an e-mail or PM on here and let me know why you deserve an account, and it will all be set up for you and i'll e-mail/PM you back with how to get started
At this stage i'll start with a limited number of seats.
Click to expand...
Click to collapse
Lasterato
Thinks is a very good offer which I really appreciate however bl0w is doing exactly the same thing at the moment and is doing a fantastic job of this. I'm not sure wether there will be a demand for this as we are very happy with bl0w's services.
See what the other chef's say anyway. I guess you can't have enough mirrors!
Regards
Shaks

shaks1979 said:
Lasterato
Thinks is a very good offer which I really appreciate however bl0w is doing exactly the same thing at the moment and is doing a fantastic job of this. I'm not sure wether there will be a demand for this as we are very happy with bl0w's services.
See what the other chef's say anyway. I guess you can't have enough mirrors!
Regards
Shaks
Click to expand...
Click to collapse
I'm willing to work side by side with bl0w, presuming that they're happy to do so...i haven't seen enough non-rapid$hare type links to be satisfied

Just thought I would say thanks to lasterato, also is this server based in Australia cause that would be tops for us aussies haha, also im with you on the NO CLEAN FILTER!!
Thanks again for your contribution!

luckyluke1137 said:
Just thought I would say thanks to lasterato, also is this server based in Australia cause that would be tops for us aussies haha, also im with you on the NO CLEAN FILTER!!
Thanks again for your contribution!
Click to expand...
Click to collapse
Sadly no...they're like millions per month. I basically just had the site for some file-storage stuff, and though i'm not going to use it all. I usually get awesome speeds off it anyways. So it's a win win situation i say!

lasterato said:
Sadly no...they're like millions per month. I basically just had the site for some file-storage stuff, and though i'm not going to use it all. I usually get awesome speeds off it anyways. So it's a win win situation i say!
Click to expand...
Click to collapse
I can offer 100Mbit hosting, yet there's no point in it since someone else is already doing a very good job at it.
My suggestion: You can always contat bl0w in order to rsync his server to yours, and that would be exactly this: mirroring.
Then again, what's the point in this, or even load balancing?
It's not like he gets overloads.
A.

adwinp said:
I can offer 100Mbit hosting, yet there's no point in it since someone else is already doing a very good job at it.
My suggestion: You can always contat bl0w in order to rsync his server to yours, and that would be exactly this: mirroring.
Then again, what's the point in this, or even load balancing?
It's not like he gets overloads.
A.
Click to expand...
Click to collapse
Very true...meh, will see if anything results from it

i can help to
Hello!
Dear friends, developers, programmers, ideological comrades!
This resource I have introduced to support you.
Yes, it’s not super-hosting, not terabytes of space for all your files … But everything here is — is provided to you free of charge.
Limited speed of traffic IN to 820 kilobytes \ s. OUT - 100mb\s
You are given, depending on the size of the project, 10GB \ 50GB \ 100GB package on the resource.
DataBase MariaDB \ php5 ZenoCacheEngine \ nginx + apache2-minded bunch.
For your domain name (if you want to move), there are two NS servers, the ability to manage DNS zone.
Available hosting control panel VESTA CONTROL PANEL.
FTP support and control include.
Each participant can carry out its project page on the main site.
=
Please write here - what candidate to hosting place or mirror you wanna see?
Whose have interesting kernel, or rom \ script \ app \ and need mirror for files without difficult access to share file service?

Torrents

Hi there.
Wouldn't it be nice if xda-developers ha a system for letting the users create torrents when they need to upload something? Many of the developers here use sites like rapidshare and similar to upload their stuff. I don't like those sites and would rather see that the uploaders used a torrent system.
I can create a separate torrent page for you if you like...

Great Idea
dongrif said:
Hi there.
Wouldn't it be nice if xda-developers ha a system for letting the users create torrents when they need to upload something? Many of the developers here use sites like rapidshare and similar to upload their stuff. I don't like those sites and would rather see that the uploaders used a torrent system.
I can create a separate torrent page for you if you like...
Click to expand...
Click to collapse
You nailed it buddy..We need exactly the same !..I am using a 16 MBPS connection and it can't do jack since it seems the ISP is using some proxy kinda system and whenever I try and download...it says your Quota is finished.. Anyways, I hope we have such a system too..I won't mind uploading ROMS using Torrents to other users..

Niice Idea

would be pretty cool and lessen the need for mirrors

Torrents are associated with warez so I don´t think this idea will be moving forwards

We do the association with loading "sinful things" using torrents...so WE as in the general Internet Users started it..I feel that if we start using torrents for sharing ROMs etc..that perception..at least between XDA users will change..and how about Private Trackers...
Explanation as per one of the websites..Looks legit enough to me..
A private tracker is a tracker which allows only registered users to download torrents. The registered users are required to maintain a certain upload-to-download ratio and are banned if the ratio is too low. This ensures that the users don’t just leech the files but also seed them after downloading it completely, which in turn ensures that you get very fast download speeds.

bamit99 said:
We do the association with loading "sinful things" using torrents...so WE as in the general Internet Users started it..I feel that if we start using torrents for sharing ROMs etc..that perception..at least between XDA users will change..and how about Private Trackers...
Explanation as per one of the websites..Looks legit enough to me..
A private tracker is a tracker which allows only registered users to download torrents. The registered users are required to maintain a certain upload-to-download ratio and are banned if the ratio is too low. This ensures that the users don’t just leech the files but also seed them after downloading it completely, which in turn ensures that you get very fast download speeds.
Click to expand...
Click to collapse
Sounds good to me

I've experimented with torrents as a distribution means, but it just doesn't work, as a huge %age of our users were in the UK, most with appalling upload speeds.
You can't expect people to seed a file to a 1:1 ratio if they are on a 30 KBps upload (absolute max), but 8 Mbps down... Bear in mind that at 30 KBps you would have your uploads saturated, so 10 KB/s is more realistic...
And XDA-Developers does not get involved in the distribution of ROMs, or their storage.

Hmm... To be fair my upload speed that I get is about 3kb/s where as download is about 200kb/s
So I won't be able to keep up that ratio

MacaronyMax said:
Hmm... To be fair my upload speed that I get is about 3kb/s where as download is about 200kb/s
So I won't be able to keep up that ratio
Click to expand...
Click to collapse
No worries buddy.. I am sure out of 3 millions XDA Users, we will find a lot with a decent bandwidth Internet...

orb3000 said:
Torrents are associated with warez so I don´t think this idea will be moving forwards
Click to expand...
Click to collapse
Thats very inaccurate. Its just a means of acquiring "illegal" content. Thats like saying cars are associated with hit and runs so nobody should drive.

Ok you words are more precise

nvm, someone beat me to the torrent thing.
Anyway, if torrents were also web-seeded I see no reason why it wouldn't work.

AdrianK said:
nvm, someone beat me to the torrent thing.
Anyway, if torrents were also web-seeded I see no reason why it wouldn't work.
Click to expand...
Click to collapse
Who would pay for the servers required for web seeding?
XDA won't get involved in distribution of ROMs, and as a server admin, I can assure you that I wouldn't want the traffic from a web seeding box on any of my servers... While it's all a lovely hypothetical idea, torrents just don't seem to be the best way to distribute ROMs. Someone ultimately has to upload the file to other peers at their broadband upload speed... Who's gonna wait an hour downloading a file from me, only to then have to seed to someone else? Starting them off would take ages

***Off topic***
Just wondering, never really been in the WM scene, and I was wondering how much roughlyin MB are the Roms?? For android they re 90MB on average, You could get like a stock cynageon rom only being 60mb and a full sense rom 110 ish lol
Thanks

Off topic reply
WM Energy ROM for Leo is about 170Mb but leaves you more free RAM than Android Froyo 2.2...

orb3000 said:
WM Energy ROM for Leo is about 170Mb but leaves you more free RAM than Android Froyo 2.2...
Click to expand...
Click to collapse
Cool thanks

Im thinking I may take this project on, If your a knowledgeable person when ity comes to roms or ideally a dev and would like to be staff on a xda releases torrent site drop me a pm.
If there is allot of interest in this I may provide the site/server/webseeds

I know this is old news but I thought I could at least put my two cents in:
I have been on other private trackers that are closer to a .8:1 ratio. To help those with severely throttled uploads, couldn't we make it more like .1:1?
I am not a coder, but love what you guys do, and I use many of your solutions. I would love to have a way to contribute back on a consistent basis. I have a 1MB upload (which I realize isn't huge) and lots of HD space, and a computer that stays on 24/7. I figure those of us that have a decent upload will help those that can't share much back.

I'd like to say that I support this fully! One more advantage that needs to be listed would the ability to resume downloads and not need to restart corrupt ones!

Still using filesharing sites?

I have a 150GB GoDaddy server itching for ROMs, kernels, themes, etc. Email me at [email protected] and I'll create an FTP account for you. Your path will be something like http://www.heylisten.org/ROM/android/your-xda-username unless you request something different. All that I request in return is that you try to recommend users to my index site, http://www.heylisten.org/ROM, when appropriate. Your ROMs can be included in said index if you tell me where to find them and how to list them (phone name, base). The Inc section is the only one that works right now, but I'm working on expanding the site.
Thanks!

Yeah Im tired of mediafire and rapidshare bull aswell, I have 4 servers 2 home and 2 on godaddy. I can mirror if anyone cares ([email protected])

i offered up space on my webdomain.. it went unheard .. devs like the media sites cause they can monitor clicks.. i monkeywrench em and download files multiple times .. your counts arent true!!!!!

To devs: with a normal FTP server, you can use ROM Manager and QR code downloads, as well as the RM web link thing. C'mon, I didn't get the 150GB plan for nothing, did I?

Mobileunderground offered free ftp space for rom chefs as long as they just mirrored their threads there aswell.
I know alot of rom chefs use it to track their downloads, or to earn some extra cash by using the sharing sites (or extra bandwith to dl other things online).
There have been tons of sites that have mirrored roms and other items, but in the long run, they end up getting shut down either by htc, google, microsoft, or the webhost's request them to go down for using them to host files when it says not to in the TOS...
read the TOS, 99.9% of webhosts will have that in there, once you start getting into tens of gigs of data, they will tell you to remove the file within 2-3days....I know, it happened to MobileUnderground multiple times since I have been a mod over there...

codybear said:
Mobileunderground offered free ftp space for rom chefs as long as they just mirrored their threads there aswell.
I know alot of rom chefs use it to track their downloads, or to earn some extra cash by using the sharing sites (or extra bandwith to dl other things online).
There have been tons of sites that have mirrored roms and other items, but in the long run, they end up getting shut down either by htc, google, microsoft, or the webhost's request them to go down for using them to host files when it says not to in the TOS...
read the TOS, 99.9% of webhosts will have that in there, once you start getting into tens of gigs of data, they will tell you to remove the file within 2-3days....I know, it happened to MobileUnderground multiple times since I have been a mod over there...
Click to expand...
Click to collapse
exactly my reason for staying with multiupload.... i have a godaddy account but they have required me to pull my files more than once... its too much of a pain to redo all of that

I have about 3TB of traffic weekly. Been running solid for about 5 months.

I looked over the ToS. As long as I don't go over 150GB I'm good, because I have unlimited bandwith.

DesireC said:
I looked over the ToS. As long as I don't go over 150GB I'm good, because I have unlimited bandwith.
Click to expand...
Click to collapse
The tens of gigs statement was only used because that's when they notice that you are using it to distribute or backup files without paying for a package that allows this. It has happened to TONS of chefs/websites that hold ROMs and other items.
It's been proven time and time again that it does not work...
Maybe if you had a dedicated server, things might be different...But the traffic to the sites that host such roms are retardedly high and the webhosts see that and shut you down....
It does not take very long.
Trust me, I've had to backup the entire ftp of MobileUnderground onto my 2tb external only to upload it to the new server and have the same thing happen to us in about a two month span with three or so webhosts.
Stick around for about 2 years and you'll see it happen to plenty of people.

DesireC said:
I looked over the ToS. As long as I don't go over 150GB I'm good, because I have unlimited bandwith.
Click to expand...
Click to collapse
No, you did not read the TOS closely enough.
DesireC said:
To devs: with a normal FTP server, you can use ROM Manager and QR code downloads, as well as the RM web link thing. C'mon, I didn't get the 150GB plan for nothing, did I?
Click to expand...
Click to collapse
Yes, you did.
codybear said:
Mobileunderground offered free ftp space for rom chefs as long as they just mirrored their threads there aswell.
I know alot of rom chefs use it to track their downloads, or to earn some extra cash by using the sharing sites (or extra bandwith to dl other things online).
There have been tons of sites that have mirrored roms and other items, but in the long run, they end up getting shut down either by htc, google, microsoft, or the webhost's request them to go down for using them to host files when it says not to in the TOS...
read the TOS, 99.9% of webhosts will have that in there, once you start getting into tens of gigs of data, they will tell you to remove the file within 2-3days....I know, it happened to MobileUnderground multiple times since I have been a mod over there...
Click to expand...
Click to collapse
Yup.
savoxis said:
I have about 3TB of traffic weekly. Been running solid for about 5 months.
Click to expand...
Click to collapse
You've just slipped under their radar so far.
codybear said:
The tens of gigs statement was only used because that's when they notice that you are using it to distribute or backup files without paying for a package that allows this. It has happened to TONS of chefs/websites that hold ROMs and other items.
It's been proven time and time again that it does not work...
Maybe if you had a dedicated server, things might be different...But the traffic to the sites that host such roms are retardedly high and the webhosts see that and shut you down....
It does not take very long.
Trust me, I've had to backup the entire ftp of MobileUnderground onto my 2tb external only to upload it to the new server and have the same thing happen to us in about a two month span with three or so webhosts.
Stick around for about 2 years and you'll see it happen to plenty of people.
Click to expand...
Click to collapse
Here's the email I got from GoDaddy when we were hosted by them months ago:
Dear Sir/Madam,
It appears as though your mobileunderground.info hosting account is being used as a file repository. You may review the Storage and Security section of our Hosting Service Agreement for further clarification. If you are interested in keeping file backups, you will need to purchase the Online File Folder, a Virtual Dedicated Server, or a Dedicated Server.
Specifically, the following currently appear to be in violation:
/_db_backups 4.12 GB
/captainback 10.93 GB
/omj09 4.76 GB
/indagroove
25.31 GB
/tmartin09 3.34 GB
/sohkis 2.02 GB
/asusmobile 1.95
GB
/ai6908 2.14 GB
/smiznith 5.91 GB
/ericwgarza1 3.32 GB
/formula84
1.73 GB
/rbolen70 1.15 GB
Please remove all files not being referenced to a website and/or explain how they are being used by the web site by July 15, 2010 at 7:00 AM MST.
We will review the mobileunderground.info hosting account again at that time. If we do not receive a reply to this notice, or the content in question is not removed, access to that content will be disabled.
Please call us at 480-505-8871 or reply to this email with the necessary information so that we can review this issue further. We appreciate your understanding and cooperation in this matter.
Sincerely,
Wayne C.
Hosting Support