Archive for the 'Mininova technical' Category

Downtime due to relocation re-scheduled for today

It was supposed to happen last friday, however due to some problems it’s scheduled for today. Hopefully the relocation it won’t take too long and without problems.

Update: The relocation has been delayed, again. It’s planned for tomorrow now. Stay tuned!

Update: The upgrade is completed! Enjoy the new and fast mininova ;)

Update: Well it seemed to be working anyway :p, still having some weird problems but we are working on it!

Final Update: Everything should be up and running like it is suppose to be! :))

Downtime due to new server

As you might have noticed, we’ve been kinda slow during the evenings. So today we are getting an extra server :)!

However this will mean some downtime as the other servers will be moved to a new rack (and get different ip addresses).

It could take up to 60 minutes for the dns servers to update. We also have to install the new server so the change will probably be noticeable tomorrow (or sooner if everything goes well).

Update: The upgrade will probably take place after the (busy) weekend, so stay tuned!

Frontpage Update

New Frontpage: As you might have seen we have changed the frontpage. The reason for this change is the incease in bandwith usage. Which is cause by the increasing number of torrents uploaded per day combined with the number of visitors which enters the site through the frontpage.

The old frontpage was an average of 150 KB (smaller in the morning, larger in the evening). With 750.000 pageviews on the frontpage, it used 107 GB per day! This is 1,27 MB per second!

The new frontpage is only 35 KB which results in 0,2 MB per second.

We are paying for a 70 mbit connection (8,75 MB per second).
As you can see in this graph this isn’t enough at the end of the day (when the frontpage gets larger than 220 KB).

This is also the reason why mininova got slower in the evenings.

We hope everybody understands this and likes the new frontpage. If you don’t you can use the today page.

Edit (Niek): Added a “show all of today” link under each category, for easier navigation.

Edit (Erik): Added a “Use the today page as my frontpage” setting to the settings page. Setting this option makes your frontpage the same as the old frontpage was.

Search Cloud (Erik): There is now a mininova search cloud. It contains the most searched words of the day. The words get bigger when they are searched for more often. The cloud is empty at the start of the day and will grow as the number of search increases for the day.

Visual passwords (Erik): I have added a cool example for visual passwords to the mininova labs. Don’t worry, we won’t use it for mininova.

Improved search performance (Erik): I have managed to increase the speed of the search queries by almost 400% using a temporary table. The bad thing is that this table is updated only once every hour, so new torrents can take up to 1 hour to show in the search results. The good thing is that we won’t have to buy a new database server because of this.

Mininova, the javascript cluster

This one time I was thinking of what would be possible with Mininova and all its visitors.
The first think I thought about was a hidden IFRAME DDOS, but this wouldn’t be cool (but maybe I’ll talk about this later).
The second thing that came to my mind was a little more complicated but also more fun.
Why not use the visitor’s browser to do some calculations.
When a page is loaded and the visitor is reading the page the browser does (almost) nothing, so why not use this time.
My Idea was to send some javascript with the page and execute it when the page is loaded.
The results could be posted back with a XMLHTTPRequest.
This request could even be used to fetch another script to execute next for if the user keeps the page open for some time.
So let’s calculate how fast this would be.

Mininova has 3,770,305 page views per day.
99.27% of the Mininova visitors have Javascript enabled.
So this is 3,742,782 page views per day with Javascript enabled.
This is 155,949 page views with javascript enabled per hour, 2,599 per minute or 43 per second.
The average time spend on a page is about 10 seconds. This is really a guess (maybe someone knows a better guess?).
Using this data we can say that at any time 43 * 10 = 430 people will have a Mininova page open (in a browser with javascript enabled).

On my machine (FireFox 1.5 Beta 2, AMD Athlon 64 3000+ ~2GHz CPU) Javascript can execute 2,000,000 IPS.
With IE this is even more, 2,500,000 IPS.
check on your machine
Because 60.32% of the Mininova visitors uses IE but not everyone has a fast CPU I’ll use an average of 2,250,000 IPS.

So using this data we can calculate that with this cluster we can do 430 * 2,250,000 = 967,500,000 IPS.
Which is 967.5 MIPS.

Now let’s compare this to some CPU’s.

486 20 MIPS
Pentium 60 MIPS
Pentium Pro 200 MIPS
Pentium 2 333 MIPS
Pentium 3 500 MIPS
Mininova cluster 967 MIPS
Pentium 3 (1Ghz) 1000 MIPS
Pentium 4 1500 MIPS
Itanium +2500 MIPS

data from here

So the Mininova javascript cluster is almost as fast as a 1Ghz Pentium 3.
Not as fast as I thought when I started writing this but still not slow.

I have never tested this for real because of 3 things:

1) I don’t know if the Mininova visitors would like it if their browser uses all their CPU power when visiting Mininova.

2) Measuring this isn’t easy.

3) Because almost every page views would result in a request to post back the data it would double the requests which the Mininova web server won’t like.

I think testing it for real would result in a somewhat slower speed.
One of the things that I haven’t taken into my calculations it the time it takes to post back the data with a XMLHTTPRequest.
Also one of the problems would be people closing the page before the request is done.

I think I will never find out if it really works and what the real speed would be.
But it was fun calculating this stuff and thinking about it.
Hope you enjoyed reading this,

Erik.

More fun can be found at the Mininova labs.

New server

We installed a new (third) server a few days ago. We moved the forum, this blog, the ads-targeting application, CodePost and some other stuff to this new box.

For those of you who’re interested, here are the specs:

Dell PE1850

Dual Xeon 2,8Ghz

2GB DDR Memory

2x 73 GB Harddisk (RAID1)

10Mbit flatfee internet connection

Hopefully this will make loading pages from the main servers even faster :)