Monty Rants or Yet Another Case Of Developers vs Users

I saw Monty's latest rant and I told myself "here we go again." I blogged before about deaf developers, and here's another symptom of the same disease.
This is another case of a developer, a brilliant one, mind you, one of the brightest in the open source arena, but a developer nonetheless, who has a non-user oriented mindset and is inflicting his view on the users.
I will stay clear of the reasons. Why the co-founder of MySQL decides to spoil the party on release day and ambushes his colleagues with a truckload of blame is beyond my understanding.
What concerns me is the patronizing plea that Monty uses against the users. "Don't use MySQL 5.1"? Really? What about the ones who have already been using it for months, if not years? Don't they count for anything? Has he ever used it?
I look around, and I find plenty of stories from people who have been using MySQL 5.1 in production for long time, starting from the people at MySQL.com itself. And then there are the ones of the Use Case competition, who have shown that 5.1 in production is not anathema. And then there is me. I haven't participated to the competition, but I have been using MySQL 5.1 with partitions for 8 months already, and it suits my needs perfectly.
I don't know of the other features, but for partitioning I have something to say. Let's look at Monty's claims:

Partitioning in MySQL 5.1 should be regarded as a step to a full partitioning feature with parallel query. Parallel query is however not scheduled even for MySQL 6.0.

Why should I care? Yes, parallel queries would help, but with partitioning I got performance gains from 10x to 100x, and I couldn't care less if the feature is not as perfect as it would satisfy a stellar developer.

For now partitioning is mainly useful in the case where you need to frequently drop a well defined part of a table (like one month of data) and when MERGE tables are too cumbersome to use.

I strongly disagree. MERGE tables were a crude hack that I used as long as there was nothing better. Partitions solve my problems nice and square. My insertion and retrieval speed increased, and that's all I need.

If one partitioned table crashes it's very hard (sometimes impossible to repair it.

Probably Monty wasn't paying attention when Bug#20129 was fixed.

If you get a server crash during ALTER TABLE of a partitioned table you may loose all your data for that table.

I would like to see proof of this.

Partitioning is very slow and can become unusable if you have a large number of partitions. This happens even if you only use a few of the underlying tables in your query.

My experience tells me otherwise.

So, it looks like Monty has never used partitioning, probably because he doesn't think that the implementation is good enough. Is this a good reason to scare away everyone else? I think not.

Cracking the Tokutek contest with MySQL 5.1

Hello, blog! Long time without posting. Here I am again.
My attention was caught a few days ago by a contest posted by Tokutek.
This is a company that makes a proprietary engine for MySQL. In their contest, they claim that it takes 20 days to insert 1 billion records into a InnoDB table, and that MyISAM is not even up to the task. In their example, InnoDB inserts 404 rows per second and MyISAM died after a few days of inserting 93 rows per second.
It looks farfetched to me. Surely you can do better than that.
Now, I don't have a quad 3.16Ghz Xeon server with 16GB RAM available like the one used by Tokutek, and so I tried with a modest quad 2.7Ghz AMD, 8GB RAM server. The problem is mostly the same. Once your index becomes bigger than the available RAM, you are in the glue. The only difference is that my server was slower.
I didn't try with InnoDB, but I tackled the MyISAM case at once. Using MySQL 5.1, I just added 400 partitions to the contest table, partitioning by range of 50 million records. And I made a simple Perl script that produces INSERT statements with 50,000 rows each.
The result? I inserted the rows into a MyISAM table at an average rate of 8,800 rows per second, with peaks of 25,000 rps. The speed did never drop below 5,000 rps anyway. The whole operation took 31.5 hours. That's much less than 20 days, and surely a hell of a lot less than the projected 167 days that should be needed for MyISAM according to the contest slides.
Should I claim the ridiculous contest prize? (a $100 Starbucks card? Gimme a break!)
No. I just wanted to make a point. The above instructions are enough for anyone savvy to reproduce the case. If the prize changes, and becomes a $2,000 donation to the FSF, for example, I will release the code that I used. In the meantime, enjoy your own tests!

While I was testing, I saw that MySQL released 5.1.30-GA. Great!. I used 5.1.29 for my experiment. Had I known about the release, I would have waited two days. Anyway, if you believe TokuTek benchmarks, using 5.1 partitions will result in a performance 200 times better than the original. There must be something wrong in their method.