Thursday, 20 December 2007

MySQL User Conference Registration Open

The 2008 MySQL Conference & Expo registration is opened!


Time flies. We’re already at our sixth Users Conference! Looking at the announcement:



Co-presented by MySQL AB and O’Reilly Media, the conference will take place April 14-17, 2008, in Santa Clara, California. The event is expected to bring together over 1,600 open source and database users from some of the most exciting and fastest-growing companies in the world, as well as from the large and active MySQL Community. The program for 2008 will include keynote presentations by Jacek Becla of Stanford Linear Accelerator and MySQL CEO Marten Mickos.



Jay Pipes, our Program Chair, has lead a huge effort in identifying the best out of the near-300 proposals for sessions. And we’re not fully done yet. A few sessions still are in the grey zone, so not all are scheduled. But I suggest you take a look at the program already!


Join us 14-17 April 2008 in Santa Clara, California for the biggest and best MySQL event yet!


Register by 26 February 2008 to save up to $200.


Links:


Saturday, 15 December 2007

Navigating categories within my blog

With 130 entries in the "MySQL" category and no MySQL-related subcategories, my blog had become impossible to search and navigate easily.



And thus I created a number of new categories for the MySQL entries within my blog. They're listed in the left navigation bar, below the months, as well as below:



I hope this will make my blog more (re)usable.

(The picture is from this summer, when navigating the way up the Großvenediger, a 3662 m high mountain in the Hohe Tauern region of Austria.)

Thursday, 13 December 2007

MySQL Connector/ODBC 5.1 Released as Beta

Today MySQL releases the first beta version of MySQL Connector/ODBC 5.1. This means that Connector/ODBC 5.1 is now feature complete. Go download it!



Connector/ODBC 5.1 is suitable for most MySQL versions in active use today. It works with MySQL 4.1, 5.0, 5.1 and 6.0. If anyone is still on 4.0 or earlier releases, you should have upgraded ages ago; consider Connector/ODBC 5.1 "the final straw".

As I wrote in September when we released the alpha version,

It is a partial rewrite of the the original MyODBC 3.51 code base, originally developed by Monty and Venu Anuganti, including individual parts of the current 65K ODBC 5 code base.

The new ODBC 5.1 driver, like 3.51, supports all relevant platforms (including Win64). While ODBC mostly is relevant for Windows, we have some (but not many) Linux based ODBC users.

Overall, ODBC 3.51 (released as 3.51.20) is now in a better state than ever. The ODBC QA team has written several hundred new tests. Without doubt, the 3.51 code base is now a good fundament for the new ODBC 5.1 version.


We've added a MSI installer for Windows 64-bit and implemented a native Windows setup library. We've implemented most parts of the section in my September blog titled "Before ODBC 5.1 goes beta, you can expect the following functions", except sqlbigint and the native Mac OS X setup library. There is no binary package for Mac OS X on 64-bit PowerPC because Apple does not currently provide a 64-bit PowerPC version of iODBC. Also, the HP-UX 11.23 IA64 binary package does not include the GUI bits because of problems building Qt on that platform.

For more details on functionality added and bugs fixed, look for Jim Winstead's announcement later today.

Our usual warning: Keep in mind that this is a beta release, and as with any other pre-production release, caution should be taken when installing on production level systems or systems with critical data.

Thank you to all who've tested Connector/ODBC 5.1 so far! And, obviously, we welcome more testing.

References:

Documentation: MySQL Server Version Reference

Stefan Hinz, MySQL's Docs Team Lead, just showed me the new restructured documentation overview page http://dev.mysql.com/doc/. The intention of the restructuring is to make it easier for you to find the information you need.


We've amended the MySQL Reference Manual section with a subsection labeled "Excerpts from the Reference Manual", examples of which are a standalone Connectors book (covering all MySQL connectors and APIs) and guides for each individual MySQL Connector.

The key new document there is the "MySQL Server Version Reference" that should make life easier for everyone who needs cross-version information. It's available for download but you can also browse it online at http://dev.mysql.com/doc/mysqld-version-reference/en/.

Here's an overview of the MySQL Server Version Reference document:


  1. The mysqld Options/Variables Reference contains all MySQL server options and variables for all MySQL versions. You can easily find out whether or not a particular option is available in a specific MySQL version, when it was introduced or deprecated, and more. For variables, that chapter gives a quick overview of core properties, such as if it's a status or a server system variable, if I can be changed dynamically, or if the scope is global or local.

  2. The Reserved Words list contains all words reserved in a particular MySQL (major) version, with annotations about minor versions. The list is created by running a script against all MySQL versions starting from 4.1.0. (Yes, our Docs Team has installed all MySQL versions on their documentation machine!)

  3. The Functions and Operators chapter shows which MySQL functions are available in which version, and when they were introduced or deprecated.

  4. The Build (configure) Options chapter should be useful for anyone building MySQL from source.

  5. Eventually, the Key changes in MySQL releases chapter lists security fixes and incompatible changes for the GA versions of MySQL 4.1 and 5.0 and for the beta and RC versions of MySQL 5.1.



The Server Version Reference is not a static document but is being recreated on a regular basis.

Kudos go to Martin C. Brown who created all this. We hope it will make your work with MySQL more productive and enjoyable!

References:

Wednesday, 12 December 2007

Ruby on Rails 2.0 Released

Ruby on Rails 2.0 was released last week, "It's done!", David Heinemeier Hansson notes.



Ruby on Rails has come a long way since Lenz Grimmer's interview with DHH in February 2006.

Browsing DHH's blog, I find snippets (bolding mine) like


Piggy-backing off the new drive for resources are a number of simplifications for controller and view methods that deal with URLs.

As you might have gathered, Action Pack in Rails 2.0 is all about getting closer with HTTP and all its glory.

We’ve also made it much easier to structure your JavaScript and stylesheet files in logical units without getting clobbered by the HTTP overhead of requesting a bazillion files.

Making it even easier to create secure applications out of the box is always a pleasure and with Rails 2.0 we’re doing it from a number of fronts.

Figuring out where your bottlenecks are with real usage can be tough, but we just made it a whole lot easier with the new request profiler that can follow an entire usage script and report on the aggregate findings.




Seems to me like Rails 2.0 goes further still in the direction of the RoR tagline "Web development that doesn't hurt" and its expanded version


Ruby on Rails is an open-source web framework that's optimized for programmer happiness and sustainable productivity. It lets you write beautiful code by favoring convention over configuration.


Congratulations, DHH and the Rails Team!

References:

Monday, 10 December 2007

Lua lessons at MySQL University -- for MySQL Proxy

Giuseppe Maxia teased me last week to see whether I could decipher the notice on the Italian web site Punto Informatico about "MySQL University, lezione gratuita su Lua".



It's about Giuseppe teaching MySQL University attendees (for free as in both beer and speech) on the Lua scripting language. The MySQL reason to learn Lua is to do scripting in MySQL Proxy. Don't worry, the MySQL University session is in English, not Italian, and Giuseppe has deciphered the Italian himself in his blog.



You can attend MySQL University virtually with slides in PDF and Wiki, audio on ogg stream, and interaction with the lecturer via IRC (at Freenode under #mysql-university).

Timing: This Thursday 13 Dec 2007 at 6:00am PST = 9:00am EST = 15:00 CET = 16:00 EET = 17:00 Moscow time

Interested?


  1. Read Giuseppe's article "Getting started with MySQL Proxy"

  2. Register for the University session



References:

Combining MySQL Proxy with MySQL Cluster

A while ago, I had a discussion with Stewart Smith, Vinay Joosery, Monty Taylor and a number of other MySQLers who know much more about MySQL Cluster than I do. The result is a model for using MySQL Proxy to offload MySQL Cluster from doing Table Scans, without touching the application.



The discussion started from me asking Stewart about the largest road block for expanding the number of use cases for MySQL Cluster. "Oh, that would probably be doing JOINs and other SELECTs requiring the scanning of large parts of the database", he replied. "There, other storage engines are faster, such as MyISAM and InnoDB."

In a very simple view, the application talks SQL with MySQL Cluster, and gets responses.



Stewart's insight can be refined into the first simplistic diagram by adding the recognition that "SQL" can consist of

  1. UPDATE, INSERT, DELETE statements (very light, usually invidual rows affected) -- unidirectional blue arrow in the diagram below

  2. Simple SELECT statements (also very light, defined as SELECTs that use indices and return invidual rows) -- bidirectional black arrow

  3. Complex SELECT statements (could be as easy as "SELECT *", but defined as those not easily using indices and usually returning multiple rows) -- dashed arrows having two arrowheads to show that plenty of data is being returned



This second figure doesn't depict any change in application architecture from the first figure; it just shows a more granular view.

Now, enter the insight that plain MySQL Server (with MyISAM or InnoDB) can deliver the complex SELECTs faster.



In the new architecture represented by the above picture, we scale the application by

  1. introducing Replication (replicating MySQL Cluster to plain MySQL Server)

  2. changing the application to direct the complex SELECTs to MySQL Server instead of MySQL Cluster



This complicates life. Not only do we need to set up replication. We also need to touch the application all over the place, to direct queries to the appropriate server.

Now, enter MySQL Proxy.



Using LUA scripts, MySQL Proxy can relieve us of the second complication, i.e. having to change the application to point to different MySQL Servers depending on the type of the SQL. Let MySQL Proxy parse the traffic and direct it to the appropriate server! The application is left untouched, and the topmost part of the picture again has a simple bidirectional arrow saying "SQL". The distinction of what type of SQL we're talking about is left to MySQL Proxy.

MySQL Proxy can also be assigned to load balancing the queries to a number of MySQL Replication Slaves.



This picture clearly is the most complex architecture depicted here, but also represents the highest level of scaling.

All of the above can be done using current versions of MySQL Cluster and MySQL Server, and the MySQL Proxy.

Sources:

Symbian enlightened by LAMP

Followers of the Forum Nokia Developer Discussion Boards saw interesting pre-announcements last month:

Nokia has ported MySQL and PHP to Symbian.



With 165 million cumulative Symbian smartphones shipped since the formation of Symbian and currently 134 Symbian smartphone models commercially available, it would be interesting to see the effects on the 8,134 third-party Symbian applications commercially available by what Nokia calls "PAMP".

Availability is planned for next month, at the CCNC 2008 (Fifth IEEE Consumer Communications &Networking Conference, 10 - 12 January 2008 in Las Vegas, Nevada).

Johan Wikman of Nokia, who has already ported the Apache web server to Nokia S60, writes in a Forum Nokia thread (emphasis mine):


I can now reveal that we at the CCNC conference in Las Vegas in January, 2008 will demonstrate and release what we call the PAMP stack.

PAMP stands for Personal Apache, MySQL, PHP, so yes, the full LAMP stack will be made available for S60 smart phones. In addition, there will be PHP extension modules that provide access to the core functionality of the phone. And on top of PAMP you can basically install any LAMP based content management system. For instance, Drupal can be installed off the shelf.

Yes, a fair amount of memory is needed and it's still pretty experimental stuff, but it runs quite nicely on E90.

So, if you are in the neighbourhood, join us in Las Vegas :)

Johan


Richard Bloor comments in Symbian Blogs (again, emphasis mine):

At first sight this may seem like an example of porting just to prove it can be done. It does, however, have the potential to create a whole new way of looking at Web application. For example, I've recently been evaluating a number of GTD applications. Several of these use PHP and MySQL and it will be interesting to see if they work on PAMP.

As such PAMP could add yet another option for developers to create S60 applications and create a new paradigm for the "off-line" Web. Hopefully by January I'll have an S60 device with the memory to run this baby.


Sources:


Perhaps its time for me to upgrade my phone?

Monday, 3 December 2007

Falcon and other Feature Previews

MySQL has introduced Previews as a new way of delivering features for early testing by our community.

Previews are for testing of features, not of versions. The two previews we offer today are the Falcon Feature Preview based on 6.0, and the GIS Feature Preview based on 5.1. However, the version number as such is not intended by us to set any expectations on when the feature will be in a production version. For instance, Falcon is going into the 6.0 code base, but the GIS geographical data won't be in before 6.1 (although the preview is based on 5.1).

Previews are by definition a bit "rough around the edges". They have no inherent level of maturity -- they aren't pre-alpha, or alpha, or beta, or RC, or GA. They are a new way to do "release early & release often" for those who want to get things early, without MySQL tying itself into a promise on the specific delivery release.

Falcon 6.0.4 is available immediately in the preview release and soon will be part of a formal beta. If you want it immediately, go get the preview version!

Our purpose of having previews is to make new features available for those who want early access. That way, we hope for more community testing. For our users, Previews are meant for testing only, absolutely not something to use in production. It's a new way for us to release early & often for experimentation, learning, and testing.

Later on, I hope we will do other feature previews for storage engines or specific features.

Links:

Monday, 26 November 2007

Designing databases with MySQL Workbench

A week ago, Mike Zinner and his team released the beta version of MySQL Workbench on http://dev.mysql.com/workbench/. MySQL Workbench is a visual database design tool that is developed by MySQL. It is the successor application of the DBDesigner4 project.



There are two different editions of MySQL Workbench at this point in time - an open source edition and a standard edition that is only available for paying customers.

This means that MySQL Workbench introduces a new concept for MySQL. Until now, MySQL products have either been open source or only available through a commercial subscription (such as MySQL Enterprise Monitor).

MySQL Workbench is the first MySQL product that will be offered in several different editions. For those interested in the logic of this decision, please read Mike Zinner's post that may help you better understand. Mike compresses the difference as follows:

If you are a MySQL expert who has the knowledge and time to manually perform some steps, MySQL Workbench OSS will be the ideal choice for you. If you want to be able to do more in less time, you would like to have the additional safety-net or you simply want to give something back to the MySQL team - the Standard Edition is your logical choice.




The open source edition of MySQL Workbench is a fully featured, non-crippled database schema designer.. The open source edition is the code base the commercial edition is built on. It has the full feature set that is needed to efficiently design database schemata and is not crippled in any way. The commercial editions only adds modules to the open source edition to help the user save time.

The commercial edition of MySQL Workbench adds a number of advanced modules that help users to work more efficiently with the tool. These modules range from workflow optimizations to extended object handling. But you cannot buy it just yet. Commercial editions of MySQL Workbench will only be offered when they have reached production quality. The Standard Edition will be available on the MySQL online shop. Until then we welcome people joining the Beta Testing Program. If you are interested please email workbench at mysql dot com, and interact with the Workbench developers on the Workbench Forum and IRC channels.

Finally, we expect to be able to create a contributor community around MySQL Workbench. Regardless if you submit a patch for a bug or a new plugin to automate a given task - all users of the MySQL Workbench will benefit from your contribution. And in return you will benefit from the work that has done by other community members. You can start by reporting bugs and answer questions on the Workbench forum. And avid members of the MySQL Workbench community get a copy of the Standard Edition in return.

MySQL Workbench will be available on Windows, Linux and OS X. However, the first release will be on the Windows platform only. Support for other platforms will be added in a short time frame due to the clear separation of back-end and front-end code. Linux and OS X will benefit from the stabilized back-end code and are treated as first class platforms.

Condensed version: MySQL Workbench is a database design tool ready for you to download!

Saturday, 10 November 2007

Release Criteria: Aligning official documentation with reality

First of all: Thank you for your positive feedback on the MySQL 5.1 Errata Sheet!

While I never doubted that publishing the 5.1 Errata Sheet was the right thing to do, I had expected a more mixed feedback. It turned out the feedback was very grateful. So thank you for your encouragement

  • Kevin Burton for commenting in Jay's blog
    "This was a big win guys. Good work.We were going to deploy 5.1.x in out or production slave configurations just to test the reliability but this give us a lot more confidence in this release."


  • Baron Schwartz for commenting in my blog
    "Kaj, this is VERY helpful. I just discovered the character set issue with mysqldump myself last week.This is a big step forward in quality, from the user’s point of view. Now it’s much easier to see the state of things."


  • Peter Zaitsev for explaining some background in the blog entry "Making bugs public - good job MySQL"


The Errata Sheet had somewhat of a difficult birth process, as our pompously titled "Release Philosophy" page optimistically refers to the ideal situation of "no known critical bugs". If we follow that policy, there is no need for an errata sheet, is there? The Errata Sheet should by definition be empty.

But it isn't.

And it wouldn't have been, for the last four years, had we just had sufficient awareness of critical bugs, and a policy of publishing them.

The only logical conclusion from our Reality Check was that wishful thinking must stop, and we need to do the right thing under the circumstances: Get the best possible quality releases out with the given resources, and set the expectations of our customers and users according to what we can and will deliver.

And hence we appointed a cross-team working group, the "Release Criteria Committee", which is defining both external and internal processes and policies when it comes to

  • releasing new releases in general

  • moving from one maturity stage to another

  • dealing with bugs of all kinds

  • aligning MySQL release policy documentation with reality


So let's first look at the circumstances:

  • Resources are finite: There is only so much time in a day, and there are only so many developers around to fix bugs.

  • Timely releases have value to our users: Earlier access to bug fixes. Earlier access to new features. Earlier testing of both, i.e. faster convergence. Reduced guesswork about when the next release will be.

  • Zero critical bugs is impossible: Sure, zero bugs is desirable. And at times, we internally glorify MySQL's past, with "production level alpha releases". Those were the days, but those days also had less complex and less interwoven features, and fewer developers to manage.


As such, it is MySQL's responsibility to create a release policy that is aligned with the needs of the customers and the user base. This will involve a number of difficult judgement calls. However, we do think that we've got a lot to learn from our customers and users, when it comes to defining the best possible release criteria. So I am inviting your input.

To structure your thinking and the work, let me briefly explain some basics:

1. We have four maturity levels: Alpha, Beta, RC and GA.
1a. Alpha is for testing and development only. Features are added, and
syntax may change. (*)
1b. Beta means feature-freeze. We fix bugs. (*)
1c. RC or Release Candidate needs solid criteria. RC has some industry definitions, and is the final stage prior to GA.
1d. GA or General Availability means the version is for production use. The name GA is somewhat silly for Open Source products, as even alpha releases are available for the general public.
(*) These are my own simplifications, and our work group may refine them.

2. Bugs are described by their Severity, Defect Classification, Impact, Risk To Fix, and Effort. They may have even more dimensions.
2a. Severity describes the damage from the original user (bug reporter) point of view. The most severe bugs are about crashes, security problems and corruption, without any known workaround. The least severe refer to unpleasant but documented behaviour, i.e. feature requests. From the severity of the individual original reporter, we derive the Defect Classification, which attempts at being a neutral classification across our user base.
2b. Impact refers to how many users may be affected. They range from
the entire user base, all the way to corner cases that affect only users of a particular feature on one platform.
2c. Risk To Fix and Effort refer to the difficulty of fixing the bug. A bug fix may destabilise the code (i.e. introduce more problems than it fixes), requiring postponing the fix to a later version. And the time for fixing the bug may vary (influencing the complexity of reviews).

Some of my personal thinking on the release policy is as follows:

(i) Maturity changes need quality criteria.

  • Progressing from Alpha to Beta is not useful if most testers find the same elementary critical bugs; basic usability should exist

  • Progressing from Beta to RC needs for the impact of critical bugs to be very low

  • Progressing from RC to GA needs the impact of critical bugs to be even lower

  • Hence, I advocate quality based (not time based) release criteria for maturity changes


(ii) New releases within the same maturity level need different quality criteria (from above).

  • Regression bugs must be avoided (what worked in release n must still work in release n+1) (**)

  • Beyond that, we mainly need for the quality of release n+1 to be higher than that of release n. In practice, if you look at today's (10 Nov 2007) version of the Errata Sheet, you will see a lot of bugs annotated with "Already fixed in 5.1.23", while others have "Target fix: 5.1.23". "Let's not wait until we can cure all illnesses, before handing out the medicine we already have!" is my logic.

  • Hence, I advocate mostly time based (seldom quality based) release criteria for new releases within the same maturity level

  • (**) By "regression bugs" I mean unintentionally changed behaviour, i.e. pure bugs. There may be situations where behaviour has to be consciously changed to enable the fixing of a bug.


(iii) Simplicity rules.

  • The definitions of the dimensions that describe bugs must be as simple as possible.

  • The rules triggering maturity changes must be as simple as possible.

  • It should be easy at any point in time for us to evaluate whether we should change a maturity level (as opposed to being a negotiation between self-proclaimed advocates of quality and timeliness, respectively)


(iv) Timeliness has a value, and RC criteria may differ from GA criteria.

  • Some claim that by definition, an RC must fulfill all GA criteria except the exposure to wide user base testing

  • Others (like me) think that it's in the best interest of the user base to converge to a GA quicker, through marginally relaxed RC criteria, whereby we can fix the remaining less-critical bugs in parallel with the RC getting exposed to massive parallel testing by the user base


When we (the members of our Release Criteria Committee) think we're done, we'll upgrade our Release Policy (and, while we're at it, no longer pompously describe it as a Philosophy). We're working at a fairly hectic pace, and expect to be done within weeks rather than months.

In the meantime, your input is welcome!

Saturday, 20 October 2007

New Blog Layout

Our Web team has just installed the newest Wordpress MU edition for blogging, and Akismet for spam fighting -- plus a new layout. I'm already relieved from lots of spam, which feels great!

I'd like to make a couple of remarks on the banner, which has pictures by my two favourite portrait photographers, James Duncan Davidson and Julian Cash.



The two leftmost pictures are by Duncan, from panels at the last two MySQL Users Conferences. The first one is from the Clash of the DB Egos at the MySQL Conference and Expo 2007 (no, I am not chewing my nails or eating sweets -- I am blowing a whistle to keep the egos apart), and the second one from the 2006 conference panel interviewing the MySQL founders.

The rightmost one is by Julian, who regularly goes not only to Burning Man but also to Open Source conferences (such as ours 2006). Julian's passion for fisheye in turn has inspired myself to buy an 8mm fisheye myself.

Thank you, Duncan and Julian, for your great photography at various Open Source conferences, and for being a great inspiration for myself as an amateur photographer!

MySQL 5.1 errata sheet

As indicated last week, we have decided to start producing a condensed report of outstanding issues in our current development release, MySQL 5.1.



This list is now available as part of our reference manual at dev.mysql.com/doc/refman/5.1/en/open-bugs.html.

The list is updated daily based on the then-current status in our bug database.

The purpose of the list is to make it possible for our users to more easily make an informed judgement about whether our development release is ready for them to test and use. While the corresponding bugs have already been openly and publicly available for searching in our bug database, I hope this list is easier to navigate, as it is compiled into one web page that is sorted into categories such as


  • C API

  • Client

  • Connector/J

  • Server: Backup

  • Server: Charsets

  • Server: Cluster

  • Server: Events

  • Server: I_S (Information Schema)

  • Server: InnoDB

  • Server: Partition

  • Server: RBR (Row-based replication)

  • Server: Replication



Hence, if you're into using Row-based replication, read through the corresponding section. Our goal is to make it easy for you to know whether any of the outstanding bugs are relevant for you.

The list also contains target fix releases (with 5.1.23 currently being the next one), and links directly into the bugs system for detailed inspection of the bug description.

Friday, 19 October 2007

Country-specific MySQL Conferences

MySQL UC JapanLocal MySQL offices have hosted country-specific user or customer conferences in a number of countries this autumn:


and we're continuing this with two more conferences


after which the following big one is The Big One, i.e.


We just concluded yesterday's German Kundenkonferenz, with a packed (despite a local transport strike) main room in the Hilton. For me, the highlight was moderating the Storage Engine panel with Ralf Gebhardt, Jan Kneschke and Kai Voigt as vocal, knowledgeable and fun panelists in front of a very interactive audience. "With delight and a dash of horror" (adapted from a Swedish saying) I'm looking forward to moderating next week's equivalent French panel. I anticipate that the my preparations will not be limited to a casual look in my Swedish-French dictionary.

The London, Munich and Paris event are/were similar in nature, with content of both technical and decision-maker nature. The Milan launch is lighter on the technical side, mainly an opportunity to meet the team.

Finally, you've still got a good week, until 30 October 2007, to propose sessions for the 2008 UC in Santa Clara.

Wednesday, 17 October 2007

MySQL 5.1.22 is out



A good two weeks ago, MySQL 5.1.22 was released. MySQL 5.1 is a new release of MySQL Server, with new features like


  • partitioning (likely the top feature in 5.1)

  • events ("crontab" triggers in the database)

  • row-based replication

  • table logs

  • some XML functions



and also with major bug fixes, such as the AUTO_INCREMENT table-lock contention in InnoDB (fixed now in 5.1.22), as well as early indications of performance improvements of up to 20 % - 40 % on dual cores in some scenarios.

Jay Pipes has written an overview that compresses all of the pointers to 5.1 into one article, MySQL 5.1 Article Recap. I recommend you read it. I also recommend the manual section What's New in MySQL 5.1.

With 5.1.22, MySQL also changed the maturity state to "RC", Release Candidate. Looking at our own Support Policies, Release Candidate (aka Gamma) release is defined as follows:

Release Candidate binaries, also known as Gamma releases, are believed stable, having passed all of MySQL's internal testing, and with all known fatal runtime bugs fixed. However this release has not been in widespread use long enough to know for sure that all bugs have been identified.

However, we recognise that this particular RC does not fit the definition exactly. We still have some fatal runtime bugs left. We are producing an errata list of these, and expect to have the list ready and published on by 23 Oct 2007. We know that we should have published this list together with the RC itself, and we are now working as fast as we can to fix this.

We know we have thus released our RC too early according to our own standards, and we are doing our best to fix it. We apologise for any inconvenience/miscommunication and are working on improving our internal guidelines to ensure it doesn't happen again.

That said, 5.1.22 is a great release, one that we're proud of, and very likely worthy of your attention!

Thursday, 4 October 2007

Users Conference 2008 Call for Papers

This is a friendly reminder that we have opened our Call for Participation for the MySQL Users Conference 15-18 April 2008 in Santa Clara, California.



Yes, 15 April next year may seem far away. But 30 October this year isn't. And that's when the Call for Paper closes.

Some UC 2008 highlights:

  • Conference theme is "Harnessing the Power of MySQL"

  • We expect to bring over 1,500 open source and database users together

  • Over 100 sessions, geared toward many skill levels (novice to expert)



Consider to present, especially if you belong to one or more of the below categories:

  • Developer or DBA at an established or up-and-coming company

  • Business manager with purchasing authority

  • Strategist, business developer, CTO, CIO

  • Technology evangelist, scout, entrepreneur pushing their enterprise boundaries

  • Researcher, academic, programmer



To learn how to phrase your submission, click on the below picture and remember


  • Be creative! Conference participants want to hear about real-world scenarios using MySQL, about ways they can be more productive, or write better code. Please submit original session and tutorial ideas that focus on hands-on instruction and real-world examples.

  • Be descriptive! Include in your proposal as much detail about the planned presentation as possible. The more we know about what you plan to present, the better.

  • Be specific! Proposals which are vague or cover too much material are unlikely to be accepted. If you think your proposal covers too much of a topic, consider submitting two proposals which split the material into different sessions.



Monday, 1 October 2007

Free the falcons!

Last Sunday, just before the MySQL Dev Meeting in Heidelberg ended, we went to see a falconry. Strictly speaking, the German Raptor Research Centre isn't a falconry (which is a commercial venture for raising falcons), but a privately funded research station dedicated to the preservation of birds of prey, or "Eagles, vultures & co" as they say on their home page at deutsche-greifenwarte.de. Their actions over the past 30 years have given concrete results for many raptors, such as the eagle-owl ("Uhu" in German and Swedish), which is no longer on the list of endangered species in Europe. The German Raptor Research Centre has successfully set out 37 sea eagles (Seeadler, havsörn) in Central Europe, an admirable investment in nature which now including second and third generation offspring has grown to about 400 eagles.



Through a donation in part from MySQL GmbH, in part from MySQL AB, but mostly from individual MySQLers attending the Heidelberg Dev Mtg, Jim Starkey and I had the pleasure of handing over a cheque of 4500 euros to Bettina Fentzloff, the wife of the research centre's founder, Claus Fentzloff, who brought another falcon for the audience of MySQLers to admire. I hope the donation will enable the Deutsche Greifenwarte to give freedom (as in speech) to many birds of prey in Europe and worldwide!



Jim, as the creator of the Falcon engine, seemed to have a good hand also with the namesake birds.

Wednesday, 26 September 2007

MySQL Heidelberg Developer Mtg: Looking back

As noted already in March and described more closely in July, we had a MySQL Developer Meeting in Heidelberg, opened up for selected members of the MySQL community. Since yesterday, I'm back and reflecting upon how it all went.

The first reflection is that I'm biased, since I was organising the meeting together with above all Patrik Backman (for the agenda) and Georg Richter (for the lion's share of all real work, such as the coordination with our venue, Marriott Hotel). But I would still like to concur with the many MySQLers who think it was "the best MySQL Developers Meeting ever".

So what made the meeting a success?

We got plenty of work done. Our motto was "Working Together", making use of finally being together while going about our already defined goals. We had next to no overall one-to-many presentations (the type where each VP in turn preaches his favourite themes and people doze off waiting for the status report to end). Instead, each small development team met with other small development teams, based on detailed advance planning on which teams really need to meet. And we had plenty of seemingly random one-on-one corridor interactions, many of which were carefully pre-planned by goal-oriented meeting attendees.

We had lots of fun. We met in five-six different restaurants in the Old Town of Heidelberg. We had a great river boat cruise. We had a good team event seeing the Falconry close to Heidelberg. We went to the Kulturbrauerei to see how beer is brewed. And we concluded the team event by a superb evening in the old castle of Heidelberg. We ended that evening by some country-wise singing, started by the Ukraine and including the largest (employee-wise) countries of Germany, Russia, the US, Canada, Sweden, Finland, Australia, and Italy.

Our infrastructure was working. Heidelberg Marriot was our best-working hotel so far. Excellent food (including edible, healthy options, but not totally forgetting the mandatory junk). Internet worked all the time, even in the rooms. And this was no coincidence. Hey, this was Germany. And we have plenty of locals in Germany, and in particular, Georg met plenty of times with whomever needed to be seen locally from an organisation standpoint -- right in front of his doorstep. (No wonder Georg got standing ovations at our closing Gala Dinner at Heidelberg Castle.)

We improved our meeting practices. We followed through on some of our innovations from smaller team meetings (like the one last December in Berlin). Team Leads were in charge of the days being but to productive use -- with over 150 people, no single individual can ensure that time is used efficiently be all Developers. We insisted on good meeting preparations, and good meeting notes being talked. We spread the last-minute notes in outdoor roll-calls each morning, and in the daily Heidelberger Nachrichten ("Heidelberg Chronicle") handouts.

We had Team Exhibitions and MySQL University Sessions. Nearly all thought that the Team Exhibitions invigorated all of us -- where proud MySQLers demoed what they themselves had identified as Cool Stuff to be highlighted for their fellow MySQLers. And the twelve University Sessions spread the knowledge of the intimate details of how to code MySQL.

We had external guests, both customers and community members. Both categories gave us positive feedback for having been invited. We are deeply thankful for the input they gave us, Keeping It Real. But not only did they keep us real -- they also made us a lot more polite and courteous than at some previous internal meetings. I heard very few negative comments or raised voices. Concerns were not wiped under the rug, but they were raised in a very constructive manner.

Thank you to all participants, external and internal, which made this event possible!

Tuesday, 11 September 2007

Ruby creator Yukihiro "Matz" Matsumoto at MySQL UC Japan

One of the most appreciated keynotes at the ongoing Japanese MySQL Users Conference was by Yukihiro "Matz" Matsumoto, the creator of the Ruby language.

Me and Matz at MySQL UC Japan Day 1 2007-09-11

Matz was preceded by an impressive set of keynoters, moderated by MySQL K.K. President Larry Stefonic, also Senior Vice President of Asia Pacific at MySQL AB :

  • His Excellency Mr. Stefan Noreén, Ambassador of Sweden to Japan

  • MÃ¥rten Mickos, CEO, MySQL AB

  • Masahiko Yoshida-san, Director, Hewlett-Packard Japan, Ltd.

  • Kenji Mukai-san, Vice President, General Manager, IT Architecture Solutions Unit, Sumisho Computer Systems Corporation

  • Hirokazu Seto-san, Senior Manager, Alliance & Solutions, Marketing & Operations, Dell Inc.



Matz carries the title of Fellow at Network Applied Communication Laboratory Ltd. For the predominantly business oriented audience, Matz described the history of Free Software, of Open Source, and of Ruby, mentioning the mandatory names: Richard Stallman, Eric Raymond, Linus Torvalds, and, for Ruby on Rails, David Heinemeier Hansson. With Linus, Matz shares the original motivation for starting their respective now-famous Free Software project: Fun!

The presentation was a great success and even Matz's jokes were translated into English, creating a time-delayed wave of laughter in the non-Japanese part of the audience. I wonder whether my own afternoon session in English will be equally well treated, when translated into Japanese.

Re-engineered ODBC 5.1 driver for MySQL

MySQL is today releasing a new ODBC driver, under the version number ODBC 5.1. It is a partial rewrite of the the original MyODBC 3.51 code base, originally developed by Monty and Venu Anuganti, including individual parts of the current 65K ODBC 5 code base. It is designed to work with all MySQL versions starting with MySQL 4.1.



The original ODBC 3.51 driver was incomplete and admittedly somewhat of a quick hack, containing some debatable code. For that reason, Connector/ODBC 5 started as a complete rewrite. After 2 years it had 65 KLOC, but no tests and was in part overdesigned. That project was started but never finished, and had some deep Qt dependencies.

In the meantime, the original ODBC 3.51 has had 6 new releases and over 200 bugs fixed. As for ODBC 5.1, debatable parts of the code have been rewritten. The new ODBC 5.1 driver thus shares almost solely its major version number with the old ODBC 5 driver, which never went past Beta stage. The development of the old ODBC 5 driver has now stopped.

The new ODBC 5.1 driver, like 3.51, supports all relevant platforms (including Win64). While ODBC mostly is relevant for Windows, we have some (but not many) Linux based ODBC users.

Overall, ODBC 3.51 (released as 3.51.20) is now in a better state than ever. The ODBC QA team has written several hundred new tests. Without doubt, the 3.51 code base is now a good fundament for the new ODBC 5.1 version.

ODBC 3.51 supports a subset of the ODBC Core interface, plus some Level 1 and Level 2 features. ODBC 5.1 should be a complete implementation of the ODBC Core interface, plus more Level 1 and 2 features.

We aren't going to reach complete Level 1 conformance because we won't be supporting asynchronous execution. and we won't support SQLProcedureColumns() as we at this point cannot get stored procedure parameter info from the server.

The stumbling blocks for full level 2 conformance are in/out and out parameters, and bookmarks, related to MySQL Server.

New features include Unicode support, and descriptors for connectors and metadata as required by Level 3. The following features are already supported in ODBC 5.1.0

  • Support for SQL_C_WCHAR.

  • Support for Unicode functions (SQLConnectW, etc.)

  • Descriptor support (SQLGetDescField, SQLGetDescRec, etc.)


Before ODBC 5.1 goes beta, you can expect the following functions


MySQL has a number of beta customers identified by the Support team. They represent user bases for which Unicode is interesting. From the Community of MySQL users, we are looking for testers who can expand upon the quality assurance done inside MySQL, which is based on automated tests.

Especially, we're looking for beta testers who work with closed source applications like Microsoft Access, PowerBuilder, and Borland tools. Testers should contact Lenz Grimmer or Kaj Arnö in the MySQL community team (at firstname@mysql.com), who will establish the contact between the ODBC team and the beta testers.

Thursday, 9 August 2007

Communication Challenges for the MySQL Community Team

With the my blog entry yesterday on Refining MySQL Community Server, did anything really change? If so, was it for the better or for worse for the community?

My answer is "not much", and "for the better", but feel free to disagree.

Judging from some reactions in the community, a lot changed, and for the worse.



Strong words.

So let me expand on the "not much" part:

The negative reactions that I experienced on the #mysql-dev channel in the Freenode IRC chat stem (like Mike Kruckenberg's blog) from the removal of the source code of MySQL Enterprise Server from ftp.mysql.com. I argue that while this may feel or appear like a step away from Open Source, the real effect on the core MySQL community member is minuscule. And this is by design. We don't intend for the change to adversely affect core MySQL community members' usage. What we do intend is related to positioning: MySQL Community Server is for our users, MySQL Enterprise Server is for our paying customers. We want people to associate MySQL Enterprise Server with a commercial relationship to MySQL as a company.

Hence, we thought about what we could do to encourage the use of MySQL Community Server, in particular by the Linux distributions. Part of that is discouraging the use of MySQL Enterprise Server, and that's why we set a signal by removing the source code of the Enterprise incarnation of the MySQL Server code.

But I want to underline that we also took a number of positive measures -- carrots, if you will -- which brings me to the "for the better" part.

First and foremost: We stopped adding new features to a GA release. No more new patches to 5.0. Adding new features to 5.0 is the "that" that I was above all referring to in IRC when saying (as now quoted by Jeremy)
<kaj> JeremyC: Our past 10 months since Oct has been a struggle to make that work, and we’ve failed


Having a stable release which gets new features is like squaring the circle. It's not doable in Euclidean geometry. This is also where I feel Jeremy's pain. There are plenty of reasons to want to have new functionality into a stable release, without the burden of the destabilisation introduced by other new functionality.

As I understand Jeremy's non-Euclidean solution (and there are other geometries than that of Euclid), he wants to pick and choose individual patches, and build an abundance of binaries with a free combination of those patches. While that may be technically feasible, it gives MySQL Community Server a different role from the one intended by MySQL AB. MySQL AB gave two mandates to MySQL Community Server: being experimental, and being the main one used by the MySQL user base. The mandate of MySQL Enterprise Server was the opposite: being stable, and being the one used by customers entering a commercial relationship with MySQL. That equation is the other "that" which hasn't worked in the past ten months at MySQL AB.

Now, yesterday's announcement recognised that these requirements are conflicting. That's where we failed. The community doesn't want to merely experiment, it wants to use a stable version. And the changes announced yesterday are about making Community Server stable. On top of that, we increase the predictability and planned frequency of Source releases (which we do to encourage Distributions to use MySQL Community Server). Jeremy correctly points out that the frequency doesn't change much compared to our actual track record on release frequency during the last 6-7-8 months. Personally, I wouldn't necessarily count it to our disadvantage that we've released more frequently than planned (which was twice a year). The more-frequent-than-planned releases is another indicator of the failure of the double mandate ("experimental" plus "main version"), that yesterday's announcements are supposed to improve.

Another solution, which I suppose would have pleased Jeremy, is to fix the situation by removing the mandate of MySQL Enterprise Server to be associated with a commercial relationship with MySQL. Then we would have one stable and one experimental variety of MySQL Server, regardless of the dimension of paying vs. non-paying. That would dissociate the brand of "MySQL Enterprise Server" with paying-only. And different branding of servers, in turn, is one of the basic current assumptions of Community versus Enterprise. I do understand that this distinction is not one that the Community particularly cares about. However, indirectly, a successful commercial company behind MySQL fuels the virtuous circle from which the community benefits in the form of new GPL features developed by MySQL AB.

Speaking of GPL: I want to make it perfectly clear that we have no intentions of moving MySQL Enterprise Server to another license. Yes, we are evaluating GPLv3 instead of GPLv2, but our plan is for both Community Server and Enterprise Server to remain GPL.

So in summary, I do say "not much" has changed. We have recognised some impossible requirements on the Community Server, and made the difference between Enterprise Server and Community Server consist mainly of the packaging (release schedule and availability). Same features, same bug fixes, different schedule, different branding.

I hope to now focus on making the best out of both Enterprise and Community server, feeding the virtuous circle of open source software.

Wednesday, 8 August 2007

Refining MySQL Community Server

Back in October 2006, we introduced MySQL Community Server. Since then, we've learnt a thing or two, spent many man hours discussing how to improve our processes, and are now refining the concept. We feel that we've come up with some good middle-ground that fulfils not only our company interests but fosters community use and growth as well.



The changes are in the areas of release policy and stability of MySQL Community Server and in the availability of MySQL Enterprise Server.

The changes start from the question: "How can we better target MySQL Community Server to the community and MySQL Enterprise Server to the paying customers?". Many of them originate from our ongoing discussions with the Linux Distributions, some of whom have been distributing MySQL Enterprise Server to their user base, since MySQL Community Server hasn't conformed to their needs of feature stability and release schedule.

Our intention is for MySQL Community Server to be very good, and for MySQL Enterprise Server to provide further value on top of that. The five changes, in short, are:



  1. New features and community contributions will go into the next development tree. The new features will not be applied to a current GA release, ensuring stability for the Community Server. At the time of writing, the development tree is MySQL 5.2.


  2. There will be at least two yearly “mature GA” (currently MySQL 5.0) binary builds. They aren't scheduled, but usually triggered by grave security vulnerabilities.


  3. When a version of MySQL initially goes GA (as 5.1 soon will), the company will release binary builds of the new GA product every month for a period of several months until it reaches a point of suitable stability/maturity to be considered a "mature GA" release -- as described above.


  4. There will be four yearly “mature GA” (currently MySQL 5.0) source releases, predictably scheduled, to be released once every quarter. These will be ideal for use by distributions shipping MySQL.


  5. The current Enterprise source tarballs will be removed from ftp.mysql.com. These will move to enterprise.mysql.com, and will be available for our paying subscribers only.



Let me expand upon the first and the last items.

Community contributions: As I know this is a topic important to our contributors and potential contributors, let me provide an example. I confess: We've been bad at incorporating new features. We haven't entered much more significant than SHOW PROFILE, and it took many months and several builds to mature. And we cannot continue with this practice of applying patches to a stable release, because new features will destabilise the release. That's what we've been hearing from both distributions and contributors. We're listening, and we're learning. By no longer going for the impossible (adding features to a feature frozen release), we hope to change this dynamic for the better.

Enterprise source code availability: I expect three questions, "Why?", "Does this conform with the GPL?" and "What keeps a paying customer from building and posting Enterprise binaries?". The rationale is to underline the positioning goal of "Community Server for community users, Enterprise Server for paying users". And the GPL requires us (like anybody else) to hand out the code to those whom we give the binaries, which in the case of MySQL Enterprise Server is just the customers. So it does conform to the GPL, something that we've verified with the FSF to eliminate any doubt. And as for the third expected question, the answer is "Nothing". Still, we feel that most business users will see the value of a MySQL Enterprise subscription that offers regularly-reliable software updates directly from the 'source', along with premium 24x7 technical support and proactive monitoring/advisory tools.

To finish off, let me repeat a number of basic facts which stay unchanged:


  1. both the Enterprise Server and the Community Server remain licensed under the GPLv2


  2. MySQL Enterprise Server has Monthly Rapid Updates (MRUs) and Quarterly Service Packs (QSPs), i.e. binaries delivered to our customers; building these binaries is a service for paying customers only


  3. bugs are first fixed in MySQL Enterprise Server builds, and while updates are delivered quicker and more frequently to paying customers, bug fixes are also added to the next source and binary builds of MySQL Community Server as stated above


  4. when scheduled, we build Community binaries for all platforms


  5. community binaries and sources are still available for download at dev.mysql.com/downloads


  6. all source trees are still available via BitKeeper


  7. the Enterprise Server is close to a full subset of the Community Server



With these changes that are intended to simplify life for the Linux Distributions that distribute MySQL, we hope to better serve the MySQL user base, while continuing to provide additional value to our paying customers in the form of more frequent, scheduled binary bug fix releases.

If you have any questions with regards to this, please do not hesitate to reply via email to community@mysql.com.

Saturday, 28 July 2007

Microsoft, MySQL and the skewed WAMP

In his OSCON keynote last Thursday (26 July 2007), Bill Hilf of Microsoft described his perspective of the "two steps forward, one step backward" relationship between Microsoft and the world of Open Source. Amongst the steps forward, I count the establishment of a web site microsoft.com/opensource, releasing some Microsoft software under an OSI compliant license, and -- specifically to MySQL -- the mentioning of the MySQL Connector for Visual Studio in his OSCON presentation.



Microsoft seems to be acknowledging that there is a heterogeneous world out there. It's not "all proprietary" versus "all Open Source". The community mixes and matches at will.

Many might think there are few commonalities between MySQL and Microsoft. But there are more than you'd think, and they're not limited to the common first letter of the alphabet.

The biggest commonality is WAMP. A simple real-world observation is that plenty of the organisations deploying LAMP applications, have developed them in whole or in part under WAMP.

MySQL doesn't have a platform agenda, so we want to make WAMP easy to use. If WAMP turns out to be good enough not just for development, but also for deployment, great! And there MySQL and Microsoft have a common interest: Making WAMP attractive enough for production deployments, as opposed to being a mere playing ground for LAMP as The Real Thing.

So while you can expect MySQL to continue to want to excel as the M in LAMP, and while I suspect it's a safe bet to expect Microsoft to continue to have a platform agenda, I don't think anybody should expect a dogmatic, black and white, all-or-nothing world. In reality, customers and users mix and match, and it's about time we all acknowledge that.

Bill Hilf made two points in his OSCON keynote that I'd like to comment upon as for how they apply to MySQL. In both cases, I'll conclude with a plea to contact me, if the thinking applies to you.

First, he pointed out the order-of-magnitude technical improvements that could very simply be made to the interface between IIS and PHP, once key IIS developers were connecting with key PHP developers. Such opportunities for improvements exist only if the products have been living in complete isolation from each other. And that's not the case with MySQL and Windows, which have peacefully coexisted since last century. This means that the technical fruits to be picked when making WAMP more attractive won't hang quite that low. But we do know that there are technical hurdles to overcome, and documentation to be improved, specifically when it relates to deployment of WAMP applications. Should you be one of the casualties thereof, I'd love to hear from you about how we should make WAMP more attractive. Email me at firstname@mysql.com!

Second, he pointed to the fear of the unknown, to the extent Linux is unknown to the Microsoft user base. "Can I go visit a Linux website?" was one of the questions he had been asked, not tongue-in-cheek, but for real. My take on this is that there is a perceived divide "Microsoft vs. Open Source" bigger than the divide out in the real world of IT. And I don't think it's in the interest of the user base to be lead to believe that solutions have to be fundamentalist. Paradoxically, I don't even think it's in Microsoft's interest. I think the world's perception of MySQL's user base may be skewed towards LAMP rather than WAMP, not just because L, A, M and P are all Open Source as opposed to W, but also because of a relative lack of acknowledgement for WAMP as a development (and deployment) stack. Should you have deployed large WAMP apps, I'd love to hear from you about WAMP specific success stories. Email me at firstname@mysql.com!

I can see history repeating itself in strange ways. In the early days, many picked MySQL "for development and testing only", fully convinced to deploy on other databases (and that was definitely the case with myself in the 1990s). But the "testing" phase dragged on and on, and in many cases MySQL turned out to be "good enough" for deployment.

That's how MySQL grew.

With proper attention from MySQL, Microsoft and our respective communities, perhaps WAMP in a growing number of scenarios can enjoy the same benefit of being "good enough for deployment"?

Friday, 27 July 2007

OSCON Lightning Talk: "State of the Dolphin"

Today at OSCON, MySQL co-founder Michael "Monty" Widenius and I presented the "State of the Dolphin" lightning talk.



My slides for this preso weren't too graphic, which makes them all the easier to reuse in this blog:

Use our new software!



  • Use MySQL 5.1, it‘s soon going RC

  • Use Falcon, it‘s soon going Beta (new transactional storage engine, faster than InnoDB on large servers)

  • Use MySQL Workbench (ER Tool), Now Beta

  • Use MySQL Proxy, just released

  • PHPers: Use mysqlnd (Native Driver)



Go test MySQL 5.1!



  • We‘re happy with the quality

    • More stable than 5.0 was four months after GA

    • RC happening very soon, GA within a few versions after that

    • A better MySQL 5.0 (thousands of small fixes)

  • We‘re happy with the new functionality

    • Table / Index Partitioning

    • Row-based replication – Transfers data instead of commands

    • Full-text indexing parser plugins – Flexible full text search

    • Disk-based Data Support for MySQL Cluster

    • Replication Support for MySQL Cluster

    • XPath Support - helps any customer wanting to better navigate and search XML documents stored in MySQL

    • Internal Task Scheduler (Events)




Other goodies coming soon (5.2, 6.0, ...)



  • Global Backup API

  • Falcon and Maria (MyISAM++) storage engine

  • Further new storage engines

  • Hash & Merge joins (faster subselects)

  • Federated tables over ODBC

  • Foreign key support for all engines




Participate in our Development!



  • Report bugs! Test them! Submit patches!

  • Hang out on Freenode IRC #mysql-dev

  • Subscribe to commits@lists.mysql.com to see our code reviews

  • Attend MySQL University, the foremost education for MySQL developers (of C/C++ code, not apps)

  • MySQL Forge: List your MySQL apps! Upload your code snippets! Fix the missing documents!

  • Go visit MySQL Forge Worklog

    • Voting for best features starting soon



Hiring!



We are hiring outstanding C/C++ developers with systems or database engineering experience.
Visit www.mysql.com/jobs or email resume to jpugh@mysql.com

Clash of the DB Egos: Seen the show? Read the article!

A good three months ago, I had the pleasure of being the referee for seven DB egos. Now, the match has appeared as a Dev Zone article:

One of the most popular keynotes of the MySQL Conference & Expo 2007 was called "The Clash of the DB Egos". It was a fight amongst seven database luminaries, all playing an important role either within MySQL AB or as providers of Storage Engines that work closely with MySQL. This article attempts at giving a picture of what the fight was about, through reciting the egos and the questions posed to them by the referee.

Saturday, 7 July 2007

OSCON Thu 26.7.2007: "The MySQL Architecture of Participation"

Coming to OSCON at the end of the month?



If so, I'd be happy to see you on Thursday 26 July 2007 5:20pm - 6:05pm in F150, where I will be talking about The MySQL Architecture of Participation.

Quoting from the O'Reilly session page:

While MySQL has been FOSS from the start and GPL since 2000, the development process at MySQL has been fairly concentrated around the company itself. One of the corporate goals for 2007 is to open up our entire development model:


  • ensuring that our already-opened tools are open enough (our bugs system at bugs.mysql.com, and our source control system with BitKeeper)

  • opening Worklog, our detailed roadmap and specifications, for commenting on MySQL Forge

  • opening internal documentation from our closed Wiki onto MySQL Forge Wiki

  • opening our internal chat sessions, moving them from our closed IRC server to Freenode



which all share the commonality of lowering the threshold for external developers to contribute to MySQL.

In this session, Arnö shares the experiences so far, concentrating on building up the network of connections between internal and external developers.

Heidelberg Dev Mtg for Community: Thu-Fri 20-21 Sep 2007

As noted already in March, we have decided to open up the MySQL Developer Meeting for selected members of the MySQL User Community, i.e. for MySQL users who have a need of interacting with our developers. In Sorrento 2006, Prag 2005 or Malta 2004, we had similar developer meetings -- but the external representation amounted to one (1) customer presentation.

In Heidelberg (one bus shuttle hour from Frankfurt airport), we'll do things differently.

Some key points:


  1. Community Days are concentrated to Thursday 20.9.2007 and Friday 21.9.2007: While we have very few sessions closed for the community even on the other days (Wed 21.9, Sat 22.9, Mon 24.9), our scheduling started from the insight that few community members can afford to stay for too many days. So meetings that are of most relevance for Community generally take place on Thu and Fri.

  2. MySQL University sessions are scheduled for the same Thursday and Friday. That's where we spread our accumulated knowledge on how to develop MySQL internally amongst those who code MySQL, the Connectors, and Tools. Invited community members are welcome to attend physically, and anyone can participate virtually at no cost.

  3. Team Exhibitions are a new form of gathering. MySQL engineering teams invite their fellow teams to celebrate success by showing what they themselves think is cool stuff -- already in production, or in beta, or just in alpha. This doesn't have to be flashy user interfaces. For instance, the Optimiser team may choose to demo a query that runs 75 % faster in 5.1 than in 5.0. We'll make sure that our non-attending Community members gets to hear about this through MySQL Forge. And naturally, any external community members are most welcome to attend the Team Exhibitions first hand, both to get an impression of what's happening, and to give us feedback.

  4. Every attending non-MySQLer will get two hosts: one Community Team Host and one Engineering Team Host. The Community Team Host is either Jay, Lenz, Colin, Giuseppe, David or myself (Lenz will tell you know who is your host), and we make sure you know where and when the meetings are, we introduce you to the proper Engineering team members, and try to ensure your expectations are met. The Engineering Team Host is whichever MySQL developer who is the domain expert on your topic. Meeting with this developer is probably the reason why you came to Heidelberg in the first place, but do take into account that such devs are likely to be busy with many meetings in Heidelberg.

  5. The detailed agenda is still under works. During the course of July, we expect to be able to publish the MySQL University schedule, as well as the tentative Team Exhibition schedule. Individual community members should also get a picture of what meetings they are likely to be interested in, through interaction at least with their Community Team hosts and in some cases even with their Engineering Team hosts.



External participants should expect to be firming up their travel schedule within the next week or two. This involves getting a firm booking from the Heidelberg Marriott Hotel (along Neckar River at Vangerowstrasse, a short distance from Heidelberg's Old Town), which is both our meeting venue and the place where nearly all MySQLers stay. Expect to be contacted by Lenz Grimmer, our Community Manager EMEA (or contact him yourself at firstname@mysql.com, if you want to be proactive). Lenz is keeping a list of the arrival and departure dates and times of our guests.

mysqlnd (the MySQL native driver for PHP) needs testers and benchmarkers

MySQL welcomes external verification by high-volume LAMP websites of the performance improvements gained by replacing libmysql with mysqlnd.

To recap some basics from the mysqlnd download page at http://dev.mysql.com/downloads/connector/php-mysqlnd/:


The MySQL native driver for PHP is an additional, alternative way to connect from PHP 5 and PHP 6 to the MySQL Server 4.1 or newer. It is a replacement for the libmysql, the MySQL Client Library. From now on you can use ext/mysqli either together with libmysql as you did in the past or with mysqlnd.

We have no plans to remove libmysql support from ext/mysqli, which would break existing applications. We just add a new, superior alternative to our PHP offerings.

For development discussion and bug reporting a new mailing list has been created php@lists.mysql.com. See also http://lists.mysql.com/.


Georg Richter has now sent the PHP extensions for mysqlnd to the PHP Core Team for reviewing. The interfaces to PHP may then be put directly into the PHP CVS repository, whereas mysqlnd remains to be downloaded from MySQL's download site.

The code base of mysqlnd has been synchronised with the PHP CVS repository of php.net. While mysqlnd was originally designed for PHP 6, Andrey Hristov has now finished the backport of mysqlnd to PHP 5 and added support for ext/mysql. Given that 60 % of the PHP user base is still on PHP 4, some heavy PHP users would like to see the backporting done all the way to PHP 4. We are evaluating that, but have no immediate plans for such a backport.

Our next ambition is to prove that mysqlnd is stable and meets the need of the most demanding PHP users. And that is where we need help from the user community. We are looking for testers among PHP users with a high server load, ideally willing to do benchmarks.

Typically, mysqlnd is at least as fast as libmysql. We are writing a MySQL Technical Whitepaper with further details on the performance of mysqlnd, which will be published soon. The document confirms previously published results for the Dell DVD Store, which show ext/mysqli with mysqlnd to be about 5% faster than libmysql. Now, we would like an external verification of our performance improvements, but a small web site is not proof enough, as differences will show up only under high load. We are curious to learn how big the overall speed-up is for a top web site, given that the time spent in the database interface is only a fraction of the overall web page generation time.

Our team has increased their Quality Assurance efforts to further stabilise mysqlnd. Code Coverage analysis with GCOV shows that our tests cover 88% of the underlying ext/mysql C-code, 93% of ext/mysqli and 83% of mysqlnd. We are proud of our improved figures. If you want to compare the values to those of PHP 4.4 and 5.2, check out http://gcov.php.net/.

Due to the help of Kent Boortz in the MySQL Build Team, we are building mysqlnd routinely on 28 platforms. We have run our test suite against more than 20 MySQL Server versions. Still, the positive lab tests need to be confirmed by field usage, quite as the performance figures.

So if you run a high load web site based on PHP and MySQL,

(i) read up on how to get mysqlnd, how to install it, and on the features & limitations as well as the FAQ on the download page

(ii) subscribe to php@lists.mysql.com at http://lists.mysql.com/

(iii) talk to Jay Pipes (at firstname@mysql.com) or Ulf Wendel (at firstname.lastname@mysql.com)! Or, better still, catch them on IRC (#mysql-dev on Freenode).

Friday, 29 June 2007

MySQL Congratulates FSF on GPLv3

We congratulate the Free Software Foundation on the release of GPLv3 and offer our thanks to the many individuals in the open source community who participated in the process of drafting the license.

It's good to see overall improvements in GPLv3 over GPLv2, when it comes to compatibility with other Free/Open Source Software licenses, to the compatibility with other legislations than the US legal system, and to strengthened incompatibility with Software Patents. I am also happy if the work of the Committee B ends up contributing to a better adoption of GPLv3. I am in awe as to the patience and skillful diplomacy with which Eben Moglen could tame the group consisting of everything from techies from comparatively small companies (like Trolltech and ourselves) to the seniormost lawyers from the biggest Fortune 500 companies.

MySQL will continue to monitor the industry's reaction and adoption of the new license, as we decide the best overall course for our community, our company, our users and our customers.

Tuesday, 26 June 2007

Forum Improvement Suggestions by Top Posters

Earlier this month, Lenz Grimmer and I approached the top Forum posters on our Forums (http://forums.mysql.com), thanking them for their contributions and soliciting them for feedback on how we could improve the Forums to make them more usable and fun to use. Especially, we asked for suggestions in these areas:


  • for easier navigation / threading / search

  • for ranking / highlighting / voting on entries

  • for showing user profiles / avatars / case studies of contributors



Of the 24 people we approached, fifteen replied. Out of these, thirteen came back with suggestions, some of which were very detailed proposals.

Here is a condensed summary of the most popular requests/suggestions:


  1. Re-enabling the display of new posts per forum on the front page was the most requested change by all.

  2. The Forum search function received a lot of criticism for being almost unusable and rarely returning relevant results. It should be possible to provide more search criteria (e.g. selecting Forums to search) and the results should highlight the selected keywords.

  3. Being able to rank/vote replies was considered a useful addition by several, also a ranking of individuals was requested, to indicate competence of posters.

  4. Many were asking for a feature to better keep track of the threads that they have contributed to, e.g. an option to see all of the forum posts that they have posted and replied to or marking threads differently if they include postings from oneself.

  5. Some requested improvements in the displaying of threads, e.g. being able to open/collapse subthreads/nodes, or moving threads to the top of the list when new posts have arrived.

  6. Some improvements for the user profile pages were requested, e.g. a list of recent postings, number of messages posted, most active forums, link to the bug database (reported bugs) or more personal information.

  7. Avatars were generally frowned upon, as being too childish and adding unnecessary overhead.



A big Thank You for good suggestions go to

  • Felix Geerinckx

  • Peter Brawley

  • Marc Castrovinci

  • Martin Sivorn

  • Anthony Willard

  • William Chiquito

  • Sam Morrison

  • Thomas Corbiere

  • Kathy Mazur Worden

  • Brian Papantonio

  • Björn Steinbrink

  • Harry Wang

  • Dave "Guelphdad" Lake


Oh, and by the way: Our Web Team is swamped with requests. So don't expect to see the good suggestions come to fruition too soon. This is just for you to know that we're listening, and we're planning some of the above improvements to take place. If you have further suggestions, email Lenz or myself at firstname@mysql.com.