Repos and Distros: Upstream and Downstream

When we launched the official MySQL repos back in October, we wanted to achieve a number of things:

First, we closed a gaping hole in our distribution on Linux. Some Linux distros do not ship all the MySQL products, and not everyone is able to or always wants to use the distro packages. And the distros naturally focus on stable releases, while some users want to test development releases from us. Second, this represented a good opportunity to get off our behinds and finally modernize our Linux packaging. And last, but definitely not least, we would put ourselves in the same position as those valuable volunteers that take our products and integrate and maintain them in the Linux distros.

When you’re upstream, you can easily lapse into a mode where you stop listening properly to those who sit downstream and have to process what you’re releasing. There will always be some amount of difference in priorities and opinions between upstream and downstream, but putting ourselves downstream and in that sense having to “eat our own dogfood” turned out to be very educational and highly useful.

This change in perspective made it easier for us to understand many of the important pain points of the distros. It quickly became clear to us that a good deal of those pain points could be fairly easily addressed, and in some of the more difficult cases, the folks involved in the repo project have acted as lobbyists internally in order to have Engineering priorities changed so we could fix some of the bigger things.

Below, I will go into some more detail on what we are doing and what we have accomplished in the Debian and Ubuntu communities, let me first just say that we hope that we are being considered useful citizens in the different distro communities, and that distro maintainers of MySQL products see us an approachable and responsive upstream. And if we aren’t, let us know!

Debian and Ubuntu

Over the last couple of weeks, we have seen our new experience and efforts bearing fruit in Debian and Ubuntu. Since late fall last year, release engineers and developers from the MySQL team have been working with the Debian and Ubuntu community to bring MySQL 5.6 into both distros. Two weeks ago, Ubuntu bug #1256931 (“please include MySQL 5.6″) got closed, and as you can see from the package status, 5.6 is now headed for Ubuntu 14.04 LTS, Trusty Tahr.

Hot on the heels of the Ubuntu news came the inclusion of 5.6 in Debian experimental. We do believe that the current packages are quite solid, and we will continue to work closely with the Debian community to deal with any reported issues and in general refine things so that 5.6 can go into Debian testing soon.

Working with the Debian and Ubuntu folks has been an absolute pleasure, and at the risk of leaving out someone, I will say a heartfelt thanks to Robie Basak, Björn Boschman, Clint Byrum, Dustin Kirkland and James Page, and to our very own Akhil Mohan and Norvald Ryeng for their hard work.

Maintenance and Updates

And now that we have 5.6 in Ubuntu, how will it get maintained there? MySQL maintenance releases come out on a bi-monthly schedule, containing fixes for important bugs reported by the community, by customers and by our own developers. This is not a perfect match for Linux distros, where the approach has historically been to select a specific MySQL release to go into a new major release of the distro. Subsequent maintenance has largely consisted of the distro maintainer fixing reported issues or security bugs by either backporting from later MySQL releases or implementing their own fixes.

There are many good reasons for this approach, and a key one is trust. If a distro maintainer were to update the distro packages more or less in sync with upstream releases (for MySQL, “upstream” is us), he or she would have to be able to trust that upstream doesn’t invite trouble by introducing risky bug fixes or new features in maintenance releases, that upgrades and forward compatibility don’t break, and that upstream in general has development and QA processes in place to prevent any surprises in new releases.

We are happy to say that we have earned that trust in Ubuntu. Earlier this month, MySQL was granted a so-called “micro release exception”, which means that new MySQL maintenance releases should appear on a regular basis in Ubuntu going forward. The relevant mail thread is on the Ubuntu tech board mailing list, and you can read much more about MySQL’s maintenance release process in an excellent post over on the MySQL Server Team blog.

What’s Next?

Right now, we’re taking all the experience and knowledge we have gathered while working with and in the Debian and Ubuntu community and applying that to our own debs. The current debs were in much need of some love, and we’ll have some very good news for our Debian and Ubuntu users in a little while.

In parallel with that, we are continuing the work to further refine the current repo packages, add more platforms and more products to the official MySQL repos, and in the process also try to reach out to distro maintainers of the same products to make sure we understand what life downstream is like. And we will continue to work with distros in areas beyond packaging.

Requests, ideas, thoughts on our packaging efforts? Anything else we should be doing together with the distros? Let us know in a comment!

Yngve Svendsen

About Yngve Svendsen

Yngve Svendsen has been part of the MySQL organization since 2008. Based in Trondheim, Norway, he is Senior Director of MySQL Engineering Services and responsible for Release Engineering, QA and development lab IT services for the MySQL org at Oracle. Back in the mists of time he majored in mathematics before the dot com wave swept him into the devops field, first at the database startup company Clustra Inc. and then for Sun Microsystems. In 2005 he joined the Dark Side by becoming a manager in Sun's Database Group which was merged into the MySQL org when Sun acquired MySQL in 2008.

19 thoughts on “Repos and Distros: Upstream and Downstream

Leave a Reply

Your email address will not be published. Required fields are marked *


2 + nine =

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>