MySQL User Camp, Bangalore – 26th June, 2015

MySQL User Camp Bangalore, organized on 26th June, 2015, was a huge success with an excellent turnout of 49 attendees. We got many users from different companies, like Flipkart, Snapdeal, CTS, Capgemini, Yahoo, VMware, HCL, Datavail, Bosch, Rakuten and more.

The event started on time with a welcome speech by Balasubramanian Kandasamy, (Principal Member Technical Staff, MySQL Release Engineering). He welcomed all the attendees, followed by a brief agenda and introduction of the attendees.

Mahesh Patil (from Rakuten, a MySQL community member) presented on MySQL Tools Usage in Rakuten and Overview of Replication GTIDs. He provided a good summary on the MySQL Tools usage in Rakuten with snapshot of inbuilt tools and how the replication GTIDs are used.

Mahesh Patil

Key points from the presentation

  • Different MySQL built-in tools used in Rakuten
  • Usage of replication GITDs

We had a quick MySQL quiz before the second presentation and some goodies were distributed in the audience.

The next presentation was on “MySQL Software Repositories” by Akhil Mohan, (Software Engineer, MySQL Release Engineering). It was followed by a demonstration of Apt repository configuration and upgrading from the MySQL 5.5 available in the distro native repos of Ubuntu 14.04 to the latest version of MySQL 5.6 available in the official MySQL software repositories. We got good feedback from users on the MySQL repos.

Akhil Mohan

Key points from the presentation

  • DevOps at MySQL
  • Different MySQL repos (Apt/Yum/SUSE)
  • Docker (currently in beta)
  • Demonstration of the Apt repo configuration.

The final presentation was on “Ansible for large scale MySQL deployment” by PR Karthik, (MySQL DBA, Yahoo). Good overview of how Ansible can be of great value in the context of large scale MySQL deployment.

PR Karthik

Key points from the presentation

  • Ansible installation and configuration
  • Using Ansible for configuration management across multiple installations of MySQL

Later, we had an open talk session where the users interacted with MySQL engineers from different teams.

Feedback from attendees:

We had great responses from the attendees. There were some requests for presentations on MySQL new features and replication in the next meetup, and we will definitely take that into consideration.

For more information follow us on:

Facebook Group : MySQL User Camp

Google Group : bangalore-mysql-user-camp

LinkedIn Group : MySQL India

Fedora 22 is out, and we’re ready

Fedora 22 arrived yesterday. With a cutting edge GCC (5.1), the new DNF package management system, and improved tooling for server administration, we congratulate the Fedora community on yet another innovative release.

We’re following up from our side, and as of yesterday our repos offer Fedora 22 packages of these products:

  • MySQL Server 5.6 (currently 5.6.24) and 5.7 (currently 5.7.7). The latter one is the latest 5.7 Development Milestone Release.
  • Connector ODBC 5.3.4
  • Connector Python 2.0.4
  • Utilities 1.5.4

We also have MySQL Workbench for Fedora 22 wending its way through QA. Keep an eye out for it in a week or two.

Head over to the Linux repo page on dev.mysql.com (the page currently refers to Yum only — we’ll fix that asap) to get the Fedora 22 setup package. And as usual, please add general comments below, or submit specific issues or feature requests to the MySQL bug tracker.

Enjoy!

MySQL Repos for Debian 8

Hi, everyone. Just a quick note here to let you all know that we’ve just added Debian 8 support to our Apt repos. We have the latest MySQL Server 5.6 ready for you, as well as the latest 5.7 Development Milestone, and more of our products for Debian 8 are in our QA pipeline as we speak.

Head over to the Apt repo page on dev.mysql.com, and we should have you up and running in next to no time at all.

MySQL in Dockerland

Around 18 months ago, we launched the first MySQL Linux package repos, marking an important milestone in our efforts to modernize and improve the way we package and deliver MySQL products to our user community. MySQL product development had gone through radical improvements in terms of quality, dependability and sheer output, but the way we delivered those products had quite frankly fallen way behind.

Now, with our repo offerings rapidly maturing, we have started looking at other new distribution and deployment technologies. We’re investigating how we can help make Puppet an even better tool for MySQL infrastructure deployment, we’re playing around with Ubuntu Snappy Core packaging of MySQL, and not least: we’re working as part of the Docker community to help make sure that the community maintained MySQL Docker images continue to be as wildly popular as they are now.

MySQL on Docker

whale-dolphin
>> Video: Whales and dolphins have been observed in playful company

The basic principle of Docker is extremely attractive: We can offer you a pre-configured image of a MySQL installation, and you can go from a blank system to a ready-to-use MySQL instance answering on port 3306 with one simple command line in the time it takes to finish a cup of coffee. Real world mileage may of course vary, but the core fact is that Docker can be a very useful alternative for many users and developers who deploy and run MySQL, either as part of a larger container based application infrastructure or as a general database service. And this is something that many of you have discovered: By early May of this year, the community maintained MySQL images had been pulled more than 4 million times. Basically: During its relatively brief existence, Docker has become an important way for many of our users to run MySQL.

MySQL and the Docker Community

When we started our adventure in Dockerland, we quickly discovered that it is in some ways quite similar to the Linux communities that we are working with: a large collection of people who are passionate about the underlying technology and the myriad uses it can be put to, and who are willing to put a lot of effort into building and improving stuff. And this is what we are gearing up to support: We want to help ensure that MySQL on Docker is as robust, easy to set up and well supported as possible.

As part of this, we’re also rolling our own, optimized Docker images. That allows us to take MySQL on Docker in some new directions, and lets us implement changes and improvements that may also be of value in the community maintained MySQL images. We will treat this as a regular release channel for MySQL Server, which means that we will apply a rigorous QA process and publish new images as part of our regular flow of update releases. We are seeing considerable interest in these images, and we’d like to say thank you to everyone who’ve taken them out for a spin.

For the community maintained images, our primary contribution so far has been revamped and much expanded documentation which aims to make MySQL deployment with Docker a more approachable task. Coming into Dockerland as relative newbies entailed climbing a learning curve that we’ve hopefully now made slightly less steep for others. We’ve also helped keep the images in sync with changes in the MySQL 5.7 development release series, and we have several other pull requests being queued up for discussion and possible inclusion into the community images.

How Docker improves MySQL

On the MySQL side, working with Docker has uncovered some areas that could use improvement. Our initialization on first startup is improving a lot in the upcoming MySQL 5.7, but we have more work to do in order to smoothly support the way Docker does things. The ability to set a far greater range of MySQL configuration parameters during runtime would also make MySQL run better inside Docker containers. Happily, we’re putting those things onto our engineering road map, so keep watching our development releases, and you should see improvements appearing.

Using Docker for Previewing and Testing MySQL

This brings us to another great use for Docker images, besides running regular production MySQL instances: testing out development releases in an isolated setting, alongside any other MySQL instances you may have running on your system. MySQL development milestone releases are of course available, and today we’re adding another image flavor: a so called MySQL Labs release. Labs releases are what we use when we have something experimental or particularly interesting to show the MySQL community in order to get very early feedback. The latest Labs release is based on the recent 5.7.7 development milestone and introduces a new data type for storing JSON data in MySQL tables, along with a range of new functions to access and manipulate JSON documents. The Docker image for this release is available in the MySQL Labs area on Docker Hub now, or for the very eager:

docker run --name my-container-name -e MYSQL_ROOT_PASSWORD=my-secret-pw -d mysql/mysql-labs:5.7.7-json

Try it out and let us know what you think!

That was a whirlwind tour of what we’re doing with Docker and other new deployment and runtime environments these days. We’d be delighted to hear from you if you have ideas or suggestions for us. Please drop us a line in the comments section below, comment on our main page on Docker Hub, or in cause you (gasp) encounter issues: file a bug in the MySQL bug database.

MySQL 5.7.6 DMR: Packages, Repos, Docker Images

We are getting closer to the next major version of MySQL, and yesterday we announced another development milestone release of what will become MySQL Server 5.7. In addition to the announcement blog post itself, more in-depth posts on specific changes and improvements in the 5.7.6 milestone release will appear on the MySQL Server Team blog over the next few days and weeks.

We have been busy in the packaging, deployment and configuration area for this milestone as well, and we have made several important improvements and changes that we’ll run through in this post.

Goodbye mysql_install_db, Hello mysqld –initialize

The MySQL bootstrap process has always been complex, relying on a Perl (shell in older MySQL versions) script called mysql_install_db to initialize the data directory, set up system tables and create the MySQL root account. This created unnecessary complexity and also required Perl to be available on the database host. As of 5.7.6 mysqld knows how to bootstrap itself, and mysql_install_db has been replaced by the much simpler –initialize option to mysqld. Or alternatively –initialize-insecure. More on the difference between those two options can be found in the section on data directory initialization in the MySQL Server 5.7 manual.

Besides a reduction in general complexity and improved robustness and error reporting, the big gain for packaging and deployment is that server bootstrap is now completely platform agnostic: no dependency on Perl any more, and in particular no need to handle Windows as a special case.

Another implication is that most of the initial configuration of MySQL is now done only with the privileges of the mysql system user. As a consequence, we are no longer able to write the random mysql root password to /root/.mysql_secret, and this now instead gets written to MySQL’s error log.

The father of –initialize, Georgi Kodinov, has a much more in-depth story on this over on the MySQL Server Team blog.

Native systemd Support

Up until this milestone, we used a somewhat suboptimal mix of mysqld_safe, mysqladmin and shell scripting to control mysqld on operating systems with systemd. As of 5.7.6, we have introduced a (platform agnostic) –daemonize option to mysqld, which allows mysqld to run as a traditional, forking UNIX daemon. This allows us to work natively with systemd’s service control and monitoring infrastructure, and the MySQL 5.7 manual has a new section about managing MySQL Server with systemd.

A related change is that we now offer native support for handling MySQL logging through syslog. This appeared in the previous 5.7.5 milestone release.

And Finally: Docker!

So what is the easiest way to try out the 5.7.6 milestone? There are packages on our download site and in our Linux repos, but there is another way that will also easily coexist with any existing MySQL instances that you may already have on your system and that you may not want to disturb: try the MySQL Docker images.

We are currently ramping up our work on Docker, both to help maintain and improve the very popular community supported set of MySQL images on Docker Hub, but also with our own set of images, which we will develop in some new directions and also use as a proving ground for changes and enhancements that we will propose for inclusion in the community images.

You’ll find instructions for pulling and running our images in the MySQL area on Docker Hub. We have 5.5, 5.6 and the latest 5.7 DMR all in there, and the tag that will get you 5.7 is simply “5.7″, or alternatively the more specific “5.7.6″. Add your desired container name and admin password to the following command line and you will be up and running with 5.7.6 in next to no time at all:

docker run --name my-container-name -e MYSQL_ROOT_PASSWORD=my-secret-pw -d mysql/mysql-server:5.7.6

Suggestions, comments, issues? Please comment below, or submit a bug report to the MySQL bug tracker

MySQL Server on SUSE 12

When we launched repos for SUSE Linux 11 back in December, we said we would be adding SUSE 12 support as soon as possible, and we are happy to announce that as of last week the repo offers MySQL Server packages for SUSE 12.

It did take us a little bit of extra time to get this done, since SUSE 12 represents a major technological leap over the previous version. Among the most important changes is the move to systemd as init system, and we wanted to make sure that we support that the right way according to SUSE guidelines (by the way, we have important systemd related improvements coming in MySQL 5.7, so this area will be getting even better.)

Another important improvement is that we now enforce strict validation of metadata signing in the repository, as per the SUSE guidelines. And not least, we worked hard to make sure that our packages work as replacement for whatever MySQL-like variant you may already have on your system. The packages have gone through stringent testing of different possible upgrade flows, and the powerful SUSE package manager will recommend different replacement options for you.

The repo has the latest MySQL Server 5.6 and the latest 5.7 development release for you, and the basic steps that will get you the latest MySQL Server 5.6 are as follows:

Go to the download page for the SUSE repository, click the Download button for the repo setup RPM package and install the package like this:

$ sudo rpm -Uvh mysql-community-release-sles12-2.noarch.rpm

Then import the key that will be used to verify the packages that come from the repo:

$ sudo rpm –import /etc/RPM-GPG-KEY-mysql

And then proceed to install the MySQL Server package from the repo:

$ sudo zypper install mysql-community-server

If you want the source rpm, enable the source subrepository you want, e.g. for MySQL 5.6:

zypper modifyrepo -e mysql56-community-source

And then proceed to install the MySQL source package from the repo:

zypper source-install  -D mysql-community-server

Alternatively, you can get source rpms manually from
http://repo.mysql.com/yum/mysql-5.6-community/sles/12/SRPMS/

And as usual, if you have general comments, leave them in the comments section here, and if you have concrete feature request or come across issues with using our packages and/or the repo in general, please submit a bug report.

Heads up: Going 100% GitHub at the End of January

Hi, all. Just a quick note to let you all know that we completed the move of MySQL Server development from Bazaar to Git some time ago.

This means that as of the upcoming, end of January batch of Server releases (5.6.23 and 5.5.42), our official source code hosting moves from Launchpad to GitHub. The Launchpad site for MySQL Server will continue to be available for some time, but 5.6.22 and 5.5.41 were the last releases to have their source code posted there.

Unfortunately, as we warned earlier, this also means that we will need to break the history of our GitHub repo. Until now, that repo has been a Git conversion of our Launchpad repo, and when we switch to publishing from our new internal Git repo, history breaks. So if you have cloned our repo, you will need to do so again and merge any changes in your old clone to the new one. Sorry for that inconvenience, but there is really no way around it.

So, in a few weeks, the place to go will be https://github.com/mysql. See you all there!

Open Source Collaboration: This is how we did it ‘together’ !

It was not long before when we all were discussing to meet in person during UDS. We did not have good enough reasons to get the logistics mobilized back then but over time we realized the vibrant MySQL Ecosystem on Debian and Ubuntu needs a brainstorming session. While we did try Google Hangout as the first option it appeared a lot more obvious to meet in person. I would appreciate how nicely Robie and James from Canonical took in the first step to host the complete MySQL community directly in touch with them for packaging various MySQL variants.

Finally, it was decided that we meet in the second week of December (8-12), 2014 to find solutions to the unique problem faced by these Linux Distros. Over time Debian and Ubuntu have seen collaboration and attendance like never before from the individual stakeholders willing to maintain their MySQL variant in the Distro backed software repositories. The increased interest also brought with it the technical challenges to allow choice amongst one of the many MySQL variants in the most meaningful way possible.

All of us namely, Me (Akhil) and Norvald from Oracle, George from Percona and Otto an independent maintainer for MariaDB agreed to meet at the Canonical’s London Office. We at Oracle have been referring this event for long as the “Canonical Sprint”. It is about time we take this name out in open as it quite aptly represents Canonical’s interest to enable maximum choices for its users.

Just because each one of us agreed to work in tandem with each other, it did not mean all was achieved. It is not so straight forward to get all the MySQL variants working seamlessly with each other allowing users to make a choice at their own pace and test a variant as they decide to use. Moreover, along with the packages it is also important that users  are allowed to make sustainable transition of configuration and data across variants when they decide to make a switch. All of this needed careful analysis of various use cases and also a well defined process that we can get implemented for changing variants in real time.

At the sprint not only did we get the packaging right for us to hit the ambitious target but we also worked on side projects listed below:

  • Got rid of the legacy packaging source not applicable any more,
  • Enabled systemd service for Debian,
  • Started using dep8 testing for the packages,
  • Established parallel builds and parallel MTR test runs to reduce development and build time.
  • Baselined common work processes and repositories to collaborate.

A lot was achieved over the week and lot more is yet to come in following months. We are looking forward to Ubutnu 15.04 which will be a great release for users of MySQL Ecosystem on Ubuntu. Thanks Canonical for hosting us all together and giving us a platform to achieve and demonstrate a kind of collaboration that has never taken shape before. Glad to be part of this sprint. Finally, towards end of the week here is one of the many happy outcomes from the event:

Canonical Sprint Team
Here we are (left to right): James (Canonical), Norvald (MySQL), George (Percona), Akhil (MySQL), Otto (independently maintains MariaDB) and Robie (Canonical).                  Graphic Courtesy: George.

 

Official MySQL Repos for SUSE Linux

Since we launched the official MySQL repos for Linux a little over a year ago, the offering has grown steadily. Starting off with support for the Yum based family of Red Hat/Fedora/Oracle Linux, we added Apt repos for Debian and Ubuntu in late spring, and throughout all of this, we have been continuously adding more and more MySQL products to each of these repos.

The one glaring exception so far has been the SUSE Linux family. So today we are launching official MySQL repos for SUSE Linux. We’re starting out with:

  • MySQL Server 5.5 (old GA version)
  • MySQL Server 5.6 (the current GA series)
  • MySQL Server 5.7 (development series)

… all for SUSE Linux Enterprise 11 (11.3 and newer).

Getting started

The basic steps that will get you the latest MySQL Server 5.6 are as follows:

Go to the download page for the SUSE repository and click the Download button for the repo setup RPM package. Then install the package like this:

$ sudo rpm -Uvh mysql-community-release-sles11-5.noarch.rpm

Then import the key that will be used to verify the packages that come from the repo:

$ sudo rpm --import /etc/RPM-GPG-KEY-mysql

And then proceed to install the MySQL Server package from the repo:

$ sudo zypper install mysql-community-server

Much more information, including how to select the Server major version and how to upgrade a pre-existing MySQL on your SUSE system, is available in the MySQL Server docs at http://dev.mysql.com/doc/mysql-sles-repo-quick-guide/en/

Enjoy!

What’s next?

The initial SUSE repo offering is limited to MySQL Server and only one SUSE variant. We will be adding more products (connectors, utilities etc.) over the coming few months, and openSUSE packages are on the horizon. Regarding SUSE Linux Enterprise 12, SUSE unfortunately chose to not provide users with a mature and robust version of MySQL as part of the distro, and we are working hard to bring that option as quickly as possible to those who want to use MySQL on SLE 12.

And finally, the usual disclaimer. Nothing in this world is perfect: if you have comments or suggestions for improvements, please let us know in a comment here, or if you run into issues using our packages and repos, please take the time to submit a bug report.

MySQL on GitHub

Today MySQL is officially joining the GitHub community, with the launch of a beta GitHub repo for MySQL Server.

Why We’re Doing It

GitHub has become the place to be for successful open source projects; it has a large and rapidly growing user base, it is robust and fast, and it provides excellent tools for building and supporting vibrant open source communities. All this, plus the fact that lots of projects that are built on top of MySQL source code are on Github, meant that the choice was pretty obvious for us when we started looking for a new code hosting service.

This also means that we are switching our day to day development activities to use Git. The MySQL org is highly distributed, with many developers working from home and others working out of Oracle offices in almost every corner of the world, so it was a given that we needed a distributed version control system. Another requirement was that the system we chose had to have a large and active community around it; as we all know so very well, a large, competent and enthusiastic user base, coupled with openness, tend to make a system much more flexible, robust and better performing. Git fits the bill very well, with good performance, lots of useful tooling from the large community of Git users and a boatload of features that will make MySQL developers’ lives easier.

How to Get Started

Want to get stuck in?

The home of MySQL on GitHub is at http://github.com/mysql, and here are the basics of using the repo:

Cloning over http: 
$ git clone https://github.com/mysql/mysql-server.git 
 
Over the native Git protocol (requires SSH key on the GitHub server): 
 
$ git clone git@github.com:mysql/mysql-server.git 
 
(The download size is ~ 410 MB)
 
Available branches: 
 
$ git branch -r 
 
origin/5.5 
origin/5.6 
origin/5.7 
origin/HEAD -> origin/5.7 
 
Checkout of MySQL 5.6: 
$ git checkout 5.6 

You can find a lot more in depth information in GitHub’s excellent documentation.

So, What Does ‘Beta’ Mean?

It means that we may change things around on GitHub for a while still, so we encourage everyone to exercise a little bit of care.

Switching source code management system is a significant change for any development project. When you have hundreds of developers and contributors, lots of surrounding infrastructure for build, test and publication of source code and binaries, with processes to coordinate all of this, and when halting development for any period of time is an absolute no-go, the scope and complexity of change becomes huge.

This means that we are not quite done yet with our switch to Git. So the code we are publishing on GitHub now is an export from our existing Bazaar source code repo. That will change when we switch development to Git; at that point our GitHub repo will become a copy of our internal development Git repo. When we do that switchover we will have to break continuity, so that if you have created your own forked MySQL repo before the switch, you will be unable to pull code updates to that repo after the switch. You will have to fork again, export your commit history from your old fork and apply that to your new fork. We will of course make sure to give a heads up when that switchover point approaches.

So the “beta” label should be taken to mean that the MySQL repo on GitHub is a correct and fully usable Git repo, but we do reserve the right to tweak and tune things for a little while, and at some point in the near future, when we go out of beta, we will need to break continuity.

Contributing

A basic question we expect to see from the GitHub community is “How do I contribute back?” For some projects, it is a simple matter of issuing a pull request to ask the upstream project to review and possibly merge in the changes you are contributing. In our case, we need to ask for a little bit more. The MySQL Server Team blog has a good step by step guide for those who want to contribute, and we encourage those who are interested to head over there for more information.

We will look for ways to better connect our contribution process with our presence on GitHub, and if you have suggestions, please let us know in the comments section below. For now, we promise to make sure that pull requests get a response with information on how to contribute “the right way”.

Let me end this posting by saying that we really hope that our presence on GitHub will be useful to everyone with an interest in our code, whether for their own development or simply as a way to keep up with the rapid pace of MySQL development in general. I would also like to thank the GitHub team for welcoming us into the fold, in particular Sam Lambert, who has been a very helpful guide along the way.

And if you have suggestions for how we can optimize our usage of GitHub, or if you should encounter issues with the repo, please let us know in the comments section.

Happy hacking!