We redesigned this site at the end of the June.

This was a complete redesign and, in addition to the new template and images, the whole process required deep changes to your site structure.

We need to change menus, modules, categories, articles, extensions options and apply many other updates. We started to worry about the complexity of the upgrade and what could go wrong during the process.

This post outlines how we approached the problem and how we managed to successfully complete a complex update inside 20 minutes.

The steps to load the new site design

Since we couldn't just copy and upload the new site, the first and most important step was done by our designer Valentin, who took care to log every single change between our live site and the development site.

Thanks to his work we were able to create a list of all the required steps for the upgrade:

• Take the site down with an offline message
• Upload and install the new template
• Apply the new template settings
• Update the settings for the current extensions
• Install updates for extensions
• Update articles
• Update categories
• Update modules
• Update menus
• Test the site thoroughly and fix any major problems
• Take the site online

The potential problems

We looked at the long list of tasks above and realized that we had several potential problems:

First, our best estimate was that we'd need to be offline for at least an hour. We'd also need to share tasks between at least 4 staff members, assuming that everything went well.

Second, we didn't have an easy way to test the migration process. Did we need to create a new test site and get 4 staff members together every time we wanted to test the migration?

The solution for the downtime

We decided to automate as many steps as possible in the upgrade process.

First we looked at creating an installable Joomla package. This works great for updating single extensions and for simple changes. However, the upgrade we were going to make was far more complex. We needed to update many extensions and also make large-scale changes to the content, modules and menus. There are also upload size limits and timeout problems that might cause problems with an installable package.

So, instead of an installable package, I decided to go with a CLI (command line interface) application. Why? This approach adds a lot of flexibility:

• I don’t need to worry about the package size because it can bypass restrictions imposed by the administration UI
• I can trigger the installer manually, instead of having it run automatically after upload
• I can get log messages in real time while it is installing
• I can take advantage of Git Submodules to grab the most recent code from each extension and template
• I still can use the Joomla! Installer libraries to do the hard job, installing extensions individually, respecting the manifest files
• At the end, I can just remove the folder, instead of uninstalling it from the Joomla! administration UI

The result

The CLI script was able to install everything in just 7 seconds! Yeah, seconds, instead of minutes. Considering the time we took testing all the site and running a few manual tasks remaning, the downtime was only 20 minutes.

The solution for the test environment

As usual, the test environment was splitted in two: online and local.

The online environment was created from a updated copy of our site, using the Akeeba Backup, uploaded to a spare domain. If you want, you can see more details about how to create a test site using Akeeba Backup.

For the local environment you can follow the same steps, restoring the backup locally. But, if you care about more confident tests, the best option is to install your site in environments as similar as possible to the online server. Virtual machines (VM) can help you with that. 

If you are not familiar with virtual machines, they are a software that allows you to run another OS, like Linux, inside your own machine without the need of a reboot. That means that you can work on both at the same time. 

We choose Vagrant and VirtualBox to run the test environment. VirtualBox is a great open source virtualization software, and Vagrant, a wonderful tool that makes easy to reuse or share your VM with the other team members.

We built our Vagrant box (the name of the Vagrant VM recipes) based on the joomla-vagrant box with some few customizations to match our live server. The next step was to get the site inside our VM. Again, we could use the Akeeba Backup to restore a backup as we did on the online environment. But the downside on this, is the fact that while we were working on the update, the live site  constantly changed, due the user interaction, content we publish or any other customization. So when you have a very dynamic site, how to make sure your local copy is always updated? To solve this problem we developed a Shell Script that is able to download the files that have changed and update the local database. That provided a easy way to "reset" our local environment after every test, so if we break anything, we can just run the command and the local site is restored/updated.

Here is the code to synchronize your site with the VM:

We are using on this example the test domain as "mylocaltest.dev". To be able to use this on your local environment, you will need to setup the hosts file according to your OS.