• Windows XP SP3
• MS SQL Server Management Studio 2008
• SQL Server 2008 R2
• Regional Setting: London BST (British Summer Time)

So I'm starting to get the impression that I no longer tender for brand new projects competing with time and cost in order to win the bids. Recently, more and more people have been asking if I can take over a project that has been abandoned by its developer.

I've started writing this article because I find myself forgetting to ask something that could have saved an infinite number of man-hours (I exaggerate a little); but seriously, I needed a checklist that works for me.

Most Difficult?
My hardest project was (still is) one created by a PhD student who graduated and left the institution. It was a custom-built site written from scratch, and it's purpose was to manage Staff and Students within the Additional Learning Needs group. These are staff who take lecture notes for people of all disabilities and assist the students for the duration of their course. It had to allow for synchronizing with the official student record system and timetabling system (done via file upload). It included it's own timesheet / session management / invoicing system.

The website was held on a virtual host running PHP and MySQL. There was no documentation, logs, notes, and any code comments were in Hungarian (later found out it was slang or a dialect not understood by most Hungarians). Fellow students and system administrators could not accurately describe what the system was for and what it does. The developer had created the system to only last during one academic year, and the system itself only just about understood academic years (required tweaking twice a year). Then there were the error logs... some 20000 errors per use of a feature over 4 seconds. Do some developers never check the errors log?

By the time, I started maintaining the project, a revamp had been agreed with another web team. This has been delayed somewhat and still after 2 years there is no new site (blamed on the customer for not knowing what their application did in the first place... tut tut. How long have you been a developer? And this is new?).

The Why
So I find myself writing increasingly complex SQL scripts and it's at the stage where we need to optimize the queries because some scripts are noticeably slow (as observed by the customer...) and then others not.

The What
I'm going to run these benchmark tests against a system that is both up and running via the front-end and back-end. It's MediaWiki CMS used by Wikipedia.org and the like. I like queries against this database because it involves linking a lot of tables and outputting... just articles and their titles.

I have another table holding the audit trail of content approvers on the system. Approvers can e-sign an article (approving it) by clicking on a button. We want to bring back the articles that aren't listed in the audit table (articles yet to be approved).

This is just a note for me as it took a while to find on the net and even then it was confusing as to why it works but it does. Not sure whether you call this an MDX Query or part of a Transact-SQL mashup. You need to go to "Fill" (of each text box in the row - unless there's a faster way) and instead of color, click on the expression button (fx) and use the following:

=IIF(RowNumber(Nothing) Mod 2, "#ffffff", "#eeeeee")

or 

=IIF(RowNumber(Nothing) Mod 2, "WhiteSmoke", "Garamond")

This is telling the report to be white (=#ffffff) in row 1 (odd row numbers) and a very light grey (=#eeeeee) in row 2 (even row numbers). The first row being of all rows in the dataset and not for a particular drill-down item. It will alternate between the two for the rest of the report. [The second example is what I use most frequently].

Update June 2013 - How to use this migration script:

1. Make a backup copy of your old Joomla website (1.5.x)
   
   
2. Download and install the latest Joomla CMS
   • Go through the Joomla Web-based Installation GUI until the process deletes the Installation folder.
   • Avoid installing Sample Data if possible (there will be some but this script has been reworked to potentially accommodate).
   
   
3. Download my migration script
   • Decide on which script to use:
     1. If you have a small site (less than 10'000 articles), then you will use the single script "migrate_j15_to_j25_new_v1_8.sql"
     2. For larger sites (~200'000 articles), then you will need to use the 3-part script "migrate_j15_to_j25_new_v1_6_pt1.sql", "migrate_j15_to_j25_new_v1_6_pt2.sql", "migrate_j15_to_j25_new_v1_6_pt3.sql". Part 2 is dedicated to content migration, the commands are split to do about 5000 articles on each iteration so as to avoid session timeouts and memory issues.
   • Edit the script, renaming the database and table names used (DO NOT CONFUSE THE FOLLOWING DATABASE NAMES OR YOU WILL END UP WITH NEITHER SITE WORKING!!!)
     1. Open the SQL file in your favorite text-editor
     2. Rename any instance of "my_old_database.jos_" to the name of your old Joomla 1.5 database or a copy of (note the "jos_" prefix to all J15 tables, if you have changed this then reflect this in the SQL file).
     3. Rename any instance of "my_new_database" to the name of your new Joomla 3.0 database (don't change the table prefix for the new Joomla and instead do this as the next step).
     4. Rename any instance of "my_new_database.my_prefix_" to the name of your new Joomla 3.0 database (note that we're reflecting the prefix to use but also the database name - by this step, "my_new_database" should have been renamed).
     5. Save the SQL file, this is now unique to your website and this process. It does not contain usernames or passwords but has the real database names and all website content, so do not redistribute without taking security measures.
   
   
4. Run the SQL against your new database
   • Open your favorite database management tool (phpMyAdmin, workBench, navicat, sqlyog?) and connect to the new database.
   • This script will use both databases, reading from the old Joomla database and writing to the new one.
   • To run this script, your user needs SELECT, INSERT and UPDATE privileges on the new database (only SELECT on the old database will do). Special care has been made to avoid the use of functions, stored procedures, temporary tables, etc. This allows broader compatibility and customers who do not have DBAs to run this themselves. Personally I grant the user who will be executing the script full privileges or at least the same as needed to install Joomla CMS.
   • Cross your fingers and use your DB management tool to run the SQL file (or copy the contents of the SQL file into the input field)
   
   
5. Re-build/Auto-correct some Joomla inconsistencies
   • Login to your Joomla Admin Panel using a System Administrator account ("Super User"?)
   • Go to Categories > click on the "REBUILD" icon (top right)
   • Go to Menus > click on the "REBUILD" icon (top right)
   
   
6. Check your Joomla Admin User
   • Each new Joomla version is trying a different method making this very diffcult to maintain. Check that the "super administrators" for Joomla 1.5 are in your new Joomla site (as "Super Users").
   • Check that the system administrator for the new Joomla site (created on installation) is still a "Super User" (e-mail address, receives sys emails) and note their ID (identifier number).
   • See if elements created on install show as created by the admin user and not by one of your users who coincidentally has the same identifier.
   • Check that your OddUser (will be last user listed in users table) can login and their respective articles are still attributed to them.
   
   
7. Prepare for Go Live
   • This migration script is not a silver bullet. It was created to take away the hours I know I would spend without it.
   • This is not an upgrade; it is likely that your client may reconsider the template they are using or may want to add new 3rd-party components.
   • If asked: "how long will it take", do not say 5 minutes. This script runs on average in 15 seconds. The remaining hours, days, weeks will be spent on bringing your website up to standard with all issues addressed and sporting the latest version of the Joomla CMS!

UPDATE 2012 - Valid for Joomla 2.5.x but only experimental for Joomla 3.0.x!
I still do not recommend using this for your site if you have no test environment, however... I have used this script continuously to migrate J1.5.x websites to J2.5.x without issue. I have managed to do whole migrations in under 2 hours (this script runs in about 20 seconds based on 1000 articles and 5000 users) and my clients are satisfied! I am so pleased I made this script; it has saved me so much time!

• Articles
• Banners
• Categories & Sections
• Menus ^{* NEW}
• Messages
• Newsfeeds
• Users ^{* UPDATED}
• Weblinks

So this is fun. We've waited and waited and Joomla 1.6 is now stable... Exciting because this is not a simple 1.5.21 to 1.5.22 upgrade. But don't believe the drama across the web, the system introduces only a few new fundamental improvements (particularly Access Control and Groups) which actually only means some minor database alterations.

I've googled, yahood and binged, but still can't find a good way to upgrade my site from Joomla version 1.5 to 1.6 so as usual I find myself writing the first migration script. Why use a script? Well apps like jUpgrade didn't work for me. This is how to manually migrate using a MySQL database management tool like PhpMyAdmin. I'm not too bothered about upgrading my own personal site (Joomla 1.5), it's just the silly number of client websites I have to upgrade.

This I find a lot simpler, my day job involves working with SQL so this script was rather basic and quick to write. You have two databases, one with your Joomla 1.5 old content, and one following a fresh install of the latest Joomla (at time of last print: 2.5.6 - preferably no sample data pre-installed). Change the database names in this script (including the new random prefix for J2.5) and run it. 20 seconds later your Joomla 1.6-2.5.x website will have all your J1.5 articles and users...

DISCLAIMER:

This is to help ME migrate Joomla 1.5 sites to Joomla 1.6 automatically. I have also been using this happily for Joomla 2.5.x websites and more recently for an experimental Joomla 3.0.x website.