Exporting/migrating a Joomla website to Drupal7, with code

I was at DrupalCon 2016 in New Orleans last week and somebody encouraged me to share this info in a blog post, so here it goes.

After building the new website for Kansas Public Radio using Drupal 7, I needed a way to export all of the old articles from our Joomla website to the new website. I could have experimented with the Feeds module for Drupal or another Joomla to Drupal solution, but our installation of Joomla was sooooo old and outdated that I couldn't trust anything. I decided to write my own bit of code.

Inspecting the two separate database architectures for Drupal and Joomla showed that they where very, very different. While Joomla had all the article data in one single table, in Drupal it was spread out between different tables for different fields. Therefor, translating the data from one database to the other would have been very, very difficult. But if I saved the database data of the one Joomla table with all the article data to a CSV file, then I could use PHP and build-in Drupal functions to create the new Drupal nodes from the CSV file.

I was successful, but it wasn't without a fair amount of effort. For example, "sections" and "categories" of content (news article, trivia article, general info article, etc.) in Joomla were designated with a number that had to be mapped to their human readable name. And each content type had to be handled differently, so I wrote a separate script for each content type. They would each parse the same CSV file, and for each row in the file, if the section ID was for the content type I was looking for - (j_sectionID2string($j_sectionID) == 'news') - then it would create a new Drupal node based on the rest of the data in that row.

I also outputed the pairs of old URLs to the new URLs and saved them in a file. Later, I used this file to create a 404 page that would automatically forward the website visitor to the location of the new article, after a period of a few seconds informing them to update their links and bookmarks.

Here is my script for migrating only our News articles:

<?php
$csvFile = './jos_content.csv';
date_default_timezone_set('America/Chicago');
 
// helper functions
function toLogFile($msg) {
        $log_file = 'output_log';
        exec("echo '" . $msg . "' >> " . $log_file);
        print $msg . '<br /> ';
}
 
// load Drupal
define('DRUPAL_ROOT', '/home3/kpr/public_html');
require_once DRUPAL_ROOT . '/includes/bootstrap.inc';
drupal_bootstrap(DRUPAL_BOOTSTRAP_FULL);
 
// get the csv data
$file = fopen($csvFile, "r");
$i = 0;
while (!feof($file)) {
 
        // do stuff with the variables
        $vars = fgetcsv($file);
 
        $j_id = $vars[0];
        $j_title = $vars[1];
        $j_alias = $vars[2];
        $j_titleAlias = $vars[3];
        $j_introText = $vars[4];
        $j_fullText = $vars[5];
        $j_state = $vars[6];
        $j_sectionID = $vars[7];
        $j_mask = $vars[8];
        $j_catID = $vars[9];
        $j_created = $vars[10];
        $j_createdBy = $vars[11];
        $j_createdByAlias = $vars[10];
        $j_metadata = $vars[25];
        $j_mp3_1 = $vars[32];
        $j_mp3_2 = $vars[33];
 
        if ((j_sectionID2string($j_sectionID) == 'news') && ($j_state == 1))
                {
                if (j_categoryID2string($j_catID) == 'kpr news')
                        {
                        $i++;
                        if ($i < 2501) {continue;}
                        //if ($i > 2500) {fclose($file); exit();}
 
                        // new music blog entry
                        toLogFile("==================================");
                        toLogFile(date('d F Y h:i:s A'));
                        toLogFile("importing ".j_categoryID2string($j_catID).": " . $j_title);
                        //toLogFile($i);
 
                        // calculate some drupal things
 
                        //$d_termID = taxonomy_get_term_by_name("featured", '')->tid;
                        $d_termID = 19; // local
 
                        $d_body = $j_introText . $j_fullText;
                        // fix the stupid images path
                        $d_body = str_replace('src="images/', '" src="/images/', $d_body);
                        $d_body = str_replace('href="images/', 'href="/images/', $d_body);
                        $d_body = str_replace('float: left', '', $d_body);
                        // text is really jacked up...
                        $d_body = str_replace('"=""', '', $d_body);
                        $d_body = str_replace('&lt;', '<', $d_body);
                        $d_body = str_replace('&gt;', '>', $d_body);
                        $d_body = str_replace('<hr />', '', $d_body);
 
                        if (($j_metadata != "" ) && ($j_metadata != NULL)) {
                                $d_summary = strip_tags($j_metadata);
                        } else {
                                $d_summary = str_replace('<img', '<span', $d_body);
                        }
                        //toLogFile($d_summary);
 
                        $d_mp3_1 = "<a href="<a href="http://129.237.213.244:8000/mp3/"">http://129.237.213.244:8000/mp3/"</a>">http://129.237.213.244:8000/mp3/"">http://129.237.213.244:8000/mp3/"</a></a> . $j_mp3_1;
                        $d_mp3_2 = "<a href="<a href="http://129.237.213.244:8000/mp3/"">http://129.237.213.244:8000/mp3/"</a>">http://129.237.213.244:8000/mp3/"">http://129.237.213.244:8000/mp3/"</a></a> . $j_mp3_2;
 
                        $d_userID = 1; // kpr
 
 
                        $node = new stdClass();
                        $node->title = $j_title;
                        $node->type = "article";
                        node_object_prepare($node); // Sets some defaults. Invokes hook_prepare() and hook_node_prepare().
                        $node->language = LANGUAGE_NONE; // Or e.g. 'en' if locale is enabled
                        $node->uid = $d_userID;
                        $node->status = 1; //(1 or 0): published or not
                        $node->promote = 0; //(1 or 0): promoted to front page
                        $node->comment = 0; // 0 for off
 
                        $node->body[LANGUAGE_NONE][0]['value'] = $d_body;
                        $node->body[LANGUAGE_NONE][0]['summary'] = $d_summary;
                        $node->body[LANGUAGE_NONE][0]['format'] = 'full_html';
 
                        // Term reference (taxonomy) field
                        $node->field_news_section[LANGUAGE_NONE][0]['tid'] = $d_termID;
 
                        // audio
                        $node->npr_audio[LANGUAGE_NONE][0]['mp3'] = $d_mp3_1;
                        $node->npr_audio[LANGUAGE_NONE][1]['mp3'] = $d_mp3_2;
 
                        // 'node' is default,
                        // Other possible values are "user" and  "taxonomy_term"
                        $node = node_submit($node); // Prepare node for saving
                        $node->created = strtotime($j_created);
                        node_save($node);
 
                        toLogFile("URL: /news/".$j_id."-".$j_alias." -> /node/".$node->nid);
 
                }
 
        }
}
 
fclose($file);
 
function j_sectionID2string($i)
        {
        switch ($i)
                {
        case 1:
                return "news";
                break;
 
        case 2:
                return "kpr administration";
                break;
 
        case 3:
                return "kpr general info";
                break;
 
        case 6:
                return "programs";
                break;
 
        case 8:
                return "music";
                break;
 
        case 10:
                return "live studio";
                break;
 
        case 11:
                return "kpr webpages";
                break;
 
        case 12:
                return "support";
                break;
 
        case 13:
                return "kpr sidecar";
                break;
 
        case 14:
                return "health";
                break;
 
        case 15:
                return "alerts";
                break;
 
        case 16:
                return "latest";
                break;
 
        default:
                return "who knows";
                break;
        }
}
 
function j_categoryID2string($i)
        {
        switch ($i)
                {
        case 43:
                return "trivia";
                break;
 
        case 3:
                return "photo of the week";
                break;
 
        case 59:
                return "slideshow";
                break;
 
        case 41:
                return "classical live";
                break;
 
        case 49:
                return "jazz live";
                break;
 
        case 51:
                return "kpr live";
                break;
 
        case 54:
                return "rch shows";
                break;
 
        case 50:
                return "trail mix live";
                break;
 
        case 25:
                return "classical";
                break;
 
        case 26:
                return "jazz";
                break;
 
        case 27:
                return "retro cocktail hour";
                break;
 
        case 28:
                return "trail mix";
                break;
 
        case 39:
                return "kpr live studio";
                break;
 
        case 66:
                return "music notes";
                break;
 
        case 47:
                return "health series";
                break;
 
        case 22:
                return "kpr news";
                break;
 
        case 23:
                return "kpr presents";
                break;
 
        case 24:
                return "commentaries";
                break;
 
        case 34:
                return "statehouse news";
                break;
 
        case 76:
                return "health news";
                break;
        }
}
 
?>

Your millage will very, a lot, but I hope this helps somebody out there!!

Attachment	Size
import.zip9.23 MB	9.23 MB

How to Create an On-Demand Feature for Your Alexa Skill

As 2020 was coming to a close and 2021 was just around the corner, I was finishing development of a new version of our skill for the Alexa smart speaker. It was a total re-write, and it changed the skill from being able to perform only a single intent (play our live stream) to providing a dialog-based menu of many intents to choose from. With the new skill, listeners can choose to either play our live stream, play our HD2 live stream, play the latest newscast recording, or play an on-demand. (One day I would like to add another intent: make a donation.)

How to use versioning with AWS Lambda and Alexa Skills Kit (ASK), a step by step guide

I learned a valuable lesson on the morning after submitting my latest Alexa skill for certification. I had an email from the Alexa certification team letting me know that their tests had failed. I thought that was odd because everything was working perfectly fine when I left work the previous day. I was using AWS Lambda to host the back-end code (within a Node.js environment) and hadn't set up the versioning system because I didn't know about it. [update: I'm now using the AWS CLI to locally host my code and push it to Lambda, which is what I recommend.

Migrating from Drupal 7 to Drupal 8: my experience, the many obstacles, and their solutions

You can't really call yourself a Drupal developer if your site is still on Drupal 7. Right? So it was high time for me to upgrade Dannix from D7 to D8. Here's how I did it, the problems I ran into, and what I did to overcome them.

Make your own conditional fields in Drupal 7 by "hacking" the hook_form_alter() function with JavaScript injection

When I needed a conditional field for a Drupal 7 content type, of course I just googled "drupal conditional fields" to see if a module exists. One did exist, but its D7 release came with a big, red flag warning any would-be downloaders that it's not supported any longer and is vulnerable to security issues. Instead of risking it on the entrerprise-level website, I decided to code up a solution myself. And I would use JavaScript inserted by the PHP code. Easy!