Debugging quickie: Objects

How do you display the contents of an object that has private and protected properties - which means dpm($object) doesn't give you what you want?

You could do this:

dpm('<pre>' . print_r($object, TRUE) . '</pre>');

But that's a lot to type and you get a huge horrible list (particularly if it's a View object).

Try this instead:

dpm((array) $object);

Works a treat.

And while we're on the subject of dpm() have you tried this:

dpm($var, __FUNCTION__);

or

dpm($var, __METHOD__);

It can help.

Field Complete and Profile2

So I thought I'd promote the latest dev version of my Field Complete module to a release candidate yesterday. There were two reasons for this: (a) it's pretty solid, and (b) people are more likely to use it if it's out of dev.

It wasn't a mistake. Within a couple of hours I had a new bug report: Doesn't work with Profile2.

Which surprised me a lot, why on earth would it not work with a popular entity-based module? My code is very standard and does nothing naughty.

The answer, of course, is that Profile2 is naughty. Or at least it's non-standard. I'm not going to go into detail but they bypassed standard edit-form processing and do odd awkward things with the display URL. So my form interceptions don't get a chance to work and the completeness progress bar (and the Incomplete fields block) can't work.

What to do?

On the one hand I don't see why I should hack my code to work with something non-standard. On the other, Profile2 is a very popular module and using Field Completeness with it is a natural thing to do. And I want people to use my module.

So I spent a couple of hours figuring out the nasty that Profile2 does and modifying my code to cope with it. It wasn't too horrible, but I can't say I enjoyed it as a process. My nice clean code now has hacks in it.

Oh well.

The History of the Field Complete module

A couple of months ago I was contracted to work on a project for a large UK organisation that provides accreditation for university, and other, further education courses of a certain type. The site had already been mostly designed and built so I had no say in its overall structure. Suffice to say: I wouldn't have done it like that.

Still, we play the hand we're dealt.

The nature of the project meant that the applicant had to fill in absolutely massive forms and provide huge amounts of evidence to show how their course delivered to the standards required for accreditation. But the form could not be submitted for review until it was complete. But the forms are so huge that nobody is going to be able to fill them in in one sitting. In some cases it's expected it would take weeks.

The original developers had decided (sensibly) to use the Content Complete module, which is a watered-down version of making a field "required" - you specify which fields need to be complete and then you can check every time the form is saved. Which was all well and good except for one tiny thing: these forms used Field Collections and Content Complete cannot cope with Field Collections.

The first task on my list was: get Content Complete working with the Field Collections.

I laughed til I stopped.

I did have a look but Content Complete is structurally incapable of handling Field Collections without a major rewrite. Anyway it was worse than that because I had to create new forms which required linking to other entities (via Entity Reference) and they needed to be checked for being complete as well.

Now there was a discussion in the Content Complete issue queue about developing a new version called Entity Complete. However the ideas were overly complex, wanted to carry on using the same interface (which I don't like), no actual work had been done, and the discussion had dried up months ago. Clearly it wasn't happening.

So I had a choice, somehow fix Content Complete, or start again and write the Entity Complete module from scratch myself. So that's what I did. Essentially it intercepts just two hooks to achieve the required result but there are plugins for specialised field types. And lots of frills.

You can find the module itself here. You'll notice it's called Field Complete instead of Entity Complete because someone had already grabbed the 'ec' short title.

And there is lots of lovely documentation with many pictures here.

Enjoy.

Export UI, Features and Taxonomy

Here's a quicky. Let's say you've created some exportable content (using CTools) which references a term ID and you have to go from your dev site to the production site. And your taxonomy is also being exported using UUID.

Somehow you have to tie them together because sure as eggs is eggs the term IDs created on the production site are not going to be the same as the local ones. That's why you used UUID in the first place. Right?

Here's what you do: in your local exportable you will have a column for the term ID so include a column for the term UUID as well.

In your add/edit form for the exportable you'll have to include some code to automatically add the selected term's UUID - I did it in the form validation. Basically you read the selected term ID, load the selected term which will have the UUID in it (because it's added to the base table). Set the term UUID value in $form_state['values']. Assuming you're using CTools Export UI the UUID will be saved automatically.

Also, in the module install, add "no export" => TRUE to the TID field, so that Export UI does not include it in the feature. (This code works for Features, it doesn't work for single imports, I'll leave that as an exercise for the reader - hint: you can specify an "import callback".)

That's the easy bit, when you export your content it will be saved with the term's UUID and not the term's ID. The difficult bit is how to link the UUID of exported content when Features loads it into the new site.

Except it's not hard at all. In your export specification, in the schema, you have the "default hook", well CTools Export UI very kindly calls an drupal_alter() on the default items after it's loaded them. So we can do this:

/**
 * Implements hook_DEFAULT_HOOK_alter().
 *
 * This intercepts any defaults picked up from code and converts
 * their UUID category into the local TID (which might be different
 * on every site).
 *
 */
function mymodule_my_default_hook_alter(&$items) {
  $uuids = db_select('taxonomy_term_data', 't')
      ->fields('t', array('uuid', 'tid'))
      ->execute()->fetchAllKeyed();

  foreach ($items as $item) {
    if (empty($item->tid) && !empty($uuids[$item->uuid])) {
      $item->tid= $uuids[$item->uuid];
    }
  }
}

The database call creates an array which maps all UUIDs to TIDs in one go. If your site uses a lot of taxonomy terms - perhaps you have user tagging - you might want to restrict this call to a specific vocabulary.

The exact item property names will depend on what you set up in your schema.

Sorted.

Entity Reference Views Widget plus Organic Groups Nightmare

Let's suppose you are using the Entity Reference Views Widget to display items to be included in an Entity Reference field. This is actually quite a cool module while being a little awkward to use - essentially it gives you a views listing of candidate entities to add, with an AJAX-driven checkbox: click the box and the entity gets moved to the list on the left to display what's been chosen.

Which is all fine.

The project I'm currently working on uses Organic Groups to group certain users within an organisation allowing them to work on very specific types of node content. I had to create a completely new type of entity (though that's not important) for them so they could add one or more of these entities to one or more of their special content.

The new entity was made subject to the OG, and that too was fine.

So then I came to build the Entity Reference Views Widget to only display the new entities that belonged to the current user's OG.

Meltdown. Either I listed everything, or nothing. Filtering OGs is not the easiest thing in the world: You have to create an OG relationship for the entity in question (easy) and then add an argument which if it has no value (the desired state) uses the OG Context module to figure out what OGs are available.

Weird fact number #87654: The OG context module allows you to base OG on the current node and on a user currently being viewed or edited, but not on the current user. So I had to build a quick context for that:

/**
 * Implements hook_og_context_negotiation_info().
 */
function mymodule_og_context_negotiation_info() {
  return array(
    'user' => array(
      'name' => t('User'),
      'description' => t("Determine context by finding the current user's OG (if any)."),
      'callback' => 'mymodule_context_handler_user',
    ),
  );
}

/**
 * Implements hook_og_context_negotiation_info_alter().
 */
function mymodule_og_context_negotiation_info_alter(&$contexts) {
  $context['node']['menu path'][] = 'node/%/edit';
}

function mymodule_context_handler_user() {
  global $user;
  $account = clone $user;
  $contexts = _group_context_handler_entity('user', $account);
  return $contexts;
}

In fact this does two things: it expands the context checking for nodes to include nodes being edited and adds a context that looks at the current user.

Okay. Next factor: Entity Reference Views Widget has this neat facility for feeding the entity IDs of entities already selected back into the view and excluding them. This is great and it also uses an argument, which needs to be the first argument.

However, and this is the nastiness, if the ERVW argument does not exist (i.e. no entities have yet to be selected for the field) the second argument fails to fire and you see all the entities without any OG filtering.

The solution is thankfully quite simple: Edit the ERVW argument so that it has a default value of "all", this means it always exists and the second argument does fire and figure out the correct OG context puts it into the query and filtering actually works.

Obviously I went through various stages of thinking each handler was broken or that there was something weird about my newly created entity. But none of those things were true. The problem was simply one of configuration.

Hope that helps.

Feed Sources and Self Nodes and Bears (oh my)

Seems it's been nearly a year since I last posted something. I can only say that I have been tied up with Drupal 6 projects and uninspiring Drupal 7 ones. However something interesting came up with a project of my own over the last day or so. So here we are:

The project in question involves creating a meta-search site on a specific subject with data gathered from various sites. Some of those sites have RSS feeds, some do not.

The Feeds module is fairly awesome if you need to get structured data from somewhere and turn it into nodes, or other entities. If you're reading, say from an RSS feed, it works out of the box and I must admit I was very impressed with what it can do. There's even a feeds crawler module that lets you scrape data.

I appreciate that may make some people feel awkward but this site does nothing except fetch information, allow someone to search and then direct the viewer to the original site. No more than Google or any other search engine.

However for this project there was an issue: even the RSS does not contain all the required information to fill the nodes created. To do that properly we have to go to the original page and scrape the relevant sections - for example, to get the full description and the tags that have been used.

There is another module Feeds Self Node Processor which allows a node to become its own feed, which means that you create an importer for the initial information and create the nodes. Then each of these new nodes can fetch its own specific information from the target URL.

I'm leaving out a lot of detail here but I hope this is enough to be understood.

Fetches can be scheduled to be performed during cron jobs or switched off completely so that's all fine. Except for one little thing:

There is no option to do a once-only fetch. No imagine you've imported 10,000 data items, the feeds_source table is now filled with 10,000 self-node rows. If you set the frequency of fetch to the maximum (4 weeks) the first self-node fetch won't happen for 4 weeks. But worse: if you set it to "as often as possible" the cron starts to cycle through the existing 10,000 nodes but (as far as I can tell) doesn't do it right and keeps repeating the same content. And new content gets ignored.

Oh dear. The initial joy of getting the feeds process functioning for three different sites was slowly eroded by the realisation that this was just not going to work.

I spent two days playing with various options. The Feeds module provides a wide array of hooks and lots of potential for customisation however none of them helped.

Of course I would not be writing this unless I had found a solution.

It became clear what was needed was to delete the relevant row from the feeds_source table. There's a nice class wrapper and method to achieve this which ensures all relevant information is also deleted (like the entry in the job_scheduler table).

And there's a "post import" hook. Theoretically just performing $source->delete() should do the job, unfortunately doing that in the "post import" hook doesn't work, the entries are simply recreated because the data is saved after the hook is run. And you can't touch the "last imported" time stamp to set it into the distant future. Simply deleting the relevant job_scheduler row is equally futile.

What was needed was a method of deleting the feeds_source row at some point after everything else had been done. The next page load was an attractive choice initially - I even toyed with the thought using $_SESSION but only for a few seconds.

The answer is that under-used hardly-mentioned-anywhere feature of Drupal 7: Queues. And here it is:


/**
 * Implements hook_feeds_after_import().
 *
 * @param $source
 *  FeedsSource object that describes the source that has been imported.
 */
function YOURMODULE_feeds_after_import(FeedsSource $source) {
  if (/* identify as a feeds source to be killed */) {
    // Get the queue (create if not existent)
    $queue = DrupalQueue::get('killJobQueue');

    // Build the required job data
    $job = array(
      'type' => $source->id,
      'id' => $source->feed_nid,
    );

    // And put it in the queue
    $queue->createItem($job);
  }
}

/**
* Implements hook_cron_queue_info().
*/
function YOURMODULE_cron_queue_info() {
  return array(
    'killJobQueue' => array(
      'worker callback' => '_YOURMODULE_kill_source',
      'time' => 5,
    ),
  );
}

function _YOURMODULE_kill_source($job) {
  feeds_source($job['type'], $job['id'])->delete();
}

We intercept the hook after the import and determine whether this is a feeds source we want to kill. I did this by using a naming convention, all my self-node importers are named "reprocess_[something]". We build the job data and add it to the queue.

It's assumed that many queue-using applications will want to process the queue during hook_cron() so the Queue API provides the functionality for you - apart from the bit that does the actual work.

Now it doesn't matter whether my cron function runs before or after the feeds cron, because somewhere between crons the feeds source rows will be correctly deleted. And it works. Apart from anything else it prevents the feeds_source and job_scheduler tables from getting clogged up with useless data.

Form API and AJAX callbacks

Here's a piece of information that's well hidden.

If you're using Drupal's AJAX functionality with forms - but you don't actually want to change the form but instead do other stuff on the page, you may run into trouble because there's something that's not clearly explained about the AJAX callback function in your PHP code.

You know you can add this:

$form['tickbox'] = array(
  '#type' => 'checkbox',
  '#title' => t('Change something on the page'),
  '#ajax' => array(
    'wrapper' => 'id-of-some-div-on-the-page',
    'callback' => 'mymodule_form_ajax_callback',
  ),
);

And when you click the box your function in the PHP gets called:

function mymodule_form_ajax_callback($form, $form_state) {
  return '<div id="">Hello!</div>';
}

Okay that just does a straight replace. But what if you want to append it instead? The documentation says that this function can return AJAX commands instead. So I can do this, right?

function mymodule_form_ajax_callback($form, $form_state) {
  return ajax_command_append(NULL, 'Hello!');
}

Nope. The documentation says I can return an array of commands. So I can do this, right?

function mymodule_form_ajax_callback($form, $form_state) {
  return array(
    ajax_command_prepend(NULL, 'Hello!'),
    ajax_command_append(NULL, 'Goodbye!'),
  );
}

Nope.

The answer is hidden around line 219 of includes/ajax.inc, this will work:

function mymodule_form_ajax_callback($form, $form_state) {
  return array(
    '#type' => 'ajax',
    '#commands => array(
      ajax_command_prepend(NULL, 'Hello!'),
      ajax_command_append(NULL, 'Goodbye!'),
    ),
  );
}

It needs to be a renderable array so this is what works.

jQuery publish/subscribe custom events

There seems to be a stuck idea with respect to jQuery which demands binding custom events and their functions to a specific DOM object (like 'document' or 'body') and triggering the event on that object which then tells whatever objects might want to know about the event using another event.


There's an example of this here: http://stackoverflow.com/questions/399867/custom-events-in-jquery and another here http://jamiethompson.co.uk/web/2008/06/17/publish-subscribe-with-jquery/ (okay, that's four years ago but it's top of the Google results on this subject).


I may be being stupid (it's been known) but that seems a completely unnecessary step - possibly inherited from OOP coding in a non-HTML environment where it may be necessary to have two stages.


So, here I am building a new carousel system for Drupal 7 - not because I'm a glutton for punishment but because none of the existing options does what I need for my current contract - and come up against this issue and something is nagging me. I've been here before.


If you imagine, a modern carousel has its little indicator buttons to show which slide we're currently on and allow the user to select a slide to view with a click. It also has forward and back arrows (which may be hidden or displayed if there's a slide to go forward or back to). And there may be an auto-change option if the user isn't selecting slides manually.


Now each of those indicators, arrows and auto-scroll items is an object which has behaviours attached. Let's call them carousel "tools". And they need to know what's going on.


Let's say the last indicator is clicked by the user, the system must scroll to the last item and then a check must be made to see if the "next" arrow should be hidden, the old indicator unhighlighted, the new indicator highlighted and the auto-scroll switched off (maybe with another timer started so the auto-scroll restarts after a period of time of user inactivity).


Or, if it's the auto-scroll in action, similar actions must be taken when a new slide is displayed.


Now you could hard-code all this into the slide function but we all know that's naughty tight coupling and will be difficult to write without lots of bugs and to maintain for anyone else. Since each object has its own behaviours this problem is ripe for proper OOP implementation.


So we could encapsulate the tools and then keep a list of those tools and hard-code a function to call in each one of them when the slide changes. Okay, that's better functionality, looser coupling but it can be done better.


Instead what we intuitively want to do is send a custom event saying "this carousel has changed its slide" to every object that needs to know (and don't forget we might have more than one carousel on a page so we also need to distinguish between the tools belonging to each carousel).


Okay, so we could bind a function for the "slideChange" event to the root carousel DOM element and then have that element triggered by the slideChange function (with data including the old and new slide IDs, plus whether this is the first slide or the last slide in the list - so that the previous/next arrows know whether to hide themselves).


But why do we have to use the root carousel element at all?


We don't. What about this:


$('.carousel-tool').trigger('slideChange.' + myCarouselID, {...slide change data...});


And in the set-up for each tool we can have this:


$(this).bind('slideChange. + myCarouselID, function(e, data) {
  var me = $(this);
  ... process the slideChange event
});


And in the HTML every tool has a "carousel-tool" class. If we do this we are completely encapsulating the actions of the tools. The slideChange function can reference every tool, without actually having to know who they are.


This uses the custom event namespacing feature available in jQuery to ensure that only the tools that belong to a specific carousel have the event triggered when that carousel slide changes. You could namespace the HTML class but that's less efficient in some respects.


Or you could modify the trigger line:


$('#' + myCarouselID).find('.carousel-tool').trigger('slideChange', {...slide change data...});


Actually this is probably the most efficient option even if it's not the most elegant, and note you wouldn't use the namespacing in the binding either if you do it this way.


In case you think that's an odd way to do the selection process, it's quicker to have the single $('#id') search on its own and then do a find() from there, than it is to combine to two. (See http://24ways.org/2011/your-jquery-now-with-less-suck.)


So there you are: how to do publish/subscribe properly with jQuery. (In my opinion.)


UPDATE: One caveat, the events propagate up through the DOM tree, which means that if you have a handler for a custom event higher up the tree it will get called as many times as there are handlers lower down the tree. You can avoid this using the e.stopPropagation() method.

Search API, Facet API and Display Suite

Just spent most of the day tracking down a nasty little issue involving these three modules.

Let's face it Search API with Facet API (and all the Search API support modules) are brilliant, the core Search is very difficult to customise and you usually have to end up with nasty core hacks if you want to do anything clever.

Search API on the other hand is lovely, and Facet API is just great with almost everything extendible. And, of course, you can display search results as a view, which adds that level of delightfulness.

Display Suite is also awesome (I may have already mentioned this).

But if you have all three together - displaying search results and facet blocks on a Display Suite configured page, you may run into the problem of the facet blocks refusing to appear.

The reason is really simple: there are no facets to display.

But you say (well, I screamed in my head) I'm displaying search results through a view, I'm looking at them right now and I know they have facets.

I finally, eventually, lit upon this issue in Facet API http://drupal.org/node/1392288 which explains the problem but doesn't come up with any solid solution. The issue is that if the page displays the blocks before the search query has been run they will be empty because search results are needed before the facets can be calculated.

Simple? Yes. Solvable? Not easily. There's no way of telling Display Suite what order you want the blocks and content displayed (it would be a nice touch but not worth for just this problem, or maybe a way to defer content rendering of blocks to the end). There is talk of having the facet run the query if it's not been run, but that's in the future if it ever happens.

The solution is legal but ugly.

The way to ensure the query is run first is to do it in hook_init() in my case by rendering the view and saving it. Then adding the view to the output when required. Yucky but it works.

UPDATE: Typical really, what I hadn't done is fully explored the DS Extras module which allows Views pages to be configured. This is very handy but the issue above still remains.

Changing view_mode mid-stream

In my current contract I have to display a hierarchy of complex taxonomy terms - each one as a page with relevant data hanging off it. There are three levels to the taxonomy: Level 1 displays one set of data and links, Level 2 is never displayed on its own (which is something I'll have to take care of) and Level 3 has a different page structure again.

Different page layouts means Display Suite (http://drupal.org/project/ds) and that's great. It works fine when you want to configure a different layout for a node - as long as it's the same for every node type. Or entity bundle.

DS works fine for taxonomy terms in just the same way as for nodes. It is one of my favourite modules.

But I needed to be able to change the view_mode on the fly: to check what level the taxonomy term is and change the view_mode based on that. Which is when I ran into trouble - and it's not DS's fault. Though I never had to do it apparently this was not a problem in D6 but in D7 there is a bug which makes it tricky to change view_mode. You can read all about it here: http://drupal.org/node/1154382

This issue gives some hints as to the solution (hook_entity_prepare_view() is not it), but there is a link to this blog here. The solution described  is for changing build mode for nodes based on the current theme (so you can change things if you're using a mobile theme).

My solution is the same except slightly more generic, instead of intercepting hook_node_...() hooks, I intercept hook_entity_...() hooks.

I'm just going to throw my code at you because I know you can work out what to do in your own situation.


/**
 * Implements hook_entity_prepare_view().
 *
 * We have to play silly games to change the view_mode, first we intercept
 * hook_entity_prepare_view() and establish what view_mode we want, and
 * save it. But there's a bug which means this has no effect on the output
 * so...
 */
function page_g2g_entity_prepare_view($entities, $entity_type, $langcode) {
  if ($entity_type!='taxonomy_term') {
    return;
  }


  foreach ($entities as $id => $entity) {
    if ($entity->vocabulary_machine_name!='gtg_tags' || $entity->view_mode!='full') {
      // wrong vocabulary or not a "full page"
      continue;
    }


    // Change the display dependent on the number of parents
    switch (count(taxonomy_get_parents_all($entity->tid))) {

      case 1:
        $entity->view_mode = 'g2g_level_1';
        break;

      case 2:
        // Hm. Need to do something else here, maybe
        // do a redirect to the parent term.
        break;

      case 3:
        $entity->view_mode = 'g2g_level_3';
        break;
    }
  }
}


/**
 * Implements hook_entity_view_alter().
 *
 * ...we intercept before the full build is enacted. You can test and see that
 * even though we changed the view_mode in the term itself, it hasn't transferred
 * to the build theme. So having verified we want to do it with this entity
 * we transfer it. And now it gets changed.
 */
function page_g2g_entity_view_alter(&$build) {
  if (isset($build['#entity_type']) && $build['#entity_type']=='taxonomy_term') {
    $build['#view_mode'] = $build['#term']->view_mode;
  }
}


Arguably you don't need the first hook, you could do it all in the second call. But it's a matter of elegance and splitting actions into their appropriate locations.

Then three come all at once

I have been doing some work on the field_extract module, a few minor fix-ups most of which wouldn't be noticed and added support for the entityreference field. You can find this module here: http://drupal.org/project/field_extract

Someone I worked with recently has put out my "deeplink" module which allows otherwise hidden content to be made available on a specific trackable URL. My version was Drupal 6 (that's what I was working on at the time) he's doing the D7 upgrade, you can find it http://drupal.org/project/deeplink

But deeplink needs my Controls module, and that's a baby that needs explanation. And that explanation is available on the project page http://drupal.org/project/controls there's both a D6 and D7 version both written by lil ole me.

Briefly: Controls is an API module which provides a similar function to CTools plugins (they have a lot in common), but requires virtually no setting up and is much easier to use. Now I always say that to people but I did seriously wonder whether it was true, so for the last commercial project I worked on I didn't use Controls, I went back to using CTools plugins instead. I desperately wished I hadn't.

So I'll stick by my statement - I think Controls is easier to use and in some ways more versatile than CTools plugins.


However they are also more easily abused. It's something for developers working on an end-client's website. Anyway I'll let you be the judge.

Entities vs Nodes

For the last six months I've been working on a Drupal 6 site professionally, but developing a couple of personal Drupal 7 sites. However I haven't been going into D7 in any great detail - except for developing entities.

Now I've moved contracts and I'm developing a new Drupal 7 site to replace an older non-Drupal site with lots of additional facilities. And the Drupal implementation is entirely up to me.

Being an OOP person at heart I love the concept of entities but when working with a commercial website you have to make some serious decisions. My personal preferences have to give way to the reality of building a website that delivers the spec and can be maintained and extended by other developers in the future.

So how do you decide what should be a node and what should be an entity?

There's a line in this blog post which says:

You can now actually create data structures specific to your application domain with their own database table (or any other storage mechanism really) plus a standardised way to add fields to them. No need to turn nodes into something they are not.

Which is technically accurate but does not always provide clear guidance, so here's my step-by-step analysis method to decide whether a specific data structure should be a node or not:

1. Is it content? Is the item definitely stuff that gets turned into HTML and displayed for the user? If you were building a review site, a review would definitely be content, so that's a node.

2. Does it need to have revisions? The node module provides revisions and the associated modules make it easy and powerful. Building revisions into custom entities is hard. So if it has to have revisions then it has to be a node.

3. Is there additional "property" (as opposed to "field") information? I had a situation where I wanted to define a "proxy", and a proxy needs a web address, a port number, maybe a username and password. These are fundamental properties of a proxy. You could add these as fields for a node, of course, but it makes more sense for them to be properties of a proxy entity. So this should be considered as an entity. (Another way of looking at this is: is there a need for a new table containing information specific to this data structure? If so, think entity.)

4. Is the structure normally invisible to the user? That should be an entity.

5. Would using an entity instead of a node obscure the function? Perhaps this is tricky to answer, after all dividing functionality out to a new object should never make things more complex. But it's worth asking the question.

Any other ways of to help make the decision?

Remember: there is virtually no overhead in creating a new entity. And there are huge advantages in additional functionality that the core, Entity API, Views, Features and other entity-related modules can give you.

Cleaning the watchdog

As a Drupal developer - do you do this?

Do you go through watchdog, find every error and fix it? You probably should - and here's why:

If there's even just a Notice, it means something's not right - checking it out might reveal something far more important.

Every error that goes to Watchdog is consuming valuable processing time. I had an instance quite recently where an error in a third party module was generating 10,000 notices per page. And the original author never noticed. That's ten thousand database writes. As you might imagine fixing it speeded things up quite dramatically.

When you can load any page on your website and have no reports added to the watchdog - you know your site is good.

Optimising JavaScript
On a similar note, I inherited a site which had a lot of JavaScript (all jQuery) which had to run on start up. It wasn't a public-facing site so all the JavaScript wasn't a problem what was a problem was that start-up time on IE7 was sloooooooooow - at least 10 seconds maybe more.

I spent a lot of time justifying this by saying "Well IE7 is slow and its JavaScript runs like a pig" which might be true but the client wasn't buying it and wanted it fixed. Fair enough.

Eventually I came to look at the code and it was possibly the most abysmally written code I have ever seen.

Imagine this code was baking 100 cookies, here's what it did.

• Fetch the ingredients from the cupboard;
• Get out enough for 1 cookie;
• Put the ingredients back in the cupboard;
• Mix the ingredients;
• Cut the cookie;
• Cook the cookie until done;

Repeat 100 times.

It worked, but it was stupid and used the most inefficient jQuery selectors imaginable, and even when it knew what object it wanted to act on (because it had already found it) it would make jQuery find it again, using exactly the same, useless, selector. The same coder, I later discovered, who in another script had written $('#x').parent().parent().parent().parent().parent().parent().parent().parent();

I re-coded it to make 100 cookies in a single batch. Start-up time <1sec.

This is a good site for jQuery optimisation: http://hungred.com/useful-information/jquery-optimization-tips-and-tricks/

Little debugging aid

I had been having a real problem tracking down a PHP error in Drupal which was passing an array to htmlspecialchars() in check_plain() instead of a string. I needed to use debug_backtrace() but the issue was related to Views which meant that the arguments being passed at higher levels were catastrophically huge.

It was a real problem, using krumo just resulted in out-of-memory errors. And the problem was also happening inside AJAX calls so any attempt to simply print the output was doomed.

I needed a way to get a function backtrace which was not too verbose, didn't crash the machine and would work in an AJAX call.

The solution is the following routine which takes a debug_backtrace() extracts only the information we need and outputs to a file. It's not Drupal-specific but you may need to set up the directory:


/**
 * Backtrace to a file for when nothing else works
 */
function debug_backtrace_file($fname = 'backtrace') {
  $file = fopen("c:\\tmp\\{$fname}.log", 'ab');
  if (!$file) {
    return;
  }
  fputs($file, '=============== ' . date('Y-m-d H:i:s') . " ===============\n");
  $line = __LINE__;
  foreach (debug_backtrace() as $entry) {
    $function = $entry['function'];
    if ($function!=__FUNCTION__) {
      if (isset($entry['class']) && $entry['class']) {
        $function = "{$entry['class']}::$function";
      }
      fputs($file, sprintf("function %s at line %d in file %s\n", $function, $line, isset($entry['file'])?$entry['file']:t('unknown')));
    }
    $line = $entry['line'];
  }
  fclose($file);
}


Happy bug hunting.

Where the **** is it?

You know how it is? You have this complex and massive PHP structure with recursive elements and somewhere in all that is a value you're looking for. Views object I'm looking at you!

Here's a PHP routine, which is not Drupal specific, that will dig down into an object to find the property or array item you're looking for - and it avoids recursion by keeping a list of structures it's already searched by MD5ing the serialized version of the structure.

The parameters are the initial structure to search; the property you're looking for - it can be partial if you're not sure what the property is called; $id would usually be the identifier of the structure you're supplying and $depth should be ignored. It returns an array of strings that show the way into the structure to find the property. It's quite common to get multiple results.

One thing to watch out for: If your structure is recursive (like a Views object) and you start the search deeper into the structure with the idea that you'll shorten the search? Well you might, but if it's recursive you might also end up going into one of the recursed objects because the routine hasn't seen it before.

Anyway, with that in mind, here it is:


function common_locate($struct, $seek, $id = '', $depth = 0) {
  static $structs = array(), $results = array();
  if (is_array($struct) || is_object($struct)) {
    $struct = (array) $struct;
    foreach ($struct as $key => $value) {
      if (strpos($key, $seek)!==FALSE) {
        $results[] = "$id => $key = $value";
      }
      if (is_array($value)) {
        $idx = str_replace('0', 'g', md5(serialize($value)));
        if (!isset($structs[$idx])) {
          $structs[$idx] = "$key:$depth";
          common_locate($value, $seek, "{$id}[$key]", $depth+1);
        }
      }
      elseif (is_object($value)) {
        $idx = str_replace('0', 'g', md5(serialize($value)));
        if (!isset($structs[$idx])) {
          $structs[$idx] = "object:$key:$depth";
          common_locate($value, $seek, "{$id}->$key", $depth+1);
        }
      }
    }
  }
  return $results;
}

Wiki filter for Drupal

One of the known problems with Drupal is no Wiki module. It is possible to put together a Wiki using various resources but the biggest stumbling block is the text filter.


There have been attempts and some successes. The current perceived wisdom is to use FlexiFilter - sorry but it's just too cumbersome. In fact nightmarish.


And I needed a Wiki filter for a project so this evening I spent 5 hours writing a Wiki filter for Drupal 7.


Does pretty much everything you could want including nested OL/UL lists which can be mixed - I only mention that particularly because it was a bitch. Otherwise it's got italics, bold, underline, strikethrough, h2-h6, blockquotes, code (pre), superscript and subscript.


So to set up a Wiki Text format on Drupal 7 you use these in this order:

• Limit allowed HTML tags - to none, to clear out any tags in the text.
• Then my filter
• Convert line breaks
• Assign IDs to anchors (From the TOC module)
• Table of contents filter module
• Freelinking module to deal with URL links

Then have WikiTools hijack Freelinking (it's an option). And configure the rest as desired.


Voila! A Drupal Wiki.


I will be putting it onto drupal.org presently.

Stable Field Value Extraction module released

Okay, after weeks of nobody complaining about any aspect of my field_extract module I have finally got around to issuing the code as the full stable version. Hurray.

Now because I'm lazy I use Eclipse for development purposes - I know, I know, how can I be a proper developer if I don't use Linux and Vim? Well, I don't. I was using command line before you were born (unless you were born before 1983). Been there, done that.

However Eclipse and Drupal Git are strange bedfellows, and it can take a bit of work learning how they can be made to work together.

Getting Drupal repos cloned locally isn't too much of an issue once you've got your SSH keys sorted out, and for cloning you can happily use http.

Pushing upstream is another matter entirely (and you will to need to use just Push... instead of Push upstream... until you get it sorted out and configured). If you try using http you may well hit a brick wall, just as I did. the trick is to use git+ssh for your protocol, and it'll work nicely. One thing, which is obvious unless you forget it, is to include Add all tags spec if you are uploading tags as well as branches. Ahem.

Hopefully it won't be so long before my next posting - I have a fun new website specifically for developers coming up. I think you're going to like it, it provides a service that all developers need from time to time, and there is only one website I've found that fulfils the need, and not as well as my version. It's written in D7 of course, leveraging it as a development framework rather than a CMS.

And with that enigma, I'll leave you.

MD5s as IDs

While there is the chance of duplication it can be handy to use MD5s for creating a "unique" ID for strings. I had this in my current project but I've done it before and there is a potential problem which is more likely than duplicate IDs.

There is a chance that you'll get an MD5 that begins with a zero and, potentially even worse, one that begins with a zero and is all decimal numeric (does not contain a-e digits).

In this instance, with the loose typing of PHP, you might find your MD5 gets converted to a number and loses its leading zero (or zeroes). In which case it's useless as an ID and it will take you a very long time to track it down. I know the first time it happened to me it took me a couple of days.

But there is a very simple solution, when you create your MD5 do an immediate search and replace to change all zeroes into 'g'.

$id = str_replace('0', 'g', md5($source));

Now you can be sure you will never lose your leading zero, because there isn't one.

(By the way, after 12 weeks my field_extract module has received no complaints or bug reports so I shall be promoting it to a full version.)

Ssssh

I've not posted recently because my current job involves Drupal 6 so I've done virtually no D7 work for a while. However I do have a new set of modules for developers coming soon which will be in both D6 and D7 varieties.

Essentially it's a system that does a similar job to CTools plugins but is lightweight and standalone. It's very good for de-coupling dependent custom modules, essentially very simple but very versatile.

There's a core module that provides the API (a very small module indeed) and then some other modules that demonstrate how to use it and provide handy facilities at the same time.

Hopefully I'll have that out by Christmas. I'm also happy to say that my field_extract module is doing very nicely  in the module usage charts currently standing at 99 sites.

Automatic file download

I have a personal project I'm working on which requires an automatic download link. You know the sort of thing - like you get on SourceForge: "Your file will download in x seconds". And then the file download dialog pops up.

All the solutions for this are JavaScript - and if you want a countdown then JS is the way to go. But I didn't want JavaScript.

The basic part of the trick isn't limited to Drupal at all, you just HTML:

<meta http-equiv="Refresh" content="1; URL=http://www.yoursite.com/system/files/downloadfile.ext" />

So here, after one second, the page refreshes - except it's a file to be downloaded, so your browser does the right thing and gives you a download dialog box instead.

Easy.