The History of the Field Complete module

A couple of months ago I was contracted to work on a project for a large UK organisation that provides accreditation for university, and other, further education courses of a certain type. The site had already been mostly designed and built so I had no say in its overall structure. Suffice to say: I wouldn't have done it like that.

Still, we play the hand we're dealt.

The nature of the project meant that the applicant had to fill in absolutely massive forms and provide huge amounts of evidence to show how their course delivered to the standards required for accreditation. But the form could not be submitted for review until it was complete. But the forms are so huge that nobody is going to be able to fill them in in one sitting. In some cases it's expected it would take weeks.

The original developers had decided (sensibly) to use the Content Complete module, which is a watered-down version of making a field "required" - you specify which fields need to be complete and then you can check every time the form is saved. Which was all well and good except for one tiny thing: these forms used Field Collections and Content Complete cannot cope with Field Collections.

The first task on my list was: get Content Complete working with the Field Collections.

I laughed til I stopped.

I did have a look but Content Complete is structurally incapable of handling Field Collections without a major rewrite. Anyway it was worse than that because I had to create new forms which required linking to other entities (via Entity Reference) and they needed to be checked for being complete as well.

Now there was a discussion in the Content Complete issue queue about developing a new version called Entity Complete. However the ideas were overly complex, wanted to carry on using the same interface (which I don't like), no actual work had been done, and the discussion had dried up months ago. Clearly it wasn't happening.

So I had a choice, somehow fix Content Complete, or start again and write the Entity Complete module from scratch myself. So that's what I did. Essentially it intercepts just two hooks to achieve the required result but there are plugins for specialised field types. And lots of frills.

You can find the module itself here. You'll notice it's called Field Complete instead of Entity Complete because someone had already grabbed the 'ec' short title.

And there is lots of lovely documentation with many pictures here.

Enjoy.

Entities vs Nodes

For the last six months I've been working on a Drupal 6 site professionally, but developing a couple of personal Drupal 7 sites. However I haven't been going into D7 in any great detail - except for developing entities.

Now I've moved contracts and I'm developing a new Drupal 7 site to replace an older non-Drupal site with lots of additional facilities. And the Drupal implementation is entirely up to me.

Being an OOP person at heart I love the concept of entities but when working with a commercial website you have to make some serious decisions. My personal preferences have to give way to the reality of building a website that delivers the spec and can be maintained and extended by other developers in the future.

So how do you decide what should be a node and what should be an entity?

There's a line in this blog post which says:

You can now actually create data structures specific to your application domain with their own database table (or any other storage mechanism really) plus a standardised way to add fields to them. No need to turn nodes into something they are not.

Which is technically accurate but does not always provide clear guidance, so here's my step-by-step analysis method to decide whether a specific data structure should be a node or not:

1. Is it content? Is the item definitely stuff that gets turned into HTML and displayed for the user? If you were building a review site, a review would definitely be content, so that's a node.

2. Does it need to have revisions? The node module provides revisions and the associated modules make it easy and powerful. Building revisions into custom entities is hard. So if it has to have revisions then it has to be a node.

3. Is there additional "property" (as opposed to "field") information? I had a situation where I wanted to define a "proxy", and a proxy needs a web address, a port number, maybe a username and password. These are fundamental properties of a proxy. You could add these as fields for a node, of course, but it makes more sense for them to be properties of a proxy entity. So this should be considered as an entity. (Another way of looking at this is: is there a need for a new table containing information specific to this data structure? If so, think entity.)

4. Is the structure normally invisible to the user? That should be an entity.

5. Would using an entity instead of a node obscure the function? Perhaps this is tricky to answer, after all dividing functionality out to a new object should never make things more complex. But it's worth asking the question.

Any other ways of to help make the decision?

Remember: there is virtually no overhead in creating a new entity. And there are huge advantages in additional functionality that the core, Entity API, Views, Features and other entity-related modules can give you.

field_extract module in beta

EDIT: Now out of beta. Fully implemented.

EDIT: You can now get the module from http://drupal.org/project/field_extract and it works properly with Drush.

My incredibly useful field_extract module is now available on drupal.org. I am slightly embarrassed because something went wrong in the project creation process so the actual URL is:

http://drupal.org/project/1158878

And when you download it the module comes inside another folder "1158878". In fact this will work just as it comes, you'll just get the folder 1158878 in your modules folder with field_extract inside that.

If you're competent you can extract the inner "field_extract" module and get rid of the outer 1158878 folder but, as I say, it should work out of the box anyway. (I may get around to re-uploading it properly at some point but I can't wait any longer to get it out to you.)

So what does it do? For a start, this is only a developer's module. It does nothing by itself and should only be downloaded if requested by another module. But what it does do is provide a couple of functions that make it much easier to extract field data from entities.

The project page provides clear instructions on how to use it.

Enjoy.

Coming to a website near you

Now I don't want you to get too excited but I have recently completed two little Drupal 7 modules designed to make my life easier - so may well help other developers:

field_extract
This takes the little function I built to extract values from fields to the extreme by attaching "extractors" to the cached field type data, and then using the appropriate extractor to extract the data from a field. So there's one extractor that gets nodes, another that gets terms, another for fields, for body, for text, for integers and so on. It follows proper Drupal guidelines and is completely extensible.

archive_stream
Makes it easy to build a downloadable Zip archive using stream_wrappers. You give it a temporary filename and an array of file info arrays (or file entities) and it gives you a file path you can use for downloading. The module also handles the downloading part when a user accesses the link.

Simples.

It may take a few days to get them on to drupal.org but I'll let you know.

Getting field data out of entities

EDIT: download the module that encapsulates this behaviour and does a lot more besides: read this blog.

The new field structure in Drupal 7 is very clever but also quite irritating when you want to get data out of a field in an entity, so I have written this handy routine which extracts field data:


/**
 * Returns field values as actual entities where possible,
 * also allows selection of individual items to be returned
 */
function field_fetch_field_values($entity_type, $entity, $field_name, $get_delta = NULL, $get_key = NULL) {
  $values = array();
  if (isset($entity->$field_name) && !empty($entity->$field_name)) {
    foreach (field_get_items($entity_type, $entity, $field_name) as $delta => $item) {
      $value = $item;
      $keys = array_keys($item);
      if (count($keys)==1) {
        $key = $keys[0];
        switch ($key) {
          case 'nid':
            $value = array_shift(entity_load('node', array($item[$key])));
            break;
          case 'uid':
            $value = array_shift(entity_load('user', array($item[$key])));
            break;
          case 'tid':
            $value = array_shift(entity_load('taxonomy_term', array($item[$key])));
            break;
          case 'vid':
            $value = array_shift(entity_load('taxonomy_vocabulary', array($item[$key])));
            break;
          case 'value':
            $value = $item['value'];
            break;
        }
      }
      else {
        if ($get_key && isset($item[$get_key])) {
          $value = $item[$get_key];
        }
        elseif (array_key_exists('value', $item)) {
          $value = isset($item['safe_value']) ? $item['safe_value'] : $item['value'];
        }
      }
      $values[$delta] = $value;
    }
  }
  if (is_numeric($get_delta)) {
    return isset($values[$get_delta]) ? $values[$get_delta] : NULL;
  }
  return $values;
}


Use it like this:

$terms = field_fetch_field_values('node', $node, 'field_myterms');

Extracts all the tids from the field and returns the actual terms, not the tids.

$term = field_fetch_field_values('node', $node, 'field_myterms', 0);

Returns just the first term.

$summary = field_fetch_field_values('node', $node, 'body', 0, 'safe_summary');


Returns only the summary from the body field, but does not create a summary if there isn't one.


$info = field_fetch_field_values('node', $node, 'field_myfiles', 0);


If field_myfiles contains uploaded file data you get the full file info array back, in this example just the first one.


There are probably cleverer ways of doing this (and you could do multiple entity_loads to make it more efficient) but I find this works well enough for normal day-to-day use.

Proper way to get field values

EDIT: download the module that encapsulates this behaviour and does a lot more besides: read this blog.

If you haven't noticed the stored structure of fields in an entity, like a node, is not simple:

$entity ->fieldname[language][delta] = [item]

The Field API does contain useful functions for manipulating these values, so if you have a $node and a field_myfield, you can get the current items like this:

$items = field_get_items('node', $node, 'field_myfield', $node->language);

Which returns an array of the items in the field, indexed by their delta value.

The actual content of each item will be dependent on what it is, a simple textfield will have array('value' => [text]) while a term reference will have array('tid' => [tid]), a "longtext with summary" (such as the standard "body" field) has several values: for the main body ('value'), for the summary ('summary'), for the filter format ('format'), and already processed "safe" values of both the body (safe_value) and the summary (safe_summary).

You can find this, and other useful functions, at Field API.

EDIT: I now have a super-duper function that extracts values for you here.

field_delete_field ooops

I'm in the process of converting a D6 module to several D7 module.

The original required manual set up of a node type and attaching various CCK fields before being able to run - a perfectly OK scenario for Drupal 6. However for D7 I wanted the module to create the node and attach the fields automagically - a perfectly acceptable scenario for Drupal 7. (I've learned only too well recently that entities, though powerful, require a lot of effort so that conversion can wait.)

One thing I've done is extracted the specialist field I need into its own module - in a similar fashion to the various field modules (text, number and so on).

So I put it together and installed on a minimum D7 installation. All good. I uninstall and then reinstall - and en exception gets thrown because the DB tables for my new field have not been renamed. Drupal 7, for speed, renames discarded tables then deletes them at its leisure during hook_cron.

I know this process works because I've seen it in action - when I've been creating fields through the User Interface. But it was failing during hook_uninstall() - in other words field_delete_field() just fails completely but without reporting an error.

Turns out this is a known bug. And it's because when a module is disabled its fields are tagged as inactive and the routine that collects the file_info does not collect information about inactive fields.

The only solution is to use this function from the field.install file:


function _update_7000_field_delete_field($field_name) {
  $table_name = 'field_data_' . $field_name;
  if (db_select($table_name)->range(0, 1)->countQuery()->execute()->fetchField()) {
    $t = get_t();
    throw new Exception($t('This function can only be used to delete fields without data'));
  }
  // Delete all instances.
  db_delete('field_config_instance')
    ->condition('field_name', $field_name)
    ->execute();


  // Nuke field data and revision tables.
  db_drop_table($table_name);
  db_drop_table('field_revision_' . $field_name);


  // Delete the field.
  db_delete('field_config')
    ->condition('field_name', $field_name)
    ->execute();
}

There is a patch on the go for fixing but this will have to do until it arrives.