Nuvole

Measure node completion in Drupal, part 2

Thu, 08 Apr 2010 20:19:19 GMT

This is the second part of the series on Content Complete. If you have missed the introduction of the Content Complete module, please read the first part in this series.

In this post, I’ll guide you to setting up views to show the completion of Drupal nodes using Views. If you don’t know how to work with Views, check out the documentation that is included within the module.

A list view on node completion

Imagine we have 10 artist nodes filled in by different users on the website and we want to have a simple list of those nodes together with their completion status.

Start by adding a new View of type ‘Node’ in your website. For a basic administration view, we add the post date and the title of the node to the ‘Fields’ and add a filter for nodes of content type ‘Album’. To show the completion, add the ‘Content Complete: Completeness’ field from the ‘Content Complete’ group. You can display the field as a Numeric Value or Bar and select the option to show a field that links to the next field to be completed. The module also provides a raw view of the data with ‘Content Complete: Completeness Data’ which will be discussed in the next paragraph. We select the Table style and make the completeness field sortable. Check out the result of this simple view in the second figure.

Total completion of a list of nodes

Another way to output data is to show the total completion on a list of selected nodes, e.g., on all the ‘Album’ nodes. For this we select the Views style ‘Content Complete’ and select the ‘Content Completeness Data’ as a field. The Style will process all the data it receives from the nodes and compute the total completeness and the next field to be completed. This is very useful if you want to show only one completion bar for all your content, as shown in the figure below.

What’s next

In the third and last part of this series, I’ll show you how to set up a user flow to drive users to complete their nodes by sending out email reminders.

Measure node completion in Drupal, part 3

Mon, 12 Apr 2010 11:53:13 GMT

This is the third part of the series on Content Complete. If you have missed the introduction of the Content Complete module, please read the first part in this series.

Thanks to the integration with the powerful Rules module, you can set up complex workflows with data completion in a couple of minutes. I’ll show you how to set up email reminders sent automatically to authors of incomplete Album nodes (< 100%).

If you’re not familiar with the Rules modules, please read this introduction and check out tutorials here.

Workflows on data completion using Rules

You need to download and install the Rules module (Rules, Rules Administration UI, Rules Scheduler), the Views module and the Token Module.

First, we will create a rule set that will continuously reschedule itself until the node is completed.

Go to “admin/rules/rule_sets/add” and fill out the label with “CC Check (Set)” and the machine-readable name with “cc_check”.
In the “Arguments” fieldset select “Content” and “User”.
Back on the Rule Set page, click on the freshly created set and click “Add a new rule”. Set the label to “CC Check”.
Every rule works with conditions (IF) and actions (THEN). Add the condition “Content Complete % is >= than x” and fill out 100 under ”% to compare”. Click on “Negate” such that our condition translates to “if Content Complete % < 100”.
Add the action “Send a mail to a user”. As recipient, select the content’s author. Fill out the subject with for example “Complete your node [node:title]” and the message with something like “Click on [node:site-url]/node/[node:nid]/edit to complete your node.” Check out the Token Replacement Patterns for more options.
We need to reschedule our rule continuously. Add another action “Schedule ‘CC Check (Set)’”, give it an identifier “CC Check ‘[node:title]’” and as scheduled evaluation date put “+1 day”. Make sure you have cron configured to have email reminders sent out daily.

We now have a rule set that checks the completion of the node and, if needed, sends an email reminder to the author of that node. Now we need to trigger that rule set on a Drupal event, for example, every time a node of type ‘Album’ is updated.

Go to “admin/rules/trigger/add” and fill out the label with “CC Check”. As a trigger event, we choose “After updating existing content”.
Add the condition “Content has type” and select the “Album” content type.
Add the action “CC Check (Set)” with Content set to “created content” and User set to “content’s author”.

Test your new workflow by writing an additional message to watchdog and setting the scheduled evaluation date to for example “+1 min”. You can also enable the debugging mode of the Rules module: at the settings page click on “Debug rule evaluation”.

Future of Content Complete: a Complete API

Checking completion for content is just one approach to having your Drupal websites as “complete” as possible. We have already experimented with different ways of “completion”, e.g. by counting user actions as steps in processes that need completion. If you’re interested in for example having your users each add 3 nodes of type ‘Album’ and every node at least filled for 75%, check out the Complete module. That module is related to the Content Complete module, but measures completeness of actions (as measured by the Rules module) instead of completeness of nodes. A combination of both modules can lead to a “Complete API” and some preliminary ideas are under discussion here. Looking forward to your feedback!

Towards a 100% database free development workflow

Mon, 24 May 2010 09:01:07 GMT

Building a website using Drupal is essentially about tuning and extending its core functionalities to meet the project’s specific requirements. During its development phase we create content types with CCK, display them using Views and extend the basic functionalities by setting up contributed modules. All these actions are recorded into the database in the form of a very heterogeneous collection of settings, each specific to the module it belongs to.

When the development phase is completed the database probably contains, besides all our valuable configuration, other Drupal “entities” such as content, users, etc… that we don’t really want to carry along once the website goes live: a careful and time consuming cleanup is then necessary.

With time our website naturally grows in size; it contains much more content and several users are active on it: soon it might be necessary to extend its functionalities. How to do it when both settings and content are now valuable, when really nothing can be lost during development?

The immediate solution could be to work on a copy of the production database and restore it afterwards. This, obviously, means that activities on the production website have to be suspended until the new development is completed. Additionally, a pre-live clean-up will probably be necessary also in this case.

When a team of developers works on the same project issues multiply: if each developer keeps and works on his own database dump the risk of overriding each other’s changes is high. Another option is to work all together the same shared database, but still hoping that nobody will mess things up.

It is clear that relying on the database makes the development flow really error prone, not easily scalable and definitely difficult to maintain.

Settings in code: the key to Drupal heaven

Keeping track of changes, working concurrently on the same project and resolving conflicts are all issues that are usually addressed by the adoption of a version control systems (VCS). Unfortunately they are not that useful when it comes to working with configuration stored in database: you can surely version a dump but change tracking and conflict solving are practically impossible.

Everything changes if configuration is stored “in code”, meaning that, instead to have settings spread across different tables, they are translated to understandable PHP code that Drupal modules can easily consume. Exportable entities are not something new in the Drupal universe: modules like Views and CCK have already supported exporting setting for quite some time. What’s missing is to streamline the export/import operations to, eventually, switch from a database to a code driven development. All this is today possible thanks to the Features project. This project introduces the concept of feature as a collection of settings and configuration that describe a very well defined functionality. In doing so it implements a very solid system of keeping track of changes that occurs to settings “owned” by a particular feature making easy to dump them to code. That’s basically it, the rest is all about wisely organizing our project in features which keeps all our settings safely in code. At this point, by using our favorite VCS, changes get traceable and conflicts resolvable, exactly how it happens with standard PHP code.

Introducing #codepower

Projects like Features, Chaos tool suite or Strongarm are at the heart of the code driven revolution, even though a lot still depends on how actually we are going to use this technology: good code conventions, well defined features boundaries and other good practices can really make the difference, at the end of the day.

What we really need is to share each other’s best practices to develop together patterns for a solid code driven development. For this reason Nuvole introduced, at Drupal VolCon in Antwerp, the #codepower Twitter tag. The idea is simple: we will tweet and tag with #codepower whatever we deem interesting to be shared with the community regarding code driven development. We ask you to do the same.

The Movement

Today we all have the chance to push Drupal even further by changing the way we work and develop applications with it. At Nuvole the adoption of a code driven development has completely revolutionized our working flow, making it more maintainable, solid and scalable. For this reason we want to contribute back to the community our experience about code driven development with a series of blog posts, starting in occasion of the first event on the matter: “Developer Session: Code Driven Development”, the 27th of May 2010, in Antwerp (Belgium) hosted by the great guys at Krimson.

First Code Driven Development meet-up in Antwerp

Fri, 28 May 2010 10:43:02 GMT

Yesterday night the first “Code Driven Development” meet-up took place at the Krimson office in Antwerp (Belgium). Around 30 people were attending the meeting, most of them to learn more about this new “hype”. Nuvole shared its experience in the field presenting “First Steps in Code Driven Development”, both an introduction and a collection of tips and examples for a quick start.

For more read our introductory blog post on Code Driven Development.

Straight after the presentation people were participating in an Open Space about the topic.

(Picture by netsensei)

Nuvole chose the following two sessions:

S.O.S. Features: beginner session about features and code driven development where we had the chance to answer people’s more specific questions about the flow showcasing real-life examples from Nuvole’s projects.
Base Features: session focused on features re-usability and followed by a brainstorming about how features should be designed in order to be more extendible and re-usable.

Then time was up and I had a train to catch. Thanks everybody for the great inspiring evening and, especially, to all the Krimson folks for putting this together.

Code driven development session proposal at DrupalCon Copenhagen 2010

Sun, 11 Jul 2010 08:08:37 GMT

At Nuvole the adoption of a code driven development has completely revolutionized our working flow making it more maintainable, solid and scalable. Projects like Features and Chaos tool suite are at the heart of it, even though a lot depends still on how actually you are going to use them: good code conventions (see the Kit project), well defined features boundaries and other good practices can really make the difference at the end of the day.

Therefore, Nuvole is proposing the session “Code Driven Development: a 100% DB-free development workflow” at DrupalCon Copenhagen 2010 with the aim of sharing good design decisions, code snippets and anything else related to a pure 100% database free development. More specifically, the session will cover:

database driven vs. code driven approach in a nutshell
what code driven development is all about: Features, Strongarm, Drush, exportables…
coding conventions: make the world a better place
your friends hook_install() and hook_update_N()
the controller feature: one feature to rule them all
lots of tips & tricks to take back home

If you are interested in the topic help this session going through the selection process by voting for it.

Measure node completion in Drupal, part 1

Mon, 05 Apr 2010 16:44:31 GMT

Ever needed to measure the completeness of your Drupal nodes? Or wanted to motivate your users to complete their content until it reaches a 100%? In this post, I’ll outline the basic configuration of Content Complete, a new Drupal module that does exactly that. Content Complete started as a simple module to show completeness of nodes, but over time has included Rules support to manage complex user flows, dynamic caching and Views integration. In this first post, I’ll guide you through the steps to configure the module, output the percentage of completeness and show you how to theme the percentage bar. In the second post of this series, I’ll show you how to output all this data using Views.

To start, you need to download and install the module from the download page or directly via Drush.

Configuring the module

Completeness of nodes is measured per content type. For each content type, you need to select the fields you want to be included in the percentage. For example, you have a content type “Artist” with the fields “Title”, “Release Date”, “Label” and “Website”. You want all of those fields to be completed by your users, such that the node is at 100%. If, for example, a user only fills in a value for “Title” and “Website”, the completeness will be at 50%.

To activate content completion checks, go to Administer > Content Management > Content types, and click ‘edit’ on the content type you wish to have checked. Enable the checks, save the content type and reload the page to find the fields you can check for completeness, as shown in Figure 1.

If you are using the Content Profile module to manage user profiles, the module will figure out the profile node that belongs to the logged in user.

Displaying completeness

Content Complete provides various ways to display data like the percentage, a percentage bar and the next field to be completed. These are the different blocks provided by the module:

Content Complete: current node: show completeness for the current node. Can be shown only at the node page.
Content Complete: Album (first node): show completeness for the first node it finds of the specified content type and for which the logged in user has edit permissions. Can be shown at any page. Use this only if you have one node of that content type, for example, the associated profile node of that user.
Views. Will be reviewed in the next post of this series.

To enable a block, go to Administer > Site building > Blocks and drag and drop Content Complete: current node to the region you want to display the block at. Then, navigate to your node to see the block appear (see Figure 2). You can also configure the block to be hidden when 100% is reached.

Who gets to see what: configuring permissions

The completeness block will be shown to every user including the anonymous user. Often, you will want to provide this block only to users who can edit the content. You can change the permissions at Administer > User management > Permissions.

Theming

The module provides several CSS classes and ids for the themer. For example, you can use '.cck-complete-percent-bar-leq-25' to style the appearance of the percent bar if the percent is lower or equal (leq) to 25. Likewise, there are classes for leq-50, leq-75 and leq-100. Absolute numbers can be styled using .cck-complete-percent-bar-x (with x replacing the actual percentage). Here is an example we used in one of our projects:

.content-complete-percent-bar-wrapper {
  background: #333333;
  border:1px solid #666666;
  padding:1px;
  margin: .5em 0;
}
.content-complete-percent-bar {
  height: 5px;
  background-color: #8CC101;
}

What’s next

In the second part of this series, I will show you how to use Views to set up different views on the completeness of your nodes.

Introducing OpenLayers Geocoder

Wed, 07 Apr 2010 14:25:28 GMT

The OpenLayers suite represents a breakthrough in the Drupal mapping solutions. The module allows to combine maps coming from different providers (Google Maps, Yahoo, and many others…) and, by using services like CloudMade or MapBox, it is finally possible to have maps perfectly integrated into the website look and feel. In order to input geospatial information using OpenLayers we have to enable the OpenLayers CCK module: the input widget provides a map where users can easily mark the desired location.

Unfortunately, this approach does not scale to deal with more complex scenarios. Imagine a website where users need to geo-locate restaurants in different cities: the input process will result in a series of very tedious drag-and-zoom operations. Ideally, the only information the user would need to provide is the address or the name of the location he or she is looking for.

Google Geocoding Web Service

The Google Geocoding Web Service V3 is a very powerful service that accepts full street addresses, location names and even known places (i.e. airports, train stations etc.), returning a complete set of information about the location we are looking for. For instance, Let’s have a closer look to the web service response for Pizzeria da Vittorio, Rome:

http://maps.google.com/maps/api/geocode/json?address=Pizzeria+Da+Vittorio,+Rome&sensor=false

results in:

{
  "status": "OK",
  "results": [ {
    "types": [ "point_of_interest", "establishment" ],
    "formatted_address": "Pizzeria da Vittorio di Martino Enzo, Via Benedetto Croce, 123, 00142 Rome, Italy",
    "address_components": [ {
      "long_name": "Pizzeria da Vittorio di Martino Enzo",
      "short_name": "Pizzeria da Vittorio di Martino Enzo",
      "types": [ "point_of_interest", "establishment" ]
    }, {
      "long_name": "123",
      "short_name": "123",
      "types": [ "street_number" ]
    }, {
      "long_name": "Via Benedetto Croce",
      "short_name": "Via Benedetto Croce",
      "types": [ "route" ]
    }, {
      "long_name": "Rome",
      "short_name": "Rome",
      "types": [ "locality", "political" ]
    }, {
      "long_name": "Rome",
      "short_name": "RM",
      "types": [ "administrative_area_level_2", "political" ]
    }, {
      "long_name": "Lazio",
      "short_name": "Lazio",
      "types": [ "administrative_area_level_1", "political" ]
    }, {
      "long_name": "Italy",
      "short_name": "IT",
      "types": [ "country", "political" ]
    }, {
      "long_name": "00142",
      "short_name": "00142",
      "types": [ "postal_code" ]
    } ],
    "geometry": {
      "location": {
        "lat": 41.8428020,
        "lng": 12.4858480
      },
      "location_type": "APPROXIMATE",
      "viewport": {
        "southwest": {
          "lat": 41.8344890,
          "lng": 12.4698406
        },
        "northeast": {
          "lat": 41.8511139,
          "lng": 12.5018554
        }
      }
    },
    "partial_match": true
  } ]
}

As you can see the web service tells us pretty much everything we want to know about the location. It’s time to use all this awesomeness to make the user’s life shining again!

Meet OpenLayers Geocoder

OpenLayers Geocoder provides a new input widget for OpenLayers CCK fields that makes location spotting a fast and painless experience. All we need to do is to select the OpenLayers Geocoder input widget from within the CCK field setting page.

After enabling the OpenLayers Geocoder widget, adding a restaurant is all about providing its name or address: OpenLayers Geocoder will provide the user with a list of possible locations.

Get the most out of the web service response

We have only used the geospatial information from the web service response so far: latitude and longitude to center the map and the boundary box to nicely fit the desired location. Besides that, the response contains really valuable information about the location we were looking for, like postal code, administrative area, city, country, etc.

OpenLayers Geocoder gives the possibility to fill CCK text fields automatically on the node submission form with data coming from the response object. This is possible thanks to the integration with the Token module: all address parts are exposed and ready to be used as replacement patterns. To enable the autofilling we have to visit the OpenLayers CCK field setting page in order to map which token is going to fill which text field.

When we will be looking for a place, information like city, country, etc. will be automatically fetched into the selected text fields.

This gives us a great flexibility: we can display our nodes by city using Views or search through them using Faceted Search.

Future development: Reverse geocoding

Another very interesting feature of the Google Geocoding Web Service is the possibility to perform reverse geocoding. As the name suggests, this time the user will spot a location on the map and the web service will return the closest address to the given point. OpenLayers Geocoder will have support for reverse geocoding in one its next releases.

Drupal database backup

Sat, 03 Apr 2010 15:50:59 GMT

In order to backup a Drupal site you need to take care of both code and database. The first problem is usually solved by placing the Drupal directory trees under Revision Control. The second requires a more careful analysis, and this post describes optimal stategies for MySQL.

Before you start: common caveats

Use a lock file

While the backup of files is usually more predictable, the database dumps can take more time than you expect: il will depend on site traffic and activity too; the usual good practice of using a lock file is a must in this case.

Keep your database clean

Drupal has known problems in clearing some cache tables, that are left unaffected by the normal cache clearing in Drupal or Drush. If you extensively use AJAX in forms, your cache tables can very quickly grow to hundreds of MBytes. Solve this by clearing expired cache elements manually; for example, with Drush, just run

drush sql-query "DELETE FROM cache_form WHERE expire < UNIX_TIMESTAMP(NOW())"

in your site directory tree to clean the DB.

What do you need the database dump for?

The best database backup stategy depends on how you actually expect to use the dump.

Use case 1: Verbatim copy of the database

This is not what you want ordinarily. Even if you do a good job of keeping your cache clear and deal manually with the tables that Drupal doesn’t clear, the database will still contain a lot of information that is useful only for forensic-like examinations, like session information and watchdog messages.

If this is what you want, meet mysqlhotcopy, the most efficient way to perform a verbatim dump.

Examples (assuming that everything happens on localhost, that your database username is drupal, password is xyz, database name is dbname):

mysqlhotcopy --user=drupal --password=xyz dbname

will (re)create a database named dbname_copy and populate it with an exact copy of the dbname database. To restore the site database exactly like it was before the dump you merely need to edit settings.php and replace dbname with dbname_copy.

mysqlhotcopy --user=drupal --password=xyz dbname directory

copies the full database contents into the specified directory. This way you can store, by changing directory name, an arbitrary number of backups. Backups are restored by copying the files back to the MySQL database directory with cp -r or rsync -a (recommended, especially for remote use, since it will compress data and it won’t replace a file with an identical copy, thus giving the same result in much less time).

Main problems with mysqlhotcopy:

You need suitable privileges on the filesystem: not applicable on shared hosting most of the times.
The dump is not human-readable: you won’t be able to easily compare two dumps.

Use case 2: Copying the meaningful tables from the database

If your backup is more for safety than for archival, then you will probably want to be more careful in selecting what to dump and how to dump it. In most cases the best solution is a selective SQL dump.

Investigate at least the following:

Drush (namely, drush sql-dump) can be enough for you. You can tell to only dump meaningful tables with the --structure-tables-key=structure-tables option (to use it, rename the file example.drushrc.php to drushrc.php), and you can enable slower but more usefully formatted dumps (one line per insert, to make the comparison of different dumps much easier) with --ordered-dump. A typical dump command would thus look like

drush sql-dump --structure-tables-key=structure-tables --ordered-dump --result-file=dumpfile.sql

Backup and migrate is an ordinary Drupal module, with very flexible backup options and scheduling. All of this is configured and used from within Drupal, so no command line hacking needed. Depending on your infrastructure, Backup and migrate will take advantage of existing server capabilities like FTP transfers and backup encryption.

Main problems with this approach:

You still need command-line access for Drush and enough flexibility for Backup and Migrate; though, you will be able to use Backup and Migrate on most shared hosting solutions.
The dump is much more computationally expensive than the “raw” dump of use case 1: you will pay for the tuning possibilities and the readable SQL output with significantly longer execution times.

Once upon a time, database dumps were the only way to transfer a site, complete with full configuration, to other developers or to develop a feature on a staging site a migrating it to production.

Fortunately, this has changed in recent times: the Features module allows to export most configuration in code and enable it on a new site as easily as enabling an ordinary module. Take the time to investigate Features, or simpler dedicated modules like the built-in export functionalities in CCK for content type definitions, or Drush Views to export and import views from database to code and vice versa; don’t rely on database dumps if at all possible.

If you do want to share a database for this purpose, take a look at dbscripts; configuration is a bit tedious, but it will allow you to define what you dump in different scenarios and how to merge the dumped database with the “production” one.

This article is released under the Creative Commons Attribution Noncommercial No Derivative Works 3.0 license.

Development sandbox with Git and Drush

Tue, 13 Apr 2010 08:19:44 GMT

I bet every Drupal developer keeps a Drupal installation as local sandbox where they try out code snippets, maintain modules or generate patches to be contributed back to the community. If not well maintained this installation will, day after day, grow into a chaotic mess of content and settings. Having separate Drupal installations for each project we work on would be a good solution here.

”Divide et impera” with Git

Git is one of the rising stars in the version control system world and the main candidate in replacing the obsolete Drupal CVS infrastructure. After installing Git, setting up a repository is a piece of cake, in your project root type:

$ git init

and there you go, you are now ready to commit, branch, merge or revert your sandbox code. Having your code under version control is generally a good practice: you can be sure that nothing will get really lost and, more important, you can branch your code according to what you are working on, to help keeping things in order. To create a branch using Git type:

$ git branch my_module

You can list all your branches by typing git branch, you’ll be working on master branch by default:

$ git branch my_module
  my_module
* master

To actually move the development to that branch type:

$ git checkout my_module

After applying your changes you can commit to the branch and move back to master to do something else. You can learn more about a normal Git workflow here, for more information about branching have a look at Basic Branching and Merging.

You can branch your sandbox per module, appending version information if you wish, and everything will be safely separated. On my local sandbox the situation looks like this:

$ git branch
  context-6.x-3.0-beta4
  faceted_search-6.x-1.x-dev
  feeds-6.x-1.x-dev
* master
  node_widget-6.x-1.x-dev
  openlayers_geocoder-6.x-1.x-dev
  openlayers_geocoder-6.x-2.x-dev
  timeline-6.x-2.x-dev
  unlimited_css-6.x-1.x-dev

The sandbox is divided per module and version: this division makes it easy to change the development focus quickly just by checking out different branches. Let’s say a new bug needs to be fixed in the OpenLayers Geocoder module, 2.x branch, we type:

$ git checkout openlayers_geocoder-6.x-2.x-dev

A CVS checkout of the module is available under sites/all/modules/devel/openlayers_geocoder/ (CVS metadata are also committed to Git). After fixing the bug we run:

$ git commit -a -m "Fixing a bug."

and we are now ready to focus on something else. What shown so far is not going to work correctly for the very simple reason: code is not (yet) the king in Drupal, the database is. All settings and content you use to develop and test your code is, obviously, stored into the database, what we need to do is to dump and commit the database:

$ git checkout master
$ drush sql-dump --result-file="dump.sql"
$ git add dump.sql
$ git commit -a -m "Adding dump."

This must be done in the master branch before branching, so we will always carry our dump along. After changing branch (e.g.: git checkout my_module) we must remember to restore the dump in order to have our Drupal sandbox work correctly. It’s clear that the database restoring must happen after every checkout.

Automation with Drush and Git hooks

Git, as other version control systems, exposes a series of hooks. Hooks are bash scripts that are ran in conjunction with a specific Git event. The main Git hooks here are located under .git/hooks directory:

  applypatch-msg.sample
  commit-msg.sample
  post-checkout.sample
  post-commit.sample
  post-receive.sample
  post-update.sample
  pre-applypatch.sample
  pre-commit.sample
  pre-rebase.sample
  prepare-commit-msg.sample
  update.sample

If we are interested in post-checkout.sample: to activate a hook we just need to remove the .sample extension. In order to automate our dump restore we paste into post-checkout the following code:

  #!/bin/bash

  # $3 is equal to 1 in case of a branch switch or after git clone, 0 otherwise
  if [ $3 ]; then
    echo "Restoring database."
    drush sql-cli < sites/all/database/development.sql
    echo "Clearing cache."
    drush cc all
    echo "Done."
  fi

This bash script restores the database and clears the cache. We are now able to switch from one branch to another carrying along all our content, settings, etc.:

$ git checkout node_widget-6.x-1.x-dev 
Checking out files: 100% (933/933), done.
Switched to branch 'node_widget-6.x-1.x-dev'
Restoring database.
Clearing cache.
'all' cache was cleared
Done.

$ git checkout openlayers_geocoder-6.x-2.x-dev
Switched to branch 'openlayers_geocoder-6.x-2.x-dev'
Restoring database.
Clearing cache.
'all' cache was cleared
Done.

On the other hand, dumping and committing the database before switching branch is not automatic since we want to have control of what changes need to stay: it will happen that you just checkout a branch, make a quick test and you want to forget about it.

Getting to the speed of light: Git bash completion

If you want to attach name code version to your branch or other information you will end up having really long branch names. If you don’t want to type them all the time get to the speed of light with Git bash completion.

Share changes with your team using Features and upgrade paths

Mon, 05 Jul 2010 11:51:03 GMT

The Features module is great for keeping configuration in code, with just a few clicks you can bundle your settings together and be sure that changes will be tracked and safely dumped to code at the next feature update.

However, often you will want to share other kinds of modifications with your team. Imagine you suddenly need to add a Taxonomy vocabulary or to enable a couple of modules your feature will be dependent on. Since your team’s workflow is completely 100% database free, you cannot pass those changes by simply sharing your database. Still, you need to push those changes to other developers, who already enabled the feature you have been modifying in their development environment. How do you do that?

Make your workflow solid: meet hook_update_N()

Features are modules and modules can have their own upgrade path by implementing [hook_update_N()](http://api.drupal.org/api/function/hook_update_N). That’s exactly what you need.

All the changes the Features module is not keeping track of must be stored in sequential implementation of the hook_update_N() to be sure that other developers will have them replicated in their database by simply visiting update.php.

Have a look at the following example: we want to add a free-tagging taxonomy vocabulary to our feature. First of all we need to enable the Taxonomy module add it as dependency in its .info file:

dependencies[] = "taxonomy"

Adding a dependency will not enable the new module on the other developers’ local copy, since the feature has already been enabled and dependencies will not be rechecked. To be sure the module will be enabled we have to write an update function:

<?php
/**
 * Enabling Taxonomy module.
 */
function feature_example_update_6001() {
  $return = array();
  drupal_install_modules(array('taxonomy'));
  $return[] = array('success' => TRUE, 'query' => 'Enabling Taxonomy module.');
  return $return;
}
?>

Now it’s time to create our free-tagging vocabulary. Since taxonomies are still not supported by the Features module we have to find another way to put this change in code. One way to do it is to create the vocabulary in an update function:

<?php
/**
 * Create "Tags" vocabulary.
 */
function feature_example_update_6002() {
  $return = array();
  $vocab = array(
    'name' => 'Tags',
    'multiple' => 0,
    'required' => 0,
    'hierarchy' => 0,
    'relations' => 0,
    'weight' => 0,
    'nodes' => array('story' => 1),
    'tags' => TRUE,
    'help' => t('Enter tags related to your post.'),
  );
  taxonomy_save_vocabulary($vocab);  
  $return[] = array('success' => TRUE, 'query' => 'Create "Tags" vocabulary.');
  return $return;
}
?>

We can now commit our changes. Once the other developers pull the latest version of our feature in their local environment all they need to do is to run a Drupal update by visiting update.php.

The comment on top of the update functions above plays an important role in the workflow since both Drush and update.php are actually able to display it when running drush updatedb or running update.php, giving valuable information to the other team members:

$ drush updatedb
The following updates are pending:

 feature_example module               
 6001 - Enabling Taxonomy module. 
 6002 - Create "Tags" vocabulary. 

Do you wish to run all pending updates? (y/n):

Make your features database free: meet hook_install()

Storing your changes in hook_update_N() will allow your team to be always up-to-date with the latest development of your feature, making your development workflow solid and maintainable. But what happens if a new developer wants to hop in? Our rule of thumb is “No database sharing”, hence the new developer needs to install our project from scratch and, still, we need to guarantee that he will get the complete latest status of the project. To do that we need to copy some of the configuration we have been placing in update functions to the hook_install():

<?php
/**
 * Implementation of hook_install()
 */
function feature_example_install() {
  $vocab = array(
    'name' => 'Tags',
    'multiple' => 0,
    'required' => 0,
    'hierarchy' => 0,
    'relations' => 0,
    'weight' => 0,
    'nodes' => array('story' => 1),
    'tags' => TRUE,
    'help' => t('Enter tags related to your post.'),
  );
  taxonomy_save_vocabulary($vocab);  
}
?>

As you might have noticed hook_install() copies code from the feature_example_update_6002() and not from feature_example_update_6001(). This is because the two updates have a different nature: 6002 is a structural update, meaning that it is something we must guarantee even if the feature will be installed from scratch; 6001 is a development update which only aims to upgrade an already working development copy.

When writing your upgrade paths, it’s good practice to distinguish between two kinds of updates, and in the case of a structural update, make sure you copy changes to the hook_install() of your feature.

Real life examples

Upgrade and install hooks are really powerful, you can put virtually anything in there. Below we list some real life example of upgrade paths we use in our projects at Nuvole.

<?php
/**
 * Implementation of hook_install()
 */
function master_site_install() {
  
  // Create a custom menu called "Manage Content".
  db_query("INSERT INTO {menu_custom} (menu_name, title, description) 
            VALUES ('%s', '%s', '%s')", 
            'menu-content', 
            'Manage Content', 
            'Manage your site content.');

  // Add "Home" menu item to Primary Links menu. 
  $item['link_title'] = t('Home');
  $item['link_path'] = '';
  $item['menu_name'] = 'primary-links';
  $item['weight'] = -10;
  menu_link_save($item);
}
?>

Add OpenId to admin account

<?php
/**
 * Add Nuvole OpenID to admin account.
 */
function nuvole_site_update_6004() {
  $return = array();
  // Delete any other association of the OpenId account to avoid conflicts.
  $return[] = update_sql("DELETE FROM {authmap} 
                          WHERE authname = 'http://nuvole.myopenid.com/'");  
  // Bind OpenId account to admin user.
  $return[] = update_sql("INSERT INTO {authmap} (uid, authname, module) 
                          VALUES (1, 'http://nuvole.myopenid.com/', 'openid')");  
  return $return; 
}
?>

Upgrading data from OpenLayers 1.x to 2.x

<?php
/**
 * Upgrade OpenLayers 1.x to 2.x: WKT data in "content_type_opera"
 */
function publicopera_site_update_6007() {

  // OpenLayers 2.x stores WKT values as geometry collection.
  // Update data accordingly.
  db_query("UPDATE {content_type_opera} SET field_opera_map_openlayers_wkt = 
            CONCAT('GEOMETRYCOLLECTION(', field_opera_map_openlayers_wkt, ')')");
  return array(array('success' => TRUE, 'query' => 'All WKT content updated.'));
}
?>

Features based development workflow

Tue, 24 Aug 2010 07:41:34 GMT

Nuvole is going to present and discuss its development workflow, entirely based on code and Features, at the DrupalCon Copenhagen 2010 in a few minutes!

Please find it as a Slideshare embed below, or attached in PDF format.

drupalcon2010.pdf (1.9 MB) download

DrupalCon Chicago training: Use Features Effectively

Wed, 03 Nov 2010 18:57:41 GMT

Nuvole was selected as a trainer for the DrupalCon Chicago 2011 pre-conference training day!

In a full-day training aimed at Drupal developers we will show how to improve the Drupal development workflow, especially regarding traceability of changes, smooth upgrade of production sites and distributed team workflow, by using code-driven development techniques and Features.

The program is a slightly condensed version of our Code-driven Drupal Development course, focused on:

how to put the whole site configuration in code, even when a module does not offer a proper Features integration;
the usage of a controller feature to manage site updates;
the update process and the difference between structural and incremental updates;
tips to deploy features faster by using Drush and an optimal shell configuration;
how to write reusable features.

For more details see the DrupalCon page about this training.

Typing less in Drush: aliases and autocompletion

Tue, 09 Nov 2010 13:55:07 GMT

While Drush is a big time saver, the textbook version of some commands can become lenghty at times:

$ drush pm-enable faceted_search
$ drush features-revert nuvole_news

An out-of-the-box Bash completion for Drush is in the works, and the Drush shell (“drush core-cli”) already offers fast access to commands and fancy features. However, if you prefer not to leave your shell and use a stock Drush, here are some tips for a faster Drush experience involving aliases and autocompletion.

Drush and Bash Aliases

In this section we’ll assume you use the Bash shell, which may not be the default on your system; however, the techniques listed here can work, or be adapted to work, on other shells too.

Drush aliases

Most Drush commands provide aliases, so it’s relatively rare that you have to type the full command. Just look them up in the Drush help page.

For example, the short form of pm-enable is just en, while the short form of pm-disable is just dis. Typing

$ drush pm-enable faceted_search
$ drush pm-disable faceted_search

can thus be shortened to simply

$ drush en faceted_search
$ drush dis faceted_search

Bash aliases for Drush

You can configure Bash in order to define shortcuts for the most frequently used commands, including Drush constructs. This is done using the alias builtin. Normally you’ll want to add the list of aliases to the .bashrc file in your home directory (or/and to .bash_profile or .profile, depending on your configuration; you might have to create it from scratch too).

Here is a sample configuration for a workflow based on Features and Drush; of course, adapt to taste.

# Drupal and Drush aliases.
# To be added at the end of .bashrc.
alias drsp='cp sites/default/default.settings.php sites/default/settings.php'
alias drcc='drush cache-clear all'
alias drdb='drush updb && drush cc all'
alias drdu='drush sql-dump --ordered-dump --result-file=dump.sql'
alias dren='drush pm-enable'
alias drdis='drush pm-disable'
alias drf='drush features'
alias drfd='drush features-diff'
alias drfu='drush -y features-update'
alias drfr='drush -y features-revert'
alias drfra='drush -y features-revert all'
alias dr='drush'

To be able to use the new aliases, you need to run source .bashrc, or just logout and login again. Note that by typing dr and then the TAB key twice you will see all aliases.

Using autocompletion

The advanced Bash Completion is a tremendously efficient and addictive extra functionality for Bash: it extends the traditional Bash TAB key completion in a smart way, so that by typing ssh and then pressing TAB you get a list of known hosts, and similarly gunzip and then TAB will show you the list of .gz files only. We want to use this powerful functionality to autocomplete modules (or themes, easily extendable) and features names in Drush.

We will first need a function that generates all project names. This could be done in a semantic way with drush pm-list --pipe, but this would involve bootstrapping Drupal and it can at times be too slow when typing. We thus choose a syntactic approach and complete based on results of the find utility, which is less smart since it cannot query the Drupal database and act accordingly, but is generally much faster.

We assume we are in the Drupal root for the time being; then we will remove this limitation. A textbook completion function is:

_drupal_modules_in_dir()
{
  COMPREPLY=( $( compgen -W '$( command find $1 -regex ".*\.module" -exec basename {} .module \; 2> /dev/null )' -- $cur  ) )
}

_drupal_modules()
{
  local cur
  COMPREPLY=()
  cur=${COMP_WORDS[COMP_CWORD]}
  # We temporarily assume we are in the Drupal root.
  _drupal_modules_in_dir "sites profiles modules"
}

To specify that this completion function is to be used to autocomplete the argument of, say, dren, we will just have to specify (a full example will follow):

complete -F _drupal_modules dren

Now let’s see how to find the Drupal root. As a byproduct we will be able to type cdd views and go from anywhere in the Drupal root to the directory containing the Views module. Again, the smart way would involve a (slow) call to Drupal, so we use filesystem-based heuristics.

_drupal_root() {
  # Go up until we find index.php
  current_dir=`pwd`;
  while [ ${current_dir} != "/" -a -d "${current_dir}" -a \
          ! -f "${current_dir}/index.php" ] ; 
  do
    current_dir=$(dirname "${current_dir}") ;
  done
  if [ "$current_dir" == "/" ] ; then
    exit 1 ;
  else
    echo "$current_dir" ;
  fi 
}

These snippets can now be assembled to obtain a full solution, ready to be added to your .bashrc file and able to handle Features and all aliases.

The full package of Drush-Bash tricks to add to your .bashrc file can be found in attachment.

But remember that everything should be adapted to your needs and configurations: for example, you might want to handle themes too, or use conventions to place features in a features/ subdirectory and thus improve performance, or add support for site aliases…

For a system-wide and cleaner implementation, you can copy the second part of the Drush-Bash tricks file to a file named /etc/bash_completion.d/drush_custom.

Examples

Time to play with our new functions! Here are a few use cases.

Change to the Drupal root from anywhere within the Drupal directory tree:

$ cdd

Autocomplete a module name (example: “fac” for “faceted_search”) and change into its directory:

$ cdd fac    # Then press TAB
$ cdd faceted_search

Autocomplete a module name (example: “fac” for “faceted_search”) and enable it:

$ dren fac    # Then press TAB
$ dren faceted_search

Autocomplete a feature name and revert it:

$ drfr n    # Then press TAB
$ drfr nuvole_
nuvole_cases      nuvole_site       nuvole_solutions

From anywhere within the Drupal tree, go back to the Drupal root, run svn update and come back again to the previous working directory:

$ (cdd && svn up)

The techniques explained in this article are part of the Nuvole Code-Driven Drupal Development course and will be covered in deeper detail in the DrupalCon Chicago 2011 training by Nuvole: Code-driven Development: Use Features Effectively.

Advanced Open Atrium customization

Fri, 17 Dec 2010 16:49:39 GMT

Open Atrium comes, out of the box, with a rather complete set of features which allow to bootstrap a functional intranet solution in a few minutes. Still, to be actually usable in a real life context, an Open Atrium installation must be adapted to each organization’s needs and structure.

In this article we will learn how to customize an Open Atrium installation in a code-driven fashion, without hacking its core and making sure that an upgrade to a newer version will not swipe away our valuable changes; we will cover:

How to add custom user profile fields;
How to override Atrium’s core configuration;
How to create custom group types.

For a smooth reading you should be familiar with Features and Exportables concepts. If you are not, just check out our Code Drive Development blog post series.

1. How to add custom user profile fields

One of the greatest changes introduced with the Features module 1.0 stable release is that, when exporting, you can actually decouple CCK fields definition from the node type they belong to, or in other words: CCK fields and node type can be exported to different features.

As a direct consequence of this approach we can store our custom user profile fields in a feature (call it custom_profile) that extends the Atrium’s core atrium_profile. Let’s have a look to its .info file:

core = "6.x"
dependencies[] = "features"
dependencies[] = "text"
description = "Custom profile feature."
features[content][] = "profile-field_profile_city"
features[content][] = "profile-field_profile_country"
name = "Custom Profile"
package = "Features"
project = "custom_profile"
version = "6.x-1.0"

This makes sure that, when Open Atrium will be upgraded, we will still find our custom fields (namely “City” and “Country”) since they belong to the custom_profile feature.

2. How to override Atrium’s core configuration

The Features module exposes several hooks that alter the default settings of exported components, like variables, permissions, contexts, spaces, etc… A correct use of these hooks makes it possible to safely override default component settings being sure that they will not get lost after an Atrium core upgrade.

Let’s say we want to change the system’s default user picture: Open Atrium stores it into a variable, so all we need to do is to implement hook_strongarm_alter() and override its value:

<?php
/**
 * Implementation of hook_strongarm_alter()
 */
function custom_profile_strongarm_alter(&$items) {
  if (isset($items['user_picture_default'])) {
    $items['user_picture_default']->value = 'sites/all/themes/custom/user.png';
  }
}
>?

After clearing the cache we will see our new user picture appearing everywhere on the site. Same goes for the default blocks we have on the dashboard of a newly created group: we can choose, for example, to replace the “Welcome” video with the “Latest discussion” block. Here is how to do that:

<?php
/**
 * Implementation of hook_spaces_presets_alter()
 */
function custom_group_spaces_presets_alter(&$items) {

  // Store a reference to our target block section.
  $blocks = &$items['atrium_groups_private']->value['context']['spaces_dashboard-custom-1:reaction:block']['blocks'];
  
  // Remove "Welcome" block.
  unset($blocks['atrium-welcome_member']);

  // Add "Latest discussions" block.
  $blocks['views-blog_listing-block_1'] = array(
    'module' => 'views',
    'delta' => 'blog_listing-block_2',
    'region' => 'content',
    'weight' => 1,
  );
}
>?

After clearing the cache all new groups (and those that didn’t have their dashboard customized by an user) will inherit the new settings. If we now have a look at our features status we’ll see that several Atrium core features have been marked as “Overridden”:

 $ drush features
 
 Name                 Feature              Status    State        
 Atrium               atrium               Enabled   Overridden   
 Atrium Activity      atrium_activity      Enabled                
 Atrium Blog          atrium_blog          Enabled      
 Atrium Notebook      atrium_book          Enabled                
 Atrium Calendar      atrium_calendar      Enabled                
 Atrium Case Tracker  atrium_casetracker   Enabled                
 Atrium Groups        atrium_groups        Enabled   Overridden   
 Atrium Members       atrium_members       Enabled   Overridden   
 Atrium News          atrium_news          Enabled                
 Atrium Pages         atrium_pages         Enabled                
 Atrium Profile       atrium_profile       Enabled

This is a totally safe situation since your configuration has been altered via code, thus it is 100% upgrade-safe. To learn more about altering default hook have a look at Feature’s feature.api.php file.

3. How to create custom group types

In the example above we learnt how to replace a block in the default “Private group” space preset. What if we need a completely new group type, with custom features and specific configurations, something that we cannot really map into one of the two group types (“Public group” and “Private group”) provided by Open Atrium? Easy, we just need to provide a custom space preset that describes the kind of group we need.

For example: we want to have a “Portal group” which will be used as the public website of our organization, it will have news and static pages and it must look slightly different than the intranet. The Spaces module is powerful enough to allow you to turn each group into a completely different website, customizing settings, menus, theme, etc…

To do that, we start by cloning the “Controlled group” space preset into our new “Portal group” preset. We then export it to a new feature called atrium_portal, having an .info file similar to:

core = "6.x"
dependencies[] = "atrium_news"
dependencies[] = "atrium_pages"
dependencies[] = "context"
dependencies[] = "menu"
dependencies[] = "spaces"
features[context][] = "layout_portal"
features[ctools][] = "context:context:3"
features[ctools][] = "spaces:ctools:3"
features[menu_custom][] = "menu-portal"
features[menu_links][] = "menu-portal:calendar"
features[menu_links][] = "menu-portal:dashboard"
features[spaces_presets][] = "atrium_portal"
name = "atrium_portal"
package = "Features"

As you might have noticed, the file contains a dependence on two features that are not part of the Open Atrium core: atrium_news and atrium_pages, which respectively add news publishing and usual static pages to our new portal.

So let’s have now a closer look at the heart of this feature, the atrium_portal Spaces preset;

<?php
/**
 * Implementation of hook_spaces_presets().
 */
function atrium_portal_spaces_presets() {
  $export = array();
  $spaces_presets = new stdClass;
  $spaces_presets->disabled = FALSE; 
  $spaces_presets->api_version = 3;
  $spaces_presets->name = 'atrium_portal';
  $spaces_presets->title = 'Portal group';
  $spaces_presets->description = 'Turn a group into a public portal. All users may view public content from this group. Users must request to join this group.';
  $spaces_presets->space_type = 'og';
  $spaces_presets->value = array(
    'variable' => array(
      'spaces_og_selective' => '1',
      'spaces_og_register' => 0,
      'spaces_og_directory' => 0,
      'spaces_og_private' => 0,
      'spaces_features' => array(
        'atrium_blog' => '0',
        'atrium_book' => '0',
        'atrium_calendar' => '1',
        'atrium_casetracker' => '0',
        'atrium_members' => '1',
        'atrium_news' => '1',
        'atrium_pages' => '1',
        'atrium_shoutbox' => '0',
        'spaces_dashboard' => '1',
      ),
      'spaces_setting_home' => 'dashboard',
      'site_frontpage' => 'dashboard',
      'menu_primary_links_source' => 'menu-portal',
      'menu_default_node_menu' => 'menu-portal',
      'designkit_color' => array(
        'background' => '#3399aa',
      ),
      'designkit_image' => array(
        'logo' => 0,
      ),
    ),
    'context' => array(
      'spaces_dashboard-custom-1:reaction:block' => array(
        'blocks' => array(
          'views-atrium_news-block_3' => array(
            'module' => 'views',
            'delta' => 'atrium_news-block_3',
            'region' => 'content',
            'weight' => 0,
          ),
          'views-calendar_listing-block_1' => array(
            'module' => 'views',
            'delta' => 'calendar_listing-block_1',
            'region' => 'right',
            'weight' => 0,
          ),
        ),
      ),
      'spaces_dashboard-custom-2:reaction:block' => array(
        'blocks' => array(),
      ),
      'spaces_dashboard-custom-3:reaction:block' => array(
        'blocks' => array(),
      ),
      'spaces_dashboard-custom-4:reaction:block' => array(
        'blocks' => array(),
      ),
      'spaces_dashboard-custom-5:reaction:block' => array(
        'blocks' => array(),
      ),
    ),
  );

  // Translatables
  // Included for use with string extractors like potx.
  t('Portal group');
  t('Turn a group into a public portal. All users may view public content from this group. Users must request to join this group.');

  $export['atrium_portal'] = $spaces_presets;
  return $export;
}
>?

The first important customization is the set of features we are enabling for our new Portal preset. Since the set of features available to each group is stored into the spaces_features variable the Spaces module can override its value when we are browsing a Portal group. Here what we have enabled:

<?php
...
'spaces_features' => array(
  'atrium_blog' => '0',
  'atrium_book' => '0',
  'atrium_calendar' => '1',
  'atrium_casetracker' => '0',
  'atrium_members' => '0',
  'atrium_news' => '1',
  'atrium_pages' => '1',
  'atrium_shoutbox' => '0',
  'spaces_dashboard' => '1',
),
...
>?

This is generally true with all those settings that are stored into variables, like default theme, primary links menu, default user picture, etc…

With a space preset we can also override the default block settings of the Open Atrium dashbaord. For example, for our Portal we want to show a news block on the content region and a calendar with the latest events on the right sidebar region. Here is how to do that:

<?php 
...
'context' => array(
  'spaces_dashboard-custom-1:reaction:block' => array(
    'blocks' => array(
      'views-atrium_news-block_3' => array(
        'module' => 'views',
        'delta' => 'atrium_news-block_3',
        'region' => 'content',
        'weight' => 0,
      ),
      'views-calendar_listing-block_1' => array(
        'module' => 'views',
        'delta' => 'calendar_listing-block_1',
        'region' => 'right',
        'weight' => 0,
      ),
    ),
  ),
...
>?

With this simple space preset we can actually turn an Open Atrium group into a real public portal for our organization.

A different way to look at Open Atrium

A stable code-driven customization joint to the powerful Spaces module could push Open Atrium much far behind its original private-intranet. This great distribution, in fact, can be seen as a versatile platform that can answer to the most diverse needs of an organization providing, for example, groups that can be turn into full featured public portals or used to organize events, etc… With the right customization each group could also be heavily customizable, letting the administrator to completely change their look and feel to serve the most different purposes.

If you are interested in learning more about the topic covered in this blog post you can check out our Drupal Con Chicago 2011 session proposal here.

Open Atrium, Beyond the Intranet

Mon, 07 Feb 2011 16:12:45 GMT

Open Atrium is designed to be a powerful Drupal-based Intranet solution, but the underlying technology allows to do much more: creating a public portal with a totally different graphic design and complementing it with a private Intranet; creating other group presets than the default “public” and “private” group; customizing the profile fields in a clean and modular way, and several other possibilities.

Here is the Nuvole presentation from the Drupal Dev Days in Brussels (Slideshare embed below, PDF attached to this post).

If you are interested in discussing Open Atrium from a developer’s point of view at DrupalCon Chicago next month, please join the BOF.

drupaldevdays_oa.pdf (2.83 MB) Download

Drupal Solutions for Public Works Monitoring and Remote Collaboration

Tue, 12 Apr 2011 16:21:36 GMT

The Drupal Government Days held last week in Brussels were a good occasion to showcase two solutions by Nuvole, born as services for local and European institutions.

Public Works Monitoring in Drupal

Nuvole built a dedicated platform, based on the widely used Drupal system, to make it easy for municipalities to monitor public works in real time and optionally share selected information with citizens.

Please find the Slideshare embed below.

Public Works Monitoring

Remote Collaboration and Institutional Intranet

Nuvole created a solution for the European Commission based on Drupal and Open Atrium to allow several hundreds experts to discuss in a Virtual Community about the harmonization of higher education in Europe. Besides the Virtual Community, several websites are created for the periodical seminars, all based on a common template and single sign-on.

Please find the Slideshare embed below.

Remote Collaboration and Institutional Intranets with Drupal and Open Atrium

Five reasons code-driven development and Features are good for you

Thu, 13 Jan 2011 15:18:40 GMT

With the DrupalCon Chicago 2011 registrations going at full speed (early bird deadline expires tomorrow, hurry up!) we are receiving questions from people who are considering to sign-up for the Code-driven Development: Use Features Effectively pre-conference training by Nuvole.

As a general answer, here are 5 benefits in adopting a code-driven development workflow. The topics below are discussed in our relevant blog posts too and all will be covered at length, with practical examples, in our DrupalCon training.

1: Keeping track of all changes

When did a certain bit of configuration (such as a variable, or the allowed values for a content type) change? Was it a side-effect of something else or an intended modification? When all your site configuration is in code instead of being buried in a huge SQL dump, and you couple it with a proper version management tool, comparing site revisions is easy and effective.

2: Working in a distributed team

Why can’t I work on setting up all content types and views for the Blog section of a website while someone else does the same for News? With no SQL dumps around, developers are free to experiment and restart from scratch without damaging each other’s work. And they can work in a modular way on the same project in parallel (thus speeding up the development pace), even from separate countries!

3: Reusing components and settings

Isn’t this site feature you are developing so similar to what you did in a previous project? Or this complex and tedious module configuration exactly the same across all your projects, so that you have resorted to write down the steps and repeat them manually? With a code-driven approach, replicating settings is just a matter of reusing the same, modular, extensible, components, avoiding tedious and error-prone manual work.

4: Updating production sites cleanly

What if a client asks for a complex modification on a production site? Do you need to take the site offline, dump the database for backup, repeat manually all the changes you had tested with an older dump and then finally put the site back online? Code-driven development allows you to use the same mechanism employed by the standard Drupal module updates, with virtually no downtime needed and a much cleaner log of changes.

5: Following the Drupal future

Worried that code-driven development could not be in the direction Drupal is taking? No need to worry! Some patches by the good guys at Development Seed made (or are making) their way into Drupal 7 and beyond, enabling a better support for a code-driven workflow out of the box. So code-driven development is not just cool, but future-proof too!

The DrupalCon training will be hands-on and will follow the model of our standard code-driven development training, with short presentations and practical assignments based on concrete examples, monitored by the Nuvole team.

Code-driven development is an approach that works in our case and addresses our needs; though, you may have different needs and thus find beneficial only selected portions of the workflow, for example; this is fine too!

Building and Maintaining Drupal Distributions

Wed, 24 Aug 2011 12:07:01 GMT

Our DrupalCon London 2011 presentation Building and Maintaining a Distribution in Drupal 7 with Features can be found as a Slideshare embed below; you can also find the final slides in PDF format on the session page on the DrupalCon site or attached to this post.

For further resources, you can refer to the code-driven development posts in the Nuvole blog or check out our trainings (next: London in October). And for those who couldn’t get a printed copy of our code-driven development cheatsheet, a PDF version is here.

nuvole-drupalcon-london-2011.pdf (13.33 MB) Download

Minisites in Open Atrium

Mon, 28 Nov 2011 13:52:29 GMT

You can use a single Open Atrium installation to create minisites, i.e., minimal websites with a simple structure, served at different domains, but sharing a centralized backend. Each minisite can be public or private and can have a specific theme and specific features. An anonymous visitor will only see a small, self-contained site, while site editors will be able to manage content on all minisites from the same backend.

This is an ideal situation, for example, when an Open Atrium installation is used as a private Intranet but you want to build small auxiliary sites for conferences or events, and possibly take advantage of the fact that editors and registered users are the same as the main Intranet.

And, best of all, this can be done with a minimal amount of coding and respecting the Open Atrium architecture.

Creating and Customizing a Minisite

Our aim is to build a feature that will allow us to create and customize a minisite in just a few minutes. We want to start from the ordinary “Create Group” page:

And this is what we get immediately upon form submission:

The minisite ships with default content (pages linked from the left side menu) and the editor can just click and add content.

We can also hook our minisite to the built-in look-and-feel customization that Open Atrium provides for its groups and give the possibility to the editor to choose background and foreground colors:

After a quick content editing, the editor can achieve something like this:

Screenshots are taken from Alfa Puentes, an international project funded by the EU that will focus on exchanging best practices between Europe and Latin America about higher education and that will include several events and conferences, each with a dedicated minisite.

Let’s now see how to build this feature in Open Atrium.

Creating a minisite group type (spaces preset)

To start, we simply define a new spaces preset type, named “minisite”: this can be done through hook_spaces_presets() using the existing definitions for public and private spaces from atrium_groups.spaces.inc as a basis.

<?php 
function atrium_minisite_spaces_presets() {
  $export = array();

  $spaces_presets = new stdClass;
  $spaces_presets->disabled = FALSE;
  $spaces_presets->api_version = 3;
  $spaces_presets->name = 'minisite';
  $spaces_presets->title = 'Minisite';
  $spaces_presets->description = 'Minisite space preset.';
  $spaces_presets->space_type = 'og';
   ...
?>

The new space preset can be exported into a feature and creating a minisite becomes as easy as creating a new group.

Assigning features to a minisite

Just like a standard Open Atrium group has some features available by default, we will want our minisites to use a certain set of features by default. In our case, we built two features for Pages and News and we enable them for our minisites, that will thus have static pages and news available.

<?php 
  ... (follows from above) ...
  $spaces_presets->value = array(
    'variable' => array(
      'spaces_features' => array(
        'atrium_blog' => 1,
        'atrium_book' => 0,
        'atrium_calendar' => 1,
        'atrium_casetracker' => 0,
        'atrium_members' => 1,
        'atrium_pages' => 1,
        'atrium_shoutbox' => 0,
        'spaces_dashboard' => 1,
        'atrium_news' => 1,
      ),
  ... (continue along the lines of atrium_groups.spaces.inc) ...
?>

Making minisites self-contained: menus and blocks

We want our minisites to be self-contained as much as possible. In order to achieve that, with some standard Drupal hooks, we create a new menu for each minisite, that takes care of updating it as soon as the respective minisite group changes. Here is one of the implemented hooks:

<?php
function atrium_minisite_nodeapi(&$node, $op, $arg = 0) {
  
  if (isset($node->spaces_preset_og) &&
      atrium_minisite_is_minisite_preset($node->spaces_preset_og)) {
    if (in_array($op, array('insert', 'update', 'delete'))) {
      $menu = array();
      $menu['title'] = $node->title;
      $menu['menu_name'] = atrium_minisite_get_menu_name($node);
      $menu['description'] = $node->og_description;
      atrium_minisite_menu_api($op, $menu);
      
      if ($op == 'insert') {
        module_invoke_all('atrium_minisite_default_content', $node, $menu);
      }
    }
  }  
}
?>

As you can notice, our minisite feature exposes an atrium_minisite_default_content() hook which allows other modules to provide default content once a new minisite is created (e.g. “About” and “Contact” pages, etc…). Shipping each new minisite with default content allows the editor to start filling up the site straight away without the need to recreate again and again the same content structure.

To make our minisite menu group-aware, we implement hook_block() in atrium_minisites.module. The implementation below relies on a couple of helper functions defined elsewhere.

<?php
function atrium_minisite_block($op = 'list', $delta = NULL, $edit = NULL) {
  
  $blocks = array();
  if ($op == 'list') {
    $blocks['primary-navigation']['info'] = t('Minisite: Primary navigation');
    $blocks['primary-navigation']['cache'] = BLOCK_NO_CACHE;

    $blocks['secondary-navigation']['info'] = t('Minisite: Secondary navigation');
    $blocks['secondary-navigation']['cache'] = BLOCK_NO_CACHE;

    $blocks['full-navigation']['info'] = t('Minisite: Full navigation');
    $blocks['full-navigation']['cache'] = BLOCK_NO_CACHE;
  }
  if ($op == 'view') {
    if (atrium_minisite_is_minisite_space()) {
      $config = atrium_minisite_get_menu_block_config($delta);
      if ($delta == 'secondary-navigation' || $delta == 'full-navigation') {
        $config['level'] = 2;        
        $config['expanded'] = 1;
        $config['depth'] = 0;
      }
      if ($delta == 'full-navigation') {
        $config['level'] = 1;        
      }
      $blocks = menu_tree_build($config);
      unset($blocks['subject']);
    }
  }
  return $blocks;
}
?>

This block is then specified as a context reaction, so that site visitors will see a self-contained navigation when browsing the minisite.

Making minisites themeable

Each minisite can be assigned a different theme. To do so from a convenient interface, we implement hook_form_alter() in atrium_minisite.module to give the possibility to select a theme for the given minisite. Again, we omit some helper functions for brevity.

<?php
/**
 * Implementation of hook_form_alter()
 */
function atrium_minisite_form_alter(&$form, &$form_state, $form_id) {
  
  if (isset($form['#node']) && $form_id == $form['#node']->type .'_node_form') {
    $node = $form['#node'];    
    
    if (og_is_group_type($node->type)) {
      module_load_include('inc', 'system', 'system.admin');      
      atrium_minisite_theme_form($form);
    }
    
    if (atrium_minisite_is_minisite_space()) {
      if ($node->type == 'page') {
        $space = spaces_get_space();
        $conf['menu_default_node_menu'] = atrium_minisite_get_menu_name($space->group);
        $menu_name = atrium_minisite_get_menu_name($space->group);
        $menu = array($menu_name => $space->group->title);
        $form['menu']['parent']['#options'] = menu_parent_options($menu, $item);
      }
    }
  }
}
?>

We can now choose the best theme for our minisite directly on the creation page:

Serving minisites on different domains

Each Open Atrium group gets its own URL path component; in our case the minisite will be reachable at something like http://alfapuentes.org/conference/. With a quick Persistent URL customization we can actually set the URL rewriting to be per domain, allowing our minisite to have its own domain. This can be achieved by:

Enabling the “Domain” URL modification type from admin/settings/purl/types;
Assigning to “Group space” the new modifier type on admin/settings/purl;
Adding a custom URL each time we are going to create a minisite group

By taking advantage of the Persistent URL domain rewriting we can really make the new minisite look completely independent from the underlying Open Atrium platform:

The techniques shown here are covered in detail in our trainings; if interested, contact us for more information.

Look bigger with Features

Wed, 18 Jan 2012 20:56:14 GMT

Nuvole built some respectably sized projects in 2011: a few large international projects (including a heavily customized Open Atrium installation for Higher Education experts and a portal for collaboration between Europe and Latin America on educational matters) and a bunch of ordinary websites, besides our other activities such as trainings and general consultancy. And people occasionally ask us how big Nuvole is.

Well, the truth is that Nuvole is a two-people company, helped by minor contributions by external consultants. Yet, it manages to run and maintain sizeable Drupal projects by using Features -the easy part- and by doing it properly - the hard part, and the main focus of our DrupalCon Denver Training.

So the key is not how many people work on a project, but how efficient their workflow is. Adopting a Features-based workflow provides a solid basis, but you still need to take a lot of design decisions: some will be easier, some will require a bit of thinking, some will prove correct or wrong only months later, when the project needs infrastructural updates.

Nuvole pioneered a Features-based development, and what in general we call the “Code-Driven Development workflow”, a couple of years ago. Our Features-based projects look much different now: we found out that our old structure was not optimal, or that technology allowed for cleaner workflows (Drupal 7 brought some substantial improvements, for example). We structured our new projects to take the new knowledge into account, found new challenges and gradually solved them.

Here are a couple of examples of the design decisions you will be happy to have taken properly (we will cover them in detail in future blog posts).

Hard and soft configuration. A site, especially when built as a distribution, should ship with complete pre-configuration. And this pre-configuration should be in code, we don’t want to carry a database around. But should all of it go into Features? No, because some pre-configuration (the soft one) is meant to be overridden in the normal site customization, while Features should only contain the hard one, i.e., configuration that defines the site structure. The soft configuration (like the default site title, or default colors) is best handled by putting the corresponding code in the installation profile rather than in Features.

Feature separation. When all your site configuration is in Features, you quickly realize that it is not feasible to put everything into one Feature: you will need multiple Features to build a website, and this means you will have to take decisions on how to split your site functionality across Features. A common mistake is to build transversal Features, like a “permissions” Feature with all site permissions: this works, but it introduces annoying interdependencies and maintenance difficulties in the long run. The problem is best solved by basing Features on content types, thus making them truly modular: in general, a Feature should contain a content type and everything that is related to it, such as views, contexts, image styles and permissions.

The Nuvole DrupalCon training, Code-driven Development: Use Features Effectively, will be held on March 19th, the day before DrupalCon Denver. The training is open to both DrupalCon attendees and externals, and must be booked from the DrupalCon registration page.

Hard and Soft configuration in Drupal Distributions

Tue, 07 Feb 2012 15:30:44 GMT

An extract from the new teaching materials we are preparing for our DrupalCon Denver 2012 pre-conference training, Code-Driven Development: Use Features Effectively.

By now the advantages of a clean Code-Driven Development workflow are clear to the majority of Drupal developers; things like separation between configuration and content, packaging and deployment of new site functionalities or sharing changes in a distributed team don’t look that scary anymore.

Still not everything in Drupal can be clearly classified into configuration or content, such as Taxonomy terms. Taxonomy terms are mostly used to categorize our content so, naturally, they are often used in site configuration: a context that reacts to a given taxonomy term is a fairly common scenario. Taxonomy terms can be generated by users (think about free-tagging), which makes them fall into the “Content realm”. Drupal, in fact, considers them as such, assigning to each term a unique numeric identifier, and that’s the problem.

Each component has its own name

One of the turning points in making configuration exportable in code was the introduction of a unique string identifier for each component: this way it’s easy to share configuration and to avoid conflicts. So what happens with taxonomy terms? If they appear in our context conditions then we’d better bundle them together: no matter where we are going to deploy our feature, they will always have to have the same values. That’s simply not possible: they have numeric IDs, forget about easy life here.

Various attempts have been made to export numerically identified components, such as UUID Features Integration or the handy Default Content module, but they only solve part of the problem: we want other components to know the unique name of our terms.

Hard and Soft configuration

At Nuvole we adopted the following terminology to classify the two kind of settings that we need to deal with everyday:

Hard configuration includes the settings under the distribution developer’s control (e.g., Views or Contexts); it has a machine name, it is easy to export, it gives no headache. We store it in Features.
Soft configuration includes the settings that are meant to be overridden by the site administrator (e.g., the default theme, or the initial terms of a vocabulary); it often gets a unique numeric ID from Drupal, it is impossible to export safely, it is painful to handle. We store it in the Installation profile.

This distinction becomes fundamental when the configuration is altered or (if you are dealing with a distribution) when the underlying distribution is upgraded. In the case of Hard configuration, altering it results in an overridden feature, which is upgrade-unsafe. In the case of Soft configuration, altering it does not change the Features state, since the corresponding settings are stored in the Installation profile, and changes are upgrade-safe.

Not only taxonomy terms

The distinction between Hard and Soft configuration goes beyond how to conveniently export taxonomy terms: it is more a design decision, especially important when dealing with distributions. We consider everything that the site builder might be entitled to change as Soft configuration. The place where to usually store Soft configuration is your hook_install(); this guarantees a safe upgrade path to the next version of the distribution. An example could be the default theme: you may ship your distribution with a specific theme but the site owner might want to change it and subsequent updates shouldn’t alter it.

Here is how our profile hook_install() might look like in a Drupal 7 distribution:

<?php 
/**
 * Implements hook_install()
 */
function example_install() {  
   $terms = array();
   $vocabulary = taxonomy_vocabulary_machine_name_load('category');  
   $terms[] = 'Solution';  
   $terms[] = 'Client';  
   $terms[] = 'Use case';  
   foreach ($terms as $name) {
     $term = new stdClass();
     $term->vid = $vocabulary->vid;
     $term->name = $name;
     taxonomy_term_save($term);
   }
   
  // Enable custom theme
  theme_enable(array('example_theme'));
  variable_set('theme_default', 'example_theme');   
}
?>

Reference for site builders

If you are customizing a distribution and you need to override some of its default Hard configuration you might want to have a look at the Features Override module and at these two related articles from Phase2 Technology:

Present and Future of Open Atrium

Thu, 03 May 2012 13:32:05 GMT

Our review of Tracy Smith’s Open Atrium book just made it to Slashdot, confirming there is a widespread interest in this popular Drupal distribution. (By the way, the book is not, and is not meant to be, useful to developers, but if you build Open Atrium projects you should definitely recommend it to your clients: it will save a lot of time to you and them).

The most interesting consideration is that the book, published in January 2011, is still perfectly current now, as of Open Atrium 1.3 released just a few weeks ago. And this is not quite good: most distributions have been significantly updated in recent months, while Open Atrium has been quite conservative in features and updates so far.

In case you haven’t heard, this is going to change. The future of Open Atrium looks bright, and, while there isn’t a roadmap yet, the guys at Phase2 have recently taken steps to make sure that Open Atrium can again be a shiny example of innovation and bleeding-edge, useful, technology:

The Open Atrium Development Group is now open for everybody to join. Open Atrium is now a true community product with a transparent development process.
A Feature and Theme Directory is now available to find in a single repository all features and themes created by the Community, such as Nuvole’s Atrium Folders.
An interesting BoF session was held at DrupalCon Denver, where we shared our experience with advanced Open Atrium customizations and we discovered that people are pushing this distribution to many different use cases: there are hundreds of organizations and companies working with Open Atrium now, and each of them can provide valuable feedback in shaping the future version.

The discussion about Open Atrium on Drupal 7 is now open, don’t miss it!

Drupal Global Training Day

Tue, 05 Jun 2012 15:10:15 GMT

Nuvole is proud to join the Drupal Global Training Day on June 22nd.

We will give a free generic introduction to Drupal. So this won’t be our usual, highly technical, training about streamlining Drupal Development that you may have seen at Drupalcon Chicago 2011, DrupalCon Denver 2012 and that is scheduled for DrupalCon Munich 2012 too. This time we will focus on Drupal as a platform, its community, what can be easily done with the available modules and what can be reached with a bit (or a lot!) of customization.

The target public is mainly people who would like to assess Drupal for use in international not-for-profit organizations: Nuvole has a long history of successful projects with this kind of organizations, and the examples we will show are all from this sector.

Topics will include:

A generic introduction to Drupal and its community
Why open-source is the best solution: low entry cost, availability of thousands of free modules, focus on customization
Drupal for distributed NGOs: Public websites with private working groups (Intranet) and file repositories
Drupal for event sites: minisites for conferences and events, easy to customize and replicate
Drupal for campaigns: high-impact sites for campaigns with social networks integration

The event is free to attend, and attendees are also welcome to stay for a buffet lunch. Registration is required. Please see the registration page for all details.

Automating Drupal Development: Makefiles, Features and beyond

Mon, 25 Jun 2012 13:16:27 GMT

Features are a well-known and very useful tool to streamline Drupal development and write reusable components. However, if you get serious about automating as much as possible in your Drupal development, you need to add other tools to your daily practices, and master them properly: forget the tedious, error-prone, point-and-click and embrace Code-Driven Development.

Automating Drupal Development, as presented by Nuvole at the Drupal Developer Days Barcelona last week, consists of the following steps:

Automating code retrieval. Use Drush Make to keep track of modules, patches and Features, and rely on Buildkit as a basic distribution with preselected modules to make your development faster. Use Drush Bake to create makefiles based on templates.
Automating installation. Customize the installation profile to perform all tasks you need during installation, especially pre-configuring your site with those “soft configuration” settings that the site administrator will be able to modify without overriding your Features.
Automating site configuration. Use Features properly, to make them truly modular, and install a Features server, where you will host and track your Features.
Automating tests. Use continuous integration to automatically ensure that your website will continue to work as expected when it is upgraded: test your codebase, installation and site configuration on every code change.

These concepts will be covered in more detail in the Nuvole DrupalCon Munich Training (if you don’t see the page, you may need to register to the DrupalCon Munich website).

The full presentation from Barcelona is below, as a Slideshare embed or a PDF file (attached).

drupal_dev_days_bcn.pdf (13.94 MB) Download

A Code-Driven Development Cheatsheet

Fri, 25 Mar 2011 16:31:54 GMT

Our Code-Driven Development with Features training in Chicago was a very interesting experience for us: people with different levels of Drupal expertise were able to appreciate the benefits of code-driven development and the most experienced attendees managed to experiment with the development workflow on their own laptops, by building and modifying sandbox websites.

As a gift to those who couldn’t attend the training because it was sold out, and to all lovers of Drush, Features and code-driven development in general, we are sharing a Code-Driven Development Cheatsheet that was part of the training materials. Please find it attached to this post, in PDF format.

We still have some printed copies available to give away; if you are attending the Drupal Government Days in two weeks in Brussels and are interested in one just let us know.

Thanks to the DrupalCon trainees for their nice and constructive feedback, to DrupalCon organizers for assistance and logistics… and for those who couldn’t make it to Chicago, there will be further opportunities in Europe in the next months!

Image credits: from DrupalCon Chicago on Flickr.

code-driven-development-cheatsheet-nuvole.pdf (1.45 MB) Download

From DrupalCon Munich: CMI, Drupal automation, Open Atrium on D7

Fri, 24 Aug 2012 12:37:56 GMT

The DrupalCon Munich 2012 is just over. There were quite a few inspiring sessions, but here are a few highlights of the moments and topics where the Nuvole team was mostly involved.

Code-Driven Development: another training… and what about Drupal 8?

We held the pre-conference Nuvole training about Code-Driven Development (Features, Makefiles, a set of techniques for effective Drupal programming) at DrupalCon Europe for the first time, after two successful editions at DrupalCon Chicago 2011 and DrupalCon Denver 2012. Nice to see that there are more and more training attendees who are already familiar with the tools but still want to see the big picture and find out how to really streamline their Drupal development.

And what’s coming with Drupal 8? We followed with great attention Greg Dunlap’s Munich session to understand the impact of the Configuration Management Initiative on the Code-Driven workflow we advocate. There will surely be disruptive changes, but in the end Drupal will be improving a lot as far as configuration management is concerned. And Features won’t die: part of its functionality (the import/export itself) will be taken over by Drupal core, but Features will be able to refocus as a tool to package configuration. And it’s time for Nuvole too to stop lurking on the CMI group and start showing up more frequently on issue queues!

BoF: Automating Drupal Development

The BoF session about Automating Drupal Development was packed, as many developers shared their secrets to effective Drupal Development.

The basic idea was how to get rid of all repetitive tasks in Drupal Development and automate as much as possible: building the code base (Drush Make), creating the installation profile from templates, storing and packaging configuration with modular Features and a Features server, rely on an automated testing framework activated at every code change. Tools we discussed are mostly the same as in our Barcelona DrupalDays presentation, with the addition of Capistrano and Webistrano.

BoF: The future of Open Atrium: getting it done

The BoF about the future of Open Atrium was to bring together the core Open Atrium team, working for Phase2 Technology, and companies and consultants that, like Nuvole, provide Open Atrium services: if we wish to move from talk to action, multiple parties will need to help. Here’s an outline:

Update core and modules to their D7 version, update the Blog (to become “Discussion”) and Book features, introduce the concept of “Organizations” to be able to apply certain criteria (e.g., inclusion in a group) to multiple users.
Adopt a new, mobile-friendly, responsive, theme.
Migrate the other Atrium features.

Development is expected to start soon with the creation of a Drupal 7 branch for all involved features. The basic Open Atrium structure will be ported to Drupal 7 as soon as possible, to give all community members the opportunity to help. A development version will be regularly made available for people to follow progress.

Lower priority items, which could appear after the Open Atrium 2.0 release (based on Drupal 7), like in version 2.1, include a full content migration path from OpenAtrium 1.x and adding new functionality. But the discussion is open and if you want to help shaping the future of Open Atrium just get involved!

Apps for Open Atrium: Atrium Folders

Wed, 25 May 2011 13:35:47 GMT

Open Atrium’s default set of features covers most of the common needs of an average organization, except one: a file repository with the familiar look and feel of a folders tree. By deploying customized versions of Open Atrium for different kinds of organizations we noticed that the built-in Notebook feature is still cumbersome for intranet users to store and categorize their documents (meeting reports, minutes, etc…): they are used to store their files in directories and subdirectories and not as books and child pages.

On the other hand, the needed technology for a more familiar documents repository is there: Drupal’s core Book module can create hierarchical relations among nodes and files can be easily attached to nodes, so it’s enough to interpret book pages as folders and their attachments as files to get the same paradigm users are familiar with.

Enter Atrium Folders: A simple file repository application

The idea behind Atrium Folders is to have a very simple file repository application with no external dependencies that users can just download, enable and start playing with. To get started, download the code directly from Nuvole’s GitHub, place it under sites/all/modules/features in your Open Atrium installation and enable the new feature. Atrium Folders is shipped as an ordinary Drupal feature so, after enabling it as a module, we need to enable it in the Open Atrium group.

We can now visit the new Folders section and start adding the first folder.

As already mentioned, folders are nodes: we just need to specify its name, press Save and we are good to go. Now, say goodbye to the ordinary Drupal node interface; from now on we will be managing our repository from an intuitive “filesystem browser”-like interface:

Managing your repository

Use the top menu to manage the folder currently being displayed; it contains the following four items.

Basic information: change name and description of the current folder:

Subfolders: easily add a bunch of subfolders by entering their names, one per line:

Attach files to this folder: just the reinterpretation of the good old Drupal file upload form, it gives the possibility to upload as many files as we need to the current folder:

Notifications: specify who will get a notification of the current changes (again, we don’t reinvent the wheel but we integrate with the default Open Atrium notifications system):

Upload multiple files: quickly upload a ZIP archive containing several files and folders: Atrium Folders will unzip it for you and place its content in the current folder (PHP must have ZIP support enabled in order for this functionality to be available)

Rearranging folders

The Reorder tab allows to rearrange the structure of our repository by simple drag-and-drop. This handy interface, borrowed from Drupal’s core Book module, is also very helpful when we have to rename several folders at once.

File toolbox

Each file has a toolbox that will allow to rename, move or delete the respective file. The toolbox uses jQuery and AJAX calls extensively, to guarantee a smooth user interaction: files will be modified and changes will take immediate effect, with no need to reload the page.

Creating your own toolbox

Atrium Folders exposes hook_folder_toolbox() to enable third party modules to expose their own toolbox. As an example, here is the default atrium_folders_folder_toolbox() implementation:

<?php 

/**
 * Implementation of hook_folder_toolbox()
 */
function atrium_folders_folder_toolbox() {
  return array(
    array('op' => 'move',
      'type' => 'file',
      'title' => t('Move'),
      'description' => t('Move the file to another location.'),
      'form callback' => 'atrium_folders_move_file',
      'ajax callback' => 'atrium_folders_move_file_ajax_callback',
    ),
    ...
  );
}
?>

Drag and Drop Upload

Atrium Folders ships with a built-in integration for the Drag’n’Drop Uploads module which gives the possibility to simply drag and drop files on the upload form to attach them to a node. If Drag’n’Drop Uploads module is enabled a Drop zone will be visible in the Attach files to this folder panel.

Drush integration: import an existing structure

When an organization wants to switch to Open Atrium, in most cases they need to migrate a shared folder to the new intranet. Atrium Folders includes the folder-import Drush command that allows to import a directory located in our local filesystem into a certain folder of a specific Open Atrium group. We just need to specify the full path of the directory we want to import and the Drupal path of the group we want it to be imported:

$ drush folder-import /tmp/meeting secretariat
Created 13 folders and 168 files.

Type $ drush help folder-import for more.

A new role at the Apache Software Foundation

Thu, 18 Oct 2012 17:39:51 GMT

Effective today, I (Andrea) have become an officer of the Apache Software Foundation, in my new role as Vice President for the Apache OpenOffice (commonly known as “OpenOffice”) project.

This doesn’t change much in my daily activity as director of Nuvole. I have been involved in the OpenOffice project for several years, starting much earlier than Drupal became a mainstream project. It has been and it will continue to be a volunteer activity, even though it will definitely happen that I attend more Apache events in the near future.

That said, this might be an occasion for the Drupal community to take a closer look at the Apache Software Foundation and its projects.

Even though there are still many people who identify Apache with its ubiquitous Apache HTTP server, the Apache Software Foundation is much larger than that: it is a huge, not-for-profit organization developing around 150 free and open source projects for the public good.

There are interesting opportunities for cooperation between Drupal and Apache projects: Apache Solr, for example, has already become a standard topic at Drupal events; Apache POI, Apache Tika and Apache PDFBox can extract text and metadata from Office or PDF documents and make them search-friendly, with initial work towards Drupal integration; Apache Traffic Server is a caching server with Drupal support. The incubating Apache OpenMeetings project has a module for Drupal integration too.

The Apache OpenOffice project itself maintains two old but functional Drupal sites for hosting its Extensions and Templates, that were recently relocated to SourceForge with technical advice from Nuvole; and sooner or later it will have to consider a deeper integration with Web technologies…

We at Nuvole are looking forward to exploring opportunities for cross-project collaboration, and the ApacheCon 2012 in Germany next month might be a nice occasion for brainstorming. If you have ideas to share, please do so in the comments below.

Configuration Management and Features: a look at Drupal 8

Tue, 03 Sep 2013 13:32:03 GMT

An update to this post is now available at https://nuvole.org/blog/2014/jul/15/packaging-and-reusing-configuration-drupal-8.

We have always told attendees of our Drupal 7 “Code-driven development” trainings that embracing the idea that configuration must be stored in files and not in the database would make their Drupal development future-ready: the specific processes would necessarily change but the concept would stay and be reinforced in Drupal 8, thanks to the nice work being done in the Configuration Management Initiative.

And this was right. The progress done in CMI is very good, to the point that in our coming DrupalCon training in Prague, Code-Driven Development: an Effective Drupal Workflow, from D7 to D8 we will explore in detail how things work in Drupal 7, but for each topic we’ll also describe what is going to improve/change in Drupal 8.

Configuration Management in a nutshell

From a site builder’s point of view, Configuration Management in Drupal 8 is visible as two items in admin/config/development: “Configuration Export” and “Configuration Import”.

The Export functionality has no further options, and it provides you with a config.tar.gz file containing hundreds (180+ on a clean installation, many more on a feature-rich test site) of files in YAML format, human- and machine-readable. All the site configuration is there: node types, their fields, permissions, Views, theme settings, variables…

The Import functionality, on the converse, will import a config.tar.gz file, show you any configuration changes between the currently active configuration and the one you are trying to import and allow you to replace the current configuration with the one in the provided files.

For many developers, this is a dream come true: with D8, Drupal Core will support a file-based configuration, allowing developers to track changes, put configuration under version control, update a production site cleanly… basically, all the advantages that we have described as selling points for the Features module in the last several years.

The Future of Features

So, if the major selling points of the Features module are now in core, does Features have a future in Drupal 8?

It still does, but it needs to be repurposed. So far, it had a dual functionality, serving as:

A machinery to export/import configuration from database to code. This will no longer be necessary in Drupal 8. Configuration lives in files and the database is used only for caching. For the same reason, an important problem in Drupal 7 development (“How do I make configuration exportable when there is no native support for Features?”) is now solved for good.
A convenient way to package and reuse configuration across multiple sites. This is extremely useful to developers and not completely covered by Configuration Management in Drupal 8, even though the basic components are there.

Poorman’s Features: emulating Features in Drupal 8

Let’s see how it is possible to package configuration and emulate, manually, what Features and Drupal 7 allow today.

As an example use case, we will package a “Blog Feature”.

1: Create and export configuration

Create the Blog content type, configure its fields (use proper naming for fields: field_blog_image and so on) and a classic view with a block and page display. Then go to admin/config/development and export your configuration into the config.tar.gz file.

2: Package relevant configuration into a module

Create a simple feature_blog module containing:

feature_blog.info.yml: a basic .info file in the Drupal 8 format.

name: Blog
type: module
description: 'A Blog content type and related configuration.'
package: Features
version: 8.x-0.1
core: 8.x

feature_blog.module: an empty module file, needed for correct Drupal operations.

<?php
  // Drupal needs this empty file.

a config subdirectory: copy to this folder all relevant files from config.tar.gz; filenames are quite helpful here, as it will be enough to copy all files that have “blog” in their name.

.
|-- config
|   |-- entity.display.comment.comment_node_blog.default.yml
|   |-- entity.display.node.blog.default.yml
|   |-- entity.display.node.blog.teaser.yml
|   |-- entity.form_display.comment.comment_node_blog.default.yml
|   |-- entity.form_display.node.blog.default.yml
|   |-- field.field.field_blog_image.yml
|   |-- field.instance.comment.comment_node_blog.comment_body.yml
|   |-- field.instance.node.blog.body.yml
|   |-- field.instance.node.blog.field_blog_image.yml
|   |-- node.type.blog.yml
|   `-- views.view.blog.yml
|-- feature_blog.info.yml
`-- feature_blog.module

3: Make the exported files portable (remove UUIDs)

The feature is ready, but it will only work if imported in the same Drupal 8 site. We need to make it portable to other sites.

The main problem here is that Drupal assigns UUIDs (Universally Unique Identifiers) to each component. When you add, say, an “image” field to a content type, you create an instance of the “image” field, and to describe this relationship Drupal will rely on the UUID of the “image” field. Since UUIDs are site-specific, trying to import this configuration on another site would result in an error message like:

Drupal\field\FieldException: Attempt to create an instance of
unknown field 84e904df-2f14-46e8-9700-e00c5ca3f7d3
in Drupal\field\Entity\FieldInstance->__construct()
(line 252 of .../core/modules/field/lib/Drupal/field/Entity/FieldInstance.php).

Fortunately, in this case configuration import is designed to accept either names or UUIDs for describing the field this is an instance of. So it is enough to edit all files containing field.instance in their name and manually replace UUIDs with machine names (or just add machine names), something like the following:

-field_uuid: 84e904df-2f14-46e8-9700-e00c5ca3f7d3
+field_name: field_blog_image

Note that this relies on the specific handling of the FieldInstance class and may not be a general solution.

Edit: also note (see comments) that it is important to include a hardcoded UUID if you want to avoid that your configuration items are seen by Drupal as different items between your development and production sites, causing configuration import/export between the two to fail.

4: Import the feature on another site

Our module/feature is now ready to be imported. Just copy it to the /modules folder on another site, or a subfolder of it (this is now the recommended location to add modules) and enable it.

Your content type and configuration will be immediately available.

Possible improvements

The Configuration Management in Drupal 8 will bring to Drupal Core a clean, unified, professional way to deal with configuration. There will still be the need for a module like Features for packaging and reusing configuration.

Some implementation details of Configuration Management do not behave well in this respect: for example, UUIDs are problematic and user permissions are stored in a packaging-unfriendly way (one file per each user role, with roles identified by UUID).

But, overall, the future looks bright and code-driven! And, as we have seen, it is already entirely possible to manually create basic “Features” (i.e., modules with configuration) that will work in Drupal 8.

Configuration Management: Drupal 7 to Drupal 8

Fri, 06 Jun 2014 20:23:29 GMT

This is a preview of Nuvole’s training at DrupalCon Amsterdam: “An Effective Development Workflow in Drupal 8”.

Nuvole gave two talks about the current status of Configuration Management in Drupal 8 at European Drupal events in the last few weeks: Drupal Days Milan 2014 and Drupal Camp Alpe Adria 2014.

Developers attending the events were mostly interested in how the future Drupal 8 Configuration Management capabilities will compare to Drupal 7, with and without the Features module (or, in general, with and without what we call the code-driven workflow).

Here is a comparison based on the current status of Drupal 8. Please note that CMI, the Configuration Management Initiative, is still under active development, and for example a couple issues mentioned in the slides have already been resolved, shortly after we gave our presentations. The very basic concepts, though, remain unchanged from our September 2013 post, so you might want to review that one if you have never heard about CMI before.

Configuration Management: Key differences between Drupal 7 and Drupal 8

Configuration is well-defined

In Drupal 7 it isn’t always clear-cut whether something belongs in the “configuration” or “content” realm. Are vocabularies configuration? Are taxonomy terms configuration? And what about a taxonomy term whose ID is used as a contextual filter in a View? Several approaches to this are possible, like soft configuration.

In Drupal 8, configuration will be less subjective: Drupal has an “Export Configuration” function that exports all the site configuration; if something is not there, it is not considered to be configuration. Concepts like soft configuration still apply, but at least it’s clear what is configuration and is not.

Configuration is stored in files

Drupal 7, generally, stores configuration in the database, together with content. Only a little subset of configuration is read from PHP files. In Drupal 8 configuration will live in text files. Then, for several reasons (performance, safety, security), those files will actually be stored in the database by default (one of the most visible recent changes), but will still be accessed and managed as files.

Unified format and approach to configuration

A particularly annoying issue with Drupal 7 is that modules are free to set their own standards for storing configuration, so it is common to find sites where configuration is scattered among variables, database tables, objects exported via CTools, Features, and other places. Modules like Features at times need some “black magic” to guess where and how relevant configuration is stored.

The unified approach in Drupal 8, where all configuration is in the form of text files in the YAML format, is a great step forward. It removes the guesswork needed by Features in Drupal 7 to understand how to find and export components.

Staging configuration

In Drupal 7 you can have “staged configuration” through Features and the Features revert operation. Drupal 8 will introduce, out of the box, an “Import Configuration” facility that you can use to import either the whole site configuration or a single configuration file. The approaches are mostly equivalent, but on one side Drupal 7 + Features offers better packaging functionality, on the other side Drupal 8 will allow to import a complete site configuration (all variables, permissions…), something that is highly unpractical to do with Drupal 7 and Features.

Interface to developers

From a developer’s point of view, writing PHP code to access and/or set a component’s configuration in Drupal 7 is tricky, since you don’t have a global interface, let alone an “entry point” that allows you to explore all the site configuration: you are forced to use a mixed bag of tricks ranging from variable_get() and variable_set() to database queries, to Ctools hooks for making configuration exportable and so on.

Drupal 8 will bring a very welcome improvement by providing a dedicated type for configuration entities and the unified interface Drupal::config($name) for getting and setting all configuration values: as a typical example Drupal::config('system.site')->get('name') will return the site name, and the same pattern can be used for everything.

The shootout: Drupal 7 vs Drupal 7 + Features vs Drupal 8

A quick feature comparison table between Drupal 7 core, Drupal 7 + Features and the current state of Drupal 8. As you see, a developer that is now using Drupal 7 without features will see spectacular advantages in Drupal 8, while experienced users of Drupal 7 and Features will still miss something.

Functionality	D7 Core	D7 Core + Features	D8 Core (current)
Export full site config (no content)	NO	NO	YES
Export selected config items	NO	YES	YES
Track config changes (full site)	NO	NO	YES
Track config changes (selected items)	NO	YES	YES
Stage configuration	NO	YES	YES
Package configuration	NO	YES	NO
Reuse configuration in other projects	NO	YES	NO
Collaborate on the same project	NO	YES	NO

What is still missing?

It’s important to understand the use case for CMI. The stated aim for CMI is not to replace Features. CMI aims at making it much simpler to transport configuration changes from development to production, and this aim was reached.

All other benefits of Features are out of scope for CMI. These include: packaging configuration, reusing configuration in other projects (and not simply moving it between the development and production version of the same site) and enabling real-time team collaboration (developer A and developer B build two separate features for the same site, like the News and Blog sections, simultaneously). So there is definitely room for a Drupal 8 version of Features.

Is CMI “Features Done Right”?

No. It is a nice way to replace and improve one use case for Features: making configuration exportable into text files.

Is CMI “Features Done Wrong”?

No. It is a huge step forward to developers and it paves the way for additional modules that could offer the same functionality of Drupal 7 + Features in a much cleaner and more reliable way.

See full versions of our Milan and Alpe Adria presentations as a Slideshare embeds or PDF files, below.

Configuration Management in Drupal 8: A preview (DrupalDays Milano 2014)
from Nuvole

Configuration Management in Drupal 8: A preview (DrupalCamp Alpe Adria 2014)
from Nuvole

Milano-2014-drupal-8-cmi-preview-en.pdf (430.48 KB) download

AlpeAdria-2014-cmi-d7-to-d8—dcaa-140517.pdf (774.4 KB) download

Packaging and reusing configuration in Drupal 8

Tue, 15 Jul 2014 13:00:00 GMT

This is a preview of Nuvole’s training at DrupalCon Amsterdam: An Effective Development Workflow in Drupal 8.

Configuration Management in Drupal 8 elegantly solves staging configuration between different environments addressing an issue that is still haunting even the most experienced Drupal 7 developer. In earlier posts we covered the new Configuration Management in Drupal 8, seeing how it compares to Drupal 7 and Features, and even investigated how to manually simulate Features in Drupal 8 last year. Recent developments and contrib modules can take us several steps closer.

Packaging configuration

Developers familiar with code-driven development practices need an equivalent in Drupal 8 + Drush 7 to what the Features module does in Drupal 7 with its features-update and features-revert Drush commands.

While Drupal 8 configuration staging capabilities are far more advanced than what Features could possibly provide, what the new Configuration Management system really lacks is the ability to package configuration.

Enter the Configuration development module

The Configuration development module, currently maintained by chx, serves two main purposes:

It automates the import of specified configuration files into the active storage.
It automates the export of specified configuration objects into files.

The module offers a simple, global UI interface where a Drupal developer can set which configuration is automatically exported and imported any time they hit the “Save” button on a configuration setting page.

In order to achieve a more modular configuration packaging it would be enough to set a specific module’s config/install directory as the actual export destination.

Nuvole contributed a patch (EDIT: now integrated) to make that possible: instead of firing an auto-export every time a “Save” button is clicked the developer can, instead, specify in the module’s info file which configuration needs to be written back to that module’s install directory and run a simple Drush command to do that.

Reusable “features” in Drupal 8

One of the main advantages of having a standardized way of dealing with configuration means that modules can now stage configuration at installation time. In a way that’s something very close to what Features allowed us to do in Drupal 7.

Say we have our news section up and running on the site we are currently working on and we would like to package it into a custom module, together with some other custom code, and ship it over a new project. The patched Config development module will help us to do just that! Here it is how:

Step 1: Download, patch and enable Configuration development module

We need to download and enable the Configuration development module and apply the patch (EDIT: already integrated as of 8.x-1.0-alpha7) attached to this Drupal.org issue.

After rebuilding the cache, we will have the config-writeback Drush command available. Let’s have a closer look at what it is meant to do:

$ drush help config-writeback

Write back configuration to a module's config/install directory. State which configuration settings you want to export in the module's info file by listing them under 'config_devel', as shown below:

config_devel:
  - entity.view_display.node.article.default
  - entity.view_display.node.article.teaser
  - field.instance.node.article.body


Examples:
 drush config-writeback MODULE_NAME        Write back configuration to the specified module, based on .info file.

Arguments:
 module                                    Module machine name.

Aliases: cwb

Step 2: Find what configuration needs to be packaged

We now look for all configuration related to our site’s news section. In Drupal 8 most of the site configuration is namespaced with related components so, if we keep on using consistent naming conventions, we can easily list all news-related configuration by simply running:

$ drush config-list | grep news

entity.form_display.node.news.default
entity.view_display.node.news.default
entity.view_display.node.news.teaser
field.instance.node.news.body
image.style.news_medium
menu.entity.node.news
node.type.news

Step 3: Package configuration

To package all the settings above we will create a module called custom_news and, in its info file, we will specify all the settings we want to export, listing them under the config_devel: directive, as follows:

$ cat modules/custom_news/custom_news.info.yml 

name: Custom News
type: module
description: 'Custom news module.'
package: Custom
core: 8.x
config_devel:
  - entity.form_display.node.news.default
  - entity.view_display.node.news.default
  - entity.view_display.node.news.teaser
  - field.instance.node.news.body
  - image.style.news_medium
  - menu.entity.node.news
  - node.type.news

After enabling the module we will run:

$ drush config-writeback custom_news

And we will have all our settings exported into the module’s install directory:

$ tree -L 3 modules/custom_news/

modules/custom_news/
├── config
│   └── install
│       ├── entity.view_display.node.news.default.yml
│       ├── entity.view_display.node.news.teaser.yml
│       ├── field.instance.node.news.body.yml
│       ├── image.style.news_medium.yml
│       ├── menu.entity.node.news.yml
│       └── node.type.news.yml
└── custom_news.info.yml

The Drush command above takes care of clearing all sensitive UUID values making sure that the module will stage the exported configuration cleanly, once enabled on a new Drupal 8 site.

To get the news section on another site we will just copy the module to the new site’s ./modules/ directory and enable it:

$ drush en custom_news

The following extensions will be enabled: custom_news
Do you really want to continue? (y/n): y
custom_news was enabled successfully.

Final evaluation: Drupal 7 versus Drupal 8

One of the main differences between working in Drupal 7 and in Drupal 8 is represented by the new Configuration Management system.

While Features was proposing a one-stop solution for both configuration staging and packaging, Drupal 8 CM does a better job in keeping them separate, allowing developers in taking a greater control over these two different and, at the same time, complementary aspect of a solid Drupal development workflow.

By using the method described above we can upgrade our comparison table between Drupal 7 and Drupal 8 introduced in one of our previous posts as follows:

Functionality	D7 Core	D7 Core + Features	D8 Core (current)	D8 Core (current) + Config Devel
Export full site config (no content)	NO	NO	YES	YES
Export selected config items	NO	YES	YES	YES
Track config changes (full site)	NO	NO	YES	YES
Track config changes (selected items)	NO	YES	YES	YES
Stage configuration	NO	YES	YES	YES
Package configuration	NO	YES	NO	YES
Reuse configuration in other projects	NO	YES	NO	YES
Collaborate on the same project	NO	YES	NO	NO

The last “NO” deserves a brief explanation: Configuration Management allows two developers to work simultaneously on different parts of the same project if they are very careful: but “merging” the work would have to be done by version control (GIT or similar), that doesn’t know about YAML or Drupal.

Some open issues

Contributed modules seem to be the best way to enhance the core Configuration Management system, much like what happened with Drupal 7 and Features. There are still several issues that should be considered for an optimal workflow, to match and improve what we already have in Drupal 7:

Piping: the ability to relate configuration components based on both hard and logic dependencies, for example: I export a content type and, automatically, I get also its fields. If piping might have been too rigid, at times, it would be still useful to have in some configurable form.
Enhanced configuration diff: it might be useful to have the possibility to review what configuration is going to be installed before enabling a module, like it is now when importing staged configuration to the active storage.
Granularity: it is still impossible to export part of a configuration file, so we still depend on the core conventions for grouping configuration into files, and we can’t export a single permission for example.
Ownership: we can’t know if another module (or “feature”) is tracking a component we wish to track; this could be useful in the perspective of maintaining several “modular” features.
Updates: we can reuse configuration by simply enabling a module, but this holds only for the initial installation; after a module is enabled, we don’t have a clean way to import changes (say, to “upgrade” to a newer version of the feature) outside the standard workflow foreseen in Configuration Management.

Git workflow for managing Drupal 8 configuration

Wed, 20 Aug 2014 17:30:26 GMT

This is a preview of Nuvole’s training at DrupalCon Amsterdam: An Effective Development Workflow in Drupal 8.

One of the new key features of Drupal 8 is the possibility to deal with configuration in code. Since configuration is now in text files, we want to put it under version control in Git to enjoy the many advantages this brings: comparing configuration states, keeping a history of configuration changes and even moving configuration between sites.

Setup

We will assume that you have a development version of Drupal 8, git and drush available on your system. You can set up your Drupal git repository in several ways. One of them is outlined in Building a Drupal site with Git on drupal.org. The document is written for Drupal 7, but can easily be adapted for Drupal 8.
Another, probably simpler method is to simply download a Drupal 8 (alpha) release and initialise a new repository with it.

In either case you should copy example.gitignore to .gitignore and adapt it to your needs to prevent settings.php and the files directory from being versioned.

The next step is to make sure that a configuration directory is versionable. By default Drupal 8 will place the staging directory under sites/default/files and it is considered a good practice to not version that location, but an alternative location can easily be specified in settings.php:

<?php
$config_directories['staging'] = 'config/staging';
?>

It is also possible and even advisable to specify a directory outside of the web root of course. In that case you would put the parent directory of your web root where drupal is under version control and use ../config/staging. We will later see that it is also possible to add more directories and keys to the $config_directories variable.

Because the configuration management of Drupal 8 only works between different instances of the same site, the different instances of the site need to be cloned. Cloning a Drupal 8 site is done the same way as cloning a Drupal 7 site. Just dump the database of the site to clone and import it in the other environment.

Development

After cloning your site you can go ahead and start configuring your site.
Once the part of the configuration you were working on is done the whole configuration of the site needs to be exported.

local$ drush config-export staging
The current contents of your export directory (config/staging) will be deleted. (y/n): y
Configuration successfully exported to config/staging.

Next, you need to merge the work of other developers. In some cases it may be enough to simply use git pull, otherwise the configuration has to be merged after it has been committed:

Add all configuration to git and commit it.
Use git pull (or git fetch and git merge) and resolve any conflicts if necessary.

Git can merge changes in text files quite well, but git does not know about Drupal and its yaml format for configuration. It is, therefore, important to verify that the merged configuration makes sense and is valid. In most cases it will probably not be an issue and just work, but it is always better to be vigilant and be on the safe side. So, after merging, you should always run:

local$ drush config-import staging

If the import went smooth you can push the configuration to the remote repository. Otherwise the configuration needs to be fixed first.

Deployment

The simplest case is when the configuration on the production site has not been changed. There is an interesting Configuration Read-only mode module that can enforce this.

If the configuration did not change deploying the new configuration is simply:

remote$ git pull
remote$ drush config-import staging

If the configuration changes on the production site, it is best to frequently export the live configuration into a dedicated directory.
Add a new config directory in settings.php:

<?php
$config_directories['to_dev'] = 'config/to_dev';
?>

remote$ drush config-export to_dev -y

Add, commit and push it to the production branch so that the developers can deal with it and integrate the changes into the configuration which will be deployed next. Exporting the configuration into a dedicated directory rather than the staging directory avoids the danger that merge conflicts happen on the production site. The deployment to the production site should be kept hassle free, so it should always be safe to pull from git and import the configuration without the risk of a conflict.

Important notes

It is important to first export the configuration changes and then pull changes from collaborators because the exporting action wipes the directory and re-populates it with the active configuration. Since everything is in git, you can recover from such a mistake without much difficulty but why make your life complicated.

Import the configuration before pushing it to the remote repository. Broken configuration breaks the site, be a nice co-worker.

Git doesn’t solve everything! Imagine Alice and Bob start with the same site, it has one content type and among others an “attachment” field. Alice deletes the attachment field, exports the configuration and pushes it to git. In the meantime, Bob creates a new content type and adds the attachment field to it. Bob exports his configuration, merges Alice’s configuration changes without a problem (the changes are separate files) and imports the merged configuration. The attentive reader sees where this leads. The commit of Alice deletes the field storage for the attachment field, but Bob added a field instance which depends on the field storage. The exported configuration now contains a field instance that can’t be imported.
At the time of writing, drush will signal a successful import but doesn’t actually import it while the UI is more helpful and complains that the attachment field instance was not imported due to the missing field storage.

Drupal 7 to Drupal 8: The Cheat Sheet

Wed, 01 Oct 2014 09:31:28 GMT

Our new An Effective Development Workflow in Drupal 8 training just made its debut at DrupalCon Amsterdam. An important part of the training is dedicated to shorten the trial-and-error period developers necessarily need to go through when converting their practices from Drupal 7 to Drupal 8: we want that proficient Drupal 7 (and Features) developers with zero, or very limited, Drupal 8 knowledge, gain a rather comprehensive knowledge of Drupal 8 (focused on configuration management) that allows them to be immediately productive when Drupal 8 is released.

As a gift to those who couldn’t attend the training because it was sold out, and to all lovers of Drush, Features and code-driven development in general, we are sharing a “Drupal 7 to Drupal 8” cheatsheet that was part of the training materials. Please find it attached to this post, in PDF format.

We still have some printed copies available to give away; find us at DrupalCon and feel free to ask us for one!

Drupal-7-to-Drupal-8-Cheatsheet.pdf (999.82 KB) download

Atrium Folders for Open Atrium 2

Mon, 08 Dec 2014 17:19:53 GMT

We received many requests to make an updated version of our Atrium Folders feature available for the latest version of Open Atrium, the excellent Drupal-based solution for Intranets developed by Phase2.

OECD sponsored the development of the new version as an open source project, in order to add a file management functionality to the Innovation Policy Platform site that it manages together with the World Bank. Atrium Folders for Open Atrium 2 is thus now available to everybody.

The usual features, a new way

Open Atrium changed completely and so did Atrium Folders. There are many differences under the hood, with a complete code rewrite, but the familiar user experience is still there.

Uploading and downloading files

Creating folders and adding files to the folders is as easy as creating any other content in Open Atrium 2: it is enough to create “Files sections”. When you are viewing a folder, specific buttons allow to create subfolders, upload files and directly download any file.

Access management

The access management works like for other nodes in Open Atrium. Access to the folder can be restricted for both viewing and editing separately, and it can be determined at a folder level.

Notifications

The notification system of Open Atrium can also be used for folders. Users can be informed when new files are added, with the same interface used by other Open Atrium features.

And much more

Media module support

The files are attached with the Media widget and it is thus possible to manage not only files, but everything Media supports, like for example YouTube videos or files attached to other content.

Multiple uploads, with drag and drop support

The multi-upload feature of Open Atrium 2 can also be used with Folders to upload several files at the same time. Drag and drop uploads are supported too.

Download folder as ZIP file

The download button for files exists also for folders and it allows to download a folder with its subfolders and all included files as a ZIP archive. This functionality is available as a submodule, bundled with Atrium Folders.

File and folder revisions

Atrium Folders supports history and revisions both for folders and files. You can view previous versions of a file and optionally restore an older version. This functionality is available as a submodule, bundled with Atrium Folders.

Download, installation and support

The Open Atrium Folders feature can be downloaded and installed like any other module.

It is available on drupal.org at https://www.drupal.org/project/oa_folders/

Please report any issues in the module’s issue queue at drupal.org

A pluggable field-handling system for the Behat Drupal Extension

Tue, 24 Mar 2015 16:15:00 GMT

On of the main goal of BDD (Behaviour Driven Development) is to be able to describe a system’s behavior using a single notation, in order to be directly accessible by product owners and developers and testable using automatic conversion tools.

In the PHP world, Behat is the tool of choice. Behat allows to write test scenarios using Gherkin step definitions and it generates the corresponding PHP code to actually run and test the defined scenarios.

Thanks to the excellent Behat Drupal Extension Drupal developers have been able to enjoy the benefits of Behavioral Driven Development for quite some time.

Essentially the project provides an integration between Drupal and Behat allowing the usage of Drupal-specific Gherkin step definitions. For example, writing a scenario that tests node authorship would look like:

Scenario: Create nodes with specific authorship
  Given users:
  | name     | mail            | status |
  | Joe User | joe@example.com | 1      |
  And "article" content:
  | title          | author   | body             |
  | Article by Joe | Joe User | PLACEHOLDER BODY |
  When I am logged in as a user with the "administrator" role
  And I am on the homepage
  And I follow "Article by Joe"
  Then I should see the link "Joe User"

Dealing with complex content types

The Gherkin scenario above is pretty straightforward and it gets the job done for simple cases. In a real-life situation, though, it’s very common to have content types with a high number of fields, often of different types and, possibly, referencing other entities.

The following scenario might be a much more common situation for a Drupal developer:

Scenario: Reference site pages from within a "Post" node
  Given "page" content:
    | title      |
    | Page one   |
    | Page two   |
    | Page three |
  When I am viewing a "post" content:
    | title                | Post title         |
    | body                 | PLACEHOLDER BODY   |
    | field_post_reference | Page one, Page two |
  Then I should see "Page one"
  And I should see "Page two"

While it is always possible to implement project specific step-definition, as show on this Gist dealing with field collections and entity references, having to do that for every specific content type might be an unnecessary burden.

Introducing field-handling for the Behat Drupal Extension

Nuvole recently contributed a field-handling system that would allow the scenario above to be ran out of the box, without having to implement any custom step definition, working both in Drupal 7 and Drupal 8. The idea behind it is to allow a Drupal developer to work with fields when writing Behat test scenarios, regardless of the entity type or of any field-specific implementation.

The code is currently available on the master branches of both the Behat Drupal Extension and the Drupal Driver projects, if you want to try it out follow the instructions at “Stand-alone installation” and make sure to grab the right code by specifying the right package versions in your composer.json file:

{
  "require": {
    "drupal/drupal-extension": "3.0.*@dev",
    "drupal/drupal-driver": "1.1.*@dev"
}

The field-handling system provides an integration with several highly-used field types, like:

Date fields

Date field values can be included in a test scenario by using the following notation:

Single date field value can be expressed as 2015-02-08 17:45:00
Start and end date are separated by a dash -, for ex. 2015-02-08 17:45:00 - 2015-02-08 19:45:00.
Multiple date field values are separated by a comma ,

For example, the following Gherkin notation will create a node with 3 date fields:

When I am viewing a "post" content:
  | title       | Post title                                |
  | field_date1 | 2015-02-08 17:45:00                       |
  | field_date2 | 2015-02-08 17:45:00, 2015-02-09 17:45:00  |
  | field_date3 | 2015-02-08 17:45:00 - 2015-02-08 19:45:00 |

Entity reference fields

Entity reference field values can be expressed by simply specifying the referenced entity’s label field (e.g. the node’s title or the term’s name). Such an approach wants to keep up with BDD’s promise: i.e. describing the system behavior by abstracting, as much as possible, any internal implementation.

For example, to reference to a content item with title “Page one” we can simply write:

When I am viewing a "post" content:
  | title           | Post title |
  | field_reference | Page one   |

Or, in case of multiple fields, titles will be separated by a comma:

When I am viewing a "post" content:
  | title           | Post title         |
  | field_reference | Page one, Page two |

Link fields

A Link field in Drupal offers quite a wide range of options, such as an optional link title or internal/external URLs. We can use the following notation to work with links in our test scenarios:

When I am viewing a "post" content:
  | title       | Post title                                              |
  | field_link1 | https://nuvole.org                                       |
  | field_link2 | Link 1 - https://nuvole.org                              |
  | field_link3 | Link 1 - https://nuvole.org, Link 2 - http://example.com |

As you can see we use always the same pattern: a dash - to separate parts of the same field value and a comma , to separate multiple field values.

We can also refer to a select list value by simply referring to its label, so to have much more readable test scenarios. For example, given the following allowed values for a select field:

option1|Single room
option2|Twin room
option3|Double room

In our test scenario, we can simply write:

When I am viewing a "post" content:
  | title      | Post title               |
  | field_room | Single room, Double room |

Working with other entity types

Field-handling works with other entity types too, such as users and taxonomy terms. We can easily have a scenario that would create a bunch of users with their relative fields by writing:

Given users:
  | name    | mail                | language | field_name | field_surname | field_country  |
  | antonio | antonio@example.com | it       | Antonio    | De Marco      | Belgium        |
  | andrea  | andrea@example.com  | it       | Andrea     | Pescetti      | Italy          |
  | fabian  | fabian@example.com  | de       | Fabian     | Bircher       | Czech Republic |

Contributing to the project

At the moment field-handling is still a work in progress and, while it does support both Drupal 7 and Drupal 8, it covers only a limited set of field types, such as:

Simple text fields
Date fields
Entity reference fields
Link fields
List text fields
Taxonomy term reference fields

If you want to contribute to the project by providing additional field type handlers you will need to implement this very simple, core-agnostic, interface:

<?php
namespace Drupal\Driver\Fields;

/**
 * Interface FieldHandlerInterface
 * @package Drupal\Driver\Fields
 */
interface FieldHandlerInterface {

  /**
   * Expand raw field values in a format compatible with entity_save().
   *
   * @param $values
   *    Raw field values array.
   * @return array
   *    Expanded field values array.
   */
  public function expand($values);
}
?>

If you need some inspiration check the current handlers implementation by inspecting classes namespaced as \Drupal\Driver\Fields\Drupal7 and \Drupal\Driver\Fields\Drupal8.

Drupal 8 Configuration Management with Features

Mon, 27 Apr 2015 13:30:09 GMT

This is a preview of Nuvole’s training at DrupalCon Los Angeles: “An Effective Development Workflow in Drupal 8”.

Features for Drupal 8 will exist and it is already in alpha state. This may be surprising to those who wrongly assumed that Configuration Management was going to supersede Features; but, as we explained before, it all comes down to understanding how to use Features the right way in Drupal 8.

If you are using Features in Drupal 8 for deployment of configuration, you are doing it wrong!™

The role of Features

While the configuration management was built to keep configuration consistent and enable deploying it between different environments of the same site it was not built for sharing configuration between different sites. This is where Features comes in. Features allows you to easily bundle related configuration and re-use it on another site.

For re-using configuration and building starter profiles the current alpha version is already great. It works just like the re-usable “feature” we were blogging about last year, except it comes with a familiar UI and lets you do everything as a site builder. It even comes with a smart system that auto detects configuration that belongs together.

Configuration changes are development

Developers and site builders working with configuration in Drupal 8 need to understand that changing configuration needs to be treated equally with development. While this may seem trivial at first, the consequences are a fundamental shift to the approach of building a Drupal 8 site.

Drupal 8 comes with the configuration management system that was built to deploy configuration between different environments of the same site. It allows the whole configuration of a site to be synchronized both through the UI for “site builders” and through drush and git for “developers”. While the system allows individual configuration objects to be exported and imported, deployment and synchronization is always done with the entire set to ensure it is consistent.

This means that when you change configuration on a production site you either opt out of using the configuration management to deploy configuration or you need to go through a more elaborate work flow to synchronize the configuration changes with the development environments similar to what you would need to do if you were to change code directly on the production server.

For the math-oriented,

Δ config ⊆ development

What if you want to be free of the temptation to edit configuration in production? Luckily there is a module for that! It is called Configuration read-only mode and allows to lock the forms where you change the configuration, thus enforcing its role as a “developers only” tool.

Of course some configuration will differ between environments. Just as the database credentials are kept separate from the rest of the code, specific configuration can be overridden in the instances’ settings.php or services.yml. The rest of the code is usually treated as a whole for consistency reasons, the same should be true for configuration.

What about not using configuration management?

Features and packaging related modules should not be regarded as a solution to deploy only partial configuration. Packaged configuration by definition is never aware of the whole site and the possible inter-dependencies of the site’s particular configuration. Deploying only partial configuration circumvents the safeguards Drupal put in place to make the deployment more robust. Of course nobody is forced to use the new tools for configuration management and you can easily opt out of using it by just ignoring that option. It is also still possible to develop a Drupal 8 site by sharing a database dump just as with Drupal 5. But remember that a Drupal 7 approach will yield the Drupal 7 headaches. We would recommend re-evaluating the deployment strategies when starting to use Drupal 8.

Managing distributions in Drupal 8

The more complicated scenario which is yet to be tackled is a feature for a richer distribution which will update over time. For example a newer version of a feature could come with an updated view or additional or updated fields for a content type.

First steps in that direction have already been taken by Configuration Update Manager (a dependency of features) and Configuration Synchronizer (a sandbox module). Configuration Update Manager compares the configuration in use on the site with the configuration which was provided by a module or installation profile. Configuration Synchronizer goes a step further and keeps a snapshot of the configuration when it was installed in order to determine whether the configuration has been customized by an administrator or whether it can safely be updated to the one provided in the new version of the module/feature.

All the modules mentioned above are used in a development environment of a particular site. Or in other words in an environment where configuration changes are expected and part of the process of updating a site.

For more, see our presentation from the Drupal Developer Days (embedded below); thanks to all the people there for the fruitful discussions we had on this topic during the event.

Drupal 8 Configuration Management with Features from Nuvole

2015-04-devdays-montpellier.pdf (711.18 KB) download

Drupal 8 Configuration Management: beware of pitfalls

Mon, 10 Aug 2015 14:35:10 GMT

Drupalaton in Hungary last weekend gave the opportunity for a 90 minutes workshop about Configuration Management (CMI) and how it changes a Drupal developers work flow with the coming Drupal 8.

CMI allows you to have the whole site configuration under control and that is a great improvement. But it also comes with its pitfalls. Developers have to be more vigilant when sharing configuration with each other: mindless exporting configuration may override the configuration shared by others, mindless importing may override one’s own work and forgetting to import merged configuration leads to ignoring the work of your co-developers at a later stage.

CMI, while it is much more awesome than anything we have in Drupal 7, is also much more unforgiving. So it would be all too easy to not use it properly and what it was designed for and then wonder why you can’t have nice things.

Fortunately, a methodical git workflow can help recover from mistakes (except importing configuration before exporting it first). So don’t forget the order of the steps:

config export
git commit
git merge
config import

And do it often to keep the merges and failures simple and easy to resolve.

Please find the full Drupalaton 2015 workshop slides attached below.

CMI-drupalaton2015.pdf (745.87 KB) download

Optimal deployment workflow for Composer-based Drupal 8 projects

Fri, 19 Aug 2016 11:20:33 GMT

This post is an excerpt from the topics covered by our DrupalCon Dublin training: Drupal 8 Development - Workflows and Tools.

During the recent Nuvole presentations at Drupal Dev Days Milan 2016 and Drupalaton Hungary 2016 we received a number of questions on how to properly setup a Drupal 8 project with Composer. An interesting case where we discovered that existing practices are completely different from each other is: “What is the best way to deploy a Composer-based Drupal 8 project?”.

We’ll quickly discuss some options and describe what works best for us.

What to commit

You should commit:

The composer.json file: this is obvious when using Composer.
The composer.lock file: this is important since it will allow you to rebuild the entire codebase at the same status it was at a given point in the past.

The fully built site is commonly left out of the repository. But this also means that you need to find a way for rebuilding and deploying the codebase safely.

Don’t run Composer on the production server

You would clearly never run composer update on the production server, as you want to be sure that you will be deploying the same code you have been developing upon. For a while, we considered it to be enough to have Composer installed on the server and run composer install to get predictable results from the (committed) composer.lock file.

Then we discovered that this approach has a few shortcomings:

The process is not robust. A transient network error or timeout might result in a failed build, thus introducing uncertainty factors in the deploy scripts. Easy to handle, but still not desirable as part of a delicate step such as deployment.
The process will inevitably take long. If you run composer install in the webroot directly, your codebase will be unstable for a few minutes. This is orders of magnitude longer than a standard update process (i.e., running drush updb and drush cim) and it may affect your site availability. This can be circumvented by building in a separate directory and then symlinking or moving directories.
Even composer install can be unpredictable, especially on servers with restrictions or running different versions of Composer or PHP; in rare circumstances, a build may succeed but yield a different codebase. This can be mitigated by enforcing (e.g., through Docker or virtualization) a dev/staging environment that matches the production environment, but you are still losing control on a relatively lengthy process.
You have no way of properly testing the newly built codebase after building it and before making it live.
Composer simply does not belong in a production server. It is a tool with a different scope, unrelated to the main tasks of a production server.

Where to build the codebase? CI to the rescue

After ruling out the production server, where should the codebase be built then?

Building it locally (i.e., using a developer’s environment) can’t work: besides the differences between the development and the production (--no-dev) setup, there is the risk of missing possible small patches applied to the local codebase. And a totally clean build is always necessary anyway.

We ended up using Continuous Integration for this task. Besides the standard CI job, which operates after any push operation to the branches under active development, performs a clean installation and runs automated tests, another CI job builds the full codebase based on the master branch and the composer.lock file. This allows sharing it between developers, a fast deployment to production through a tarball or rsync, and opportunities for actually testing the upgrade (with a process like: automatically import the production database, run database updates, import the new configuration, run a subset of automated tests to ensure that basic site functionality has no regressions) for maximum safety.

Slides from our recent presentations, mostly focused on Configuration Management but covering part of this discussion too, are below.

Configuration Management in Drupal 8 - DDD2016.pdf (1.1 MB) download

Maintain local development configuration with Configuration Split

Tue, 23 Aug 2016 12:21:17 GMT

This post is an excerpt from the topics covered by our DrupalCon Dublin training: Drupal 8 Development - Workflows and Tools.

For the last two years we have been giving trainings and presentations at various Drupal events about configuration management and its new workflows in Drupal 8. One of the recurring questions has been:

How do I keep some configuration from being deployed? For example the devel module and its configuration?

Until now we have answered to use drush with the --skip-modules flag and then to gitignore the configuration for devel. But then you can’t share the development configuration and the process is error prone when there are more modules and other configuration items that depend on the configuration you gitignore.

For some simpler cases where configuration needs to be different between environments (for example the error verbosity) the configuration override system allows to override the configuration in settings.php. This is a solution for some cases, however, you can not override which modules are enabled or override configuration that doesn’t exist.

Enter: Configuration split!

Our new module Configuration split splits the configuration when exporting it with a special Drupal Console command. It can be configured to split out enabled modules and given blacklisted configuration and it will then separate all the configuration that is dependent on the listed modules or configuration. The module’s settings also allow to set a folder to which the separated configuration will be exported to.

That way the configuration set which you use to deploy configuration between different environments is a subset of your development configuration and the trusty configuration system of Drupal 8 can be used unharmed. Of course when importing with the special command the split configuration is merged back, allowing you to keep your development configuration in place.

“To split” is a synonym for “to break” and as such “Configuration split” has a dangerous ring to it. This is on purpose because the exported subset is on purpose not what you have on your development site and not what you have locally tested. So you need to compensate that with a workflow that imports and verifies the configuration you are going to deploy. This is better than to import and export individual configuration because Drupal needs the whole set of configuration to do its checks.

How do I use it?

Download and enable the Configuration split module like any other module. Then configure it under admin/config/development/configuration/config_split. Set the split folder to a different folder than your normal config sync folder. If you want a prettier interface, consider using Chosen. Then use the Drupal Console command config_split:export and config_split:import to export and import respectively.

Let’s look at the devel example from above. We typically version the configuration outside of the webroot in ../config/sync as seen by Drupal. (In the project root when starting a project with drupal-composer/drupal-project.) So the folder we specify for config_split would be ../config/dev. The module to filter would be Devel and the rest can be left empty. The devel.settings will be detected automatically. Note though that the system.menu.devel does not depend on the devel module and can not be detected automatically, but it is easy to add it to the blacklisted config and it could theoretically be deployed without breaking anything.

The resulting config_split.settings.yml could look something like this when exporting it:

folder: ../config/dev
module:
  devel: 0
theme: {  }
blacklist:
  - system.menu.devel

Finally the following command will export the configuration to the default sync directory without the devel module enabled and export the devel configuration into the dev directory.

web$ ../vendor/bin/drupal config_split:export

Now if you would import the configuration through the UI or with drush cim the devel module would be un-installed, and you can do that to see the site without its development configuration. However, If you want the development configuration to stay or become active and the devel module installed use the following command:

web$ ../vendor/bin/drupal config_split:import

How does it work internally?

We implemented a StorageWrapper that allows filters to interact with the configuration after it has been read and before it is written to the wrapped FileStorage during the import and export operation. The SplitFilter has a secondary storage and decides where to read from or write to. This is a very similar concept to what drush does with its --skip-modules flag since we will want to easily integrate this with drush in the future.

What comes next?

The module is still in an early development stage and some more additions in the scope of splitting configuration could be added. The subset without the split configuration could be verified after exporting it and we could warn the user if it couldn’t be imported. Or for example we currently use only config_split.settings but if the need arises we could support multiple split configurations. Or we could add a “gray list” to ignore some configuration that exists rather than removing it when splitting, essentially making it a configuration override outside of the scope of Drupal’s runtime. This could be useful when maintaining several sites that are almost the same but all have their little “special snowflake” configuration which in turn could be synchronized with the normal workflow.

It is important to understand that the configuration system of Drupal has limitations that are there for a good reason. Most of them are to ensure data integrity and robustness and reliability of the synchronisation and so on. In other words measures to protect you from accidentally breaking your site. Using tools like config_split or drush --skip-modules you circumvent some of these security and integrity checks so use them with caution.

Since the module is not required for the regular functioning of a Drupal 8 site (it can even be used to blacklist itself) it can already be used in the development process of your current projects already. Your feedback is welcome, see you in the issue queue.

Pimp your Behat Drupal Extension and rule the world

Mon, 19 Sep 2016 14:00:00 GMT

This post is an excerpt from the topics covered by our DrupalCon Dublin training: Drupal 8 Development - Workflows and Tools.

At Nuvole we consider writing good tests as a fundamental part of development and, when it comes to testing a complex site, there is nothing better than extensive behavioral tests using Behat. The benefits of such a choice are quite obvious:

Tests are very easy to write.
Behat scenarios serve as a solid communication mean between business and developers.

As a site grows in complexity, however, the default step definitions provided by the excellent Behat Drupal Extension might not be specific enough and you will quickly find yourself adding custom step to your FeatureContext or creating custom Behat contexts, as advocated by all official documentation.

This is all fine except that your boilerplate test code might soon start to grow into a non-reusable, non-tested bunch of code.

Enter Nuvole’s Behat Drupal Extension.

Nuvole’s Behat Drupal Extension

Nuvole’s Behat Drupal Extension is built on the shoulders of the popular Behat Drupal Extension and it focuses on step re-usability and testability by allowing developers to:

Organize their code in services by providing a YAML service description file, pretty much like we all are used to do nowadays with Drupal 8.
Override default Drupal Behat Extension services with their own.
Benefit of many ready-to-use contexts that are provided by the extension out of the box.

Installation and setup

Install Nuvole’s Behat Drupal Extension with Composer by running:

$ composer require nuvoleweb/drupal-behat

Setup the extension by following the Quick start section available on the original Behat Drupal Extension page, just use NuvoleWeb\Drupal\DrupalExtension instead of the native Drupal\DrupalExtension in your behat.yml as shown below:

default:
  suites:
    default:
      contexts:
        - Drupal\DrupalExtension\Context\DrupalContext
        - NuvoleWeb\Drupal\DrupalExtension\Context\DrupalContext
        ...
  extensions:
    Behat\MinkExtension:
      goutte: ~
      ...
    # Use "NuvoleWeb\Drupal\DrupalExtension" instead of "Drupal\DrupalExtension".
    NuvoleWeb\Drupal\DrupalExtension:
      api_driver: "drupal"
      ...
      services: "tests/my_services.yml"
      text:
        node_submit_label: "Save and publish"

“Service container”-aware Contexts

All contexts extending \NuvoleWeb\Drupal\DrupalExtension\Context\RawDrupalContext and \NuvoleWeb\Drupal\DrupalExtension\Context\RawMinkContext are provided with direct access to the current Behat service container. Developers can also define their own services by adding a YAML description file to their project and setting the services: parameter to point to its current location (as shown above).

The service description file can describe both custom services and override already defined services. For example, given a tests/my_services.yml containing:

services:
  your.own.namespace.hello_world:
    class: Your\Own\Namespace\HelloWorldService

Then all contexts extending \NW\D\DE\C\RawDrupalContext or \NW\D\DE\C\RawMinkContext will be able to access that service by just calling:

<?php

class TestContext extends RawDrupalContext {

  /**
   * Assert service.
   *
   * @Then I say hello
   */
  public function assertHelloWorld() {
    $this->getContainer()->get('your.own.namespace.hello_world')->sayHello();
  }

}
?>

The your.own.namespace.hello_world service class itself can be easily tested using PHPUnit. Also, since Behat uses Symfony’s Service Container you can list services your service depends on as arguments so to remove any hardcoded dependency, following Dependency Injection best practices.

Override existing services

Say that, while working on your Drupal 7 project, you have defined a step that publishes a node given its content type and title and you want to use the same exact step on your Drupal 8 project, something like:

Given I publish the node of type "page" and title "My page title"

The problem here is that the actual API calls to load and save a node differs between Drupal 7 and Drupal 8.

The solution is to override the default Drupal core services specifying your own classes in your tests/my_services.yml:

parameters:
  # Overrides Nuvole's Drupal Extension Drupal 7 core class.
  drupal.driver.cores.7.class: Your\Own\Namespace\Driver\Cores\Drupal7
  # Overrides Nuvole's Drupal Extension Drupal 8 core class.
  drupal.driver.cores.8.class: Your\Own\Namespace\Driver\Cores\Drupal8

services:
  your.own.namespace.hello_world:
    class: Your\Own\Namespace\HelloWorldService

You’ll then delegate the core-specific business logic to the new core classes allowing your custom step to be transparently run on both Drupal 7 and Drupal 8. Such a step would look like:

<?php
class TestContext extends RawDrupalContext {

  /**
   * @Given I publish the node of type :type and title :title
   */
  public function iPublishTheNodeOfTypeAndTitle($type, $title) {
    $this->getCore()->publishNode($type, $title);
  }

...
?>

Ready to use contexts

The extension also provides some utility contexts that you can use right away in your tests. Below a quick overview of what’s currently available:

Context	Description
`NuvoleWeb\Drupal\DrupalExtension\Context\DrupalContext`	Standard Drupal context. You want to use this one next to (and not instead of) `Drupal\DrupalExtension\Context\DrupalContext`.
`NuvoleWeb\Drupal\DrupalExtension\Context\ContentContext`	Perform operations on Content.
`NuvoleWeb\Drupal\DrupalExtension\Context\CKEditorContext`	Allows to interact with CKEditor components on your page.
`yaml NuvoleWeb\Drupal\DrupalExtension\Context\ResponsiveContext: devices: mobile_portrait: 360x640 mobile_landscape: 640x360 tablet_portrait: 768x1024 tablet_landscape: 1024x768 laptop: 1280x800 desktop: 2560x1440`	Resize the browser according to the specified devices, useful for testing responsive behaviors.
`NuvoleWeb\Drupal\DrupalExtension\Context\PositionContext`	Check position of elements on the page.
`NuvoleWeb\Drupal\DrupalExtension\Context\ChosenFieldContext`	Interact with Chosen elements on the page.

We will share more steps in the future enriching the current contexts as well as providing new ones so keep an eye on the project repository!

Disclaimer

At the moment only Drupal 8 is supported but we will add Drupal 7 support ASAP (yes, it’s as easy as providing missing Drupal 7 driver core methods and adding tests).

Configuration Management: Theory and Practice

Tue, 27 Sep 2016 16:23:53 GMT

Developers often want to use Configuration Management outside its intended use case. New workflows and practices have to be established.

We’ve just presented a gallery of problems and solutions at DrupalCon Dublin 2016.

The room was packed, and many people were denied admission for security reasons. For them, and for those who didn’t manage to come to this DrupalCon, we are making the full presentation available here:

PDF slides for download
Full video of the presentation

Configuration-Management-Theory-and-Practice.pdf (2.41 MB) download

Configuration Split: first beta release at Drupal IronCamp

Mon, 28 Nov 2016 11:30:44 GMT

One of the nice things that happened during Drupal Ironcamp in Prague last week was the first beta release of the Configuration Split that I started developing a few months ago.

As explained in the previous blog post about Configuration Split, the typical use case of Configuration Split is to be able to maintain a “development” configuration and a “production” configuration, where some of the modules or settings you use in development are not copied to production upon configuration export/import.

A typical use case

We assume that you have a production and development version of the same site and that they are aligned.

Step 1: Enable anything you need in development and work normally

In the development environment we enable Devel and UI modules enabled and we can create new content types, add fields and do all things we like to do while developing a Drupal site. Some of this configuration work is meant to go to the production site and some (e.g., the fact that Devel is enabled and related configuration) is not. We thus need to split our configuration.

Step 2: Create a split definition

Choose a name for your “local” configuration and select all the configuration you don’t want to have on the live site (even though, if you wish, you can still share it with colleagues). This can include modules you would not want to be enabled or configuration that is for development only. The path here is a folder next to the normal config sync directory outside of the web root. In the example above, we are splitting out to a configuration set named “Development” the Devel, Fields UI, Views UI modules and the system.menu.devel configuration.

Step 3: Export your configuration with `config_split`

$ drush config-split-export

In the most basic use case we just replace config-export with config-split-export in our workflow. This exports the elements we selected above to ../config/dev and the rest (the configuration we want to deplay) to our “standard” configuration folder, in this case ../config/sync.

Step 4: Import the “clean” configuration in production

On all environments we use the configuration import of config_split:

$ drush config-split-import

So in our usual workflow we replace config-import with config-split-import

In order not to have different workflows for development and production we simply deactivate the split on production by overriding the configuration in settings.php on production: $config['config_split.config_split.development']['status'] = FALSE;

This will result in the split definition to be deactivated.

And consequently the import will have the configuration blacklisted in the split removed and development modules deactivated.

What’s new in the beta version

The following features are new or changed in the beta release:

The drush command works and has some text to warn users to which directories configuration will be written to.
The drupal console command is disabled, but there is a patch/branch you can use or help to make it functional again.
Split entities now have a status and the default command only uses the active splits. Of course you can override that in settings.php as with almost all configuration.
You can switch out the default config.storage.sync service in your services.yml file so that the default configuration sync UI uses the split import. Details are in the README.txt file.

More information and acknowledgments

On Friday I gave a presentation about my favorite topic in Drupal: Configuration Management, attached here are the slides.

I wish to thank IronCamp participants and especially Swentel and mr.baileys who worked on patches, and I’d like to thank IronCamp organizers for a very well organized event where I met a lot of old and new friends. Swentel also took care of documenting more advanced use cases of Configuration Split on his blog.

For any further information see the module page and its issue queue.

ConfigurationManagement_ironcamp2016.pdf (1.28 MB) download

Introducing the UI Patterns module: use atomic UI components everywhere in Drupal 8

Mon, 23 Jan 2017 16:09:20 GMT

Update: We have setup a Gitter chat room if you have any specific questions about the module’s functionality:

Establishing an atomic design system in your project is one of the most effective way to build a consistent and maintainable user interface.

Over the past years projects like PatternLab or the Component Libraries module aimed at lowering the cost of maintaining (PatternLab) and re-using (Component Libraries) UI patterns in your projects, be it a generic PHP application or a brand new Drupal 8 site.

But, as we all know, when it comes to presenting content the Drupal 8 landscape is quite diverse: you can layout your pages using Panels or Views, style your entities using Display Suite view modes, group your fields with Field group, etc.

Such a diversification can surely present some challenges when it comes at reusing a well-designed and consistent UI library. In other words: how can I transparently use the same UI pattern in my views, layouts, field formatters, etc.?

Enter the UI Patterns module

The UI Patterns module allows you to define and expose self-contained UI patterns as Drupal 8 plugins and use them seamlessly as drop-in templates for panels, field groups, views, Display Suite view modes and field templates.

The module also generates a pattern library page that can be used as documentation for content editors or as a showcase for business and clients (the following example is styled using the Bootstrap theme):

The UI Patterns module also easily integrates with with tools like PatternLab or modules like Component Libraries.

Define your patterns

Patterns can be defined using YAML in files named MY_MODULE.ui_patterns.yml or MY_THEME.ui_patterns.yml using the following format:

blockquote:
  label: Blockquote
  description: Display a quote with attribution information.
  fields:
    quote:
      type: text
      label: Quote
      description: Quote text.
      preview: Life is like riding a bicycle. To keep your balance, you must keep moving.
    attribution:
      type: text
      label: Attribution
      description: Quote attribution.
      preview: Albert Einstein

After defining the pattern you have to provide a Twig template to render it which, in our case, could look like that:

<blockquote>
  <p>{{ quote }}</p>
  <footer>{{ attribution }}</footer>
</blockquote>

Once you are done you can visit the pattern library page and check your new Blockquote pattern in action:

We have much more options available to make sure pattern definition can fit your use case (i.e. template overrides, etc.), make sure you check the documentation for the full list.

Use your patterns everywhere

After exposing your patterns you are ready to use them anywhere thanks to the sub-modules bundled within UI Patterns, namely:

UI Patterns Field Group: allows to use patterns to format field groups provided by the Field group module. Read the documentation.
UI Patterns Layouts: allows to use patterns as layouts provided by the Layout plugin module. This allows patterns to be used on Display Suite view modes or on panels out of the box. Read the documentation.
UI Patterns Display Suite: allows to use patterns to format Display Suite field templates. Read the documentation.
UI Patterns Views: allows to use patterns as Views row templates. Read the documentation.

Example: style links as call-to-action buttons

One of the most ordinary situation is styling a group of links as call-to-action buttons. This can be easily achieved using UI Patters.

Say we have defined the following Button pattern:

button:
 label: Button
 description: A simple button.
 fields:
   title:
     type: text
     label: Label
     description: The button label
     preview: Submit
   url:
     type: text
     label: URL
     description: The button URL
     preview: http://example.com

On the entity display setting page we access the link field setting by clicking on the gear icon:

Then, after selecting the Pattern field template and the Button pattern, we map the link field columns to the pattern’s fields defined above:

Each value of our multi-valued link field will be then formatted using the Button pattern, as shown below:

Conclusions

The UI Patterns module aims at integrating your pattern library with the most used Drupal 8 rendering systems. It also makes easy to use third-party tools such as PatternLab.

The project is currently under active maintenance, please file issues and/or support requests using this GitHub repository.

P.S. Special thanks to aleksip for getting the integration with PatternLab and the Component Libraries to work!

IMPORTANT UPDATE: If you are using version 8.x-1.0-beta2 make sure you read this change note for a safe upgrade to newer versions.

Introducing Config Filter

Mon, 27 Feb 2017 14:12:28 GMT

Our solution for advanced configuration management workflows has just become more powerful! The core of what makes Configuration Split work so nicely with drush and the Drupal UI has been split off in a new module: Config Filter! Config Filter exposes a ConfigFilter plugin and swaps the sync storage applying all the active ConfigFilters to it. This means it is no longer necessary to manually swap the service as we recommended to do in the past and it works also when installing a site, as the swapping happens only when the module is installed. As of drush 8.1.10 this works with the default config import and export commands.

Update Config Split

For current users of the Config Split module it means that they should remove the custom service swapping as part of the update and then apply the database updates which will install Config Filter (you already downloaded it with composer update, right?).

New command-line options and new workflows

With the re-factoring we also changed the options of the drush and console commands. We added a new option for specifying a split and then export to and import from only that split directory; this supersedes the previous options to specify separate export destinations and import sources - they are not needed for the simple work flow we advocate for, and the behaviour can easily be achieved by modifying settings.php as needed.

With the addition of the single split export, a new workflow becomes possible: export only the “production” split, before importing the whole configuration which will have it merged back in.

Improved Graylists

In addition to the new architecture the graylist feature has been improved so that dependencies can be graylisted as well and optionally only items that differ from the sync storage are split off. More details and more documentation will come in future.

Roadmap

The ultimate goal is to improve the export API in Drupal 8 core so that more advanced workflows are possible. Config Filter is our proposal for a solution in contrib, but we feel that this functionality should belong to Core.

On the way to Core, here are some more immediate steps:

Clean up after the big refactoring, adding automated tests for some special cases.
Improve the documentation.
Integrate with Config Readonly to allow locking the site except for the configuration defined in splits.
For those that want to do configuration management without seatbelts, integrate Config Ignore with Config Filter so that it can work smoothly next to Config Split.
…and eventually propose Config Filter for inclusion in Core!

More information and acknowledgments

The re-factoring started during the Drupal Mountain Camp in Davos and I would like to thank the organizing committee for the great event. On Friday I gave a presentation about my favorite topic in Drupal: Configuration Management, attached here are the slides based on the Nuvole presentation at DrupalCon Dublin 2016.

For any further information see the module pages for Config Split and Config Filter and their issue queues.

ConfigurationManagement_no-problems_davos17.pdf (1.15 MB) download

Our presentations at Drupal Dev Days Seville

Thu, 30 Mar 2017 15:25:55 GMT

Last week we attended and sponsored the Drupal Developer Days in Seville where we also had two well attended sessions. Introducing the UI Patterns module: use atomic UI components everywhere in Drupal 8 and Advanced Configuration Management with Config Split et al. attached here are the promised slides, as well as a few updates about the modules.

What’s new in Config Split

Drush

As we posted about before, drush supports the Config Split workflow since 8.1.10. In the next version drush will drop the support for its --skip-modules flag and people using it should upgrade to using Config Split.

Split Storage in Database

Previous versions of Config Split allowed to set an empty split folder which resulted in the configuration to be lost. To avoid saving the configuration into files under version control one would therefore have to set up a temporary directory and save the split there. But with the last development release a separate database storage is used when not specifying a split folder. This allows configuration to be “stashed” in the database for a deployment. A specific export first is still and will always be necessary by design.

Advanced Configuration Management with Config Split et al. from Nuvole

Click here to download the slides.

What’s new in UI Patterns

Following a productive BOF meeting at DDD, it was decided to move everything that concerns defining and displaying patterns locally into a separate module.

This will allow for a better and more solid architecture: the main UI Patterns module will be solely responsible to provide plugins and other glue code: it will then be responsibility of modules implementing component library integrations to expose their components using pattern derivers.

Discussion is ongoing at https://github.com/nuvoleweb/ui_patterns/issues/86 and more generic future plans are being dicussed at https://github.com/nuvoleweb/ui_patterns/issues/76

Introducing the UI Patterns module: use atomic UI components everywhere in Drupal 8 from Nuvole

Click here to download the slides.

Configuration Management dimensions

Tue, 25 Apr 2017 12:07:28 GMT

Unfortunately none of us from Nuvole are attending DrupalCon Baltimore and can’t, therefore, attend the “hallway track” and join discussions in person. We have held presentations about advanced configuration management in Drupal 8 at all Drupal events we attended in the last year including the last DrupalCon in Dublin. So I’ll try to cover some concepts here that could help the discussion about how the configuration management can be improved.

To me there are at least two important dimensions to configuration management in Drupal 8 and different contrib project have sprouted to address:

Vertical: transfer configuration between different environments of the same site.
Horizontal: transfer configuration between different sites.

Following are a few contrib solutions and core issues that address the different itches. This is not meant to be an exhaustive or definitive list but highlight the different problem spaces. Many more contrib solutions address some issues in this space to accommodate various different workflows.

Vertical

Drupal 8 core only addresses this use case, however, there are some important cases not covered (yet). The management of configuration between the different environments is best taken care of by importing and exporting the whole sites configuration together.

Horizontal

Due to the fact that the same tools can be used for both dimensions and the fact that in Drupal 7 features was abused for doing the vertical configuration management as well this concept may not be so clear. Ideally configuration between sites is shared from the development environment of one site to the development environment of another, and the vertical tools with drupal core are used for deployment. Moving configuration between different sites is done by moving a subset of configuration between environments.

Re-using a set of configuration in another site

Contrib Solution: Features
There are many more modules designed to deal with distributions and dealing with the paradigm that sites now own the configuration.

Inheriting installation profiles

Core issue: Allow profiles to provide a base/parent profile

Multisite with large portion of shared configuration

Contrib Solution: Config Split
While this is not the original problem it tries to solve, it is reported to be used for it..

Conclusion

I may have an obvious bias towards config split since we maintain that module but it is just one of many config related modules. I hope there is a fruitful discussion in Baltimore about configuration management in Drupal 8. BoF

Stable release for Config Split and Config Filter

Mon, 21 Aug 2017 08:39:04 GMT

One year ago we released the first public version of Config Split with the goal to simplify working with Drupal cores configuration management. The main motivation was to find a solution for having development modules in local environments and avoiding to deploy their configuration. To achieve this in the cleanest way possible we decided to interact with Drupal only during the configuration import and export operations by altering what is read from and written to the \Drupal\Core\Config\StorageInterface.

We quickly realized that this is a powerful way to interact with how Drupal sees the configuration to import and so we split off the code that does the heavy lifting into its own module and now Config Ignore and Config Role Split use the same mechanism to do their thing.

Config Split now has documentation pages you are welcome to contribute to and there will be a session at DrupalCon in Vienna in which I will show how it can be used to manage a sites configuration in interesting ways.

If you were an early adopter (pre beta3) but have not updated to a recent version, you will need to install Config Filter first or patch core. The workaround and legacy code has now been removed and the current code is going to be backwards compatible in the future. So if you use the APC class loader, make sure to clear the apc/apcu cache or you might get an error after updating.

How to maintain Drush commands for Drush 8 and 9 and Drupal console with the same code base.

Fri, 13 Oct 2017 08:48:40 GMT

Drupal 8.4 and its upgrade to Symfony 3 has made the compatibility of the global Drush 8 a bit more challenging. Drush 9 works with Drupal 8.4 but it is not stable yet and the format of how third party Drush commands are made has changed significantly.

While Drush 9 comes with a command that helps porting Drush 8 commands you will still end up maintaining very similar code in two places one with calls to drush_confirm('...') and one with $this->io()->confirm('...'). If you decide to also provide your commands for Drupal console you now have three times the burden.

Because we tried to provide the commands for Config Split for both Drush and Drupal console early on we faced this problem already more than a year ago. And now it has paid off because porting the commands to Drush 9 was very quick.

The solution is actually really simple and brings the added benefit of being able to test the business logic of the commands in the absence of Drush or Drupal console. It is all about separating the command discovery from the command logic. Drush 8, 9 and Drupal console all have a bit different ways to discover and invoke commands, but the business logic you want to implement is the same so all we have to do is to extract a common “interface” our custom service can implement and then make the command definitions wrap that and keep things DRY.

The CliService

Config Split defines a config_split.cli service with the class ConfigSplitCliService with all its dependencies injected. It has the methods \Drupal\config_split\ConfigSplitCliService::ioExport and \Drupal\config_split\ConfigSplitCliService::ioImport that implements all the commands logic and delegate the actual importing and exporting to specific methods.

The method signature for both the export and import method are more or less the same: CliService::ioMethod($arguments, $io, callable $t).

$arguments: The arguments passed to the command.
$io: This is an object that interacts with the command line, in Drush 9 and Drupal console this comes from the Symfony console component, for Drush 8 we created a custom wrapper around drush_confirm and drush_log called ConfigSplitDrush8Io.
$t: This is essentially a t function akin to how Drupal translates strings. Because Drupal console translates things differently we had to be a bit creative with that by adding a t method to the command.

Commands wrap the service

The Drush 8 command is essentially:

<?php
function drush_config_split_export($split = NULL) {
  // Make the magic happen.
  \Drupal::service('config_split.cli')->ioExport($split, new ConfigSplitDrush8Io(), 'dt');
}

For Drush 9 we can use dependency injection and the Drush 9 command becomes essentially:

<?php
class ConfigSplitCommands extends DrushCommands {
  public function splitExport($split = NULL) {
    $this->cliService->ioExport($split, $this->io(), 'dt');
  }
}

And very similar the Drupal console command:

<?php
class ExportCommand extends SplitCommandBase {
  protected function execute(InputInterface $input, OutputInterface $output) {
    $this->setupIo($input, $output);
    // Make the magic happen.
    $this->cliService->ioExport($input->getOption('split'), $this->getIo(), [$this, 't']);
  }
}

Testing

The ConfigSplitCliServiceTest is a KernelTest which asserts that the export works as expected by exporting to a virtual file system. The test coverage is not 100% (patches welcome) but the most important aspects for the complete and conditional splitting (blacklist/graylist) is thoroughly tested. There are no limitations on what or how you can test your CliService since it is self contained in your module and does not depend on Drush or the Drupal console. For example one could write a unit test with a mocked $io object that asserts that the messages printed to the cli are correct.

Config Distro

Mon, 09 Apr 2018 16:06:25 GMT

Drupal core workflow limitations

It is not a secret that the configuration management in Drupal 8 core was made for only one specific use case: move configuration from one environment to another for the same site. Everything else was left for contrib to find solutions for. Config Installer is necessary to set up a site starting from existing configuration and Config Split is needed to have environment specific configuration that go beyond simple configuration overrides, like development modules enabled only locally. But the whole workflow still assumes that developers - most often in the same team - have control over the environments and deployment of the site.

Distributions are not covered

Distributions are very different. When maintaining a distribution one doesn’t develop a specific site but rather the template to build different sites with. Deploying configuration as we do it for a single site does not have any meaning because there is no one production site to deploy to. Each site based off a distribution is its own site with its own site uuid and its own possible customisations and possibly its own development and production environments. At the same time as a consumer of a distribution - a site owner of a site based off a distribution - one wants to update the distributions features while keeping the customisations.

Update hooks?

Different distributions handle this in different ways. A first approach to solve this issue is treating it the same way you would a schema update and change the configuration on a site with an update hook. The problem with this is that update hooks are meant to fix the database to be able to run with the version of code that is in use on the site. The first thing one needs to do after updating the code base is to run the update hooks to realign the database. Configuration management has different needs and when updating the configuration in an update hooks means that one has to be careful to check that the configuration is in the expected state. And the only recourse is to log and make sure the user is informed what manual actions he needs to do to update the configuration himself. As there is no place to involve the user with a choice while update hooks are run.

Config Distro

We propose a new kind of workflow, in essence it allows developers of a distribution to say how the “distributions configuration” is supposed to be updated and it allows site owners of a site based on a distribution to treat the distribution maintainers as a developer on their team. At the same time this does not interfere with the configuration management workflow for staging and deploying configuration to the production site.

The primary function of Config Distro is to allow updating a distributions configuration in a new workflow and give the site owners more control and insight on how the configuration of their site changes. It is a new way to imagine the configuration management system in a way that lets sites own their configuration but allowing distributions to provide updated configuration for existing sites.

A new dimension

In a blog post one year ago I talked about different dimensions of configuration management;

vertical: moving configuration between environments but for the same site
horizontal: moving configuration between different sites. for example for distribution updates.

Config Distro is a UI and a drush command for that workflow. It is intended to be used by site builders who base their site on a distribution and want to import configuration updates from the updated distribution. The UI is essentially a parallel to configuration import screen for the workflow supported by core. But instead of importing the configuration from the files used to move/deploy configuration between environments you import from a special configuration storage which starts out with the sites active configuration and has the distributions updates applied to it. This means you do not have problems with mismatched site uuids or things being removed that you added and were not part of the distribution. And updates can apply (or not) to translations even if the distribution did not have languages.

Where do updates come from

If you only install Config Distro, the import screen eternally shows that there is nothing to import. This is because the module only provides the framework for this to work, ie the UI and the drush command. The module is not opinionated on how or what should be updated. All the complexity can be addressed by a distribution maintainer with the help of a ConfigFilter plugin (the same way Config Split and Config Ignore work). One such module is Config Sync. All the complexity of finding out what what configuration a module has originally shipped with, what it does now and whether a user has changed the originally installed configuration is left to Config Sync and its dependencies.

Choice

Just like Config Ignore allows you to opt out of some of the configuration management workflows, Config Distro has a Config Distro Ignore module that lets you retain certain configuration from being changed when you hit the “import” button. The “Retain configuration” button is available right next to the “View differences” button.

Clicking it leads to a form that lets you choose to retain the configuration permanently or only for this specific update. It also allows you to ignore a update for a specific language.

Example

In our team we set up a site based on a distribution. We added our own login module for our single sign on and added a few webforms. Now there is a new version of the distribution with some new features and I would like upgrade the site to using the new features. I am tasked with updating the site, here is what I do:

I update the code of the distribution by specifying the new version in composer.json and do a composer update to get the updated code *
I run the database updates drush updb to align the database with the code *
I go to the distro update screen provided by Config Distro
- This screen looks very familiar, it looks the same way as when I import configuration changes my teammates made after I get their code with git pull.
- I see a new module is going to be installed and a few views and settings from the distribution are updated.
- Our webforms are not going to be removed and our custom modules not uninstalled.
I click “import all”.
Now my site has the updated code and configuration, so I export the configuration for deployment: drush cex and commit *
- git pull to get the code.
- composer install to get the external code (vendor, contrib, etc).
- drush updb to align the database with the new code base.
- drush cim to import the sync configuration.

This is the same procedure as for any module update on any site.
Distributions may add a “Auto upgrade” setting and then import the Config Distro in a post_update_hook to bypass the manual step required of site administrators upgrading the distribution.

Conclusion

Config Distro provides a new workflow for updating distributions with a familiar feel. It recognizes that update hooks are not an adequate solution for updating a sites configuration when a site owns the configuration and site administrators may have changed the initially provided configuration.
It allows developers of distributions to alter the sites configuration through ConfigFilter plugins and it gives site administrators a choice of what to import.
Config Distro is just the framework for extending cores configuration management to allow managing configuration changes to get into the site from a third party such as the distribution maintainers. It does not interfere with the traditional workflow and importing the configuration updates from a distribution should be seen as any other configuration change such as adding a new view or changing permissions: You go to an admin page and you click some things in a form, then you export the configuration and deploy it.

Future - CMI 2.0

While it already works, Config Distro is still in an alpha state. Config Distro is part of a larger effort to improve the configuration management in Drupal. You can find further information and participate in the discussion on drupal.org

CMI 2.0 issues need your review.

Wed, 06 Mar 2019 09:08:56 GMT

The configuration management initiative 2.0 needs your help.

I will be presenting updates of CMI 2.0 at the upcoming Drupalcon Seattle together with Mike Potter.

Some of the highlights of the CMI 2.0 road map for inclusion in Drupal core are an improved version of the concept of Config Filter (in the form of the config storage transformation api) and a simplified version of Config Split (in the form of config environments).
Unfortunately those big things will not make it into 8.7, but we could lay the ground work for 8.8.

But the deadline for some patches for Drupal 8.7 is this Friday!

It would be great if you could help us get the following issues to RTBC and committed:

#3036193: Add ExportStorageFactory to allow config export in third party tools
This would allow us to add a service to core that drush, drupal console and other tools and modules can use to export configuration. If this lands in 8.7 we will be able to patch Drush and Drupal Console between 8.7 and 8.8 and make improvements to configuration management such as adding a Config Environment module to core without then patching the cli tools again afterwards.

#3016429: Add a config storage copy utility trait
This adds a new utility trait that would make dealing with configuration storages easier. Currently there are a bunch of modules that implement this logic by themselves and not all of them do it correctly. This lead to bugs in Drush and Drupal Console and even Drupal core has a bug (which is fixed by this issue).

Thanks already in advance.

PS: If you are interested in more CMI 2.0 issues to review or work on check the CMI 2.0 candidate issue tag.

CMI 2.0 at Drupalcon Seattle 2019

Tue, 09 Apr 2019 17:08:51 GMT

Tomorrow Wednesday at 13:45 (Pacific Daylight Time) in room 6C we have the CMI 2.0 updates session at the Drupalcon.

Together with Mike Potter, we will briefly present the work that has been done so far and lay out the road map for what is to come. The session will be recorded and the slides are attached to this post (subject to change).

I will not go more into details of the session in this post but I would like to mention a challenge for which we need to find a solution and I would like everyone to invite to join the discussion on drupal.org #3046903 or here in the comments.

The challenge is how to deal with update hooks that change configuration which run only in certain environments and thus make the configuration deployment more difficult.

Click here to download the presentation slides.

CMI 2.0 session at Drupaldevdays in Cluj-Napoca

Thu, 13 Jun 2019 14:25:47 GMT

Today I presented the CMI 2.0 updates at the drupal dev days in Cluj-Napoca. The session went well and I received good feedback and had interesting conversations about configuration management after it.

Attached are the slides I presented.

There are plenty of issues to work on, join us in the sprint room or on the #config drupal slack. Find more information on the CMI 2 project page and issues tagged with CMI 2.0 candiate

CMI 2.0 session at Drupal DevDays in Cluj-Napoca from Nuvole

Click here to download the presentation slides.

Highlights from CMI 2.0 in Drupal 8.8.0

Thu, 10 Oct 2019 08:35:59 GMT

Nuvole offers a training at DrupalCon Amsterdam: “Drupal 8 Migration as a process” - register until October 27.

There are a few significant changes that landed in Drupal 8.8 with respect to configuration management:

The sync directory is defined in $settings and not $config_directories

The first one concerns everybody using Drupal 8 and especially going forward Drupal 9. So far the config directory was set up with in settings.php with the variable $config_directories. This variable was an array for historical reasons when both the active configuration and the sync configuration were stored in files. But since Drupal 8.0.0 that was not the case any more, yet the array remained. Drupal only ever used one key, but developers saw an array and naturally assumed that there should be more than one key.

In the effort to avoid confusion and to streamline the configuration management we have now corrected this and added the config directory to the settings variable as $settings['config_sync_directory']. If for some reason someone still wants to have a workflow with multiple sync directories then that can still be achieved with a small amount of code since setting.php is a php file and the settings can be set conditionally. In my opinion though one sync directory is sufficient for the vast majority of workflows and this change will help to not confuse developers.

Event is dispatched before configuration import and export to transform the configuration

For me this is the biggest achievement for CMI 2. This allows contrib and custom modules to interact with the configuration synchronization in a very straight forward way. It is Config Filter in core but better, it is what I have been talking about at many Drupal events and finally it is in Drupal core, stable and available to everyone. There is already a patch for Config Filter 2.x to be the bridge between modules depending on Config Filter and the new core API. Config Filter 2.x will be api compatible with Config Filter 1.x but instead of decorating the sync storage it will apply the filter plugins to the new core API. In addition there is an issue for Config Distro on the way to model its API after the new event driven core API.

Modules can be excluded from the configuration synchronization

The latest major achievement which will be celebrated by many will be that it is finally possible to have devel installed on the development environment and not having to worry about it being deployed. This feature is already available in contrib since December 2017 with the contrib module Config Exclude. It uses Config Filter and is a relatively simple module with just one configuration option. This is why it was a good candidate to add to core as a proof that the new API works. The contrib module and the new core feature (yes you don’t need to install a module it just works) both use the same setting: $settings['config_exclude_modules'] = ['devel', 'stage_file_proxy']; They work both alongside together so if you want to use this feature you don’t have to wait for 8.8, you can use it on 8.6 and then when you upgrade to 8.8 you can simply uninstall Config Exclude and it just continues to work.

There has been some confusion about this feature and how it can be dangerous. The excluded modules, (ie the modules listed in the settings array) are removed from core.extension.yml and all the config depending on them is not exported. When importing the configuration and the module is installed already the configuration from the site is added to the configuration to import just before validating the configuration for the import. That means that devel will not be uninstalled when you import the configuation that doesn’t have devel in it. But it also means that the configuration that you exported is no longer the configuration you had when you devloped your site and if you exclude modules that are central to the site such as required modules such as system or user or modules that change a lot of configuration such as language or field then the configuration can not be imported on the target site any more. Your site will not break, but you can not deploy the new configuration.

The CMI 2.0 Session at DrupalCon Amsterdam will contain more about all of the above and what else is planned to come.

A middle-format approach for complex and incremental Drupal 8 migrations

Sun, 20 Oct 2019 16:29:30 GMT

Nuvole offers a training at DrupalCon Amsterdam: “Drupal 8 Migration as a process” - register until October 27.

When having to import data onto a Drupal 8 site there is no other choice than relying on core’s Migrate API and its contrib ecosystem. The Migrate API in Drupal 8 implements a rather classic Extract, Transform, Load (ETL) process, with the following “Drupal-lingo” twists:

The extract phase is called “source” and it uses a source plugin to read data from external systems, be it a Drupal 7 database, a CSV file, a REST web service, etc.
The transform phase is called “process” and it uses process plugins to process and transform data
The load phase is called “destination” and it uses destination plugins to import data into specific Drupal 8 entity storages (e.g. nodes, taxonomy terms, etc.)

To recap, Drupal core implements the ETL process as follows:

Source plugins extract the data from the source.
Process plugins transform the data.
Destination plugins save the data as Drupal 8 entities.

Limitations of standard Drupal 8 ETL process

In the process described above the three steps are executed at the same moment in time, sequentially, every time we run a migration import. In this scenario we are to lose one of the most valuable aspects of an ETL process: testing and validating data prior to import.

Also, the process above easily accommodates for multiple data sources to be consolidated into one, coherent, dataset. With Drupal Migrate API this is only possible by using chained source plugin, and it can only run on the destination site. This is quite a limitation in complex enterprise scenarios, where displayed data is often the result of complex and convoluted transformations in the backend.

A middle-format approach

At Nuvole we have adopted a so called “middle-format approach”. The process simply moves the Extract and Transform parts outside Drupal so to ease the production of an easy to import dataset, in a well known middle-format (such as JSON API).

This approach has proved very successful in complex scenarios and, while completely leveraging the standard Drupal 8 migration process, it also allows to:

Aggregate data from different sources (not only Drupal 7 databases)
Test the data transformation process
Test imported Drupal 8 data
Easily automate the points above

The process looks like the following:

The approach outlined above brings the following benefits:

Exported data can be reviewed by the client and iteratively refined
Since Drupal 8 is not a precondition to export and transform data, data export site building can run in parallel, making the whole migration process much more efficient
Exported data can be presented to the different stakeholders using a user friendly UI, even before starting any Drupal 8 development
Since the data uses a well-known middle-format building the import process (as Drupal core’s Migrate plugins) is straightforward and allows to maximize code reusability

Middle-format migration at work

At Nuvole we have successfully used the middle-format approach to incrementally transform, review and import data over several years, on complex sites such as the World Food Programme main website. In that scenario we had to consolidate data coming from two Drupal 7 sites (plus a number of external data sources) in 13 different languages, over a two years period.

We have recently open-sourced a simplified version of the tool we have used to export and transform data, you can find its boilerplate code here.

We will run an hands-on session on how to use such a tool to plan and execute data exports in our upcoming DrupalCon Amsterdam training.

DrupalCon CMI 2 Session slides

Tue, 29 Oct 2019 13:42:38 GMT

Yesterday I presented the updates for the Configuration Management Initiative 2 at DrupalCon Amsterdam.

The main takeaways are:

There were 180 people in the small room and many who wanted to join the session simply couldn’t fit any more.

This is the recording from youtube:

There is still a lot of work to be done for CMI 2, join us on the contribution day at DrupalCon.

Attached are the slides of the session.

Have I seen you at DrupalCon before?

Thu, 31 Oct 2019 21:56:01 GMT

Amsterdam is the first city to host three European DrupalCons: 2005 (first DrupalCon ever), 2014 and 2019 (ongoing). How many community members attended all three of them?

We analyzed all user profiles published on drupal.org to find out this and much more.

Disclaimer: this is not serious research, just a fun experiment. It uses data from drupal.org user profiles that are known to be both unverified and outdated. If you take these data seriously, then Dries has never attended a DrupalCon. Who was that man on stage then? 😅

The flow: Amsterdam to Amsterdam to Amsterdam

You can hover/tap on the chart to see details. Apparently, less than 200 people from the current DrupalCon Amsterdam had never been seen before at a DrupalCon in Amsterdam, and a huge amount of those seen in Amsterdam for the first time in 2014 did not return. But this may well be due to people not updating their profile, so this should be assessed again once everybody has had time to put their attendance on record.

Out of the (not so few, it seems; but beware of cheaters, see below) pioneers from 2005, about 75% never returned. But some were still attending in 2014, some skipped 2014 but are back this year, and three heroes attended all DrupalCons Amsterdam (watch for the almost invisible segment bottom right in the chart). Kudos to them! Here they are, together with other statistics about DrupalCon Amsterdam.

Amsterdam DrupalCons attended in total	How many people	Who
Attended one DrupalCon in Amsterdam	2130 people	Many!
Attended two DrupalCons in Amsterdam	191 people	Many!
Attended all three DrupalCons in Amsterdam	3 people	Heine, bertboerland, drumm

Extending to Europe: attendees through 15 years of DrupalCon in Europe

Amsterdam 2019 is the 15th DrupalCon held in Europe. What about extending the analysis above to cover the entire history of DrupalCon in Europe?

Here is the entire flow from 2005 to 2019. The representation is a bit overwhelming due to the very high number of combinations, but hovering or tapping will help you see what goes where.

You can explore your path through the European DrupalCons and check how many people made your same choice at each step. For example, 58 community members decided, like me, to go from Barcelona 2007 to Paris 2009 (skipping 2008) and 534 community members decided to go from Paris 2009 to Copenhagen 2010.

Can we assess the community growth associated to each conference?

Left: how many new people joined at each conference (first DrupalCon presence); right: how many people left after each conference (last DrupalCon presence). So, e.g., after the recent Barcelona, Dublin and Vienna we lost more people than the conference attracted; while after London, Prague, Paris and Munich the members we gained with the event were more than those we lost after the event. Again, it is likely that the Amsterdam 2005 figures are altered by cheaters as discussed in the final paragraph.

And here’s the overall ranking as far as European DrupalCons are concerned.

DrupalCons attended in Europe (as of 2019)	How many people	Who
1 DrupalCon	2903 people	Many!
2 DrupalCons	1107 people	Many!
3 DrupalCons	647 people	Many!
4 DrupalCons	417 people	Many!
5 DrupalCons	326 people	Many!
6 DrupalCons	228 people	Many!
7 DrupalCons	113 people	Many!
8 DrupalCons	77 people	Many!
9 DrupalCons	38 people	Helpermedia, Ivo.Radulovski, Jeremy, Jo Wouters, John Morahan, Jose Reyero, Nick_vh, PieterDC, SqyD, StryKaizer, aburrows, askibinski, bircher, chipway, chx, cocomore, demeester_roel, douggreen, e.huijs, emmajane, gaele, h2cm, idevit, kgtzs, kostask, lslinnet, meichr, ohthehugemanatee, pdjohnson, pixelite, pmarciano, rachel_norfolk, ratbagash, reglogge, skwashd, steveparks, valthebald, vesapalmu
10 DrupalCons	24 people	BarisW, Crell, MatthewS, MickeA, R.Hendel, Wim Leers, ademarco, alexpott, dasjo, dokumori, gdemet, hagen, ifrik, jfhovinne, kvantomme, mikl, mkalkbrenner, nicholasThompson, pescetti, pwolanin, rszrama, stBorchert, swentel, yngvewb
11 DrupalCons	12 people	Pol, Wimmmmm, add1sun, agentrickard, alanburke, aschiwi, beltofte, betz, claudiu.cristea, klausi, michaelemeyers, mortendk
12 DrupalCons	9 people	Robert Castelo, ekes, fago, fgm, horncologne, populist, sanduhrs, stella, webchick
13 DrupalCons	5 people	Gábor Hojtsy, Heine, Sutharsan, criz, moshe weitzman
14 DrupalCons	0 people
15 DrupalCons	2 people	bertboerland, drumm

In red Nuvole team members, just for a bit of company pride.

The fine print

Only DrupalCons held in Europe are considered. Drupal Europe 2018 is considered to be the European DrupalCon for the year 2018.
Data were obtained using the little-known but very handy Drupal.org API as explained in a session, more focused on community demographics, we gave at Drupal Europe 2018.
Like most d.o profile data, attendance to Drupal events is manually entered by each user. There are obvious cases of missing information (Dries being a notable example) and of cheaters (many very recent accounts claim an unlikely number of attended events).
Data were algorithmically sanitized to remove suspicious entries (e.g., accounts created last month that self-proclaimed presence at all DrupalCons ever held) as much as possible. This was done for the tables but not for the charts. If you are affected, please point out any errors in the comments.
If you are in Amsterdam now but forgot to tick the checkbox - don’t worry, we had forgotten about it too. Go edit your profile (“Drupal” tab) and we’ll update this post with fresh numbers this weekend.

CMI 2 at DrupalCon Europe 2020

Mon, 21 Dec 2020 07:30:42 GMT

Last week was the virtual DrupalCon Europe and I had the pleasure to present CMI 2 updates together with Moshe Weitzman. Attached to this post are the slides.

It was a strange DrupalCon, no travelling and home made food and good coffee are of course a nice change but I am not sure it makes up for not meeting Drupal friends in person. I enjoyed the hallway track and visited the “virtual booth” of sponsors and it was a nice way to randomly meet people like one would at a in-person event. But presenting a presentation to just the screen without seeing the audience was a new experience.

I presented the CMI updates under the overarching goal of standardising the way configuration management is approached in Drupal. After the big API addition last year we focused on updating contrib to use this new API. A particular highlight is the testing framework for modules upgrading from config filter to the new core API.

Another highlight is of course the new drush deploy command which Moshe presented. It executes the steps to update a drupal site where new code has been pulled. In particular it executes a database update, config import and running of deploy hooks.

Deploy hooks are the same as post_update hooks, but they are meant to be run after the configuration is imported. But instead of putting them in your mymodule.post_update.php you put the code in mymodule.deploy.php. They are not meant for contrib modules, but rather for custom code that a site relies on. For example it allows you to add content to fields that you created via config. More information can be found in the deploy hook documentation

Lastly I attended a very interesting BOF around distributions. I did not initiate it but I was very happy to observe the exchange of ideas and approaches between maintainers of different distributions. I hope more collaboration will happen next year in this space and hopefully more distributions join forces to find a way core can assist the development of distributions.

CMI 2 at DrupalCon Europe 2020.pdf

Changes coming to Config Split

Wed, 04 Aug 2021 08:11:58 GMT

Config Split has worked mostly the same way as it did when the first beta release was tagged in 2016. The 8.x-1.x branch is using Config Filter as the API to hook into the configuration management of Drupal. But since Drupal 8.8 we have a much better API in Drupal core which allows us to do more complex things with simpler code than the Config Filter API did. The core API was designed with the lessons learned from Config Filter in mind after all.

Config Split 2.0.0-beta2 already uses the core API, but it still does the same thing as Config Split 1.x. I think now would be a great time to change things up a bit and do the things that were too hard to do before.

But first let’s revisit what Config Split does.

On export these two things happen:
Complete Split: Modules and config and all the config that depends on it is split off into the split storage and removed from the sync storage when the export is complete.
Conditional Split: The config is compared against the sync storage and is split to the split storage if it differs, the original config in the sync storage remains.

On import the config from the split storage is merged before Drupal imports the config from the sync storage.

Now what do I propose 2.x should do?

On export these very similar but different things happen:
Complete Split: Modules and config are removed from the sync storage as if it had been uninstalled. The difference of how other config (which no longer depends on the modules) changes is kept as a quasi-config-patch* together with the config of the split modules in the split storage.
Partial Split: Config is compared against the sync storage but instead of splitting the whole config to the split storage only the difference is kept in the split storage as a quasi-config-patch*.

On import the config from the split storage is merged and the quasi-patches applied before Drupal imports the config from the sync storage.

What now?

I think these changes make sense and make Config Split a better module. But the behavior is quite different, so if you have gotten accustomed to the stability of Config Split and depend on some specific way it works then this could be upsetting. And right now there is still a way to have the cake and eat it too: We keep 2.x the way it is now (in beta2) and maintain the behavior and only add “non-breaking” changes and do what I proposed in 3.x. Here is the catch though: I am not interested in supporting two branches, especially if I don’t know that it is useful to anyone.

Please let me know what you think about this in a comment on the issue on drupal.org.

* What is a “quasi-patch” or a config patch?

The word quasi-patch was already introduced to the drupal world #3055872. But what I have in mind is a bit different. I think it will be easiest to explain with an example:
Imagine you install the demo Umami profile and then uninstall the Menu UI module. The recipe node type will be updated to have the third party configuration removed. A patch for the exported config file would look like this:

diff --git a/node.type.recipe.yml b/node.type.recipe.yml
index 974c2ce..8d0de1f 100644
--- a/node.type.recipe.yml
+++ b/node.type.recipe.yml
@@ -1,13 +1,6 @@
 uuid: 40b74b76-6f31-4999-87bb-9700b85d43a6
 langcode: en
 status: true
-dependencies:
-  module:
-    - menu_ui
-third_party_settings:
-  menu_ui:
-    available_menus: {  }
-    parent: ''
 _core:
   default_config_hash: '-3P1yuM15pcimXHBBhevjW-4aqojpmZbUgAqlVIeweo'
 name: Recipe

The quasi-patch or config patch would be a file called config_split.partial.node.type.recipe.yml with the content:

added: {  }
removed:
  dependencies:
    module:
      - menu_ui
  third_party_settings:
    menu_ui:
      available_menus: {  }
      parent: ''

Disclaimer: The details about this are subject to change. There are likely some compromises we need to make.

Config Split at DrupalCon Europe 2021

Tue, 05 Oct 2021 13:25:50 GMT

This morning was the session on Config Split 2.x. It introduced the new features of Config Split 2.x and why they are relevant for the planned new core module.

Attached are the slides of the presentation

Drupal for European Universities

Fri, 08 Oct 2021 19:23:38 GMT

Thank you for a great DrupalCon! Here are a few key findings and PDF slides from our Thursday talk, Drupal for European Universities: Data-based perspectives.

Drupal is not leading (and not even the runner-up)

Wordpress is the most popular CMS for European Universities (restricted to EEA Countries; data retrieved last week from all the the 1830 HEIs from those 30 countries). Surprise, Drupal is not the runner-up either: Typo3 beats it by a few installations.

But many countries still love Drupal

Wordpress (green in this map) leads in 15 countries, Drupal (blue) in 12, Typo3 in 2 (Germany, Austria), Joomla in Slovakia. This is a good sign for the expansion potential: Typo3’s success is based on a couple countries only. The most Drupalized countries are Estonia (44% of HEIs sites are Drupal sites), Iceland (43%), Finland (34%), Italy (33%), Belgium (33%).

European Universities like Drupal 7

Drupal 7 (red in this map) is still the most common Drupal version for University sites in 16 countries. In 9 countries the most common version is Drupal 8 (yellow), only in 2 countries it’s Drupal 9 (green); other EEA countries are omitted due to non-significant data.

Overall, Drupal 9 sites account for only about 10% of the total, with Drupal 6 (yes, 6!) still beating Drupal 9 in four countries. 45% of the Drupal websites used by European Universities are running on a version that will be unsupported in one month.

And the future?

Drupal 9 has a lot of solid selling points for Universities, see the slides below for a summary and a case study. But it needs to be marketed well to people who are still on Drupal 7, and still facing problems the Drupal community solved years ago.

It will be interesting to re-run the analysis in one month, perhaps in an extended version including the non-EEA countries that were left out of this initial survey, like the UK and Switzerland. Stay tuned!

Drupal-for-Universities-DrupalCon-Europe-2021.pdf

PSA: Config Split and Drupal 9.3

Tue, 14 Dec 2021 09:38:38 GMT

Drupal 9.3.0 was just released! It contains a lot of cool new things we are excited about but for users of Config Split there is one change that will impact on what is split off by Config Split.

For those unfamiliar with it, Config Split is a Drupal module developed by Nuvole, which splits the configuration when it is exported and merges it back together when it is imported. It is useful in many scenarios but best tailored to having some configuration only in some environments (for example the Devel module only in development environments).

The important change in Drupal 9.3+ is that permissions must exist. This is enforced with configuration dependencies which roles can now have.

Config Split 1.x will split off all the config which depends on the module you split off. Which means that if a role has a permission from the module you are splitting off, it will then in turn also be split. This is how Config Split 1.x works and it is not considered a bug. However, most likely this is not what you wanted, so what is the solution?

Use Config Split 2.x! The new version is currently still in beta but I aim to tag a new release soon. In particular Config Split 2.x splits off modules as if they had been uninstalled. Which means that instead of splitting the role completely, it will just remove the permissions and save them in a patch for the role.

There are still a few open issues though and probably more testing is needed too. One of the issues I would like to merge before 2.0.0 is #3238855. So if you have some moment to give it a try and mark it as RTBC then that would be great! Also of course don’t hesitate to report new bugs too or contribute with patches/MRs. And if coding or testing is not your strength then we also need still a lot of documentation.

I hope to tag a stable 2.0.0 release for Config Split soon and then Config Split 1.x will become unsupported when Drupal 9.2 becomes unsupported.

Config Role Split (one of the obscure modules in the long tail of config_* modules) has an even deeper existential crisis and may become unsupported altogether unless someone wants to become a maintainer. Probably the main use case for Config Role Split is now automatically addressed with Config Split 2. since roles have dependencies. But if you rely on some other feature you should reach out and start maintaining it.

New stable releases for config filter split and ignore

Tue, 07 Nov 2023 15:12:43 GMT

The last two weeks the dust settled after an energetic and productive Drupalcon. Now there is a new stable release for all of the three most popular contrib modules Nuvole maintains: Config Filter, Config Split and Config Ignore.

Config Filter

This is the most stable module, its new releases just switch over to Gitlab CI and fix a small inconsistency in how config storages are expected to behave. The first iteration introduced a bug which the second release fixed. Thank you to the early adopters who spotted the bug. Config Filter will remain supported but its relevance is probably going to decrease with the release of stable versions of the modules which used to depend on it but no longer do. For example Config Split 2.x and Config Ignore 3.x do not depend on Config FIlter and both now have a stable release. Modules are encouraged to switch to the Config Storage Transformation API added to Drupal 8.8. Both branches remain supported and recommended on drupal.org since most commits can be cherry picked between the branches. But for performance I would recommend the original 1.x branch. Both branches have exactly the same API, just the behaviour is different when importing or exporting. Other config modules can use either branch for the test traits which facilitate writing tests that pass before and after refactoring from Config Filters API to the core API.

If your site indirectly depends on Config Filter because you use a module which depends on it, you need to explicitly require Config Filter when you upgrade the modules. Drupal can not uninstall a module that is no longer in the codebase. So explicitly require Config Filter, uninstall it and then in a subsequent deployment remove Config Filter.

Config Split

Our oldest Drupal 8 config module has a new stable 2.0.0 release. In it, bugs and edge cases discovered in last year’s release were fixed. It also contains a “new” feature which brings back the functionality of the 1.x branch. This should help sites holding back on upgrading because the functionality changed. With that out of the way the plan is to deprecate the 1.x branch and end support with the end of Drupal 10 support. There will not be any features added to 1.x.

Config Ignore

Previous releases of the 3.x branch have not been feature compatible with 2.x. But the stable 3.0 release has been re-written from previous 3.x beta versions. It is configurable so that most use cases can be catered for with enough creativity. One can configure the configuration to ignore for create, update and delete for both import and export. But one can also just keep it simple and then it will be as all the versions before. In particular, however, it can be configured easily to behave like: the last 2.x release, the last 2.x release with the popular patch to allow filtering on export and the previous 3.x release. Because of that the 2.x branch is deprecated as of now and it will be marked as unsupported by the end of the year on Drupal.org. There is a new hook replacing the one which existed in 2.x and which addresses the new capabilities (the old hook is still invoked and will be removed in 4.0.0). With the end of Drupal 9 being supported, PHP 8.1 is now the minimum required version, but 3.0 does not yet take advantage of the new PHP language features. So the plan for 4.0.0 is to switch the string constants to enums and switch to semantic versioning and remove the old hook.

How the pizza is made

Our Drupal 8 modules have always been maintained “like a php library”. The development of UI Patterns was initially hosted on Github among other things for that reason. Config Split and Config Filter shipped with their docker compose files and scripts to symlink the module into the Drupal site. Later the custom scripts were replaced by drupal-spoons. Since the now generally available drupal-gitlab-ci is inspired by spoons and DDEV is likely becoming the recommended development environment for contributing to Drupal, I switched my local environment for maintaining the contrib modules to DDEV with the ddev-drupal-contrib plugin, the spiritual successor of drupal-spoons. For contributing to Drupal core I can only recommend the ddev-drupal-core-dev plugin created by justafish during Drupalcon. I helped beta test it and it works like a charm. That said my PhpStorm configuration had to be updated a bit even though I installed the ddev plugin for it. In particular I had to add a second server mapping for Xdebug to work as described in a comment on the issue. The release notes for the contrib modules were generated with drupal-mrn.dev.

Many thanks to everyone who contributed with feedback, code, ideas or even just listened to my ramblings as I discovered untested edge cases.

More peace of mind when applying recipes or letting AI configure your site

Fri, 25 Apr 2025 18:10:23 GMT

Last week I attended the Drupal Dev Days in Leuven and many of the sessions and also conversations in the hallway and the contribution room were about Drupal CMS and AI.

The foundational engine behind Drupal CMS is the recipe system, which allows a bunch of configuration changes to be bundled and applied to your site. This is really cool, but it may do things you are not completely aware of. Unfortunately there is no way to “undo” a recipe. The same problem occurs if one is letting AI change configuration.

DDEV snapshots and git commits

The proper solution of course is to export the configuration and version it in git before starting to play with recipes, as well as periodically in between applying recipes to have more save points to go back to. Since recipes can also add default content and some configuration can not be deleted when there is content for it, it means that sometimes a previous git checkout of the configuration can not be imported. Therefore, the best solution is to create database backups and roll them back when needed. DDEV makes this very easy to do, and so that would be the first recommendation to address the problem and it will always work.

However, the ddev snapshots do not really give one a lot of information of what changed. The configuration in git is a lot better, but it requires an extra manual step to create the export and commit it to git. That may not be instinctive to most drupal users, especially not the target audience of Drupal CMS.

What if Drupal could help us with that?

It turns out that Drupal core introduced the concept of configuration checkpoints when recipes were added. The idea is that before a recipe is applied a checkpoint is set, then when the recipe application fails the configuration is rolled back to the checkpoint.

There is an issue for adding a command to revert to checkpoints but no work has even started.

I had the idea already at Drupalcon Barcelona 2024, but after discussions in Leuven I decided to implement a UI for the configuration checkpoints.

How does Config Checkpoint UI work?

So far the first version is very simple: It exposed some of the basic API to the UI. One can create new checkpoints, delete checkpoints and revert the site configuration to an older checkpoint. The reverting is done essentially the same way as the core command does: importing from the checkpoint storage. This is very similar to what Config Split does and the reason for which I thought it was a good idea to try.

The UI is pretty straight forward:

Reverting to a checkpoint is very similar to the familiar configuration import screen:

Future and Limitations

There are a few bugs in other projects to watch out for: Project browser does not set a checkpoint, and core has a bug when deleting checkpoints. Of course there may be lots of bugs in the new module itself. In particular one of the things I have ran into is that the uninstall validation is a bit broken, so some of the recipes can not be rolled back when they add fields that depend on a module added at the same time. I will have to check where exactly this bug is hiding. More advanced things could also be explored, for example merging checkpoints or deleting checkpoints without deleting everything that came before.

Implementing fast end-to-end tests on Drupal with Docker and JSON-RPC

Mon, 07 Jul 2025 12:24:16 GMT

“We’ll add the tests later”. Most of us have said this at some point in our software development career. But then “later” never arrives.

That is because implementing tests is hard. Setting up the test runners, writing the test cases, keeping them running; all of it requires extra effort. Even more effort is needed for end-to-end tests (E2E) since they touch every part of an application. However, this effort always pays off and in this post, I will share how we implemented end-to-end tests (E2E) on one of our Drupal projects so that you can save some of the effort needed to get started. We are very happy with how this approach turned out and we plan to use it on future projects too.

Meet ILO Live

ILO Live is the International Labour Organization (ILO)‘s online video streaming platform. It hosts both live streams and recordings of events organized all across the world by the ILO.

Under the hood, ILO Live is powered by Next.js and a headless Drupal CMS. Most pages on the site are statically served by Next.js.

In this post, I will take you through how we implemented E2E tests on ILO Live.

The challenges of implementing E2E tests

A common approach when writing frontend tests is to use mocked API responses. But with ILO Live, we wanted to go a step further and use the actual CMS in the tests. By using the actual CMS in the tests, in addition to verifying whether the site works as intended, we can also verify whether changes to the CMS have not caused any functionality on the site to break. But to do this, we had to solve 2 major problems.

How do we orchestrate the CMS into different testing states from the test runner?
How do we quickly start the CMS during each test run?

Orchestrating the Drupal CMS using a private JSON-RPC API

The simplest way to orchestrate a Drupal CMS is to have a test runner click on various elements to create new entities and change their state. But this takes a lot of time to run, especially when multiple entities need to be created during each test run.

To solve this problem, we decided to create a private API using the JSON-RPC module. This API would only be enabled on development and testing instances of the CMS and it exposed several operations which the test runner could use to orchestrate the CMS into different states.

For example, the JSON-RPC API exposed a create.event method for creating new events.

{
    "jsonrpc": "2.0",
    "method": "create.event",
    "params": {
        "type": "meeting",
        "title": "Aragorn meets Gandalf",
        "start_date": "2021-06-01 8:00:00",
        "end_date": "2021-06-01 17:00:00"
    },
    "id": 1
}

And a clean.content method for resetting the CMS to the initial state with no content set.

{
    "jsonrpc": "2.0",
    "method": "clean.content",
    "params": {},
    "id": 1
}

We added similar methods for creating other entities on the site and for admin tasks such as reindexing the search index. ILO Live used AWS AppSync to receive real-time updates about the current state of events so to simulate this in the tests, we set up a testing instance of AppSync and implemented methods like the update.livestream method shown below to change the state of this instance.

{
    "jsonrpc": "2.0",
    "method": "update.livestream",
    "params": {
        "event_id": "1",
        "status": "live",
    },
    "id": 1
}

Speeding up the Drupal CMS’s startup time using Docker

Now we had to get the Drupal CMS to run while the test runner was running. We were familiar with using Cypress for implementing frontend tests so we decided to use the same for implementing the E2E tests. Since the E2E tests ran assertions on elements on the site / frontend, we decided to store the E2E tests within the frontend repo and use Github Actions to run them since the frontend repo was hosted on Github.

We used GNU Make and Docker Compose during development to run the CMS and its services (i.e. Maria DB and Redis) so initially we tried to clone the CMS and run the commands to start it. Docker is preinstalled by default on the ubuntu-latest runner on GitHub Actions so we were able to easily use it there.

# Simplified Github actions workflow for running the E2E tests
name: Tests
on: push
jobs:
  cypress:
    name: Run Cypress tests
    runs-on: ubuntu-latest
    steps:
      - name: Checkout
        uses: actions/checkout@v4
      # Clone the CMS into a subfolder
      - name: Clone CMS
        uses: actions/checkout@v4
        with:
          repository: github-org/ilo-live-cms
          #legacyPath: ./cms
          # actions/checkout@v2 only has access to the current repo by default
          # A custom token needs to be provided to it for it to access a different repo
          # https://docs.github.com/en/authentication/keeping-your-account-and-data-secure/managing-your-personal-access-tokens
          token: ${{ secrets.CMS_ACCESS_TOKEN }}
          ref: develop
      - name: Start CMS
        run: |
          cd cms
          make
      # More steps to actually run the tests

# Simplified Makefile from the CMS
default: build install

up:
  @echo "Starting up containers for ilo_live..."
 docker compose up -d --remove-orphans

build: up
 @echo "Building ilo_live project development environment."
 docker compose exec -T php bash -c "composer install"

install:
 docker compose exec -T php bash -c "vendor/bin/drush si -y --existing-config --account-pass=\"admin\""
 docker compose exec -T php bash -c "vendor/bin/drush deploy"

# Simplified docker-compose.yml for the CMS and its services
services:
    mariadb:
        image: mariadb-image

    php:
        image: php-image
        volumes:
            - ./:/var/www/html

    redis:
        image: redis-image

This setup worked but it took around 2 - 4 minutes to run. Most of this was due to Composer and the Drupal site install.

This delayed each test run so we started looking into ways to improve it. The solution that we came up with was to create a self-contained Docker image for the Drupal CMS. This way, the test runner only needed to pull and run a single image to start the CMS.

To do this, we set up a Github Actions workflow on the CMS repo to build and push a development Docker image. This image had the JSON-RPC API enabled and it used a SQLite database instead of MariaDB.

# Simplied Dockerfile for building the development docker image
FROM drupal-image

# Replace the site files in the image with our own
RUN rm -rf /opt/drupal/*
COPY ./ /opt/drupal/

# Install SQLite and dependencies required by composer
RUN apt update && apt install -y sqlite3 git zip

WORKDIR /opt/drupal
RUN composer install
# Use custom settings for the development image
RUN cp docker/settings.bundle.php web/sites/default/settings.local.php
RUN vendor/bin/drush site:install -y --existing-config --account-pass=admin
RUN vendor/bin/drush deploy

// Simplified settings.bundle.php for the development image
<?php
    // Enable the config split for the JSONRPC-API for the development image
    $config['config_split.config_split.jsonrpc']['status'] = TRUE;

    // Use SQLite instead of MariaDB
    $databases['default']['default'] = array (
        'database' => 'sites/default/files/.ht.sqlite',
        'prefix' => '',
        'driver' => 'sqlite',
        'namespace' => 'Drupal\\sqlite\\Driver\\Database\\sqlite',
        'autoload' => 'core/modules/sqlite/src/Driver/Database/sqlite/',
    );

We used the Configuration Split module by our own Fabian Bircher to ensure that the JSON-RPC module is only enabled during development.

Here is the Github Actions workflow we used to build and publish the image to the Github Container registry.

# Simplified Github actions workflow for building and publishing the development docker image
name: Build and push development docker image
on:
  push:
    # Usually the E2E tests can use the image built from the develop branch of the repo
    # But to test upcoming changes, we can prefix a branch with docker- to have this workflow build an image for it
    branches: ['develop', 'docker-*']
jobs:
  build-and-push-image:
    runs-on: ubuntu-latest
    # Grant the default `GITHUB_TOKEN` permission to read the current repo and push images to the Github container registry
    permissions:
      contents: read
      packages: write
    steps:
      - name: Log in to the Container registry
        uses: docker/login-action@v3
        with:
          registry: ghcr.io
          username: ${{ github.actor }}
          password: ${{ secrets.GITHUB_TOKEN }}
      # This action extracts the tags and labels that should be set on the published image
      # It exposes them as an output which is consumed by the next step through steps.meta
      - name: Extract metadata (tags, labels) for Docker
        id: meta
        uses: docker/metadata-action@v5
        with:
          images: ghcr.io/github-org/ilo-live-cms
      # This action uses the git repo as the build context by default
      - name: Build and push Docker image
        uses: docker/build-push-action@v6
        with:
          push: true
          file: docker/Dockerfile
          tags: ${{ steps.meta.outputs.tags }}
          labels: ${{ steps.meta.outputs.labels }}

This is the updated workflow on the frontend repo for using the development image.

# Simplified Github actions workflow for running the E2E tests using the development docker image
name: Tests
on: push
jobs:
  cypress:
    name: Run Cypress tests
    runs-on: ubuntu-latest
    steps:
      - name: Checkout
        uses: actions/checkout@v4
      - name: Log in to the Container registry
        uses: docker/login-action@v3
        with:
          registry: ghcr.io
          username: ${{ github.actor }}
          password: ${{ secrets.ACCESS_TOKEN }}
      - name: Run CMS container
        run: |
          docker run \
          --detach \
          -p 80:80 \
          ghcr.io/github-org/ilo-live-cms:develop

With this setup, the development image is built within 5 minutes each time a pull request is merged into the develop branch of the CMS and the E2E tests workflow can start the CMS in less than a minute.

An added benefit of this approach is that it was much simpler to run the CMS. Previously, the only way to run the CMS was to clone the CMS repo and set up the project locally alongside the database server. Now it just needed a single command.

docker run --detach -p 80:80 ghcr.io/github-org/ilo-live-cms:develop

Putting everything together

To use the JSON-RPC API more conveniently within the Cypress tests, we defined several Custom Commands.

const getRPCPayload = (method, params = {}) => {
  // Get basic authorization header.
  const authorization = Buffer.from(
    [Cypress.env("CMS_USER"), Cypress.env("CMS_PASSWORD")].join(":")
  ).toString("base64");

  return {
    method: "POST",
    url: `${Cypress.env("CMS_URL")}/path/to/jsonrpc`,
    headers: {
      Accept: "application/json",
      Authorization: `Basic ${authorization}`,
    },
    body: {
      jsonrpc: "2.0",
      method,
      params,
      id: Math.floor(Math.random() *1000),
    },
  };
};

Cypress.Commands.add("rpc", (method, params = {}) => {
  cy.request(getRPCPayload(method, params)).then((response) => {
    // Run assertions on the response from the CMS to ensure that the call ran successfully
    expect(response.status).to.eq(200);
    expect(response.body).to.have.property("result");
    return response.body.result;
  });
});

// Cypress commands can call other commands and build on top of each other
// So we created several utility functions to reduce repeated logic
Cypress.Commands.add(
  "createPastEvent",
  ({
    id = 1,
    type = "meeting",
    startDate,
    startHour = 8
  } = {}) => {
    const event = {
      type,
      title: `Past ${type} ${id}`,
      // Not the most robust logic but sufficient for testing
      start_date: `2023-01-${startDate || id} ${startHour}:00:00`,
      end_date: `2023-01-${startDate || id} ${startHour + 1}:00:00`,
    };
    cy.rpc("create.event", event);
  }
);

// Command that should be run before each test run
Cypress.Commands.add("setup", () => {
  // Cypress runs each test in isolation so that they can't interfere with each other
  // To enforce that isolation across the entire system, we use this command to reset CMS to its initial state  
  cy.rpc("clean.content");

  // Enable preview mode on Next.js to make it regenerate pages during each request
  // https://nextjs.org/docs/pages/guides/preview-mode
  cy.request("/path/to/enable-preview");
});

These custom commands made it much easier to orchestrate the CMS as required.

describe("Event page", () => {
  beforeEach(() => {
    cy.setup();
  });

  it("should display past meetings correctly", () => {
    // Call the JSONRPC-API to setup the content
    cy.createPastEvent().then((res) => {
      // Visit the page corresponding to the newly created content
      cy.visit(res.path);

      // Check if the elements on the page are rendered correctly
      cy.get("h1").should("contain", "Past meeting 1");
      cy.contains("1 January 2023");
      cy.contains("08:00 - 09:00 GMT+1");
    });
  });
});

Final results

Ultimately we implemented over 50 end-to-end tests for this project. The final test suite takes between 7 - 8 minutes to run on a standard Github hosted runner. (i.e. Not a large runner) This runtime is acceptable for us for now but it can be improved even further by using Parallelization.

While these tests took us some effort to implement, they definitely paid off in the long term. ILO Live had live event functionality which was mission-critical but very hard to orchestrate manually and with the E2E tests, we were able to ensure that it always worked. These E2E tests gave us the confidence to make big improvements to the codebase since we knew that bugs in key functionality would be instantly revealed and this is the biggest benefit of them all.

Written by Prabashwara Seneviratne, frontend developer at Nuvole and author of Frontend undefined.

Drupal Dev Days Athens

Tue, 05 May 2026 11:00:00 GMT

Last week was the annual Drupal Developer Days, and as always it was a fantastic event.

Config Ignore 3.4

Thanks to having some time away from regular projects, I released Config Ignore 3.4. The biggest change is that ignoring the settings for Config Ignore now works. By default, Config Ignore now has this bug fixed, but you can configure which of its settings to use. The README contains more information on the available options, but I would recommend leaving it at the default. The other options allow configuring the behaviour of previous 3.x and even 2.x releases, but they come with their own drawbacks.

Refactor to `#[Hook]`

In the last session of the first day of the conference, I presented Refactor to #[Hook]. While at Nuvole we have been using the PHP attribute-based approach for hooks since 2022 thanks to the Hux module, this style of hooks landed in core only a little over a year ago.

The talk went briefly over what a hook is and what a PHP attribute is, but it also explained how simple it is to refactor an existing codebase to the new style. Despite (or maybe because of) rehearsing the demo twice, there was a hiccup when showing how to refactor the code on stage. Unfortunately, I didn’t think of the most obvious solution of restarting DDEV, which would have solved the problem.

The demo would have concluded by showing the form still working, but I hope participants still took away how accessible the steps are to use Rector to refactor the code.

Attached are the slides with the links to the documentation.

Drupal Global Training Day in Italy

Wed, 12 Sep 2012 10:08:08 GMT

After the successful experience from last June, Nuvole is proud to join again the Drupal Global Training Day initiative on September 14th.

This time we will hold the event in Parma, Italy, and, following the initiative guidelines, we will give a free generic introduction to Drupal, focusing on Drupal as a platform, its community, what can be easily done with the available modules and what can be reached with a bit (or a lot!) of customization.

Since there’s nothing better than real use cases to show what Drupal can do, we will also show and discuss in detail some recent Nuvole projects. If you can’t make it to Parma, here are the fact sheets, with screenshots and short texts in Italian, we prepared to describe our most interesting projects:

The event is free to attend, and attendees are also welcome to stay for a buffet lunch. Registration is required. Please see the registration page for all details.

factsheet-1-alfapuentes.pdf (1.98 MB) download

factsheet-2-bolognaexperts.pdf (1.75 MB) download

factsheet-3-esncard.pdf (1.96 MB) download

factsheet-4-orgalime.pdf (3 MB) download

Open Atrium: a one-stop solution for international cooperation projects

Fri, 27 Jul 2012 14:06:35 GMT

At Nuvole we have always supported the idea that Open Atrium can deal with complex use cases. Modules like Spaces, PURL and Organic groups can push the limit of the platform far beyond being a simple-yet-powerful intranet software. We were already experimenting with building public websites and simple distributions-like mini-sites with Open Atrium for quite some time, but now the new Alfa Puentes project gave us the opportunity to blend together all those customizations in one powerful platform.

These concepts will be covered in more detail in the Nuvole DrupalCon Munich Training.

The Alfa Puentes project

The Alfa Puentes project’s main aim is to enhance the international cooperation between the European and the Latino-American Higher Education environments by creating a community of teachers, rectors and other related stakeholders. The members of this community need to share data and information both online, using an intranet platform, and offline, by participating to a series of mid-size seminars and conference spread across the two interested regions. Given the diversity of this community, multi-language support was also a strong requirement.

Open Atrium to the rescue

The platform is organized in three main parts: a public portal, a series of private self-managed intranet groups and mini-sites for the organization of real-life events. Since it’s developed on a single Open Atrium installation users of the platform can have different roles in each components, e.g., a rector can start a private discussion group while a project partner may help with translations.

Public portal

The portal is based on our mini-site features set, it contains information about the project, like funding periods and partners involved. All content is available in three languages: English, Spanish and Portuguese. Site managers can easily assign content translation to partners or collaboratively work on a specific section of the website. Even though the portal is implemented as a specific Open Atrium group type, the site look and feel does not resemble in any way its default theme: visitors perceive the portal as being an independent site.

All portal content is available in three languages:

Even though it is built as a group the portal does contain highly customized sections:

Private intranet groups

In a cooperation project it is essential to provide a platform where users can easily engage in a conversation and share relevant information: nothing can be simpler with Open Atrium. Each partner can start a private discussion group and can easily invite users to participate. To facilitate information sharing each group can be powered with our Atrium Folders feature, which provides a more user friendly and familiar way of sharing documents and files than the built-in Notebook feature. The language of each group can be set at its creation, chosen among English, Spanish and Portuguese.

Group owners can invite users to join a group by using powerful search tools:

Atrium Folders enhances the document sharing user experience:

Mini-sites for event organization

The mini-site features set also powers the creation and management of mini-sites for organization of events. Each event site is an independent Open Atrium public group, with a specific Spaces preset which, at every site creation, enables features like static pages, custom navigation menu pre-filled with default content, news section and a spotlight area on the front page. Each mini-site must also provide a customizable per-event registration form which will add specific information (like participation to dinners, workshops, etc…) to the usual user profile data. To implement such a functionality we have integrated the Webform module with Open Atrium, ensuring a smooth user experience in managing such a complex use case.

Managers can deploy and customize a fully fledged event website in minutes:

Users can integrate their profile with event-specific information:

Glue all together: the user dashboard

The default Open Atrium user profile section has been customized to be the starting point for the user to find her way trough the system. After the login the user is redirected to her profile page, from where she can have direct access to update her personal information, get to the groups she is member of and manage her event registrations.

Nuvole

Measure node completion in Drupal, part 2

A list view on node completion

Total completion of a list of nodes

What’s next

Measure node completion in Drupal, part 3

Workflows on data completion using Rules

Future of Content Complete: a Complete API

Towards a 100% database free development workflow

Settings in code: the key to Drupal heaven

Introducing #codepower

The Movement

First Code Driven Development meet-up in Antwerp

Code driven development session proposal at DrupalCon Copenhagen 2010

Measure node completion in Drupal, part 1

Configuring the module

Displaying completeness

Who gets to see what: configuring permissions

Theming

What’s next

Introducing OpenLayers Geocoder

Google Geocoding Web Service

Meet OpenLayers Geocoder

Get the most out of the web service response

Future development: Reverse geocoding

Drupal database backup

Before you start: common caveats

Use a lock file

Keep your database clean

What do you need the database dump for?

Use case 1: Verbatim copy of the database

Use case 2: Copying the meaningful tables from the database

Use case 3: Sharing a database with other sites or developers

Development sandbox with Git and Drush

”Divide et impera” with Git

Automation with Drush and Git hooks

Getting to the speed of light: Git bash completion

Share changes with your team using Features and upgrade paths

Make your workflow solid: meet hook_update_N()

Make your features database free: meet hook_install()

Real life examples

Add a menu and menu items

Add OpenId to admin account

Upgrading data from OpenLayers 1.x to 2.x

Features based development workflow

DrupalCon Chicago training: Use Features Effectively

Typing less in Drush: aliases and autocompletion

Drush and Bash Aliases

Drush aliases

Bash aliases for Drush

Using autocompletion

Examples

Advanced Open Atrium customization

1. How to add custom user profile fields

2. How to override Atrium’s core configuration

3. How to create custom group types

A different way to look at Open Atrium

Open Atrium, Beyond the Intranet

Drupal Solutions for Public Works Monitoring and Remote Collaboration

Public Works Monitoring in Drupal

Remote Collaboration and Institutional Intranet

Five reasons code-driven development and Features are good for you

1: Keeping track of all changes

2: Working in a distributed team

3: Reusing components and settings

4: Updating production sites cleanly

5: Following the Drupal future

Building and Maintaining Drupal Distributions

Minisites in Open Atrium

Creating and Customizing a Minisite

Creating a minisite group type (spaces preset)

Assigning features to a minisite

Making minisites self-contained: menus and blocks

Making minisites themeable

Serving minisites on different domains

Look bigger with Features

Hard and Soft configuration in Drupal Distributions

Each component has its own name

Hard and Soft configuration

Not only taxonomy terms