Packaging and reusing configuration in Drupal 8

Antonio De Marco
15 Jul 2014
11 Comments
Antonio De Marco
15 Jul 2014
11 Comments
Antonio De Marco, 15 Jul 2014 - 11 Comments

Packaging and reusing configuration in Drupal 8

Bringing "reusable features" to Drupal 8.

This is a preview of Nuvole's training at DrupalCon Amsterdam: An Effective Development Workflow in Drupal 8.

Configuration Management in Drupal 8 elegantly solves staging configuration between different environments addressing an issue that is still haunting even the most experienced Drupal 7 developer. In earlier posts we covered the new Configuration Management in Drupal 8, seeing how it compares to Drupal 7 and Features, and even investigated how to manually simulate Features in Drupal 8 last year. Recent developments and contrib modules can take us several steps closer.

Packaging configuration

Developers familiar with code-driven development practices need an equivalent in Drupal 8 + Drush 7 to what the Features module does in Drupal 7 with its features-update and features-revert Drush commands.

While Drupal 8 configuration staging capabilities are far more advanced than what Features could possibly provide, what the new Configuration Management system really lacks is the ability to package configuration.

Enter the Configuration development module

The Configuration development module, currently maintained by chx, serves two main purposes:

  • It automates the import of specified configuration files into the active storage.
  • It automates the export of specified configuration objects into files.

The module offers a simple, global UI interface where a Drupal developer can set which configuration is automatically exported and imported any time they hit the “Save” button on a configuration setting page.

In order to achieve a more modular configuration packaging it would be enough to set a specific module’s config/install directory as the actual export destination.

Nuvole contributed a patch (EDIT: now integrated) to make that possible: instead of firing an auto-export every time a “Save” button is clicked the developer can, instead, specify in the module’s info file which configuration needs to be written back to that module’s install directory and run a simple Drush command to do that.

Reusable “features” in Drupal 8

One of the main advantages of having a standardized way of dealing with configuration means that modules can now stage configuration at installation time. In a way that’s something very close to what Features allowed us to do in Drupal 7.

Say we have our news section up and running on the site we are currently working on and we would like to package it into a custom module, together with some other custom code, and ship it over a new project. The patched Config development module will help us to do just that! Here it is how:

Step 1: Download, patch and enable Configuration development module

We need to download and enable the Configuration development module and apply the patch (EDIT: already integrated as of 8.x-1.0-alpha7) attached to this Drupal.org issue.

After rebuilding the cache, we will have the config-writeback Drush command available. Let's have a closer look at what it is meant to do:

$ drush help config-writeback

Write back configuration to a module's config/install directory. State which configuration settings you want to export in the module's info file by listing them under 'config_devel', as shown below:

config_devel:
  - entity.view_display.node.article.default
  - entity.view_display.node.article.teaser
  - field.instance.node.article.body


Examples:
drush config-writeback MODULE_NAME        Write back configuration to the specified module, based on .info file.

Arguments:
module                                    Module machine name.

Aliases: cwb

Step 2: Find what configuration needs to be packaged

We now look for all configuration related to our site’s news section. In Drupal 8 most of the site configuration is namespaced with related components so, if we keep on using consistent naming conventions, we can easily list all news-related configuration by simply running:

$ drush config-list | grep news

entity.form_display.node.news.default
entity.view_display.node.news.default
entity.view_display.node.news.teaser
field.instance.node.news.body
image.style.news_medium
menu.entity.node.news
node.type.news

Step 3: Package configuration

To package all the settings above we will create a module called custom_news and, in its info file, we will specify all the settings we want to export, listing them under the config_devel: directive, as follows:

$ cat modules/custom_news/custom_news.info.yml

name: Custom News
type: module
description: 'Custom news module.'
package: Custom
core: 8.x
config_devel:
  - entity.form_display.node.news.default
  - entity.view_display.node.news.default
  - entity.view_display.node.news.teaser
  - field.instance.node.news.body
  - image.style.news_medium
  - menu.entity.node.news
  - node.type.news

After enabling the module we will run:

$ drush config-writeback custom_news

And we will have all our settings exported into the module’s install directory:

$ tree -L 3 modules/custom_news/

modules/custom_news/
├── config
│   └── install
│       ├── entity.view_display.node.news.default.yml
│       ├── entity.view_display.node.news.teaser.yml
│       ├── field.instance.node.news.body.yml
│       ├── image.style.news_medium.yml
│       ├── menu.entity.node.news.yml
│       └── node.type.news.yml
└── custom_news.info.yml

The Drush command above takes care of clearing all sensitive UUID values making sure that the module will stage the exported configuration cleanly, once enabled on a new Drupal 8 site.

To get the news section on another site we will just copy the module to the new site's ./modules/ directory and enable it:

$ drush en custom_news

The following extensions will be enabled: custom_news
Do you really want to continue? (y/n): y
custom_news was enabled successfully.     

Final evaluation: Drupal 7 versus Drupal 8

One of the main differences between working in Drupal 7 and in Drupal 8 is represented by the new Configuration Management system.

While Features was proposing a one-stop solution for both configuration staging and packaging, Drupal 8 CM does a better job in keeping them separate, allowing developers in taking a greater control over these two different and, at the same time, complementary aspect of a solid Drupal development workflow.

By using the method described above we can upgrade our comparison table between Drupal 7 and Drupal 8 introduced in one of our previous posts as follows:

Functionality D7 Core D7 Core + Features D8 Core (current) D8 Core (current) + Config Devel
Export full site config (no content) NO NO YES YES
Export selected config items NO YES YES YES
Track config changes (full site) NO NO YES YES
Track config changes (selected items) NO YES YES YES
Stage configuration NO YES YES YES
Package configuration NO YES NO YES
Reuse configuration in other projects NO YES NO YES
Collaborate on the same project NO YES NO NO

The last "NO" deserves a brief explanation: Configuration Management allows two developers to work simultaneously on different parts of the same project if they are very careful: but "merging" the work would have to be done by version control (GIT or similar), that doesn't know about YAML or Drupal.

Some open issues

Contributed modules seem to be the best way to enhance the core Configuration Management system, much like what happened with Drupal 7 and Features. There are still several issues that should be considered for an optimal workflow, to match and improve what we already have in Drupal 7:

  • Piping: the ability to relate configuration components based on both hard and logic dependencies, for example: I export a content type and, automatically, I get also its fields. If piping might have been too rigid, at times, it would be still useful to have in some configurable form.
  • Enhanced configuration diff: it might be useful to have the possibility to review what configuration is going to be installed before enabling a module, like it is now when importing staged configuration to the active storage.
  • Granularity: it is still impossible to export part of a configuration file, so we still depend on the core conventions for grouping configuration into files, and we can't export a single permission for example.
  • Ownership: we can't know if another module (or "feature") is tracking a component we wish to track; this could be useful in the perspective of maintaining several "modular" features.
  • Updates: we can reuse configuration by simply enabling a module, but this holds only for the initial installation; after a module is enabled, we don't have a clean way to import changes (say, to "upgrade" to a newer version of the feature) outside the standard workflow foreseen in Configuration Management.

Comments

Comments

Iztok
7 Aug 2014

"The last "NO" deserves a brief explanation: Configuration Management allows two developers to work simultaneously on different parts of the same project if they are very careful: but "merging" the work would have to be done by version control (GIT or similar), that doesn't know about YAML or Drupal."

Can you please explain this even in more details? For me it sounds like its going to be super complicated (and it was not easy with Features in D7).

"The last "NO" deserves a brief explanation: Configuration Management allows two developers to work simultaneously on different parts of the same project if they are very careful: but "merging" the work would have to be done by version control (GIT or similar), that doesn't know about YAML or Drupal."

Can you please explain this even in more details? For me it sounds like its going to be super complicated (and it was not easy with Features in D7).

Iztok, 7 Aug 2014

"The last "NO" deserves a brief explanation: Configuration Management allows two developers to work simultaneously on different parts of the same project if they are very careful: but "merging" the work would have to be done by version control (GIT or similar), that doesn't know about YAML or Drupal."

Can you please explain this even in more details? For me it sounds like its going to be super complicated (and it was not easy with Features in D7).

Iztok, 7 Aug 2014
Andrea Pescetti
8 Aug 2014

We are going to explain this in detail in a dedicated post next week or so... Stay tuned!

We are going to explain this in detail in a dedicated post next week or so... Stay tuned!

Andrea Pescetti, 8 Aug 2014

We are going to explain this in detail in a dedicated post next week or so... Stay tuned!

Andrea Pescetti, 8 Aug 2014
StryKaizer
21 Aug 2014

If I'm not mistaken, the D7 core + Features way of collaborating also depends on GIT merge, when working on the same components, no?

If I'm not mistaken, the D7 core + Features way of collaborating also depends on GIT merge, when working on the same components, no?

StryKaizer, 21 Aug 2014

If I'm not mistaken, the D7 core + Features way of collaborating also depends on GIT merge, when working on the same components, no?

StryKaizer, 21 Aug 2014
Andrea Pescetti
21 Aug 2014

Yes, but a key difference is that with D7 + Features it is much easier to separate features; so in a well-planned development one can avoid that two developers work simultaneously on the same feature (they will work simultaneously, but on different features, and there won't be the risk of conflicts).

In the D8 case, we now have a follow-up post: http://nuvole.org/blog/2014/aug/20/git-workflow-managing-drupal-8-config... where you can see that in D8, even if two developers are not working on the same component, merges can be problematic and result in broken configuration; so in this sense it is quite tricky to rely on GIT for merges in D8.

Yes, but a key difference is that with D7 + Features it is much easier to separate features; so in a well-planned development one can avoid that two developers work simultaneously on the same feature (they will work simultaneously, but on different features, and there won't be the risk of conflicts).

In the D8 case, we now have a follow-up post: http://nuvole.org/blog/2014/aug/20/git-workflow-managing-drupal-8-config... where you can see that in D8, even if two developers are not working on the same component, merges can be problematic and result in broken configuration; so in this sense it is quite tricky to rely on GIT for merges in D8.

Andrea Pescetti, 21 Aug 2014

Yes, but a key difference is that with D7 + Features it is much easier to separate features; so in a well-planned development one can avoid that two developers work simultaneously on the same feature (they will work simultaneously, but on different features, and there won't be the risk of conflicts).

In the D8 case, we now have a follow-up post: http://nuvole.org/blog/2014/aug/20/git-workflow-managing-drupal-8-config... where you can see that in D8, even if two developers are not working on the same component, merges can be problematic and result in broken configuration; so in this sense it is quite tricky to rely on GIT for merges in D8.

Andrea Pescetti, 21 Aug 2014
Jay Callicott
25 Dec 2014

Using Drupal 8 beta 4 + whatever the latest config_devel update I was able to do a POC using the steps you provided to create a Drupal 8 "feature". This is very exciting!

Using Drupal 8 beta 4 + whatever the latest config_devel update I was able to do a POC using the steps you provided to create a Drupal 8 "feature". This is very exciting!

Jay Callicott, 25 Dec 2014

Using Drupal 8 beta 4 + whatever the latest config_devel update I was able to do a POC using the steps you provided to create a Drupal 8 "feature". This is very exciting!

Jay Callicott, 25 Dec 2014
Raúl Cano
9 Feb 2015

Thanks for a great post.
I have started with Drupal 8 development recently and I was following this approach already (without success, as I'll explain below). Learning about Config_devel module made things much easier!

The problem I am getting with this approach is that, after putting the necessary configuration files in my module's config/install folder, if I enable the module with drush:
drush en my_module -y
I get an exception:
Drupal\Core\Config\PreExistingConfigException with message 'Configuration objects (...) provided by my_module already exist in active configuration in ....

and hence I cannot import the config from my module. Did anybody face this problem?
Thanks in advance!

Thanks for a great post.
I have started with Drupal 8 development recently and I was following this approach already (without success, as I'll explain below). Learning about Config_devel module made things much easier!

The problem I am getting with this approach is that, after putting the necessary configuration files in my module's config/install folder, if I enable the module with drush:
drush en my_module -y
I get an exception:
Drupal\Core\Config\PreExistingConfigException with message 'Configuration objects (...) provided by my_module already exist in active configuration in ....

and hence I cannot import the config from my module. Did anybody face this problem?
Thanks in advance!

Raúl Cano, 9 Feb 2015

Thanks for a great post.
I have started with Drupal 8 development recently and I was following this approach already (without success, as I'll explain below). Learning about Config_devel module made things much easier!

The problem I am getting with this approach is that, after putting the necessary configuration files in my module's config/install folder, if I enable the module with drush:
drush en my_module -y
I get an exception:
Drupal\Core\Config\PreExistingConfigException with message 'Configuration objects (...) provided by my_module already exist in active configuration in ....

and hence I cannot import the config from my module. Did anybody face this problem?
Thanks in advance!

Raúl Cano, 9 Feb 2015
Raúl Cano
9 Feb 2015

The situation is as follows:
1.- Enable my_module, which adds field_X and vocabulary_Y
--EVERYTHING OK--
2.- Uninstall my_module, which **does not delete** the field_X and vocabulary_Y
--EVERYTHING OK--
3.- Work on my_module to add some changes to field_X and vocabulary_Y
...
4.- Try to enable my_module, which should update field_X and vocabulary_Y with new values
--ERROR: "Configuration objects (...) provided by my_module already exist..."--

After reading the Open Issues section, more precisely:

"Updates: we can reuse configuration by simply enabling a module, but this holds only for the initial installation; after a module is enabled, we don't have a clean way to import changes (say, to "upgrade" to a newer version of the feature) outside the standard workflow foreseen in Configuration Management."

I am afraid that the exception thrown in step 4 is the expected behaviour by design, or at least at this point in time. Am I right?

The situation is as follows:
1.- Enable my_module, which adds field_X and vocabulary_Y
--EVERYTHING OK--
2.- Uninstall my_module, which **does not delete** the field_X and vocabulary_Y
--EVERYTHING OK--
3.- Work on my_module to add some changes to field_X and vocabulary_Y
...
4.- Try to enable my_module, which should update field_X and vocabulary_Y with new values
--ERROR: "Configuration objects (...) provided by my_module already exist..."--

After reading the Open Issues section, more precisely:

"Updates: we can reuse configuration by simply enabling a module, but this holds only for the initial installation; after a module is enabled, we don't have a clean way to import changes (say, to "upgrade" to a newer version of the feature) outside the standard workflow foreseen in Configuration Management."

I am afraid that the exception thrown in step 4 is the expected behaviour by design, or at least at this point in time. Am I right?

Raúl Cano, 9 Feb 2015

The situation is as follows:
1.- Enable my_module, which adds field_X and vocabulary_Y
--EVERYTHING OK--
2.- Uninstall my_module, which **does not delete** the field_X and vocabulary_Y
--EVERYTHING OK--
3.- Work on my_module to add some changes to field_X and vocabulary_Y
...
4.- Try to enable my_module, which should update field_X and vocabulary_Y with new values
--ERROR: "Configuration objects (...) provided by my_module already exist..."--

After reading the Open Issues section, more precisely:

"Updates: we can reuse configuration by simply enabling a module, but this holds only for the initial installation; after a module is enabled, we don't have a clean way to import changes (say, to "upgrade" to a newer version of the feature) outside the standard workflow foreseen in Configuration Management."

I am afraid that the exception thrown in step 4 is the expected behaviour by design, or at least at this point in time. Am I right?

Raúl Cano, 9 Feb 2015
Andrea Pescetti
9 Feb 2015

In short, you are correct. As a rule of thumb, keep in mind that once something is imported into Configuration Management then Configuration Management (as things are now in D8 previews) takes over and expects to be the only "owner" of that configuration.

In short, you are correct. As a rule of thumb, keep in mind that once something is imported into Configuration Management then Configuration Management (as things are now in D8 previews) takes over and expects to be the only "owner" of that configuration.

Andrea Pescetti, 9 Feb 2015

In short, you are correct. As a rule of thumb, keep in mind that once something is imported into Configuration Management then Configuration Management (as things are now in D8 previews) takes over and expects to be the only "owner" of that configuration.

Andrea Pescetti, 9 Feb 2015
Raúl Cano
9 Feb 2015

Ok, I understand. I hope that in futue versions of Drupal 8 this kind of configuration updates are allowed since they would add an incredible flexibility to the platform.
Thanks for the quick answer and have a nice evening!

Ok, I understand. I hope that in futue versions of Drupal 8 this kind of configuration updates are allowed since they would add an incredible flexibility to the platform.
Thanks for the quick answer and have a nice evening!

Raúl Cano, 9 Feb 2015

Ok, I understand. I hope that in futue versions of Drupal 8 this kind of configuration updates are allowed since they would add an incredible flexibility to the platform.
Thanks for the quick answer and have a nice evening!

Raúl Cano, 9 Feb 2015
Raúl Cano
10 Feb 2015

Hi again, after installing the config_devel module I have seen that the command in this article is no longer valid, or at least in the version of config_devel I just installed (8.x-1.0-beta4).
As I see from the module files, instead of using "config-writeback", the command is now "config-devel-export-module". Cheers.

Hi again, after installing the config_devel module I have seen that the command in this article is no longer valid, or at least in the version of config_devel I just installed (8.x-1.0-beta4).
As I see from the module files, instead of using "config-writeback", the command is now "config-devel-export-module". Cheers.

Raúl Cano, 10 Feb 2015

Hi again, after installing the config_devel module I have seen that the command in this article is no longer valid, or at least in the version of config_devel I just installed (8.x-1.0-beta4).
As I see from the module files, instead of using "config-writeback", the command is now "config-devel-export-module". Cheers.

Raúl Cano, 10 Feb 2015
Admin
24 Jun 2015

Features for Drupal 8 is now available. See http://nuvole.org/blog/2015/apr/27/drupal-8-configuration-management-fea... for an updated version of this post.

Features for Drupal 8 is now available. See http://nuvole.org/blog/2015/apr/27/drupal-8-configuration-management-fea... for an updated version of this post.

Admin, 24 Jun 2015

Features for Drupal 8 is now available. See http://nuvole.org/blog/2015/apr/27/drupal-8-configuration-management-fea... for an updated version of this post.

Admin, 24 Jun 2015

Get your project started today!

Contact us