Once we have removed an item from a BitBake variable, we cannot re-append it. The evaluation of the following three assignments of the variable BB_VAR yields the same result, no matter in which order the assignments are executed.

# Code under someone else's control
BB_VAR ?= "x z"
BB_VAR:remove = "a y"

# Code under our control
BB_VAR:append = " y"

### Result: BB_VAR = "x z" instead of "x y z"

Context

The Torizon OS Minimal image is an excellent starting point for our own products. It comes with automotive-grade OTA updates based on Aktualizr and OsTree. Through the remote access client, the torizon-minimal image provides remote access over SSH and VNC and device monitoring over FluentBit. Additionally, it has monthly and quarterly releases based on the Yocto LTS releases. These features are necessary to achieve conformity with the EU Cyber Resilience Act.

With all this goodness freely available, I wanted to extend the torizon-minimal image by Wayland, its default window manager Weston and example Qt applications. My first step was to add wayland to DISTRO_FEATURES in a distro configuration or in local.conf.

DISTRO_FEATURES:append = " wayland"

Although I had added weston to IMAGE_FEATURES in the image recipe if the distro feature wayland was present, no weston package was built.

IMAGE_FEATURES += " \
    ${@bb.utils.contains('DISTRO_FEATURES', 'wayland', 'weston', '', d)} \
    "

The reason could only be that wayland was not contained in DISTRO_FEATURES. Running

$ bitbake-getvar DISTRO_FEATURES

confirmed my observation, but also showed that my appending of wayland was done after its removal.

#   :remove meta-toradex-torizon/conf/distro/include/base-distro.inc:44
#     "3g ... pulseaudio wayland x11 ..."
#   :append meta-b4-apps/conf/distro/b4-torizon.conf:9
#     " seccomp wayland"

Therefore, the execution order couldn't play a role. I conjectured that all append's were evaluated before all the remove's. The section Removal (Override Style Syntax) of the BitBake manual corroborated my conjecture.

Override application order may not match variable parse history, i.e. the output of [bitbake-getvar or bitbake -e] may contain “:remove” before “:append”, but the result will be removed string, because “:remove” is handled last. [...]

The overrides are applied in this order, “:append”, “:prepend”, “:remove”. This implies it is not possible to re-append previously removed strings.

This explains it: Appending a distro feature once it is removed is not possible.

This would close most images and distros for extension and hence violate the open-closed principle: "software entities [...] should be open for extension, but closed for modification". It would be a pity, because the torizon-minimal image is such a good basis for extension. Fortunately, there is a good solution.

Solution

The solution is to make the removed distro features configurable. We introduce a variable DISTRO_FEATURES_REMOVE and initialise it with the removed distro features in the distro configuration (here: torizon.conf).

# In torizon.conf through torizon.inc
DISTRO_FEATURES_REMOVE ?= "3g ... pulseaudio wayland x11 ..."

At the place, where we removed the above distro features originally, we now remove DISTRO_FEATURES_REMOVE.

# In torizon.conf through base-distro.inc and torizon.inc
DISTRO_FEATURES:remove = "${DISTRO_FEATURES_REMOVE}"

In our extension of the basic distro configuration or in local.conf, we remove wayland from DISTRO_FEATURES_REMOVE and add it to DISTRO_FEATURES.

DISTRO_FEATURES_REMOVE:remove = "wayland"
DISTRO_FEATURES:append = " wayland"

Now, DISTRO_FEATURES contains wayland. Slightly indirect, but it works!

DISTRO_FEATURES_REMOVE is evaluated in this order:

set?: DISTRO_FEATURES_REMOVE ?= "3g ... pulseaudio wayland x11 ..."
remove: DISTRO_FEATURES_REMOVE:remove = "wayland"
### Result: DISTRO_FEATURES_REMOVE ?= "3g ... x11 ..."

DISTRO_FEATURES is evaluated in this order:

set?: DISTRO_FEATURES ?= "acl bluetooth"
append: DISTRO_FEATURES:append = " seccomp wayland"
remove: DISTRO_FEATURES:remove = "${DISTRO_FEATURES_REMOVE}"
        DISTRO_FEATURES:remove = "3g ... pulseaudio x11 ..."
### Result: DISTRO_FEATURES = "acl bluetooth seccomp wayland"

We created an extension point with the variable DISTRO_FEATURES_REMOVE. Clients of the original distro configuration torizon can use  DISTRO_FEATURES_REMOVE to extend their distros by features like wayland, nfc or bluetooth. Extension points open the distro configuration for extension and keep it closed for modification - as required by the open-closed principle.

The original authors must foresee and provide such extensions points. The authors of the layer meta-toradex-torizon didn't foresee extensions for DISTRO_FEATURES, but they did for IMAGE_FSTYPES and IMAGE_BOOT_FILES. The authors of the meta-freescale-* layers use the same trick several times.

Summary

Let us go back to our introductory example, where we couldn't undo the removal of y. With the help of BB_VAR_REMOVE, we can now append y although we removed it before.

# Code under our control
BB_VAR_REMOVE:remove = "y"
BB_VAR:append = " y"

# Extendable code under someone else's control
BB_VAR_REMOVE = "a y"
BB_VAR ?= "x z"
BB_VAR:remove = "${BB_VAR_REMOVE}"

### Result: BB_VAR = "x y z"

Note: The best news comes last. I upstreamed my changes to the meta-toradex-torizon repository. The Torizon maintainers merged my pull request into the default branch scarthgap-7.x.y in no time. So, you can already use the change with the nightly and monthly pre-releases and soon with the quarterly production releases. Many thanks to the Torizon maintainers for such quick action!