> ## Content Index
> Fetch the complete content index at: https://www.smallstepsystems.com/llms.txt
> Use this file to discover other available public pages before exploring further.

# DISTRO_FEATURES:append After DISTRO_FEATURES:remove Has No Effect
- URL: https://www.smallstepsystems.com/distro_featuresappend-after-distro_featuresremove-has-no-effect/
- Published: 2025-07-22T15:51:53.000Z
- Updated: 2025-11-07T21:46:20.000Z
- Author: Burkhard Stubert
- Tags: Embedded Linux, yocto

Once we have removed an item from a BitBake variable, we cannot re-append it. The evaluation of the following three assignments of the variable `BB_VAR` yields the same result, no matter in which order the assignments are executed.

```
# Code under someone else's control
BB_VAR ?= "x z"
BB_VAR:remove = "a y"

# Code under our control
BB_VAR:append = " y"

### Result: BB_VAR = "x z" instead of "x y z"
```

## Context

The [Torizon OS Minimal image](https://developer.toradex.com/torizon/torizoncore/torizon-os-reference-minimal-image/?ref=smallstepsystems.com) is an excellent starting point for our own products. It comes with automotive-grade OTA updates based on [Aktualizr](https://github.com/uptane/aktualizr?ref=smallstepsystems.com) and [OsTree](https://ostreedev.github.io/ostree/introduction/?ref=smallstepsystems.com). Through the [remote access client](https://www.torizon.io/remote-access?ref=smallstepsystems.com), the `torizon-minimal` image provides remote access over SSH and VNC and device monitoring over [FluentBit](https://fluentbit.io/?ref=smallstepsystems.com). Additionally, it has [monthly and quarterly releases](https://developer.toradex.com/software/toradex-embedded-software/toradex-embedded-linux-support-strategy/?ref=smallstepsystems.com) based on the Yocto LTS releases. These features are necessary to achieve conformity with the [EU Cyber Resilience Act](https://www.smallstepsystems.com/tag/eu-cyber-resilience-act/).

With all this goodness freely available, I wanted to extend the `torizon-minimal` image by Wayland, its default window manager Weston and example Qt applications. My first step was to add `wayland` to `DISTRO_FEATURES` in a distro configuration or in `local.conf`.

```
DISTRO_FEATURES:append = " wayland"
```

Although I had added `weston` to `IMAGE_FEATURES` in the image recipe if the distro feature `wayland` was present, no `weston` package was built.

```
IMAGE_FEATURES += " \
    ${@bb.utils.contains('DISTRO_FEATURES', 'wayland', 'weston', '', d)} \
    "
```

The reason could only be that `wayland` was not contained in `DISTRO_FEATURES`. Running

```
$ bitbake-getvar DISTRO_FEATURES
```

confirmed my observation, but also showed that my appending of `wayland` was done after its removal.

```
#   :remove meta-toradex-torizon/conf/distro/include/base-distro.inc:44
#     "3g ... pulseaudio wayland x11 ..."
#   :append meta-b4-apps/conf/distro/b4-torizon.conf:9
#     " seccomp wayland"
```

Therefore, the execution order couldn't play a role. I conjectured that all append's were evaluated before all the remove's. The section [Removal (Override Style Syntax)](https://docs.yoctoproject.org/bitbake/2.12/bitbake-user-manual/bitbake-user-manual-metadata.html?ref=smallstepsystems.com#removal-override-style-syntax) of the BitBake manual corroborated my conjecture.

> Override application order may not match variable parse history, i.e. the output of `[bitbake-getvar` or `bitbake -e]` may contain “:remove” before “:append”, but the result will be removed string, because “:remove” is handled last. \[...\]  
>  
> The overrides are applied in this order, “:append”, “:prepend”, “:remove”. This implies it is not possible to re-append previously removed strings.

This explains it: Appending a distro feature once it is removed is not possible.

This would close most images and distros for extension and hence violate the [open-closed principle](https://en.wikipedia.org/wiki/Open%E2%80%93closed%5Fprinciple?ref=smallstepsystems.com): *"software entities \[...\] should be open for extension, but closed for modification"*. It would be a pity, because the `torizon-minimal` image is such a good basis for extension. Fortunately, there is a good solution.

## Solution

The solution is to make the removed distro features configurable. We introduce a variable DISTRO\_FEATURES\_REMOVE and initialise it with the removed distro features in the distro configuration (here: `torizon.conf`).

```
# In torizon.conf through torizon.inc
DISTRO_FEATURES_REMOVE ?= "3g ... pulseaudio wayland x11 ..."
```

At the place, where we removed the above distro features originally, we now remove DISTRO\_FEATURES\_REMOVE.

```
# In torizon.conf through base-distro.inc and torizon.inc
DISTRO_FEATURES:remove = "${DISTRO_FEATURES_REMOVE}"
```

In our extension of the basic distro configuration or in `local.conf`, we remove `wayland` from DISTRO\_FEATURES\_REMOVE and add it to DISTRO\_FEATURES.

```
DISTRO_FEATURES_REMOVE:remove = "wayland"
DISTRO_FEATURES:append = " wayland"
```

Now, `DISTRO_FEATURES` contains `wayland`. Slightly indirect, but it works!

`DISTRO_FEATURES_REMOVE` is evaluated in this order:

```
set?: DISTRO_FEATURES_REMOVE ?= "3g ... pulseaudio wayland x11 ..."
remove: DISTRO_FEATURES_REMOVE:remove = "wayland"
### Result: DISTRO_FEATURES_REMOVE ?= "3g ... x11 ..."
```

`DISTRO_FEATURES` is evaluated in this order:

```
set?: DISTRO_FEATURES ?= "acl bluetooth"
append: DISTRO_FEATURES:append = " seccomp wayland"
remove: DISTRO_FEATURES:remove = "${DISTRO_FEATURES_REMOVE}"
        DISTRO_FEATURES:remove = "3g ... pulseaudio x11 ..."
### Result: DISTRO_FEATURES = "acl bluetooth seccomp wayland"
```

We created an extension point with the variable `DISTRO_FEATURES_REMOVE`. Clients of the original distro configuration `torizon` can use `DISTRO_FEATURES_REMOVE` to extend their distros by features like `wayland`, `nfc` or `bluetooth`. Extension points open the distro configuration for extension and keep it closed for modification - as required by the open-closed principle.

The original authors must foresee and provide such extensions points. The authors of the layer `meta-toradex-torizon` didn't foresee extensions for `DISTRO_FEATURES`, but they did for `IMAGE_FSTYPES` and `IMAGE_BOOT_FILES`. The authors of the `meta-freescale-*` layers use the same trick several times.

## Summary

Let us go back to our introductory example, where we couldn't undo the removal of `y`. With the help of `BB_VAR_REMOVE`, we can now append `y` although we removed it before.

```
# Code under our control
BB_VAR_REMOVE:remove = "y"
BB_VAR:append = " y"

# Extendable code under someone else's control
BB_VAR_REMOVE = "a y"
BB_VAR ?= "x z"
BB_VAR:remove = "${BB_VAR_REMOVE}"

### Result: BB_VAR = "x y z"
```

**Note**: The best news comes last. I upstreamed my changes to the [meta-toradex-torizon repository](https://github.com/torizon/meta-toradex-torizon/?ref=smallstepsystems.com). The Torizon maintainers merged [my pull request](https://github.com/torizon/meta-toradex-torizon/commit/3ec6a441ea45666e745e60a759e926a8be186f6e?ref=smallstepsystems.com) into the default branch `scarthgap-7.x.y` in no time. So, you can already use the change with the nightly and monthly pre-releases and soon with the quarterly production releases. Many thanks to the Torizon maintainers for such quick action!