How to Set the Toolchain Version in Yocto

Some time ago I was confronted by the question: how to set the version of gcc used in a Yocto build (or g++, or any other software commonly referred to as the "toolchain" - a set of compilers and other tools used for the sole purpose, of building the target, which itself isn't installed on the target). This is a trivial issue for a developer working on a simple application without a complicated dependency tree, but in embedded systems development, where we build entire system images, even a slight modification to the toolchain often has enormous impact on the entire build, to the effect that the system will often fail to compile with that new toolchain.

Even though the title of this post is "How to Set the Toolchain Version in Yocto", the intention of it isn't that it be used as a tutorial. Rather, I'm trying to clear up the confusion that I had had over this issue, for myself, and also to describe the issue in a way that allows one to appreciate its complexity.

Short Answer: Don't

If for some reason you find yourself wanting to change the version of gcc (I'm using gcc as a concrete example but a lot of what I'm saying generalizes to other components of the toolchain) used for your Yocto build, don't. As you'll see below, when stated like this, the problem is ambiguous because there are actually multiple instances of gcc used during the process of building an image, but in any case, trying to fiddle with the version likely will cause more issues than it solves.

Rather, aim at a standardized environment that you can easily replicate between physical and virtual machines. Yocto already takes care of most of that, but it still needs some binaries from your host to kick-start the build (see the description below). This means that the build still needs specific versions of those programs and may not work with others.

There's an easy answer to this problem - use a Docker container. Base your Docker image on a distribution that's known to work with your build. For example, if you're building Poky (Yocto project's reference-distribution), you can look up the list of such distributions in meta-poky/conf/distro/poky.conf (of course, use the poky.conf file present in your revision of meta-poky rather than the one here):

SANITY_TESTED_DISTROS ?= " \
            poky-2.7 \n \
            poky-3.0 \n \
            ubuntu-16.04 \n \
            ubuntu-18.04 \n \
            ubuntu-19.04 \n \
            fedora-28 \n \
            fedora-29 \n \
            fedora-30 \n \
            centos-7 \n \
            debian-8 \n \
            debian-9 \n \
            debian-10 \n \
            opensuseleap-15.1 \n \
            "

A Docker image based on any one of those should do. Run your build inside a Docker volume so that you can persist your work and so that you can access the metadata from your standard programming environment. An exact tutorial on how to set up such a container is out of scope of this post.

Now on to why setting the version of gcc is such a complex problem.

Cross-compiling & Bootstrapping

Here's what happens when you build an image for an embedded platform. Assume your build machine is x86 and your target (i.e. your embedded platform) is ARM. The build system can't just call gcc from a shell. The CPU instruction set on the target (ARM) is different from that on the build machine (x86), so it's going to build another gcc instance specifically for compiling your target. This instance of gcc is called the cross-compiler and the technique where you compile sources on one system, to be deployed to another system is called cross-compiling.

There's nothing special about the cross-compiler gcc. Why not just install one from your host distribution's repositories, and use it to build the target? I've already said that in order to achieve maximum reproducibility, Yocto is aiming for the build environment to be as consistent as possible. The way it works is that the build system will first build all tools needed to build the target (compilers and so on, that is, the toolchain), with versions specified by the Yocto metadata, using the exact instructions specificed by the metadata.

But first, Yocto will need to build those tools. Or: a toolchain to build the toolchain that's going to be used for your platform. This is known as the bootstrapping problem - for example in order to build a compiler, you need to compile the sources of that compiler... With a compiler.

Different embedded Linux projects solve this problem differently. Yocto will simply use the tools already available on your build machine (e.g. /usr/bin/gcc for the C compiler; which is why it is useful to have a consistent user-space base, for example a Docker image).

The Initial Toolchain / Host Toolchain

For example, here are the steps the build system takes to build busybox:

  1. Build the toolchain needed for busybox. The packages forming the toolchain are usually suffixed with -native. Those will be built using the tools available on the host.
  2. Build busybox using the packages from step (1).

How to set the version of GCC used for step (1)? To do this, you set the BUILD_CC bitbake variable (by default set to gcc).

If you add the following to your local.conf

BUILD_CC = "gcc-8"

The build system is going to call gcc-8 whenever it needs to compile a -native package (of course, gcc-8 must be in your $PATH). If you put this in your local.conf

BUILD_CC = "/usr/local/bin/my-gcc-version"

It will directly reference that path. If you take a look at meta/classes/native.bbclass, you can see the variables used for the other initial tools:

# set the compiler as well. It could have been set to something else
export CC = "${BUILD_CC}"
export CXX = "${BUILD_CXX}"
export FC = "${BUILD_FC}"
export CPP = "${BUILD_CPP}"
export LD = "${BUILD_LD}"
export CCLD = "${BUILD_CCLD}"
export AR = "${BUILD_AR}"
export AS = "${BUILD_AS}"
export RANLIB = "${BUILD_RANLIB}"
export STRIP = "${BUILD_STRIP}"
export NM = "${BUILD_NM}"

See meta/classes/native.bbclass for details.

The Toolchain Used to Build the Target

How do you set the GCC version used for step (2)? Taking our busybox example again, the busybox recipe will depend on a virtual package called virtual/${TARGET_PREFIX}gcc. Any cross-compiler recipe will provide this virtual package.

In Yocto Zeus, there is only gcc cross-compiler recipe, namely, meta/recipes-devtools/gcc/gcc-cross_9.2.bb. If you want to use a different compiler, you'll have to import a recipe for it first. For example, if you want to use gcc 8, you could retrieve the recipe for it from a previous version of OpenEmbedded, and copy it over to your layer.

Assume you've added the recipes for gcc 8. Next, you will need to indicate to the build system which recipe you want to use for the cross-compiler. This is done by setting GCCVERSION.

In our example, we put the following in our local.conf (% is a wildcard):

GCCVERSION = "8.%"

This will in turn set PREFERRED_VERSION overrides to pick up the right recipes for the virtual packages for GCC.

And that's it!

Now, it is often a bad idea to set GCCVERSION to something different than what's already provided by OpenEmbedded. If you set this to an arbitrary version, it's possible that the other recipes provided by OpenEmbedded will refuse to build.

Conclusion

What I wrote about about gcc generalizes to other development tools. However, it's usually easier to port whatever you're trying to add to the development tools present in poky rather than the other way around. You're likely to break your build if you try to change the toolchain.