summaryrefslogtreecommitdiff
path: root/yocto-poky/documentation/ref-manual/technical-details.xml
diff options
context:
space:
mode:
Diffstat (limited to 'yocto-poky/documentation/ref-manual/technical-details.xml')
-rw-r--r--yocto-poky/documentation/ref-manual/technical-details.xml1460
1 files changed, 0 insertions, 1460 deletions
diff --git a/yocto-poky/documentation/ref-manual/technical-details.xml b/yocto-poky/documentation/ref-manual/technical-details.xml
deleted file mode 100644
index f06382ab5..000000000
--- a/yocto-poky/documentation/ref-manual/technical-details.xml
+++ /dev/null
@@ -1,1460 +0,0 @@
-<!DOCTYPE chapter PUBLIC "-//OASIS//DTD DocBook XML V4.2//EN"
-"http://www.oasis-open.org/docbook/xml/4.2/docbookx.dtd"
-[<!ENTITY % poky SYSTEM "../poky.ent"> %poky; ] >
-
-<chapter id='technical-details'>
-<title>Technical Details</title>
-
- <para>
- This chapter provides technical details for various parts of the
- Yocto Project.
- Currently, topics include Yocto Project components,
- cross-toolchain generation, shared state (sstate) cache,
- x32, Wayland support, and Licenses.
- </para>
-
-<section id='usingpoky-components'>
- <title>Yocto Project Components</title>
-
- <para>
- The
- <ulink url='&YOCTO_DOCS_DEV_URL;#bitbake-term'>BitBake</ulink>
- task executor together with various types of configuration files form
- the OpenEmbedded Core.
- This section overviews these components by describing their use and
- how they interact.
- </para>
-
- <para>
- BitBake handles the parsing and execution of the data files.
- The data itself is of various types:
- <itemizedlist>
- <listitem><para><emphasis>Recipes:</emphasis> Provides details
- about particular pieces of software.
- </para></listitem>
- <listitem><para><emphasis>Class Data:</emphasis> Abstracts
- common build information (e.g. how to build a Linux kernel).
- </para></listitem>
- <listitem><para><emphasis>Configuration Data:</emphasis> Defines
- machine-specific settings, policy decisions, and so forth.
- Configuration data acts as the glue to bind everything
- together.
- </para></listitem>
- </itemizedlist>
- </para>
-
- <para>
- BitBake knows how to combine multiple data sources together and refers
- to each data source as a layer.
- For information on layers, see the
- "<ulink url='&YOCTO_DOCS_DEV_URL;#understanding-and-creating-layers'>Understanding and
- Creating Layers</ulink>" section of the Yocto Project Development Manual.
- </para>
-
- <para>
- Following are some brief details on these core components.
- For additional information on how these components interact during
- a build, see the
- "<link linkend='closer-look'>A Closer Look at the Yocto Project Development Environment</link>"
- Chapter.
- </para>
-
- <section id='usingpoky-components-bitbake'>
- <title>BitBake</title>
-
- <para>
- BitBake is the tool at the heart of the OpenEmbedded build system
- and is responsible for parsing the
- <ulink url='&YOCTO_DOCS_DEV_URL;#metadata'>Metadata</ulink>,
- generating a list of tasks from it, and then executing those tasks.
- </para>
-
- <para>
- This section briefly introduces BitBake.
- If you want more information on BitBake, see the
- <ulink url='&YOCTO_DOCS_BB_URL;#bitbake-user-manual'>BitBake User Manual</ulink>.
- </para>
-
- <para>
- To see a list of the options BitBake supports, use either of
- the following commands:
- <literallayout class='monospaced'>
- $ bitbake -h
- $ bitbake --help
- </literallayout>
- </para>
-
- <para>
- The most common usage for BitBake is <filename>bitbake <replaceable>packagename</replaceable></filename>, where
- <filename>packagename</filename> is the name of the package you want to build
- (referred to as the "target" in this manual).
- The target often equates to the first part of a recipe's filename
- (e.g. "foo" for a recipe named
- <filename>foo_1.3.0-r0.bb</filename>).
- So, to process the <filename>matchbox-desktop_1.2.3.bb</filename> recipe file, you
- might type the following:
- <literallayout class='monospaced'>
- $ bitbake matchbox-desktop
- </literallayout>
- Several different versions of <filename>matchbox-desktop</filename> might exist.
- BitBake chooses the one selected by the distribution configuration.
- You can get more details about how BitBake chooses between different
- target versions and providers in the
- "<ulink url='&YOCTO_DOCS_BB_URL;#bb-bitbake-preferences'>Preferences</ulink>"
- section of the BitBake User Manual.
- </para>
-
- <para>
- BitBake also tries to execute any dependent tasks first.
- So for example, before building <filename>matchbox-desktop</filename>, BitBake
- would build a cross compiler and <filename>glibc</filename> if they had not already
- been built.
- </para>
-
- <para>
- A useful BitBake option to consider is the <filename>-k</filename> or
- <filename>--continue</filename> option.
- This option instructs BitBake to try and continue processing the job
- as long as possible even after encountering an error.
- When an error occurs, the target that
- failed and those that depend on it cannot be remade.
- However, when you use this option other dependencies can still be
- processed.
- </para>
- </section>
-
- <section id='usingpoky-components-metadata'>
- <title>Metadata (Recipes)</title>
-
- <para>
- Files that have the <filename>.bb</filename> suffix are "recipes"
- files.
- In general, a recipe contains information about a single piece of
- software.
- This information includes the location from which to download the
- unaltered source, any source patches to be applied to that source
- (if needed), which special configuration options to apply,
- how to compile the source files, and how to package the compiled
- output.
- </para>
-
- <para>
- The term "package" is sometimes used to refer to recipes. However,
- since the word "package" is used for the packaged output from the OpenEmbedded
- build system (i.e. <filename>.ipk</filename> or <filename>.deb</filename> files),
- this document avoids using the term "package" when referring to recipes.
- </para>
- </section>
-
- <section id='usingpoky-components-classes'>
- <title>Classes</title>
-
- <para>
- Class files (<filename>.bbclass</filename>) contain information that
- is useful to share between
- <ulink url='&YOCTO_DOCS_DEV_URL;#metadata'>Metadata</ulink> files.
- An example is the
- <link linkend='ref-classes-autotools'><filename>autotools</filename></link>
- class, which contains common settings for any application that
- Autotools uses.
- The "<link linkend='ref-classes'>Classes</link>" chapter provides
- details about classes and how to use them.
- </para>
- </section>
-
- <section id='usingpoky-components-configuration'>
- <title>Configuration</title>
-
- <para>
- The configuration files (<filename>.conf</filename>) define various configuration variables
- that govern the OpenEmbedded build process.
- These files fall into several areas that define machine configuration options,
- distribution configuration options, compiler tuning options, general common configuration
- options, and user configuration options in <filename>local.conf</filename>, which is found
- in the
- <ulink url='&YOCTO_DOCS_DEV_URL;#build-directory'>Build Directory</ulink>.
- </para>
- </section>
-</section>
-
-<section id="cross-development-toolchain-generation">
- <title>Cross-Development Toolchain Generation</title>
-
- <para>
- The Yocto Project does most of the work for you when it comes to
- creating
- <ulink url='&YOCTO_DOCS_DEV_URL;#cross-development-toolchain'>cross-development toolchains</ulink>.
- This section provides some technical background on how
- cross-development toolchains are created and used.
- For more information on toolchains, you can also see the
- <ulink url='&YOCTO_DOCS_SDK_URL;'>Yocto Project Software Development Kit (SDK) Developer's Guide</ulink>.
- </para>
-
- <para>
- In the Yocto Project development environment, cross-development
- toolchains are used to build the image and applications that run on the
- target hardware.
- With just a few commands, the OpenEmbedded build system creates
- these necessary toolchains for you.
- </para>
-
- <para>
- The following figure shows a high-level build environment regarding
- toolchain construction and use.
- </para>
-
- <para>
- <imagedata fileref="figures/cross-development-toolchains.png" width="8in" depth="6in" align="center" />
- </para>
-
- <para>
- Most of the work occurs on the Build Host.
- This is the machine used to build images and generally work within the
- the Yocto Project environment.
- When you run BitBake to create an image, the OpenEmbedded build system
- uses the host <filename>gcc</filename> compiler to bootstrap a
- cross-compiler named <filename>gcc-cross</filename>.
- The <filename>gcc-cross</filename> compiler is what BitBake uses to
- compile source files when creating the target image.
- You can think of <filename>gcc-cross</filename> simply as an
- automatically generated cross-compiler that is used internally within
- BitBake only.
- <note>
- The extensible SDK does not use
- <filename>gcc-cross-canadian</filename> since this SDK
- ships a copy of the OpenEmbedded build system and the sysroot
- within it contains <filename>gcc-cross</filename>.
- </note>
- </para>
-
- <para>
- The chain of events that occurs when <filename>gcc-cross</filename> is
- bootstrapped is as follows:
- <literallayout class='monospaced'>
- gcc -> binutils-cross -> gcc-cross-initial -> linux-libc-headers -> glibc-initial -> glibc -> gcc-cross -> gcc-runtime
- </literallayout>
- <itemizedlist>
- <listitem><para><filename>gcc</filename>:
- The build host's GNU Compiler Collection (GCC).
- </para></listitem>
- <listitem><para><filename>binutils-cross</filename>:
- The bare minimum binary utilities needed in order to run
- the <filename>gcc-cross-initial</filename> phase of the
- bootstrap operation.
- </para></listitem>
- <listitem><para><filename>gcc-cross-initial</filename>:
- An early stage of the bootstrap process for creating
- the cross-compiler.
- This stage builds enough of the <filename>gcc-cross</filename>,
- the C library, and other pieces needed to finish building the
- final cross-compiler in later stages.
- This tool is a "native" package (i.e. it is designed to run on
- the build host).
- </para></listitem>
- <listitem><para><filename>linux-libc-headers</filename>:
- Headers needed for the cross-compiler.
- </para></listitem>
- <listitem><para><filename>glibc-initial</filename>:
- An initial version of the Embedded GLIBC needed to bootstrap
- <filename>glibc</filename>.
- </para></listitem>
- <listitem><para><filename>gcc-cross</filename>:
- The final stage of the bootstrap process for the
- cross-compiler.
- This stage results in the actual cross-compiler that
- BitBake uses when it builds an image for a targeted
- device.
- <note>
- If you are replacing this cross compiler toolchain
- with a custom version, you must replace
- <filename>gcc-cross</filename>.
- </note>
- This tool is also a "native" package (i.e. it is
- designed to run on the build host).
- </para></listitem>
- <listitem><para><filename>gcc-runtime</filename>:
- Runtime libraries resulting from the toolchain bootstrapping
- process.
- This tool produces a binary that consists of the
- runtime libraries need for the targeted device.
- </para></listitem>
- </itemizedlist>
- </para>
-
- <para>
- You can use the OpenEmbedded build system to build an installer for
- the relocatable SDK used to develop applications.
- When you run the installer, it installs the toolchain, which contains
- the development tools (e.g., the
- <filename>gcc-cross-canadian</filename>),
- <filename>binutils-cross-canadian</filename>, and other
- <filename>nativesdk-*</filename> tools,
- which are tools native to the SDK (i.e. native to
- <link linkend='var-SDK_ARCH'><filename>SDK_ARCH</filename></link>),
- you need to cross-compile and test your software.
- The figure shows the commands you use to easily build out this
- toolchain.
- This cross-development toolchain is built to execute on the
- <link linkend='var-SDKMACHINE'><filename>SDKMACHINE</filename></link>,
- which might or might not be the same
- machine as the Build Host.
- <note>
- If your target architecture is supported by the Yocto Project,
- you can take advantage of pre-built images that ship with the
- Yocto Project and already contain cross-development toolchain
- installers.
- </note>
- </para>
-
- <para>
- Here is the bootstrap process for the relocatable toolchain:
- <literallayout class='monospaced'>
- gcc -> binutils-crosssdk -> gcc-crosssdk-initial -> linux-libc-headers ->
- glibc-initial -> nativesdk-glibc -> gcc-crosssdk -> gcc-cross-canadian
- </literallayout>
- <itemizedlist>
- <listitem><para><filename>gcc</filename>:
- The build host's GNU Compiler Collection (GCC).
- </para></listitem>
- <listitem><para><filename>binutils-crosssdk</filename>:
- The bare minimum binary utilities needed in order to run
- the <filename>gcc-crosssdk-initial</filename> phase of the
- bootstrap operation.
- </para></listitem>
- <listitem><para><filename>gcc-crosssdk-initial</filename>:
- An early stage of the bootstrap process for creating
- the cross-compiler.
- This stage builds enough of the
- <filename>gcc-crosssdk</filename> and supporting pieces so that
- the final stage of the bootstrap process can produce the
- finished cross-compiler.
- This tool is a "native" binary that runs on the build host.
- </para></listitem>
- <listitem><para><filename>linux-libc-headers</filename>:
- Headers needed for the cross-compiler.
- </para></listitem>
- <listitem><para><filename>glibc-initial</filename>:
- An initial version of the Embedded GLIBC needed to bootstrap
- <filename>nativesdk-glibc</filename>.
- </para></listitem>
- <listitem><para><filename>nativesdk-glibc</filename>:
- The Embedded GLIBC needed to bootstrap the
- <filename>gcc-crosssdk</filename>.
- </para></listitem>
- <listitem><para><filename>gcc-crosssdk</filename>:
- The final stage of the bootstrap process for the
- relocatable cross-compiler.
- The <filename>gcc-crosssdk</filename> is a transitory compiler
- and never leaves the build host.
- Its purpose is to help in the bootstrap process to create the
- eventual relocatable <filename>gcc-cross-canadian</filename>
- compiler, which is relocatable.
- This tool is also a "native" package (i.e. it is
- designed to run on the build host).
- </para></listitem>
- <listitem><para><filename>gcc-cross-canadian</filename>:
- The final relocatable cross-compiler.
- When run on the
- <link linkend='var-SDKMACHINE'><filename>SDKMACHINE</filename></link>,
- this tool
- produces executable code that runs on the target device.
- Only one cross-canadian compiler is produced per architecture
- since they can be targeted at different processor optimizations
- using configurations passed to the compiler through the
- compile commands.
- This circumvents the need for multiple compilers and thus
- reduces the size of the toolchains.
- </para></listitem>
- </itemizedlist>
- </para>
-
- <note>
- For information on advantages gained when building a
- cross-development toolchain installer, see the
- "<ulink url='&YOCTO_DOCS_SDK_URL;#sdk-building-an-sdk-installer'>Building an SDK Installer</ulink>"
- section in the Yocto Project Software Development Kit (SDK) Developer's
- Guide.
- </note>
-</section>
-
-<section id="shared-state-cache">
- <title>Shared State Cache</title>
-
- <para>
- By design, the OpenEmbedded build system builds everything from scratch unless
- BitBake can determine that parts do not need to be rebuilt.
- Fundamentally, building from scratch is attractive as it means all parts are
- built fresh and there is no possibility of stale data causing problems.
- When developers hit problems, they typically default back to building from scratch
- so they know the state of things from the start.
- </para>
-
- <para>
- Building an image from scratch is both an advantage and a disadvantage to the process.
- As mentioned in the previous paragraph, building from scratch ensures that
- everything is current and starts from a known state.
- However, building from scratch also takes much longer as it generally means
- rebuilding things that do not necessarily need to be rebuilt.
- </para>
-
- <para>
- The Yocto Project implements shared state code that supports incremental builds.
- The implementation of the shared state code answers the following questions that
- were fundamental roadblocks within the OpenEmbedded incremental build support system:
- <itemizedlist>
- <listitem><para>What pieces of the system have changed and what pieces have
- not changed?</para></listitem>
- <listitem><para>How are changed pieces of software removed and replaced?</para></listitem>
- <listitem><para>How are pre-built components that do not need to be rebuilt from scratch
- used when they are available?</para></listitem>
- </itemizedlist>
- </para>
-
- <para>
- For the first question, the build system detects changes in the "inputs" to a given task by
- creating a checksum (or signature) of the task's inputs.
- If the checksum changes, the system assumes the inputs have changed and the task needs to be
- rerun.
- For the second question, the shared state (sstate) code tracks which tasks add which output
- to the build process.
- This means the output from a given task can be removed, upgraded or otherwise manipulated.
- The third question is partly addressed by the solution for the second question
- assuming the build system can fetch the sstate objects from remote locations and
- install them if they are deemed to be valid.
- </para>
-
- <note>
- The OpenEmbedded build system does not maintain
- <link linkend='var-PR'><filename>PR</filename></link> information
- as part of the shared state packages.
- Consequently, considerations exist that affect maintaining shared
- state feeds.
- For information on how the OpenEmbedded build system
- works with packages and can
- track incrementing <filename>PR</filename> information, see the
- "<ulink url='&YOCTO_DOCS_DEV_URL;#incrementing-a-package-revision-number'>Incrementing a Package Revision Number</ulink>"
- section.
- </note>
-
- <para>
- The rest of this section goes into detail about the overall incremental build
- architecture, the checksums (signatures), shared state, and some tips and tricks.
- </para>
-
- <section id='overall-architecture'>
- <title>Overall Architecture</title>
-
- <para>
- When determining what parts of the system need to be built, BitBake
- works on a per-task basis rather than a per-recipe basis.
- You might wonder why using a per-task basis is preferred over a per-recipe basis.
- To help explain, consider having the IPK packaging backend enabled and then switching to DEB.
- In this case, the
- <link linkend='ref-tasks-install'><filename>do_install</filename></link>
- and
- <link linkend='ref-tasks-package'><filename>do_package</filename></link>
- task outputs are still valid.
- However, with a per-recipe approach, the build would not include the
- <filename>.deb</filename> files.
- Consequently, you would have to invalidate the whole build and rerun it.
- Rerunning everything is not the best solution.
- Also, in this case, the core must be "taught" much about specific tasks.
- This methodology does not scale well and does not allow users to easily add new tasks
- in layers or as external recipes without touching the packaged-staging core.
- </para>
- </section>
-
- <section id='checksums'>
- <title>Checksums (Signatures)</title>
-
- <para>
- The shared state code uses a checksum, which is a unique signature of a task's
- inputs, to determine if a task needs to be run again.
- Because it is a change in a task's inputs that triggers a rerun, the process
- needs to detect all the inputs to a given task.
- For shell tasks, this turns out to be fairly easy because
- the build process generates a "run" shell script for each task and
- it is possible to create a checksum that gives you a good idea of when
- the task's data changes.
- </para>
-
- <para>
- To complicate the problem, there are things that should not be
- included in the checksum.
- First, there is the actual specific build path of a given task -
- the <link linkend='var-WORKDIR'><filename>WORKDIR</filename></link>.
- It does not matter if the work directory changes because it should
- not affect the output for target packages.
- Also, the build process has the objective of making native
- or cross packages relocatable.
- <note>
- Both native and cross packages run on the build host.
- However, cross packages generate output for the target
- architecture.
- </note>
- The checksum therefore needs to exclude
- <filename>WORKDIR</filename>.
- The simplistic approach for excluding the work directory is to set
- <filename>WORKDIR</filename> to some fixed value and create the
- checksum for the "run" script.
- </para>
-
- <para>
- Another problem results from the "run" scripts containing functions that
- might or might not get called.
- The incremental build solution contains code that figures out dependencies
- between shell functions.
- This code is used to prune the "run" scripts down to the minimum set,
- thereby alleviating this problem and making the "run" scripts much more
- readable as a bonus.
- </para>
-
- <para>
- So far we have solutions for shell scripts.
- What about Python tasks?
- The same approach applies even though these tasks are more difficult.
- The process needs to figure out what variables a Python function accesses
- and what functions it calls.
- Again, the incremental build solution contains code that first figures out
- the variable and function dependencies, and then creates a checksum for the data
- used as the input to the task.
- </para>
-
- <para>
- Like the <filename>WORKDIR</filename> case, situations exist where dependencies
- should be ignored.
- For these cases, you can instruct the build process to ignore a dependency
- by using a line like the following:
- <literallayout class='monospaced'>
- PACKAGE_ARCHS[vardepsexclude] = "MACHINE"
- </literallayout>
- This example ensures that the
- <link linkend='var-PACKAGE_ARCHS'><filename>PACKAGE_ARCHS</filename></link>
- variable does not
- depend on the value of
- <link linkend='var-MACHINE'><filename>MACHINE</filename></link>,
- even if it does reference it.
- </para>
-
- <para>
- Equally, there are cases where we need to add dependencies BitBake is not able to find.
- You can accomplish this by using a line like the following:
- <literallayout class='monospaced'>
- PACKAGE_ARCHS[vardeps] = "MACHINE"
- </literallayout>
- This example explicitly adds the <filename>MACHINE</filename> variable as a
- dependency for <filename>PACKAGE_ARCHS</filename>.
- </para>
-
- <para>
- Consider a case with in-line Python, for example, where BitBake is not
- able to figure out dependencies.
- When running in debug mode (i.e. using <filename>-DDD</filename>), BitBake
- produces output when it discovers something for which it cannot figure out
- dependencies.
- The Yocto Project team has currently not managed to cover those dependencies
- in detail and is aware of the need to fix this situation.
- </para>
-
- <para>
- Thus far, this section has limited discussion to the direct inputs into a task.
- Information based on direct inputs is referred to as the "basehash" in the
- code.
- However, there is still the question of a task's indirect inputs - the
- things that were already built and present in the
- <ulink url='&YOCTO_DOCS_DEV_URL;#build-directory'>Build Directory</ulink>.
- The checksum (or signature) for a particular task needs to add the hashes
- of all the tasks on which the particular task depends.
- Choosing which dependencies to add is a policy decision.
- However, the effect is to generate a master checksum that combines the basehash
- and the hashes of the task's dependencies.
- </para>
-
- <para>
- At the code level, there are a variety of ways both the basehash and the
- dependent task hashes can be influenced.
- Within the BitBake configuration file, we can give BitBake some extra information
- to help it construct the basehash.
- The following statement effectively results in a list of global variable
- dependency excludes - variables never included in any checksum:
- <literallayout class='monospaced'>
- BB_HASHBASE_WHITELIST ?= "TMPDIR FILE PATH PWD BB_TASKHASH BBPATH DL_DIR \
- SSTATE_DIR THISDIR FILESEXTRAPATHS FILE_DIRNAME HOME LOGNAME SHELL TERM \
- USER FILESPATH STAGING_DIR_HOST STAGING_DIR_TARGET COREBASE PRSERV_HOST \
- PRSERV_DUMPDIR PRSERV_DUMPFILE PRSERV_LOCKDOWN PARALLEL_MAKE \
- CCACHE_DIR EXTERNAL_TOOLCHAIN CCACHE CCACHE_DISABLE LICENSE_PATH SDKPKGSUFFIX"
- </literallayout>
- The previous example excludes
- <link linkend='var-WORKDIR'><filename>WORKDIR</filename></link>
- since that variable is actually constructed as a path within
- <link linkend='var-TMPDIR'><filename>TMPDIR</filename></link>, which is on
- the whitelist.
- </para>
-
- <para>
- The rules for deciding which hashes of dependent tasks to include through
- dependency chains are more complex and are generally accomplished with a
- Python function.
- The code in <filename>meta/lib/oe/sstatesig.py</filename> shows two examples
- of this and also illustrates how you can insert your own policy into the system
- if so desired.
- This file defines the two basic signature generators <filename>OE-Core</filename>
- uses: "OEBasic" and "OEBasicHash".
- By default, there is a dummy "noop" signature handler enabled in BitBake.
- This means that behavior is unchanged from previous versions.
- <filename>OE-Core</filename> uses the "OEBasicHash" signature handler by default
- through this setting in the <filename>bitbake.conf</filename> file:
- <literallayout class='monospaced'>
- BB_SIGNATURE_HANDLER ?= "OEBasicHash"
- </literallayout>
- The "OEBasicHash" <filename>BB_SIGNATURE_HANDLER</filename> is the same as the
- "OEBasic" version but adds the task hash to the stamp files.
- This results in any
- <ulink url='&YOCTO_DOCS_DEV_URL;#metadata'>Metadata</ulink>
- change that changes the task hash, automatically
- causing the task to be run again.
- This removes the need to bump <link linkend='var-PR'><filename>PR</filename></link>
- values, and changes to Metadata automatically ripple across the build.
- </para>
-
- <para>
- It is also worth noting that the end result of these signature generators is to
- make some dependency and hash information available to the build.
- This information includes:
- <itemizedlist>
- <listitem><para><filename>BB_BASEHASH_task-</filename><replaceable>taskname</replaceable>:
- The base hashes for each task in the recipe.
- </para></listitem>
- <listitem><para><filename>BB_BASEHASH_</filename><replaceable>filename</replaceable><filename>:</filename><replaceable>taskname</replaceable>:
- The base hashes for each dependent task.
- </para></listitem>
- <listitem><para><filename>BBHASHDEPS_</filename><replaceable>filename</replaceable><filename>:</filename><replaceable>taskname</replaceable>:
- The task dependencies for each task.
- </para></listitem>
- <listitem><para><filename>BB_TASKHASH</filename>:
- The hash of the currently running task.
- </para></listitem>
- </itemizedlist>
- </para>
- </section>
-
- <section id='shared-state'>
- <title>Shared State</title>
-
- <para>
- Checksums and dependencies, as discussed in the previous section, solve half the
- problem of supporting a shared state.
- The other part of the problem is being able to use checksum information during the build
- and being able to reuse or rebuild specific components.
- </para>
-
- <para>
- The
- <link linkend='ref-classes-sstate'><filename>sstate</filename></link>
- class is a relatively generic implementation of how to "capture"
- a snapshot of a given task.
- The idea is that the build process does not care about the source of a task's output.
- Output could be freshly built or it could be downloaded and unpacked from
- somewhere - the build process does not need to worry about its origin.
- </para>
-
- <para>
- There are two types of output, one is just about creating a directory
- in <link linkend='var-WORKDIR'><filename>WORKDIR</filename></link>.
- A good example is the output of either
- <link linkend='ref-tasks-install'><filename>do_install</filename></link>
- or
- <link linkend='ref-tasks-package'><filename>do_package</filename></link>.
- The other type of output occurs when a set of data is merged into a shared directory
- tree such as the sysroot.
- </para>
-
- <para>
- The Yocto Project team has tried to keep the details of the
- implementation hidden in <filename>sstate</filename> class.
- From a user's perspective, adding shared state wrapping to a task
- is as simple as this
- <link linkend='ref-tasks-deploy'><filename>do_deploy</filename></link>
- example taken from the
- <link linkend='ref-classes-deploy'><filename>deploy</filename></link>
- class:
- <literallayout class='monospaced'>
- DEPLOYDIR = "${WORKDIR}/deploy-${PN}"
- SSTATETASKS += "do_deploy"
- do_deploy[sstate-inputdirs] = "${DEPLOYDIR}"
- do_deploy[sstate-outputdirs] = "${DEPLOY_DIR_IMAGE}"
-
- python do_deploy_setscene () {
- sstate_setscene(d)
- }
- addtask do_deploy_setscene
- do_deploy[dirs] = "${DEPLOYDIR} ${B}"
- </literallayout>
- In this example, we add some extra flags to the task, a name field ("deploy"), an
- input directory where the task sends data, and the output
- directory where the data from the task should eventually be copied.
- We also add a <filename>_setscene</filename> variant of the task and add the task
- name to the <filename>SSTATETASKS</filename> list.
- </para>
-
- <para>
- If you have a directory whose contents you need to preserve, you can do this with
- a line like the following:
- <literallayout class='monospaced'>
- do_package[sstate-plaindirs] = "${PKGD} ${PKGDEST}"
- </literallayout>
- This method, as well as the following example, also works for multiple directories.
- <literallayout class='monospaced'>
- do_package[sstate-inputdirs] = "${PKGDESTWORK} ${SHLIBSWORKDIR}"
- do_package[sstate-outputdirs] = "${PKGDATA_DIR} ${SHLIBSDIR}"
- do_package[sstate-lockfile] = "${PACKAGELOCK}"
- </literallayout>
- These methods also include the ability to take a lockfile when manipulating
- shared state directory structures since some cases are sensitive to file
- additions or removals.
- </para>
-
- <para>
- Behind the scenes, the shared state code works by looking in
- <link linkend='var-SSTATE_DIR'><filename>SSTATE_DIR</filename></link> and
- <link linkend='var-SSTATE_MIRRORS'><filename>SSTATE_MIRRORS</filename></link>
- for shared state files.
- Here is an example:
- <literallayout class='monospaced'>
- SSTATE_MIRRORS ?= "\
- file://.* http://someserver.tld/share/sstate/PATH \n \
- file://.* file:///some/local/dir/sstate/PATH"
- </literallayout>
- <note>
- The shared state directory (<filename>SSTATE_DIR</filename>) is
- organized into two-character subdirectories, where the subdirectory
- names are based on the first two characters of the hash.
- If the shared state directory structure for a mirror has the
- same structure as <filename>SSTATE_DIR</filename>, you must
- specify "PATH" as part of the URI to enable the build system
- to map to the appropriate subdirectory.
- </note>
- </para>
-
- <para>
- The shared state package validity can be detected just by looking at the
- filename since the filename contains the task checksum (or signature) as
- described earlier in this section.
- If a valid shared state package is found, the build process downloads it
- and uses it to accelerate the task.
- </para>
-
- <para>
- The build processes use the <filename>*_setscene</filename> tasks
- for the task acceleration phase.
- BitBake goes through this phase before the main execution code and tries
- to accelerate any tasks for which it can find shared state packages.
- If a shared state package for a task is available, the shared state
- package is used.
- This means the task and any tasks on which it is dependent are not
- executed.
- </para>
-
- <para>
- As a real world example, the aim is when building an IPK-based image,
- only the
- <link linkend='ref-tasks-package_write_ipk'><filename>do_package_write_ipk</filename></link>
- tasks would have their
- shared state packages fetched and extracted.
- Since the sysroot is not used, it would never get extracted.
- This is another reason why a task-based approach is preferred over a
- recipe-based approach, which would have to install the output from every task.
- </para>
- </section>
-
- <section id='tips-and-tricks'>
- <title>Tips and Tricks</title>
-
- <para>
- The code in the build system that supports incremental builds is not
- simple code.
- This section presents some tips and tricks that help you work around
- issues related to shared state code.
- </para>
-
- <section id='debugging'>
- <title>Debugging</title>
-
- <para>
- When things go wrong, debugging needs to be straightforward.
- Because of this, the Yocto Project includes strong debugging
- tools:
- <itemizedlist>
- <listitem><para>Whenever a shared state package is written
- out into the
- <link linkend='var-SSTATE_DIR'><filename>SSTATE_DIR</filename></link>,
- a corresponding <filename>.siginfo</filename> file is
- also written.
- This file contains a pickled Python database of all
- the Metadata that went into creating the hash for a
- given shared state package.
- Whenever a stamp is written into the stamp directory
- <link linkend='var-STAMP'><filename>STAMP</filename></link>,
- a corresponding <filename>.sigdata</filename> file
- is created that contains the same hash data that
- represented the executed task.
- </para></listitem>
- <listitem><para>You can use BitBake to dump out the
- signature construction information without executing
- tasks by using either of the following BitBake
- command-line options:
- <literallayout class='monospaced'>
- &dash;&dash;dump-signatures=<replaceable>SIGNATURE_HANDLER</replaceable>
- -S <replaceable>SIGNATURE_HANDLER</replaceable>
- </literallayout>
- <note>
- Two common values for
- <replaceable>SIGNATURE_HANDLER</replaceable> are
- "none" and "printdiff" to only dump the signature
- or to compare the dumped signature with the
- cached one, respectively.
- </note>
- Using BitBake with either of these options causes
- BitBake to dump out <filename>.sigdata</filename> files
- in the stamp directory for every task it would have
- executed instead of building the specified target
- package.
- </para></listitem>
- <listitem><para>There is a
- <filename>bitbake-diffsigs</filename> command that
- can process <filename>.sigdata</filename> and
- <filename>.siginfo</filename> files.
- If you specify one of these files, BitBake dumps out
- the dependency information in the file.
- If you specify two files, BitBake compares the two
- files and dumps out the differences between the two.
- This more easily helps answer the question of "What
- changed between X and Y?"</para></listitem>
- </itemizedlist>
- </para>
- </section>
-
- <section id='invalidating-shared-state'>
- <title>Invalidating Shared State</title>
-
- <para>
- The OpenEmbedded build system uses checksums and shared state
- cache to avoid unnecessarily rebuilding tasks.
- Collectively, this scheme is known as "shared state code."
- </para>
-
- <para>
- As with all schemes, this one has some drawbacks.
- It is possible that you could make implicit changes to your
- code that the checksum calculations do not take into
- account.
- These implicit changes affect a task's output but do not trigger
- the shared state code into rebuilding a recipe.
- Consider an example during which a tool changes its output.
- Assume that the output of <filename>rpmdeps</filename> changes.
- The result of the change should be that all the
- <filename>package</filename> and
- <filename>package_write_rpm</filename> shared state cache
- items become invalid.
- However, because the change to the output is
- external to the code and therefore implicit,
- the associated shared state cache items do not become
- invalidated.
- In this case, the build process uses the cached items rather
- than running the task again.
- Obviously, these types of implicit changes can cause problems.
- </para>
-
- <para>
- To avoid these problems during the build, you need to
- understand the effects of any changes you make.
- Realize that changes you make directly to a function
- are automatically factored into the checksum calculation.
- Thus, these explicit changes invalidate the associated area of
- shared state cache.
- However, you need to be aware of any implicit changes that
- are not obvious changes to the code and could affect the output
- of a given task.
- </para>
-
- <para>
- When you identify an implicit change, you can easily take steps
- to invalidate the cache and force the tasks to run.
- The steps you can take are as simple as changing a function's
- comments in the source code.
- For example, to invalidate package shared state files, change
- the comment statements of
- <link linkend='ref-tasks-package'><filename>do_package</filename></link>
- or the comments of one of the functions it calls.
- Even though the change is purely cosmetic, it causes the
- checksum to be recalculated and forces the OpenEmbedded build
- system to run the task again.
- </para>
-
- <note>
- For an example of a commit that makes a cosmetic change to
- invalidate shared state, see this
- <ulink url='&YOCTO_GIT_URL;/cgit.cgi/poky/commit/meta/classes/package.bbclass?id=737f8bbb4f27b4837047cb9b4fbfe01dfde36d54'>commit</ulink>.
- </note>
- </section>
- </section>
-</section>
-
-<section id='x32'>
- <title>x32</title>
-
- <para>
- x32 is a processor-specific Application Binary Interface (psABI) for x86_64.
- An ABI defines the calling conventions between functions in a processing environment.
- The interface determines what registers are used and what the sizes are for various C data types.
- </para>
-
- <para>
- Some processing environments prefer using 32-bit applications even when running
- on Intel 64-bit platforms.
- Consider the i386 psABI, which is a very old 32-bit ABI for Intel 64-bit platforms.
- The i386 psABI does not provide efficient use and access of the Intel 64-bit processor resources,
- leaving the system underutilized.
- Now consider the x86_64 psABI.
- This ABI is newer and uses 64-bits for data sizes and program pointers.
- The extra bits increase the footprint size of the programs, libraries,
- and also increases the memory and file system size requirements.
- Executing under the x32 psABI enables user programs to utilize CPU and system resources
- more efficiently while keeping the memory footprint of the applications low.
- Extra bits are used for registers but not for addressing mechanisms.
- </para>
-
- <section id='support'>
- <title>Support</title>
-
- <para>
- This Yocto Project release supports the final specifications of x32
- psABI.
- Support for x32 psABI exists as follows:
- <itemizedlist>
- <listitem><para>You can create packages and images in x32 psABI format on x86_64 architecture targets.
- </para></listitem>
- <listitem><para>You can successfully build many recipes with the x32 toolchain.</para></listitem>
- <listitem><para>You can create and boot <filename>core-image-minimal</filename> and
- <filename>core-image-sato</filename> images.</para></listitem>
- </itemizedlist>
- </para>
- </section>
-
- <section id='completing-x32'>
- <title>Completing x32</title>
-
- <para>
- Future Plans for the x32 psABI in the Yocto Project include the following:
- <itemizedlist>
- <listitem><para>Enhance and fix the few remaining recipes so they
- work with and support x32 toolchains.</para></listitem>
- <listitem><para>Enhance RPM Package Manager (RPM) support for x32 binaries.</para></listitem>
- <listitem><para>Support larger images.</para></listitem>
- </itemizedlist>
- </para>
- </section>
-
- <section id='using-x32-right-now'>
- <title>Using x32 Right Now</title>
-
- <para>
- Follow these steps to use the x32 spABI:
- <itemizedlist>
- <listitem><para>Enable the x32 psABI tuning file for <filename>x86_64</filename>
- machines by editing the <filename>conf/local.conf</filename> like this:
- <literallayout class='monospaced'>
- MACHINE = "qemux86-64"
- DEFAULTTUNE = "x86-64-x32"
- baselib = "${@d.getVar('BASE_LIB_tune-' + (d.getVar('DEFAULTTUNE', True) \
- or 'INVALID'), True) or 'lib'}"
- #MACHINE = "genericx86"
- #DEFAULTTUNE = "core2-64-x32"
- </literallayout></para></listitem>
- <listitem><para>As usual, use BitBake to build an image that supports the x32 psABI.
- Here is an example:
- <literallayout class='monospaced'>
- $ bitbake core-image-sato
- </literallayout></para></listitem>
- <listitem><para>As usual, run your image using QEMU:
- <literallayout class='monospaced'>
- $ runqemu qemux86-64 core-image-sato
- </literallayout></para></listitem>
- </itemizedlist>
- </para>
- </section>
-</section>
-
-<section id="wayland">
- <title>Wayland</title>
-
- <para>
- <ulink url='http://en.wikipedia.org/wiki/Wayland_(display_server_protocol)'>Wayland</ulink>
- is a computer display server protocol that
- provides a method for compositing window managers to communicate
- directly with applications and video hardware and expects them to
- communicate with input hardware using other libraries.
- Using Wayland with supporting targets can result in better control
- over graphics frame rendering than an application might otherwise
- achieve.
- </para>
-
- <para>
- The Yocto Project provides the Wayland protocol libraries and the
- reference
- <ulink url='http://en.wikipedia.org/wiki/Wayland_(display_server_protocol)#Weston'>Weston</ulink>
- compositor as part of its release.
- This section describes what you need to do to implement Wayland and
- use the compositor when building an image for a supporting target.
- </para>
-
- <section id="wayland-support">
- <title>Support</title>
-
- <para>
- The Wayland protocol libraries and the reference Weston compositor
- ship as integrated packages in the <filename>meta</filename> layer
- of the
- <ulink url='&YOCTO_DOCS_DEV_URL;#source-directory'>Source Directory</ulink>.
- Specifically, you can find the recipes that build both Wayland
- and Weston at <filename>meta/recipes-graphics/wayland</filename>.
- </para>
-
- <para>
- You can build both the Wayland and Weston packages for use only
- with targets that accept the
- <ulink url='http://dri.freedesktop.org/wiki/'>Mesa 3D and Direct Rendering Infrastructure</ulink>,
- which is also known as Mesa DRI.
- This implies that you cannot build and use the packages if your
- target uses, for example, the
- <trademark class='registered'>Intel</trademark> Embedded Media and
- Graphics Driver (<trademark class='registered'>Intel</trademark>
- EMGD) that overrides Mesa DRI.
- </para>
-
- <note>
- Due to lack of EGL support, Weston 1.0.3 will not run directly on
- the emulated QEMU hardware.
- However, this version of Weston will run under X emulation without
- issues.
- </note>
- </section>
-
- <section id="enabling-wayland-in-an-image">
- <title>Enabling Wayland in an Image</title>
-
- <para>
- To enable Wayland, you need to enable it to be built and enable
- it to be included in the image.
- </para>
-
- <section id="enable-building">
- <title>Building</title>
-
- <para>
- To cause Mesa to build the <filename>wayland-egl</filename>
- platform and Weston to build Wayland with Kernel Mode
- Setting
- (<ulink url='https://wiki.archlinux.org/index.php/Kernel_Mode_Setting'>KMS</ulink>)
- support, include the "wayland" flag in the
- <link linkend="var-DISTRO_FEATURES"><filename>DISTRO_FEATURES</filename></link>
- statement in your <filename>local.conf</filename> file:
- <literallayout class='monospaced'>
- DISTRO_FEATURES_append = " wayland"
- </literallayout>
- </para>
-
- <note>
- If X11 has been enabled elsewhere, Weston will build Wayland
- with X11 support
- </note>
- </section>
-
- <section id="enable-installation-in-an-image">
- <title>Installing</title>
-
- <para>
- To install the Wayland feature into an image, you must
- include the following
- <link linkend='var-CORE_IMAGE_EXTRA_INSTALL'><filename>CORE_IMAGE_EXTRA_INSTALL</filename></link>
- statement in your <filename>local.conf</filename> file:
- <literallayout class='monospaced'>
- CORE_IMAGE_EXTRA_INSTALL += "wayland weston"
- </literallayout>
- </para>
- </section>
- </section>
-
- <section id="running-weston">
- <title>Running Weston</title>
-
- <para>
- To run Weston inside X11, enabling it as described earlier and
- building a Sato image is sufficient.
- If you are running your image under Sato, a Weston Launcher appears
- in the "Utility" category.
- </para>
-
- <para>
- Alternatively, you can run Weston through the command-line
- interpretor (CLI), which is better suited for development work.
- To run Weston under the CLI, you need to do the following after
- your image is built:
- <orderedlist>
- <listitem><para>Run these commands to export
- <filename>XDG_RUNTIME_DIR</filename>:
- <literallayout class='monospaced'>
- mkdir -p /tmp/$USER-weston
- chmod 0700 /tmp/$USER-weston
- export XDG_RUNTIME_DIR=/tmp/$USER-weston
- </literallayout></para></listitem>
- <listitem><para>Launch Weston in the shell:
- <literallayout class='monospaced'>
- weston
- </literallayout></para></listitem>
- </orderedlist>
- </para>
- </section>
-</section>
-
-<section id="licenses">
- <title>Licenses</title>
-
- <para>
- This section describes the mechanism by which the OpenEmbedded build system
- tracks changes to licensing text.
- The section also describes how to enable commercially licensed recipes,
- which by default are disabled.
- </para>
-
- <para>
- For information that can help you maintain compliance with various open
- source licensing during the lifecycle of the product, see the
- "<ulink url='&YOCTO_DOCS_DEV_URL;#maintaining-open-source-license-compliance-during-your-products-lifecycle'>Maintaining Open Source License Compliance During Your Project's Lifecycle</ulink>" section
- in the Yocto Project Development Manual.
- </para>
-
- <section id="usingpoky-configuring-LIC_FILES_CHKSUM">
- <title>Tracking License Changes</title>
-
- <para>
- The license of an upstream project might change in the future.
- In order to prevent these changes going unnoticed, the
- <filename><link linkend='var-LIC_FILES_CHKSUM'>LIC_FILES_CHKSUM</link></filename>
- variable tracks changes to the license text. The checksums are validated at the end of the
- configure step, and if the checksums do not match, the build will fail.
- </para>
-
- <section id="usingpoky-specifying-LIC_FILES_CHKSUM">
- <title>Specifying the <filename>LIC_FILES_CHKSUM</filename> Variable</title>
-
- <para>
- The <filename>LIC_FILES_CHKSUM</filename>
- variable contains checksums of the license text in the source code for the recipe.
- Following is an example of how to specify <filename>LIC_FILES_CHKSUM</filename>:
- <literallayout class='monospaced'>
- LIC_FILES_CHKSUM = "file://COPYING;md5=xxxx \
- file://licfile1.txt;beginline=5;endline=29;md5=yyyy \
- file://licfile2.txt;endline=50;md5=zzzz \
- ..."
- </literallayout>
- </para>
-
- <para>
- The build system uses the
- <filename><link linkend='var-S'>S</link></filename> variable as
- the default directory when searching files listed in
- <filename>LIC_FILES_CHKSUM</filename>.
- The previous example employs the default directory.
- </para>
-
- <para>
- Consider this next example:
- <literallayout class='monospaced'>
- LIC_FILES_CHKSUM = "file://src/ls.c;beginline=5;endline=16;\
- md5=bb14ed3c4cda583abc85401304b5cd4e"
- LIC_FILES_CHKSUM = "file://${WORKDIR}/license.html;md5=5c94767cedb5d6987c902ac850ded2c6"
- </literallayout>
- </para>
-
- <para>
- The first line locates a file in
- <filename>${S}/src/ls.c</filename>.
- The second line refers to a file in
- <filename><link linkend='var-WORKDIR'>WORKDIR</link></filename>.
- </para>
- <para>
- Note that <filename>LIC_FILES_CHKSUM</filename> variable is
- mandatory for all recipes, unless the
- <filename>LICENSE</filename> variable is set to "CLOSED".
- </para>
- </section>
-
- <section id="usingpoky-LIC_FILES_CHKSUM-explanation-of-syntax">
- <title>Explanation of Syntax</title>
- <para>
- As mentioned in the previous section, the
- <filename>LIC_FILES_CHKSUM</filename> variable lists all the
- important files that contain the license text for the source code.
- It is possible to specify a checksum for an entire file, or a specific section of a
- file (specified by beginning and ending line numbers with the "beginline" and "endline"
- parameters, respectively).
- The latter is useful for source files with a license notice header,
- README documents, and so forth.
- If you do not use the "beginline" parameter, then it is assumed that the text begins on the
- first line of the file.
- Similarly, if you do not use the "endline" parameter, it is assumed that the license text
- ends with the last line of the file.
- </para>
-
- <para>
- The "md5" parameter stores the md5 checksum of the license text.
- If the license text changes in any way as compared to this parameter
- then a mismatch occurs.
- This mismatch triggers a build failure and notifies the developer.
- Notification allows the developer to review and address the license text changes.
- Also note that if a mismatch occurs during the build, the correct md5
- checksum is placed in the build log and can be easily copied to the recipe.
- </para>
-
- <para>
- There is no limit to how many files you can specify using the
- <filename>LIC_FILES_CHKSUM</filename> variable.
- Generally, however, every project requires a few specifications for license tracking.
- Many projects have a "COPYING" file that stores the license information for all the source
- code files.
- This practice allows you to just track the "COPYING" file as long as it is kept up to date.
- </para>
-
- <tip>
- If you specify an empty or invalid "md5" parameter, BitBake returns an md5 mis-match
- error and displays the correct "md5" parameter value during the build.
- The correct parameter is also captured in the build log.
- </tip>
-
- <tip>
- If the whole file contains only license text, you do not need to use the "beginline" and
- "endline" parameters.
- </tip>
- </section>
- </section>
-
- <section id="enabling-commercially-licensed-recipes">
- <title>Enabling Commercially Licensed Recipes</title>
-
- <para>
- By default, the OpenEmbedded build system disables
- components that have commercial or other special licensing
- requirements.
- Such requirements are defined on a
- recipe-by-recipe basis through the
- <link linkend='var-LICENSE_FLAGS'><filename>LICENSE_FLAGS</filename></link>
- variable definition in the affected recipe.
- For instance, the
- <filename>poky/meta/recipes-multimedia/gstreamer/gst-plugins-ugly</filename>
- recipe contains the following statement:
- <literallayout class='monospaced'>
- LICENSE_FLAGS = "commercial"
- </literallayout>
- Here is a slightly more complicated example that contains both an
- explicit recipe name and version (after variable expansion):
- <literallayout class='monospaced'>
- LICENSE_FLAGS = "license_${PN}_${PV}"
- </literallayout>
- In order for a component restricted by a <filename>LICENSE_FLAGS</filename>
- definition to be enabled and included in an image, it
- needs to have a matching entry in the global
- <link linkend='var-LICENSE_FLAGS_WHITELIST'><filename>LICENSE_FLAGS_WHITELIST</filename></link>
- variable, which is a variable
- typically defined in your <filename>local.conf</filename> file.
- For example, to enable
- the <filename>poky/meta/recipes-multimedia/gstreamer/gst-plugins-ugly</filename>
- package, you could add either the string
- "commercial_gst-plugins-ugly" or the more general string
- "commercial" to <filename>LICENSE_FLAGS_WHITELIST</filename>.
- See the
- "<link linkend='license-flag-matching'>License Flag Matching</link>" section
- for a full explanation of how <filename>LICENSE_FLAGS</filename> matching works.
- Here is the example:
- <literallayout class='monospaced'>
- LICENSE_FLAGS_WHITELIST = "commercial_gst-plugins-ugly"
- </literallayout>
- Likewise, to additionally enable the package built from the recipe containing
- <filename>LICENSE_FLAGS = "license_${PN}_${PV}"</filename>, and assuming
- that the actual recipe name was <filename>emgd_1.10.bb</filename>,
- the following string would enable that package as well as
- the original <filename>gst-plugins-ugly</filename> package:
- <literallayout class='monospaced'>
- LICENSE_FLAGS_WHITELIST = "commercial_gst-plugins-ugly license_emgd_1.10"
- </literallayout>
- As a convenience, you do not need to specify the complete license string
- in the whitelist for every package.
- You can use an abbreviated form, which consists
- of just the first portion or portions of the license string before
- the initial underscore character or characters.
- A partial string will match
- any license that contains the given string as the first
- portion of its license.
- For example, the following
- whitelist string will also match both of the packages
- previously mentioned as well as any other packages that have
- licenses starting with "commercial" or "license".
- <literallayout class='monospaced'>
- LICENSE_FLAGS_WHITELIST = "commercial license"
- </literallayout>
- </para>
-
- <section id="license-flag-matching">
- <title>License Flag Matching</title>
-
- <para>
- License flag matching allows you to control what recipes the
- OpenEmbedded build system includes in the build.
- Fundamentally, the build system attempts to match
- <link linkend='var-LICENSE_FLAGS'><filename>LICENSE_FLAGS</filename></link>
- strings found in recipes against
- <link linkend='var-LICENSE_FLAGS_WHITELIST'><filename>LICENSE_FLAGS_WHITELIST</filename></link>
- strings found in the whitelist.
- A match causes the build system to include a recipe in the
- build, while failure to find a match causes the build system to
- exclude a recipe.
- </para>
-
- <para>
- In general, license flag matching is simple.
- However, understanding some concepts will help you
- correctly and effectively use matching.
- </para>
-
- <para>
- Before a flag
- defined by a particular recipe is tested against the
- contents of the whitelist, the expanded string
- <filename>_${PN}</filename> is appended to the flag.
- This expansion makes each <filename>LICENSE_FLAGS</filename>
- value recipe-specific.
- After expansion, the string is then matched against the
- whitelist.
- Thus, specifying
- <filename>LICENSE_FLAGS = "commercial"</filename>
- in recipe "foo", for example, results in the string
- <filename>"commercial_foo"</filename>.
- And, to create a match, that string must appear in the
- whitelist.
- </para>
-
- <para>
- Judicious use of the <filename>LICENSE_FLAGS</filename>
- strings and the contents of the
- <filename>LICENSE_FLAGS_WHITELIST</filename> variable
- allows you a lot of flexibility for including or excluding
- recipes based on licensing.
- For example, you can broaden the matching capabilities by
- using license flags string subsets in the whitelist.
- <note>When using a string subset, be sure to use the part of
- the expanded string that precedes the appended underscore
- character (e.g. <filename>usethispart_1.3</filename>,
- <filename>usethispart_1.4</filename>, and so forth).
- </note>
- For example, simply specifying the string "commercial" in
- the whitelist matches any expanded
- <filename>LICENSE_FLAGS</filename> definition that starts with
- the string "commercial" such as "commercial_foo" and
- "commercial_bar", which are the strings the build system
- automatically generates for hypothetical recipes named
- "foo" and "bar" assuming those recipes simply specify the
- following:
- <literallayout class='monospaced'>
- LICENSE_FLAGS = "commercial"
- </literallayout>
- Thus, you can choose to exhaustively
- enumerate each license flag in the whitelist and
- allow only specific recipes into the image, or
- you can use a string subset that causes a broader range of
- matches to allow a range of recipes into the image.
- </para>
-
- <para>
- This scheme works even if the
- <filename>LICENSE_FLAGS</filename> string already
- has <filename>_${PN}</filename> appended.
- For example, the build system turns the license flag
- "commercial_1.2_foo" into "commercial_1.2_foo_foo" and would
- match both the general "commercial" and the specific
- "commercial_1.2_foo" strings found in the whitelist, as
- expected.
- </para>
-
- <para>
- Here are some other scenarios:
- <itemizedlist>
- <listitem><para>You can specify a versioned string in the
- recipe such as "commercial_foo_1.2" in a "foo" recipe.
- The build system expands this string to
- "commercial_foo_1.2_foo".
- Combine this license flag with a whitelist that has
- the string "commercial" and you match the flag along
- with any other flag that starts with the string
- "commercial".</para></listitem>
- <listitem><para>Under the same circumstances, you can
- use "commercial_foo" in the whitelist and the
- build system not only matches "commercial_foo_1.2" but
- also matches any license flag with the string
- "commercial_foo", regardless of the version.
- </para></listitem>
- <listitem><para>You can be very specific and use both the
- package and version parts in the whitelist (e.g.
- "commercial_foo_1.2") to specifically match a
- versioned recipe.</para></listitem>
- </itemizedlist>
- </para>
- </section>
-
- <section id="other-variables-related-to-commercial-licenses">
- <title>Other Variables Related to Commercial Licenses</title>
-
- <para>
- Other helpful variables related to commercial
- license handling exist and are defined in the
- <filename>poky/meta/conf/distro/include/default-distrovars.inc</filename> file:
- <literallayout class='monospaced'>
- COMMERCIAL_AUDIO_PLUGINS ?= ""
- COMMERCIAL_VIDEO_PLUGINS ?= ""
- </literallayout>
- If you want to enable these components, you can do so by making sure you have
- statements similar to the following
- in your <filename>local.conf</filename> configuration file:
- <literallayout class='monospaced'>
- COMMERCIAL_AUDIO_PLUGINS = "gst-plugins-ugly-mad \
- gst-plugins-ugly-mpegaudioparse"
- COMMERCIAL_VIDEO_PLUGINS = "gst-plugins-ugly-mpeg2dec \
- gst-plugins-ugly-mpegstream gst-plugins-bad-mpegvideoparse"
- LICENSE_FLAGS_WHITELIST = "commercial_gst-plugins-ugly commercial_gst-plugins-bad commercial_qmmp"
- </literallayout>
- Of course, you could also create a matching whitelist
- for those components using the more general "commercial"
- in the whitelist, but that would also enable all the
- other packages with
- <link linkend='var-LICENSE_FLAGS'><filename>LICENSE_FLAGS</filename></link>
- containing "commercial", which you may or may not want:
- <literallayout class='monospaced'>
- LICENSE_FLAGS_WHITELIST = "commercial"
- </literallayout>
- </para>
-
- <para>
- Specifying audio and video plug-ins as part of the
- <filename>COMMERCIAL_AUDIO_PLUGINS</filename> and
- <filename>COMMERCIAL_VIDEO_PLUGINS</filename> statements
- (along with the enabling
- <filename>LICENSE_FLAGS_WHITELIST</filename>) includes the
- plug-ins or components into built images, thus adding
- support for media formats or components.
- </para>
- </section>
- </section>
-</section>
-</chapter>
-<!--
-vim: expandtab tw=80 ts=4
--->