diff options
Diffstat (limited to 'yocto-poky/documentation/ref-manual/closer-look.xml')
-rw-r--r-- | yocto-poky/documentation/ref-manual/closer-look.xml | 1426 |
1 files changed, 0 insertions, 1426 deletions
diff --git a/yocto-poky/documentation/ref-manual/closer-look.xml b/yocto-poky/documentation/ref-manual/closer-look.xml deleted file mode 100644 index 84ff584ba..000000000 --- a/yocto-poky/documentation/ref-manual/closer-look.xml +++ /dev/null @@ -1,1426 +0,0 @@ -<!DOCTYPE chapter PUBLIC "-//OASIS//DTD DocBook XML V4.2//EN" -"http://www.oasis-open.org/docbook/xml/4.2/docbookx.dtd" -[<!ENTITY % poky SYSTEM "../poky.ent"> %poky; ] > - -<chapter id='closer-look'> -<title>A Closer Look at the Yocto Project Development Environment</title> - - <para> - This chapter takes a more detailed look at the Yocto Project - development environment. - The following diagram represents the development environment at a - high level. - The remainder of this chapter expands on the fundamental input, output, - process, and - <ulink url='&YOCTO_DOCS_DEV_URL;#metadata'>Metadata</ulink>) blocks - in the Yocto Project development environment. - </para> - - <para id='general-yocto-environment-figure'> - <imagedata fileref="figures/yocto-environment-ref.png" align="center" width="8in" depth="4.25in" /> - </para> - - <para> - The generalized Yocto Project Development Environment consists of - several functional areas: - <itemizedlist> - <listitem><para><emphasis>User Configuration:</emphasis> - Metadata you can use to control the build process. - </para></listitem> - <listitem><para><emphasis>Metadata Layers:</emphasis> - Various layers that provide software, machine, and - distro Metadata.</para></listitem> - <listitem><para><emphasis>Source Files:</emphasis> - Upstream releases, local projects, and SCMs.</para></listitem> - <listitem><para><emphasis>Build System:</emphasis> - Processes under the control of - <ulink url='&YOCTO_DOCS_DEV_URL;#bitbake-term'>BitBake</ulink>. - This block expands on how BitBake fetches source, applies - patches, completes compilation, analyzes output for package - generation, creates and tests packages, generates images, and - generates cross-development tools.</para></listitem> - <listitem><para><emphasis>Package Feeds:</emphasis> - Directories containing output packages (RPM, DEB or IPK), - which are subsequently used in the construction of an image or - SDK, produced by the build system. - These feeds can also be copied and shared using a web server or - other means to facilitate extending or updating existing - images on devices at runtime if runtime package management is - enabled.</para></listitem> - <listitem><para><emphasis>Images:</emphasis> - Images produced by the development process. - </para></listitem> - <listitem><para><emphasis>Application Development SDK:</emphasis> - Cross-development tools that are produced along with an image - or separately with BitBake.</para></listitem> - </itemizedlist> - </para> - - <section id="user-configuration"> - <title>User Configuration</title> - - <para> - User configuration helps define the build. - Through user configuration, you can tell BitBake the - target architecture for which you are building the image, - where to store downloaded source, and other build properties. - </para> - - <para> - The following figure shows an expanded representation of the - "User Configuration" box of the - <link linkend='general-yocto-environment-figure'>general Yocto Project Development Environment figure</link>: - </para> - - <para> - <imagedata fileref="figures/user-configuration.png" align="center" /> - </para> - - <para> - BitBake needs some basic configuration files in order to complete - a build. - These files are <filename>*.conf</filename> files. - The minimally necessary ones reside as example files in the - <ulink url='&YOCTO_DOCS_DEV_URL;#source-directory'>Source Directory</ulink>. - For simplicity, this section refers to the Source Directory as - the "Poky Directory." - </para> - - <para> - When you clone the <filename>poky</filename> Git repository or you - download and unpack a Yocto Project release, you can set up the - Source Directory to be named anything you want. - For this discussion, the cloned repository uses the default - name <filename>poky</filename>. - <note> - The Poky repository is primarily an aggregation of existing - repositories. - It is not a canonical upstream source. - </note> - </para> - - <para> - The <filename>meta-poky</filename> layer inside Poky contains - a <filename>conf</filename> directory that has example - configuration files. - These example files are used as a basis for creating actual - configuration files when you source the build environment - script - (i.e. - <link linkend='structure-core-script'><filename>&OE_INIT_FILE;</filename></link> - or - <link linkend='structure-memres-core-script'><filename>oe-init-build-env-memres</filename></link>). - </para> - - <para> - Sourcing the build environment script creates a - <ulink url='&YOCTO_DOCS_DEV_URL;#build-directory'>Build Directory</ulink> - if one does not already exist. - BitBake uses the Build Directory for all its work during builds. - The Build Directory has a <filename>conf</filename> directory that - contains default versions of your <filename>local.conf</filename> - and <filename>bblayers.conf</filename> configuration files. - These default configuration files are created only if versions - do not already exist in the Build Directory at the time you - source the build environment setup script. - </para> - - <para> - Because the Poky repository is fundamentally an aggregation of - existing repositories, some users might be familiar with running - the <filename>&OE_INIT_FILE;</filename> or - <filename>oe-init-build-env-memres</filename> script in the context - of separate OpenEmbedded-Core and BitBake repositories rather than a - single Poky repository. - This discussion assumes the script is executed from within a cloned - or unpacked version of Poky. - </para> - - <para> - Depending on where the script is sourced, different sub-scripts - are called to set up the Build Directory (Yocto or OpenEmbedded). - Specifically, the script - <filename>scripts/oe-setup-builddir</filename> inside the - poky directory sets up the Build Directory and seeds the directory - (if necessary) with configuration files appropriate for the - Yocto Project development environment. - <note> - The <filename>scripts/oe-setup-builddir</filename> script - uses the <filename>$TEMPLATECONF</filename> variable to - determine which sample configuration files to locate. - </note> - </para> - - <para> - The <filename>local.conf</filename> file provides many - basic variables that define a build environment. - Here is a list of a few. - To see the default configurations in a <filename>local.conf</filename> - file created by the build environment script, see the - <filename>local.conf.sample</filename> in the - <filename>meta-poky</filename> layer: - <itemizedlist> - <listitem><para><emphasis>Parallelism Options:</emphasis> - Controlled by the - <link linkend='var-BB_NUMBER_THREADS'><filename>BB_NUMBER_THREADS</filename></link>, - <link linkend='var-PARALLEL_MAKE'><filename>PARALLEL_MAKE</filename></link>, - and - <ulink url='&YOCTO_DOCS_BB_URL;#var-BB_NUMBER_PARSE_THREADS'><filename>BB_NUMBER_PARSE_THREADS</filename></ulink> - variables.</para></listitem> - <listitem><para><emphasis>Target Machine Selection:</emphasis> - Controlled by the - <link linkend='var-MACHINE'><filename>MACHINE</filename></link> - variable.</para></listitem> - <listitem><para><emphasis>Download Directory:</emphasis> - Controlled by the - <link linkend='var-DL_DIR'><filename>DL_DIR</filename></link> - variable.</para></listitem> - <listitem><para><emphasis>Shared State Directory:</emphasis> - Controlled by the - <link linkend='var-SSTATE_DIR'><filename>SSTATE_DIR</filename></link> - variable.</para></listitem> - <listitem><para><emphasis>Build Output:</emphasis> - Controlled by the - <link linkend='var-TMPDIR'><filename>TMPDIR</filename></link> - variable.</para></listitem> - </itemizedlist> - <note> - Configurations set in the <filename>conf/local.conf</filename> - file can also be set in the - <filename>conf/site.conf</filename> and - <filename>conf/auto.conf</filename> configuration files. - </note> - </para> - - <para> - The <filename>bblayers.conf</filename> file tells BitBake what - layers you want considered during the build. - By default, the layers listed in this file include layers - minimally needed by the build system. - However, you must manually add any custom layers you have created. - You can find more information on working with the - <filename>bblayers.conf</filename> file in the - "<ulink url='&YOCTO_DOCS_DEV_URL;#enabling-your-layer'>Enabling Your Layer</ulink>" - section in the Yocto Project Development Manual. - </para> - - <para> - The files <filename>site.conf</filename> and - <filename>auto.conf</filename> are not created by the environment - initialization script. - If you want the <filename>site.conf</filename> file, you need to - create that yourself. - The <filename>auto.conf</filename> file is typically created by - an autobuilder: - <itemizedlist> - <listitem><para><emphasis><filename>site.conf</filename>:</emphasis> - You can use the <filename>conf/site.conf</filename> - configuration file to configure multiple build directories. - For example, suppose you had several build environments and - they shared some common features. - You can set these default build properties here. - A good example is perhaps the packaging format to use - through the - <link linkend='var-PACKAGE_CLASSES'><filename>PACKAGE_CLASSES</filename></link> - variable.</para> - <para>One useful scenario for using the - <filename>conf/site.conf</filename> file is to extend your - <link linkend='var-BBPATH'><filename>BBPATH</filename></link> - variable to include the path to a - <filename>conf/site.conf</filename>. - Then, when BitBake looks for Metadata using - <filename>BBPATH</filename>, it finds the - <filename>conf/site.conf</filename> file and applies your - common configurations found in the file. - To override configurations in a particular build directory, - alter the similar configurations within that build - directory's <filename>conf/local.conf</filename> file. - </para></listitem> - <listitem><para><emphasis><filename>auto.conf</filename>:</emphasis> - The file is usually created and written to by - an autobuilder. - The settings put into the file are typically the same as - you would find in the <filename>conf/local.conf</filename> - or the <filename>conf/site.conf</filename> files. - </para></listitem> - </itemizedlist> - </para> - - <para> - You can edit all configuration files to further define - any particular build environment. - This process is represented by the "User Configuration Edits" - box in the figure. - </para> - - <para> - When you launch your build with the - <filename>bitbake <replaceable>target</replaceable></filename> - command, BitBake sorts out the configurations to ultimately - define your build environment. - It is important to understand that the OpenEmbedded build system - reads the configuration files in a specific order: - <filename>site.conf</filename>, <filename>auto.conf</filename>, - and <filename>local.conf</filename>. - And, the build system applies the normal assignment statement - rules. - Because the files are parsed in a specific order, variable - assignments for the same variable could be affected. - For example, if the <filename>auto.conf</filename> file and - the <filename>local.conf</filename> set - <replaceable>variable1</replaceable> to different values, because - the build system parses <filename>local.conf</filename> after - <filename>auto.conf</filename>, - <replaceable>variable1</replaceable> is assigned the value from - the <filename>local.conf</filename> file. - </para> - </section> - - <section id="metadata-machine-configuration-and-policy-configuration"> - <title>Metadata, Machine Configuration, and Policy Configuration</title> - - <para> - The previous section described the user configurations that - define BitBake's global behavior. - This section takes a closer look at the layers the build system - uses to further control the build. - These layers provide Metadata for the software, machine, and - policy. - </para> - - <para> - In general, three types of layer input exist: - <itemizedlist> - <listitem><para><emphasis>Policy Configuration:</emphasis> - Distribution Layers provide top-level or general - policies for the image or SDK being built. - For example, this layer would dictate whether BitBake - produces RPM or IPK packages.</para></listitem> - <listitem><para><emphasis>Machine Configuration:</emphasis> - Board Support Package (BSP) layers provide machine - configurations. - This type of information is specific to a particular - target architecture.</para></listitem> - <listitem><para><emphasis>Metadata:</emphasis> - Software layers contain user-supplied recipe files, - patches, and append files. - </para></listitem> - </itemizedlist> - </para> - - <para> - The following figure shows an expanded representation of the - Metadata, Machine Configuration, and Policy Configuration input - (layers) boxes of the - <link linkend='general-yocto-environment-figure'>general Yocto Project Development Environment figure</link>: - </para> - - <para> - <imagedata fileref="figures/layer-input.png" align="center" width="8in" depth="7.5in" /> - </para> - - <para> - In general, all layers have a similar structure. - They all contain a licensing file - (e.g. <filename>COPYING</filename>) if the layer is to be - distributed, a <filename>README</filename> file as good practice - and especially if the layer is to be distributed, a - configuration directory, and recipe directories. - </para> - - <para> - The Yocto Project has many layers that can be used. - You can see a web-interface listing of them on the - <ulink url="http://git.yoctoproject.org/">Source Repositories</ulink> - page. - The layers are shown at the bottom categorized under - "Yocto Metadata Layers." - These layers are fundamentally a subset of the - <ulink url="http://layers.openembedded.org/layerindex/layers/">OpenEmbedded Metadata Index</ulink>, - which lists all layers provided by the OpenEmbedded community. - <note> - Layers exist in the Yocto Project Source Repositories that - cannot be found in the OpenEmbedded Metadata Index. - These layers are either deprecated or experimental in nature. - </note> - </para> - - <para> - BitBake uses the <filename>conf/bblayers.conf</filename> file, - which is part of the user configuration, to find what layers it - should be using as part of the build. - </para> - - <para> - For more information on layers, see the - "<ulink url='&YOCTO_DOCS_DEV_URL;#understanding-and-creating-layers'>Understanding and Creating Layers</ulink>" - section in the Yocto Project Development Manual. - </para> - - <section id="distro-layer"> - <title>Distro Layer</title> - - <para> - The distribution layer provides policy configurations for your - distribution. - Best practices dictate that you isolate these types of - configurations into their own layer. - Settings you provide in - <filename>conf/distro/<replaceable>distro</replaceable>.conf</filename> override - similar - settings that BitBake finds in your - <filename>conf/local.conf</filename> file in the Build - Directory. - </para> - - <para> - The following list provides some explanation and references - for what you typically find in the distribution layer: - <itemizedlist> - <listitem><para><emphasis>classes:</emphasis> - Class files (<filename>.bbclass</filename>) hold - common functionality that can be shared among - recipes in the distribution. - When your recipes inherit a class, they take on the - settings and functions for that class. - You can read more about class files in the - "<link linkend='ref-classes'>Classes</link>" section. - </para></listitem> - <listitem><para><emphasis>conf:</emphasis> - This area holds configuration files for the - layer (<filename>conf/layer.conf</filename>), - the distribution - (<filename>conf/distro/<replaceable>distro</replaceable>.conf</filename>), - and any distribution-wide include files. - </para></listitem> - <listitem><para><emphasis>recipes-*:</emphasis> - Recipes and append files that affect common - functionality across the distribution. - This area could include recipes and append files - to add distribution-specific configuration, - initialization scripts, custom image recipes, - and so forth.</para></listitem> - </itemizedlist> - </para> - </section> - - <section id="bsp-layer"> - <title>BSP Layer</title> - - <para> - The BSP Layer provides machine configurations. - Everything in this layer is specific to the machine for which - you are building the image or the SDK. - A common structure or form is defined for BSP layers. - You can learn more about this structure in the - <ulink url='&YOCTO_DOCS_BSP_URL;'>Yocto Project Board Support Package (BSP) Developer's Guide</ulink>. - <note> - In order for a BSP layer to be considered compliant with the - Yocto Project, it must meet some structural requirements. - </note> - </para> - - <para> - The BSP Layer's configuration directory contains - configuration files for the machine - (<filename>conf/machine/<replaceable>machine</replaceable>.conf</filename>) and, - of course, the layer (<filename>conf/layer.conf</filename>). - </para> - - <para> - The remainder of the layer is dedicated to specific recipes - by function: <filename>recipes-bsp</filename>, - <filename>recipes-core</filename>, - <filename>recipes-graphics</filename>, and - <filename>recipes-kernel</filename>. - Metadata can exist for multiple formfactors, graphics - support systems, and so forth. - <note> - While the figure shows several <filename>recipes-*</filename> - directories, not all these directories appear in all - BSP layers. - </note> - </para> - </section> - - <section id="software-layer"> - <title>Software Layer</title> - - <para> - The software layer provides the Metadata for additional - software packages used during the build. - This layer does not include Metadata that is specific to the - distribution or the machine, which are found in their - respective layers. - </para> - - <para> - This layer contains any new recipes that your project needs - in the form of recipe files. - </para> - </section> - </section> - - <section id="sources-dev-environment"> - <title>Sources</title> - - <para> - In order for the OpenEmbedded build system to create an image or - any target, it must be able to access source files. - The - <link linkend='general-yocto-environment-figure'>general Yocto Project Development Environment figure</link> - represents source files using the "Upstream Project Releases", - "Local Projects", and "SCMs (optional)" boxes. - The figure represents mirrors, which also play a role in locating - source files, with the "Source Mirror(s)" box. - </para> - - <para> - The method by which source files are ultimately organized is - a function of the project. - For example, for released software, projects tend to use tarballs - or other archived files that can capture the state of a release - guaranteeing that it is statically represented. - On the other hand, for a project that is more dynamic or - experimental in nature, a project might keep source files in a - repository controlled by a Source Control Manager (SCM) such as - Git. - Pulling source from a repository allows you to control - the point in the repository (the revision) from which you want to - build software. - Finally, a combination of the two might exist, which would give the - consumer a choice when deciding where to get source files. - </para> - - <para> - BitBake uses the - <link linkend='var-SRC_URI'><filename>SRC_URI</filename></link> - variable to point to source files regardless of their location. - Each recipe must have a <filename>SRC_URI</filename> variable - that points to the source. - </para> - - <para> - Another area that plays a significant role in where source files - come from is pointed to by the - <link linkend='var-DL_DIR'><filename>DL_DIR</filename></link> - variable. - This area is a cache that can hold previously downloaded source. - You can also instruct the OpenEmbedded build system to create - tarballs from Git repositories, which is not the default behavior, - and store them in the <filename>DL_DIR</filename> by using the - <link linkend='var-BB_GENERATE_MIRROR_TARBALLS'><filename>BB_GENERATE_MIRROR_TARBALLS</filename></link> - variable. - </para> - - <para> - Judicious use of a <filename>DL_DIR</filename> directory can - save the build system a trip across the Internet when looking - for files. - A good method for using a download directory is to have - <filename>DL_DIR</filename> point to an area outside of your - Build Directory. - Doing so allows you to safely delete the Build Directory - if needed without fear of removing any downloaded source file. - </para> - - <para> - The remainder of this section provides a deeper look into the - source files and the mirrors. - Here is a more detailed look at the source file area of the - base figure: - <imagedata fileref="figures/source-input.png" align="center" width="7in" depth="7.5in" /> - </para> - - <section id='upstream-project-releases'> - <title>Upstream Project Releases</title> - - <para> - Upstream project releases exist anywhere in the form of an - archived file (e.g. tarball or zip file). - These files correspond to individual recipes. - For example, the figure uses specific releases each for - BusyBox, Qt, and Dbus. - An archive file can be for any released product that can be - built using a recipe. - </para> - </section> - - <section id='local-projects'> - <title>Local Projects</title> - - <para> - Local projects are custom bits of software the user provides. - These bits reside somewhere local to a project - perhaps - a directory into which the user checks in items (e.g. - a local directory containing a development source tree - used by the group). - </para> - - <para> - The canonical method through which to include a local project - is to use the - <link linkend='ref-classes-externalsrc'><filename>externalsrc</filename></link> - class to include that local project. - You use either the <filename>local.conf</filename> or a - recipe's append file to override or set the - recipe to point to the local directory on your disk to pull - in the whole source tree. - </para> - - <para> - For information on how to use the - <filename>externalsrc</filename> class, see the - "<link linkend='ref-classes-externalsrc'><filename>externalsrc.bbclass</filename></link>" - section. - </para> - </section> - - <section id='scms'> - <title>Source Control Managers (Optional)</title> - - <para> - Another place the build system can get source files from is - through an SCM such as Git or Subversion. - In this case, a repository is cloned or checked out. - The - <link linkend='ref-tasks-fetch'><filename>do_fetch</filename></link> - task inside BitBake uses - the <link linkend='var-SRC_URI'><filename>SRC_URI</filename></link> - variable and the argument's prefix to determine the correct - fetcher module. - </para> - - <note> - For information on how to have the OpenEmbedded build system - generate tarballs for Git repositories and place them in the - <link linkend='var-DL_DIR'><filename>DL_DIR</filename></link> - directory, see the - <link linkend='var-BB_GENERATE_MIRROR_TARBALLS'><filename>BB_GENERATE_MIRROR_TARBALLS</filename></link> - variable. - </note> - - <para> - When fetching a repository, BitBake uses the - <link linkend='var-SRCREV'><filename>SRCREV</filename></link> - variable to determine the specific revision from which to - build. - </para> - </section> - - <section id='source-mirrors'> - <title>Source Mirror(s)</title> - - <para> - Two kinds of mirrors exist: pre-mirrors and regular mirrors. - The <link linkend='var-PREMIRRORS'><filename>PREMIRRORS</filename></link> - and - <link linkend='var-MIRRORS'><filename>MIRRORS</filename></link> - variables point to these, respectively. - BitBake checks pre-mirrors before looking upstream for any - source files. - Pre-mirrors are appropriate when you have a shared directory - that is not a directory defined by the - <link linkend='var-DL_DIR'><filename>DL_DIR</filename></link> - variable. - A Pre-mirror typically points to a shared directory that is - local to your organization. - </para> - - <para> - Regular mirrors can be any site across the Internet that is - used as an alternative location for source code should the - primary site not be functioning for some reason or another. - </para> - </section> - </section> - - <section id="package-feeds-dev-environment"> - <title>Package Feeds</title> - - <para> - When the OpenEmbedded build system generates an image or an SDK, - it gets the packages from a package feed area located in the - <ulink url='&YOCTO_DOCS_DEV_URL;#build-directory'>Build Directory</ulink>. - The - <link linkend='general-yocto-environment-figure'>general Yocto Project Development Environment figure</link> - shows this package feeds area in the upper-right corner. - </para> - - <para> - This section looks a little closer into the package feeds area used - by the build system. - Here is a more detailed look at the area: - <imagedata fileref="figures/package-feeds.png" align="center" width="7in" depth="6in" /> - </para> - - <para> - Package feeds are an intermediary step in the build process. - The OpenEmbedded build system provides classes to generate - different package types, and you specify which classes to enable - through the - <link linkend='var-PACKAGE_CLASSES'><filename>PACKAGE_CLASSES</filename></link> - variable. - Before placing the packages into package feeds, - the build process validates them with generated output quality - assurance checks through the - <link linkend='ref-classes-insane'><filename>insane</filename></link> - class. - </para> - - <para> - The package feed area resides in the Build Directory. - The directory the build system uses to temporarily store packages - is determined by a combination of variables and the particular - package manager in use. - See the "Package Feeds" box in the illustration and note the - information to the right of that area. - In particular, the following defines where package files are - kept: - <itemizedlist> - <listitem><para><link linkend='var-DEPLOY_DIR'><filename>DEPLOY_DIR</filename></link>: - Defined as <filename>tmp/deploy</filename> in the Build - Directory. - </para></listitem> - <listitem><para><filename>DEPLOY_DIR_*</filename>: - Depending on the package manager used, the package type - sub-folder. - Given RPM, IPK, or DEB packaging and tarball creation, the - <link linkend='var-DEPLOY_DIR_RPM'><filename>DEPLOY_DIR_RPM</filename></link>, - <link linkend='var-DEPLOY_DIR_IPK'><filename>DEPLOY_DIR_IPK</filename></link>, - <link linkend='var-DEPLOY_DIR_DEB'><filename>DEPLOY_DIR_DEB</filename></link>, - or - <link linkend='var-DEPLOY_DIR_TAR'><filename>DEPLOY_DIR_TAR</filename></link>, - variables are used, respectively. - </para></listitem> - <listitem><para><link linkend='var-PACKAGE_ARCH'><filename>PACKAGE_ARCH</filename></link>: - Defines architecture-specific sub-folders. - For example, packages could exist for the i586 or qemux86 - architectures. - </para></listitem> - </itemizedlist> - </para> - - <para> - BitBake uses the <filename>do_package_write_*</filename> tasks to - generate packages and place them into the package holding area (e.g. - <filename>do_package_write_ipk</filename> for IPK packages). - See the - "<link linkend='ref-tasks-package_write_deb'><filename>do_package_write_deb</filename></link>", - "<link linkend='ref-tasks-package_write_ipk'><filename>do_package_write_ipk</filename></link>", - "<link linkend='ref-tasks-package_write_rpm'><filename>do_package_write_rpm</filename></link>", - and - "<link linkend='ref-tasks-package_write_tar'><filename>do_package_write_tar</filename></link>" - sections for additional information. - As an example, consider a scenario where an IPK packaging manager - is being used and package architecture support for both i586 - and qemux86 exist. - Packages for the i586 architecture are placed in - <filename>build/tmp/deploy/ipk/i586</filename>, while packages for - the qemux86 architecture are placed in - <filename>build/tmp/deploy/ipk/qemux86</filename>. - </para> - </section> - - <section id='bitbake-dev-environment'> - <title>BitBake</title> - - <para> - The OpenEmbedded build system uses - <ulink url='&YOCTO_DOCS_DEV_URL;#bitbake-term'>BitBake</ulink> - to produce images. - You can see from the - <link linkend='general-yocto-environment-figure'>general Yocto Project Development Environment figure</link>, - the BitBake area consists of several functional areas. - This section takes a closer look at each of those areas. - </para> - - <para> - Separate documentation exists for the BitBake tool. - See the - <ulink url='&YOCTO_DOCS_BB_URL;#bitbake-user-manual'>BitBake User Manual</ulink> - for reference material on BitBake. - </para> - - <section id='source-fetching-dev-environment'> - <title>Source Fetching</title> - - <para> - The first stages of building a recipe are to fetch and unpack - the source code: - <imagedata fileref="figures/source-fetching.png" align="center" width="6.5in" depth="5in" /> - </para> - - <para> - The - <link linkend='ref-tasks-fetch'><filename>do_fetch</filename></link> - and - <link linkend='ref-tasks-unpack'><filename>do_unpack</filename></link> - tasks fetch the source files and unpack them into the work - directory. - <note> - For every local file (e.g. <filename>file://</filename>) - that is part of a recipe's - <link linkend='var-SRC_URI'><filename>SRC_URI</filename></link> - statement, the OpenEmbedded build system takes a checksum - of the file for the recipe and inserts the checksum into - the signature for the <filename>do_fetch</filename>. - If any local file has been modified, the - <filename>do_fetch</filename> task and all tasks that - depend on it are re-executed. - </note> - By default, everything is accomplished in the - <ulink url='&YOCTO_DOCS_DEV_URL;#build-directory'>Build Directory</ulink>, - which has a defined structure. - For additional general information on the Build Directory, - see the - "<link linkend='structure-core-build'><filename>build/</filename></link>" - section. - </para> - - <para> - Unpacked source files are pointed to by the - <link linkend='var-S'><filename>S</filename></link> variable. - Each recipe has an area in the Build Directory where the - unpacked source code resides. - The name of that directory for any given recipe is defined from - several different variables. - You can see the variables that define these directories - by looking at the figure: - <itemizedlist> - <listitem><para><link linkend='var-TMPDIR'><filename>TMPDIR</filename></link> - - The base directory where the OpenEmbedded build system - performs all its work during the build. - </para></listitem> - <listitem><para><link linkend='var-PACKAGE_ARCH'><filename>PACKAGE_ARCH</filename></link> - - The architecture of the built package or packages. - </para></listitem> - <listitem><para><link linkend='var-TARGET_OS'><filename>TARGET_OS</filename></link> - - The operating system of the target device. - </para></listitem> - <listitem><para><link linkend='var-PN'><filename>PN</filename></link> - - The name of the built package. - </para></listitem> - <listitem><para><link linkend='var-PV'><filename>PV</filename></link> - - The version of the recipe used to build the package. - </para></listitem> - <listitem><para><link linkend='var-PR'><filename>PR</filename></link> - - The revision of the recipe used to build the package. - </para></listitem> - <listitem><para><link linkend='var-WORKDIR'><filename>WORKDIR</filename></link> - - The location within <filename>TMPDIR</filename> where - a specific package is built. - </para></listitem> - <listitem><para><link linkend='var-S'><filename>S</filename></link> - - Contains the unpacked source files for a given recipe. - </para></listitem> - </itemizedlist> - </para> - </section> - - <section id='patching-dev-environment'> - <title>Patching</title> - - <para> - Once source code is fetched and unpacked, BitBake locates - patch files and applies them to the source files: - <imagedata fileref="figures/patching.png" align="center" width="6in" depth="5in" /> - </para> - - <para> - The - <link linkend='ref-tasks-patch'><filename>do_patch</filename></link> - task processes recipes by - using the - <link linkend='var-SRC_URI'><filename>SRC_URI</filename></link> - variable to locate applicable patch files, which by default - are <filename>*.patch</filename> or - <filename>*.diff</filename> files, or any file if - "apply=yes" is specified for the file in - <filename>SRC_URI</filename>. - </para> - - <para> - BitBake finds and applies multiple patches for a single recipe - in the order in which it finds the patches. - Patches are applied to the recipe's source files located in the - <link linkend='var-S'><filename>S</filename></link> directory. - </para> - - <para> - For more information on how the source directories are - created, see the - "<link linkend='source-fetching-dev-environment'>Source Fetching</link>" - section. - </para> - </section> - - <section id='configuration-and-compilation-dev-environment'> - <title>Configuration and Compilation</title> - - <para> - After source code is patched, BitBake executes tasks that - configure and compile the source code: - <imagedata fileref="figures/configuration-compile-autoreconf.png" align="center" width="7in" depth="5in" /> - </para> - - <para> - This step in the build process consists of three tasks: - <itemizedlist> - <listitem><para><emphasis><filename>do_configure</filename>:</emphasis> - This task configures the source by enabling and - disabling any build-time and configuration options for - the software being built. - Configurations can come from the recipe itself as well - as from an inherited class. - Additionally, the software itself might configure itself - depending on the target for which it is being built. - </para> - - <para>The configurations handled by the - <link linkend='ref-tasks-configure'><filename>do_configure</filename></link> - task are specific - to source code configuration for the source code - being built by the recipe.</para> - - <para>If you are using the - <link linkend='ref-classes-autotools'><filename>autotools</filename></link> - class, - you can add additional configuration options by using - the <link linkend='var-EXTRA_OECONF'><filename>EXTRA_OECONF</filename></link> - variable. - For information on how this variable works within - that class, see the - <filename>meta/classes/autotools.bbclass</filename> file. - </para></listitem> - <listitem><para><emphasis><filename>do_compile</filename>:</emphasis> - Once a configuration task has been satisfied, BitBake - compiles the source using the - <link linkend='ref-tasks-compile'><filename>do_compile</filename></link> - task. - Compilation occurs in the directory pointed to by the - <link linkend='var-B'><filename>B</filename></link> - variable. - Realize that the <filename>B</filename> directory is, by - default, the same as the - <link linkend='var-S'><filename>S</filename></link> - directory.</para></listitem> - <listitem><para><emphasis><filename>do_install</filename>:</emphasis> - Once compilation is done, BitBake executes the - <link linkend='ref-tasks-install'><filename>do_install</filename></link> - task. - This task copies files from the <filename>B</filename> - directory and places them in a holding area pointed to - by the - <link linkend='var-D'><filename>D</filename></link> - variable.</para></listitem> - </itemizedlist> - </para> - </section> - - <section id='package-splitting-dev-environment'> - <title>Package Splitting</title> - - <para> - After source code is configured and compiled, the - OpenEmbedded build system analyzes - the results and splits the output into packages: - <imagedata fileref="figures/analysis-for-package-splitting.png" align="center" width="7in" depth="7in" /> - </para> - - <para> - The - <link linkend='ref-tasks-package'><filename>do_package</filename></link> - and - <link linkend='ref-tasks-packagedata'><filename>do_packagedata</filename></link> - tasks combine to analyze - the files found in the - <link linkend='var-D'><filename>D</filename></link> directory - and split them into subsets based on available packages and - files. - The analyzing process involves the following as well as other - items: splitting out debugging symbols, - looking at shared library dependencies between packages, - and looking at package relationships. - The <filename>do_packagedata</filename> task creates package - metadata based on the analysis such that the - OpenEmbedded build system can generate the final packages. - Working, staged, and intermediate results of the analysis - and package splitting process use these areas: - <itemizedlist> - <listitem><para><link linkend='var-PKGD'><filename>PKGD</filename></link> - - The destination directory for packages before they are - split. - </para></listitem> - <listitem><para><link linkend='var-PKGDATA_DIR'><filename>PKGDATA_DIR</filename></link> - - A shared, global-state directory that holds data - generated during the packaging process. - </para></listitem> - <listitem><para><link linkend='var-PKGDESTWORK'><filename>PKGDESTWORK</filename></link> - - A temporary work area used by the - <filename>do_package</filename> task. - </para></listitem> - <listitem><para><link linkend='var-PKGDEST'><filename>PKGDEST</filename></link> - - The parent directory for packages after they have - been split. - </para></listitem> - </itemizedlist> - The <link linkend='var-FILES'><filename>FILES</filename></link> - variable defines the files that go into each package in - <link linkend='var-PACKAGES'><filename>PACKAGES</filename></link>. - If you want details on how this is accomplished, you can - look at the - <link linkend='ref-classes-package'><filename>package</filename></link> - class. - </para> - - <para> - Depending on the type of packages being created (RPM, DEB, or - IPK), the <filename>do_package_write_*</filename> task - creates the actual packages and places them in the - Package Feed area, which is - <filename>${TMPDIR}/deploy</filename>. - You can see the - "<link linkend='package-feeds-dev-environment'>Package Feeds</link>" - section for more detail on that part of the build process. - <note> - Support for creating feeds directly from the - <filename>deploy/*</filename> directories does not exist. - Creating such feeds usually requires some kind of feed - maintenance mechanism that would upload the new packages - into an official package feed (e.g. the - Ångström distribution). - This functionality is highly distribution-specific - and thus is not provided out of the box. - </note> - </para> - </section> - - <section id='image-generation-dev-environment'> - <title>Image Generation</title> - - <para> - Once packages are split and stored in the Package Feeds area, - the OpenEmbedded build system uses BitBake to generate the - root filesystem image: - <imagedata fileref="figures/image-generation.png" align="center" width="6in" depth="7in" /> - </para> - - <para> - The image generation process consists of several stages and - depends on several tasks and variables. - The - <link linkend='ref-tasks-rootfs'><filename>do_rootfs</filename></link> - task creates the root filesystem (file and directory structure) - for an image. - This task uses several key variables to help create the list - of packages to actually install: - <itemizedlist> - <listitem><para><link linkend='var-IMAGE_INSTALL'><filename>IMAGE_INSTALL</filename></link>: - Lists out the base set of packages to install from - the Package Feeds area.</para></listitem> - <listitem><para><link linkend='var-PACKAGE_EXCLUDE'><filename>PACKAGE_EXCLUDE</filename></link>: - Specifies packages that should not be installed. - </para></listitem> - <listitem><para><link linkend='var-IMAGE_FEATURES'><filename>IMAGE_FEATURES</filename></link>: - Specifies features to include in the image. - Most of these features map to additional packages for - installation.</para></listitem> - <listitem><para><link linkend='var-PACKAGE_CLASSES'><filename>PACKAGE_CLASSES</filename></link>: - Specifies the package backend to use and consequently - helps determine where to locate packages within the - Package Feeds area.</para></listitem> - <listitem><para><link linkend='var-IMAGE_LINGUAS'><filename>IMAGE_LINGUAS</filename></link>: - Determines the language(s) for which additional - language support packages are installed. - </para></listitem> - <listitem><para><link linkend='var-PACKAGE_INSTALL'><filename>PACKAGE_INSTALL</filename></link>: - The final list of packages passed to the package manager - for installation into the image. - </para></listitem> - </itemizedlist> - </para> - - <para> - With - <link linkend='var-IMAGE_ROOTFS'><filename>IMAGE_ROOTFS</filename></link> - pointing to the location of the filesystem under construction and - the <filename>PACKAGE_INSTALL</filename> variable providing the - final list of packages to install, the root file system is - created. - </para> - - <para> - Package installation is under control of the package manager - (e.g. smart/rpm, opkg, or apt/dpkg) regardless of whether or - not package management is enabled for the target. - At the end of the process, if package management is not - enabled for the target, the package manager's data files - are deleted from the root filesystem. - As part of the final stage of package installation, postinstall - scripts that are part of the packages are run. - Any scripts that fail to run - on the build host are run on the target when the target system - is first booted. - If you are using a - <ulink url='&YOCTO_DOCS_DEV_URL;#creating-a-read-only-root-filesystem'>read-only root filesystem</ulink>, - all the post installation scripts must succeed during the - package installation phase since the root filesystem is - read-only. - </para> - - <para> - The final stages of the <filename>do_rootfs</filename> task - handle post processing. - Post processing includes creation of a manifest file and - optimizations. - </para> - - <para> - The manifest file (<filename>.manifest</filename>) resides - in the same directory as the root filesystem image. - This file lists out, line-by-line, the installed packages. - The manifest file is useful for the - <link linkend='ref-classes-testimage*'><filename>testimage</filename></link> - class, for example, to determine whether or not to run - specific tests. - See the - <link linkend='var-IMAGE_MANIFEST'><filename>IMAGE_MANIFEST</filename></link> - variable for additional information. - </para> - - <para> - Optimizing processes run across the image include - <filename>mklibs</filename>, <filename>prelink</filename>, - and any other post-processing commands as defined by the - <link linkend='var-ROOTFS_POSTPROCESS_COMMAND'><filename>ROOTFS_POSTPROCESS_COMMAND</filename></link> - variable. - The <filename>mklibs</filename> process optimizes the size - of the libraries, while the - <filename>prelink</filename> process optimizes the dynamic - linking of shared libraries to reduce start up time of - executables. - </para> - - <para> - After the root filesystem is built, processing begins on - the image through the <filename>do_image</filename> task. - The build system runs any pre-processing commands as defined - by the - <link linkend='var-IMAGE_PREPROCESS_COMMAND'><filename>IMAGE_PREPROCESS_COMMAND</filename></link> - variable. - This variable specifies a list of functions to call before - the OpenEmbedded build system creates the final image output - files. - </para> - - <para> - The <filename>do_image</filename> task dynamically creates - other <filename>do_image_*</filename> tasks as needed, which - include compressing the root filesystem image to reduce the - overall size of the image. - The process turns everything into an image file or a set of - image files. - The formats used for the root filesystem depend on the - <link linkend='var-IMAGE_FSTYPES'><filename>IMAGE_FSTYPES</filename></link> - variable. - </para> - - <para> - The final task involved in image creation is the - <filename>do_image_complete</filename> task. - This task completes the image by applying any image - post processing as defined through the - <link linkend='var-IMAGE_POSTPROCESS_COMMAND'><filename>IMAGE_POSTPROCESS_COMMAND</filename></link> - variable. - The variable specifies a list of functions to call once the - OpenEmbedded build system has created the final image output - files. - </para> - - <note> - The entire image generation process is run under Pseudo. - Running under Pseudo ensures that the files in the root - filesystem have correct ownership. - </note> - </section> - - <section id='sdk-generation-dev-environment'> - <title>SDK Generation</title> - - <para> - The OpenEmbedded build system uses BitBake to generate the - Software Development Kit (SDK) installer script for both the - standard and extensible SDKs: - <imagedata fileref="figures/sdk-generation.png" align="center" /> - </para> - - <note> - For more information on the cross-development toolchain - generation, see the - "<link linkend='cross-development-toolchain-generation'>Cross-Development Toolchain Generation</link>" - section. - For information on advantages gained when building a - cross-development toolchain using the - <link linkend='ref-tasks-populate_sdk'><filename>do_populate_sdk</filename></link> - task, see the - "<ulink url='&YOCTO_DOCS_SDK_URL;#sdk-building-an-sdk-installer'>Building an SDK Installer</ulink>" - section in the Yocto Project Software Development Kit (SDK) - Developer's Guide. - </note> - - <para> - Like image generation, the SDK script process consists of - several stages and depends on many variables. - The <filename>do_populate_sdk</filename> and - <filename>do_populate_sdk_ext</filename> tasks use these - key variables to help create the list of packages to actually - install. - For information on the variables listed in the figure, see the - "<link linkend='sdk-dev-environment'>Application Development SDK</link>" - section. - </para> - - <para> - The <filename>do_populate_sdk</filename> task helps create - the standard SDK and handles two parts: a target part and a - host part. - The target part is the part built for the target hardware and - includes libraries and headers. - The host part is the part of the SDK that runs on the - <link linkend='var-SDKMACHINE'><filename>SDKMACHINE</filename></link>. - </para> - - <para> - The <filename>do_populate_sdk_ext</filename> task helps create - the extensible SDK and handles host and target parts - differently than its counter part does for the standard SDK. - For the extensible SDK, the task encapsulates the build system, - which includes everything needed (host and target) for the SDK. - </para> - - <para> - Regardless of the type of SDK being constructed, the - tasks perform some cleanup after which a cross-development - environment setup script and any needed configuration files - are created. - The final output is the Cross-development - toolchain installation script (<filename>.sh</filename> file), - which includes the environment setup script. - </para> - </section> - </section> - - <section id='images-dev-environment'> - <title>Images</title> - - <para> - The images produced by the OpenEmbedded build system - are compressed forms of the - root filesystem that are ready to boot on a target device. - You can see from the - <link linkend='general-yocto-environment-figure'>general Yocto Project Development Environment figure</link> - that BitBake output, in part, consists of images. - This section is going to look more closely at this output: - <imagedata fileref="figures/images.png" align="center" width="5.5in" depth="5.5in" /> - </para> - - <para> - For a list of example images that the Yocto Project provides, - see the - "<link linkend='ref-images'>Images</link>" chapter. - </para> - - <para> - Images are written out to the - <ulink url='&YOCTO_DOCS_DEV_URL;#build-directory'>Build Directory</ulink> - inside the <filename>tmp/deploy/images/<replaceable>machine</replaceable>/</filename> - folder as shown in the figure. - This folder contains any files expected to be loaded on the - target device. - The - <link linkend='var-DEPLOY_DIR'><filename>DEPLOY_DIR</filename></link> - variable points to the <filename>deploy</filename> directory, - while the - <link linkend='var-DEPLOY_DIR_IMAGE'><filename>DEPLOY_DIR_IMAGE</filename></link> - variable points to the appropriate directory containing images for - the current configuration. - <itemizedlist> - <listitem><para><filename><replaceable>kernel-image</replaceable></filename>: - A kernel binary file. - The <link linkend='var-KERNEL_IMAGETYPE'><filename>KERNEL_IMAGETYPE</filename></link> - variable setting determines the naming scheme for the - kernel image file. - Depending on that variable, the file could begin with - a variety of naming strings. - The <filename>deploy/images/<replaceable>machine</replaceable></filename> - directory can contain multiple image files for the - machine.</para></listitem> - <listitem><para><filename><replaceable>root-filesystem-image</replaceable></filename>: - Root filesystems for the target device (e.g. - <filename>*.ext3</filename> or <filename>*.bz2</filename> - files). - The <link linkend='var-IMAGE_FSTYPES'><filename>IMAGE_FSTYPES</filename></link> - variable setting determines the root filesystem image - type. - The <filename>deploy/images/<replaceable>machine</replaceable></filename> - directory can contain multiple root filesystems for the - machine.</para></listitem> - <listitem><para><filename><replaceable>kernel-modules</replaceable></filename>: - Tarballs that contain all the modules built for the kernel. - Kernel module tarballs exist for legacy purposes and - can be suppressed by setting the - <link linkend='var-MODULE_TARBALL_DEPLOY'><filename>MODULE_TARBALL_DEPLOY</filename></link> - variable to "0". - The <filename>deploy/images/<replaceable>machine</replaceable></filename> - directory can contain multiple kernel module tarballs - for the machine.</para></listitem> - <listitem><para><filename><replaceable>bootloaders</replaceable></filename>: - Bootloaders supporting the image, if applicable to the - target machine. - The <filename>deploy/images/<replaceable>machine</replaceable></filename> - directory can contain multiple bootloaders for the - machine.</para></listitem> - <listitem><para><filename><replaceable>symlinks</replaceable></filename>: - The <filename>deploy/images/<replaceable>machine</replaceable></filename> - folder contains - a symbolic link that points to the most recently built file - for each machine. - These links might be useful for external scripts that - need to obtain the latest version of each file. - </para></listitem> - </itemizedlist> - </para> - </section> - - <section id='sdk-dev-environment'> - <title>Application Development SDK</title> - - <para> - In the - <link linkend='general-yocto-environment-figure'>general Yocto Project Development Environment figure</link>, - the output labeled "Application Development SDK" represents an - SDK. - The SDK generation process differs depending on whether you build - a standard SDK - (e.g. <filename>bitbake -c populate_sdk</filename> <replaceable>imagename</replaceable>) - or an extensible SDK - (e.g. <filename>bitbake -c populate_sdk_ext</filename> <replaceable>imagename</replaceable>). - This section is going to take a closer look at this output: - <imagedata fileref="figures/sdk.png" align="center" width="9in" depth="7.25in" /> - </para> - - <para> - The specific form of this output is a self-extracting - SDK installer (<filename>*.sh</filename>) that, when run, - installs the SDK, which consists of a cross-development - toolchain, a set of libraries and headers, and an SDK - environment setup script. - Running this installer essentially sets up your - cross-development environment. - You can think of the cross-toolchain as the "host" - part because it runs on the SDK machine. - You can think of the libraries and headers as the "target" - part because they are built for the target hardware. - The environment setup script is added so that you can initialize - the environment before using the tools. - </para> - - <note> - <para> - The Yocto Project supports several methods by which you can - set up this cross-development environment. - These methods include downloading pre-built SDK installers - or building and installing your own SDK installer. - </para> - - <para> - For background information on cross-development toolchains - in the Yocto Project development environment, see the - "<link linkend='cross-development-toolchain-generation'>Cross-Development Toolchain Generation</link>" - section. - For information on setting up a cross-development - environment, see the - <ulink url='&YOCTO_DOCS_SDK_URL;#sdk-manual'>Yocto Project Software Development Kit (SDK) Developer's Guide</ulink>. - </para> - </note> - - <para> - Once built, the SDK installers are written out to the - <filename>deploy/sdk</filename> folder inside the - <ulink url='&YOCTO_DOCS_DEV_URL;#build-directory'>Build Directory</ulink> - as shown in the figure at the beginning of this section. - Depending on the type of SDK, several variables exist that help - configure these files. - The following list shows the variables associated with a standard - SDK: - <itemizedlist> - <listitem><para><link linkend='var-DEPLOY_DIR'><filename>DEPLOY_DIR</filename></link>: - Points to the <filename>deploy</filename> - directory.</para></listitem> - <listitem><para><link linkend='var-SDKMACHINE'><filename>SDKMACHINE</filename></link>: - Specifies the architecture of the machine - on which the cross-development tools are run to - create packages for the target hardware. - </para></listitem> - <listitem><para><link linkend='var-SDKIMAGE_FEATURES'><filename>SDKIMAGE_FEATURES</filename></link>: - Lists the features to include in the "target" part - of the SDK. - </para></listitem> - <listitem><para><link linkend='var-TOOLCHAIN_HOST_TASK'><filename>TOOLCHAIN_HOST_TASK</filename></link>: - Lists packages that make up the host - part of the SDK (i.e. the part that runs on - the <filename>SDKMACHINE</filename>). - When you use - <filename>bitbake -c populate_sdk <replaceable>imagename</replaceable></filename> - to create the SDK, a set of default packages - apply. - This variable allows you to add more packages. - </para></listitem> - <listitem><para><link linkend='var-TOOLCHAIN_TARGET_TASK'><filename>TOOLCHAIN_TARGET_TASK</filename></link>: - Lists packages that make up the target part - of the SDK (i.e. the part built for the - target hardware). - </para></listitem> - <listitem><para><link linkend='var-SDKPATH'><filename>SDKPATH</filename></link>: - Defines the default SDK installation path offered by the - installation script. - </para></listitem> - </itemizedlist> - This next list, shows the variables associated with an extensible - SDK: - <itemizedlist> - <listitem><para><link linkend='var-DEPLOY_DIR'><filename>DEPLOY_DIR</filename></link>: - Points to the <filename>deploy</filename> directory. - </para></listitem> - <listitem><para><link linkend='var-SDK_EXT_TYPE'><filename>SDK_EXT_TYPE</filename></link>: - Controls whether or not shared state artifacts are copied - into the extensible SDK. - By default, all required shared state artifacts are copied - into the SDK. - </para></listitem> - <listitem><para><link linkend='var-SDK_INCLUDE_PKGDATA'><filename>SDK_INCLUDE_PKGDATA</filename></link>: - Specifies whether or not packagedata will be included in - the extensible SDK for all recipes in the "world" target. - </para></listitem> - <listitem><para><link linkend='var-SDK_LOCAL_CONF_WHITELIST'><filename>SDK_LOCAL_CONF_WHITELIST</filename></link>: - A list of variables allowed through from the build system - configuration into the extensible SDK configuration. - </para></listitem> - <listitem><para><link linkend='var-SDK_LOCAL_CONF_BLACKLIST'><filename>SDK_LOCAL_CONF_BLACKLIST</filename></link>: - A list of variables not allowed through from the build - system configuration into the extensible SDK configuration. - </para></listitem> - <listitem><para><link linkend='var-SDK_INHERIT_BLACKLIST'><filename>SDK_INHERIT_BLACKLIST</filename></link>: - A list of classes to remove from the - <link linkend='var-INHERIT'><filename>INHERIT</filename></link> - value globally within the extensible SDK configuration. - </para></listitem> - </itemizedlist> - </para> - </section> - -</chapter> -<!-- -vim: expandtab tw=80 ts=4 ---> |