diff options
author | Andrew Geissler <geissonator@yahoo.com> | 2021-06-25 22:25:14 +0300 |
---|---|---|
committer | Brad Bishop <bradleyb@fuzziesquirrel.com> | 2021-06-28 15:35:59 +0300 |
commit | 0903674e2d7bafcf89cf75adbcf34cac5ce4b938 (patch) | |
tree | c46ac7b80c4e559224df62f95931ed8a2ab51435 /poky/documentation/test-manual | |
parent | a1a6aefba3ae965f2447b102663b2a6a40aa968a (diff) | |
download | openbmc-0903674e2d7bafcf89cf75adbcf34cac5ce4b938.tar.xz |
poky: subtree update:9d1b332292..2834c2f853
Alex Stewart (3):
opkg-utils: upgrade to version 0.4.5
opkg: upgrade to version 0.4.5
opkg: add QA check for openssl feed verification
Alexander Kanavin (37):
virglrenderer: explicitly depend on libgbm
elfutils: update 0.183 -> 0.185
libcap: update 2.49 -> 2.50
perl: split perl-cross into its own recipe
perl-cross: 1.3.5 -> 1.3.6
perl: update 5.32.1 -> 5.34.0
libgcrypt: upgrade 1.9.2 -> 1.9.3
erofs-utils: correct upstream version check
m4: correct ptest failures
ovmf: update 2021.02 -> 2021.05
apt: update 2.2.3 -> 2.2.4
util-linux: update 2.36.2 -> 2.37
cross-canadian: correct the location of pkg-config files
nettle: update 3.7.2 -> 3.7.3
glib-2.0: update 2.68.2 -> 2.68.3
meson: upgrade 0.58.0 -> 0.58.1
ell: upgrade 0.40 -> 0.41
erofs-utils: upgrade 1.2.1 -> 1.3
grub: upgrade 2.04+2.06~rc1 -> 2.06
gptfdisk: upgrade 1.0.7 -> 1.0.8
connman: update 1.39 -> 1.40
libksba: upgrade 1.5.1 -> 1.6.0
libnss-mdns: upgrade 0.15 -> 0.15.1
libwpe: upgrade 1.10.0 -> 1.10.1
puzzles: upgrade to latest revision
rng-tools: upgrade 6.12 -> 6.13
stress-ng: upgrade 0.12.09 -> 0.12.10
python3-magic: upgrade 0.4.23 -> 0.4.24
sudo: upgrade 1.9.7 -> 1.9.7p1
wpebackend-fdo: upgrade 1.8.4 -> 1.10.0
xkeyboard-config: upgrade 2.32 -> 2.33
bitbake.conf: enable debuginfod in native/nativesdk
gdb-cross: enable debuginfod
util-linux: backport a patch to address mkswap hangs
selftest: do not hardcode /tmp/sdk
glibc: do not enable memory tagging on aarch64 just yet
mesa: enable gallium intel drivers when building for x86
Alexandre Belloni (1):
runqemu: time the copy to tmpfs
Alexey Brodkin (3):
gcc: Fixes for ARC
gdb: Add native GDB support for ARC
gcc: Apply multilib fix to ARC as well
Alistair Francis (3):
recipes-bsp/opensbi: Disable FW_PIC
recipes-bsp/u-boot: Allow deploying the u-boot DTB
recipes-bsp/opensbi: Add support for specifying a device tree
Anders Wallin (1):
coreutils: remove NOSTAT_LEAF_OPTIMIZATION
Andrea Adami (1):
kernel.bbclass: fix do_sizecheck() comparison
Andreas Müller (19):
mesa: upgrade 21.1.1 -> 21.1.2
systemd: Add more ugly casts to fix build with musl
alsa-lib: upgrade 1.2.4 -> 1.2.5
alsa-plugins: upgrade 1.2.2 -> 1.2.5
alsa-tools: upgrade 1.2.2 -> 1.2.5
alsa-topology-conf: upgrade 1.2.4 -> 1.2.5
alsa-ucm-conf: upgrade 1.2.4 -> 1.2.5
alsa-utils(-scripts): upgrade 1.2.4 -> 1.2.5
libinput: upgrade 1.17.3 -> 1.18.0
xf86-input-libinput: upgrade 0.30.0 -> 1.0.1
epiphany: upgrade 40.1 -> 40.2
vala: upgrade 0.52.3 -> 0.52.4
p11-kit: upgrade 0.23.22 -> 0.23.24
xorgproto: upgrade 2021.4.99.1 -> 2021.4.99.2
mpg123: 1.27.2 -> 1.28.0
libx11: upgrade 1.7.1 -> 1.7.2
libx11: remove CPPFLAGS_FOR_BUILD += "-D_GNU_SOURCE"
libpcap: upgrade 1.10.0 -> 1.10.1
mesa: upgrade 21.1.2 -> 21.1.3
Bruce Ashfield (10):
linux-yocto/5.10: update to v5.10.42
linux-yocto/5.10: temporarily revert aufs
linux-yocto-dev: base AUTOREV on specified version
linux-yocto/5.4: update to v5.4.124
linux-yocto/5.10: restore aufs
linux-yocto/5.10: update to v5.10.43
linux-yocto/5.4: update to v5.4.125
linux-yocto/5.10: cgroup1: fix leaked context root causing sporadic NULL deref in LTP
btrfs-tools: include linux/const.h to fix build with 5.12+ headers
bsps/5.10: update to v5.10.43
Changqing Li (1):
libjpeg-turbo: fix do_compile error on arm
Chris Laplante (1):
bitbake: build: warn on setting noexec/nostamp/fakeroot flag to any value besides '1'
Daniel Wagenknecht (5):
ref-manual: variables: update examples refering to DEPLOY_DIR_IMAGE
ref-manual: variables: document IMGDEPLOYDIR
ref-manual: migration-2.2: add note about IMGDEPLOYDIR
ref-manual: variables: fixup example in IMAGE_CMD
ref-manual: variables: fixup class reference in IMAGE_MANIFEST
Joe Slater (1):
tcf-agent: change license to EPL/EDL
Joshua Watt (2):
classes/buildhistory: Add option to strip path prefix
classes/reproducible_build: Use atomic rename for SDE file
Justin Bronder (1):
populate_sdk_ext: copy BBMULTICONFIG files
Kai Kang (1):
valgrind: fix a typo
Khem Raj (14):
harfbuzz: Fix unused-variable warning
arch-armv4: Allow -march=armv4
ffmpeg: Link in libatomic on riscv32
libssp-nonshared: Use a different implementation for __stack_chk_fail
qemuriscv: Enable 4 core emulation
gcompat: Add recipe
musl: Do not package glibc loader
musl: Set UPSTREAM_CHECK_COMMITS
Revert "libgcc-initial: Do not build fp128 to decimal ppc functions"
qemu: Provide float128 via hwcaps2 on ppc64le
linuxloader: Be aware of riscv32 ldso
linuxloader.bbclass: Add entry for ppc64 LE glibc loader
gcompat: Create symlinks to glibc ldso locations
sdk: Enable do_populate_sdk with multilibs
Luca Boccassi (1):
systemd: install new sysext tool via systemd-extra-utils
Marcus Comstedt (1):
conf/machine-sdk: Add ppc64 SDK machine
Matt Spencer (1):
systemd-conf: Prevent systemd-network from managing veth interfaces
Michael Halstead (1):
releases: update to include 3.1.8
Michael Opdenacker (12):
bitbake: docs: Add BB_HASHSERVE definition to glossary
bitbake: doc: bitbake-user-manual: fix erroneous statement in glossary intro
manuals: fix epub export warnings
ref-manual: move migration guides to separate document
releases: clarify supported and outdated releases
releases: put release number after "Release Series"
sdk-manual: fix broken references
migration guides: remove index reference to BB_SETSCENE_VERIFY_FUNCTION2
manuals: fix issues related to trailing dots
sdk-manual: add missing quoting around "devtool upgrade"
sdk-manual: fix wrong word
sdk-manual: add missing index references
Ming Liu (2):
u-boot-tools: fix a mkimage signature issue
uboot-sign.bbclass: fix some install commands
Mingli Yu (2):
sysstat: make the service start automatically
boost: fix wrong type for mutex in regex v5
Nicolas Dechesne (3):
index: remove the link/section to 'mega manual' from main page
index: remove links to releases manual and index
index: split releases manuals and indexes into two sections in the tree
Paul Barker (2):
bitbake: asyncrpc: Add ping method
bitbake: asyncrpc: Reduce verbosity
Quentin Schulz (6):
docs: ref-manual: migration-3.0: remove reference to non-existing BB_SETSCENE_VERIFY_FUNCTION2
docs: ref-manual: variables: add missing links to terms glossary
bitbake: doc: user-manual: remove mentions to BBVERSIONS
bitbake: doc: user-manual: ref-manual: remove mentions to BB_SETSCENE_VERIFY_FUNCTION2
documentation: Makefile: turn warnings into errors by default
docs: replace ``FOO`` by :term:`FOO` where possible
Richard Purdie (11):
lttng-tools: upgrade 2.12.3 -> 2.12.4
qemurunner: Try to ensure mmap'd libs are paged in
qemurunner: Increase startup timeout 120 -> 300
build-appliance-image: Update to master head revision
test-manual: add initial reproducible builds documentation
test-manual: Add initial YP Compatible documentation
README: Tweak as the website isn't really new now
README: Move to using markdown as the format
perf: Use python3targetconfig to ensure we use target libraries
ltp: Reinstate 'hanging' tests for evaluation
README.poky: Formatting and content cleanup
Richard Weinberger (1):
Document erofs filesystem targets
Robert P. J. Day (2):
ref-manual: add SRCTREECOVEREDTASKS to variable glossary
ref-manual: add glossary entry for NON_MULTILIB_RECIPES
Ross Burton (11):
mx: remove from Openembedded Core
core-image-weston: remove Clutter examples
Remove Clutter and Cogl
oeqa: remove Clutter usage
meta-poky: remove clutter references
Remove Clutter references
gcc: enable branch protection by standard
image_types: add zsync conversions
avahi: apply fix for CVE-2021-3468
qemu: fix virtio vhost-user-gpu CVEs
gcc: replace gdb helper install revert with the upstream fix
Sakib Sajal (3):
oeqa/core/target/qemu.py: display contents of dumped files
oe-time-dd-test.sh: improve output formatting
oe-time-dd-test.sh: add iostat command
Saul Wold (1):
qemurunner: add second qmp port
Scott Weaver (1):
bitbake: fetch2: add check for empty SRC_URI hash string
Tim Orling (8):
maintainers.inc: update email address
python3-scons: upgrade 3.1.2 -> 4.1.0; simplify
python3-hypothesis: upgrade 6.13.7 -> 6.13.14
at-spi2-core: upgrade 2.40.1 -> 2.40.2
python3-importlib-metadata: upgrade 4.4.0 -> 4.5.0
python3-manifest: add statistics subpackage
python3-hypothesis: upgrade 6.13.14 -> 6.14.0
python3: skip tests requiring tools-sdk
Tony Battersby (1):
glibc: fix path to place zdump in the tzcode package
Tony Tascioglu (3):
valgrind: Improve non-deterministic ptest reliability
valgrind: remove buggy ptest from arm64
valgrind: Actually install list of non-deterministic ptests
hongxu (1):
nativesdk-libdnf: fix installed and not shipped files
wangmy (21):
cmake: upgrade 3.20.2 -> 3.20.3
mtools: upgrade 4.0.27 -> 4.0.29
python3-magic: upgrade 0.4.22 -> 0.4.23
less: upgrade 586 -> 589
python3-libarchive-c: upgrade 3.0 -> 3.1
diffoscope: upgrade 175 -> 177
dtc: upgrade 1.6.0 -> 1.6.1
git: upgrade 2.31.1 -> 2.32.0
gnutls: upgrade 3.7.1 -> 3.7.2
go: upgrade 1.16.4 -> 1.16.5
less: upgrade 589 -> 590
ethtool: upgrade 5.10 -> 5.12
m4: upgrade 1.4.18 -> 1.4.19
alsa-lib: upgrade 1.2.5 -> 1.2.5.1
alsa-utils: upgrade 1.2.5 -> 1.2.5.1
alsa-topology-conf: upgrade 1.2.5 -> 1.2.5.1
alsa-ucm-conf: upgrade 1.2.5 -> 1.2.5.1
blktrace: upgrade 1.2.0 -> 1.3.0
enchant2: upgrade 2.2.15 -> 2.3.0
librepo: upgrade 1.14.0 -> 1.14.1
createrepo-c: upgrade 0.17.2 -> 0.17.3
zangrc (1):
python3-pycairo: upgrade 1.20.0 -> 1.20.1
zhengruoqin (6):
python3-importlib-metadata: upgrade 4.3.0 -> 4.4.0
libogg: upgrade 1.3.4 -> 1.3.5
liburcu: upgrade 0.12.2 -> 0.13.0
libcomps: upgrade 0.1.16 -> 0.1.17
python3-dbusmock: upgrade 0.23.0 -> 0.23.1
nfs-utils: upgrade 2.5.3 -> 2.5.4
Signed-off-by: Andrew Geissler <geissonator@yahoo.com>
Change-Id: Iac124e214336beb9cab7fb3b67a6968d4e34d06f
Diffstat (limited to 'poky/documentation/test-manual')
-rw-r--r-- | poky/documentation/test-manual/index.rst | 2 | ||||
-rw-r--r-- | poky/documentation/test-manual/reproducible-builds.rst | 135 | ||||
-rw-r--r-- | poky/documentation/test-manual/yocto-project-compatible.rst | 124 |
3 files changed, 261 insertions, 0 deletions
diff --git a/poky/documentation/test-manual/index.rst b/poky/documentation/test-manual/index.rst index e2198c4c3..4c590a6aa 100644 --- a/poky/documentation/test-manual/index.rst +++ b/poky/documentation/test-manual/index.rst @@ -13,6 +13,8 @@ Yocto Project Test Environment Manual intro test-process understand-autobuilder + reproducible-builds + yocto-project-compatible history .. include:: /boilerplate.rst diff --git a/poky/documentation/test-manual/reproducible-builds.rst b/poky/documentation/test-manual/reproducible-builds.rst new file mode 100644 index 000000000..e13583c0b --- /dev/null +++ b/poky/documentation/test-manual/reproducible-builds.rst @@ -0,0 +1,135 @@ +.. SPDX-License-Identifier: CC-BY-SA-2.0-UK + +******************* +Reproducible Builds +******************* + +================ +How we define it +================ + +The Yocto Project defines reproducibility as where a given input build +configuration will give the same binary output regardless of when it is built +(now or in 5 years time), regardless of the path on the filesystem the build is +run in, and regardless of the distro and tools on the underlying host system the +build is running on. + +============== +Why it matters +============== + +The project aligns with the `Reproducible Builds project +<https://reproducible-builds.org/>`_, which shares information about why +reproducibility matters. The primary focus of the project is the ability to +detect security issues being introduced. However, from a Yocto Project +perspective, it is also hugely important that our builds are deterministic. When +you build a given input set of metadata, we expect you to get consistent output. +This has always been a key focus but, :yocto_docs:`since release 3.1 ("dunfell") +</ref-manual/migration-3.1.html#reproducible-builds-now-enabled-by-default>`, +it is now true down to the binary level including timestamps. + +For example, at some point in the future life of a product, you find that you +need to rebuild to add a security fix. If this happens, only the components that +have been modified should change at the binary level. This would lead to much +easier and clearer bounds on where validation is needed. + +This also gives an additional benefit to the project builds themselves, our hash +equivalence for :ref:`Shared State <overview-manual/concepts:Shared State>` +object reuse works much more effectively when the binary output remains the +same. + +.. note:: + + We strongly advise you to make sure your project builds reproducibly + before finalizing your production images. It would be too late if you + only address this issue when the first updates are required. + +=================== +How we implement it +=================== + +There are many different aspects to build reproducibility, but some particular +things we do within the build system to ensure reproducibility include: + +- Adding mappings to the compiler options to ensure debug filepaths are mapped + to consistent target compatible paths. This is done through the + ``DEBUG_PREFIX_MAP`` variable which sets the ``-fmacro-prefix-map`` and + ``-fdebug-prefix-map`` compiler options correctly to map to target paths. +- Being explicit about recipe dependencies and their configuration (no floating + configure options or host dependencies creeping in). In particular this means + making sure :term:`PACKAGECONFIG` coverage covers configure options which may + otherwise try and auto-detect host dependencies. +- Using recipe specific sysroots to isolate recipes so they only see their + dependencies. These are visible as ``recipe-sysroot`` and + ``recipe-sysroot-native`` directories within the :term:`WORKDIR` of a given + recipe and are populated only with the dependencies a recipe has. +- Build images from a reduced package set: only packages from recipes the image + depends upon. +- Filtering the tools available from the host's ``PATH`` to only a specific set + of tools, set using the :term:`HOSTTOOLS` variable. + +========================================= +Can we prove the project is reproducible? +========================================= + +Yes, we can prove it and we regularly test this on the Autobuilder. At the +time of writing (release 3.3, "hardknott"), :term:`OpenEmbedded-Core (OE-Core)` +is 100% reproducible for all its recipes (i.e. world builds) apart from the Go +language and Ruby documentation packages. Unfortunately, the current +implementation of the Go language has fundamental reproducibility problems as +it always depends upon the paths it is built in. + +.. note:: + + Only BitBake and :term:`OpenEmbedded-Core (OE-Core)`, which is the ``meta`` + layer in Poky, guarantee complete reproducibility. The moment you add + another layer, this warranty is voided, because of additional configuration + files, ``bbappend`` files, overridden classes, etc. + +To run our automated selftest, as we use in our CI on the Autobuilder, you can +run:: + + oe-selftest -r reproducible.ReproducibleTests.test_reproducible_builds + +This defaults to including a ``world`` build so, if other layers are added, it would +also run the tests for recipes in the additional layers. The first build will be +run using :ref:`Shared State <overview-manual/concepts:Shared State>` if +available, the second build explicitly disables +:ref:`Shared State <overview-manual/concepts:Shared State>` and builds on the +specific host the build is running on. This means we can test reproducibility +builds between different host distributions over time on the Autobuilder. + +If ``OEQA_DEBUGGING_SAVED_OUTPUT`` is set, any differing packages will be saved +here. The test is also able to run the ``diffoscope`` command on the output to +generate HTML files showing the differences between the packages, to aid +debugging. On the Autobuilder, these appear under +https://autobuilder.yocto.io/pub/repro-fail/ in the form ``oe-reproducible + +<date> + <random ID>``, e.g. ``oe-reproducible-20200202-1lm8o1th``. + +The project's current reproducibility status can be seen at +:yocto_home:`/reproducible-build-results/` + +You can also check the reproducibility status on supported host distributions: + +- CentOS: :yocto_ab:`/typhoon/#/builders/reproducible-centos` +- Debian: :yocto_ab:`/typhoon/#/builders/reproducible-debian` +- Fedora: :yocto_ab:`/typhoon/#/builders/reproducible-fedora` +- Ubuntu: :yocto_ab:`/typhoon/#/builders/reproducible-ubuntu` + +=============================== +Can I test my layer or recipes? +=============================== + +Once again, you can run a ``world`` test using the +:ref:`oe-selftest <ref-manual/release-process:Testing and Quality Assurance>` +command provided above. This functionality is implemented +in :oe_git:`meta/lib/oeqa/selftest/cases/reproducible.py +</openembedded-core/tree/meta/lib/oeqa/selftest/cases/reproducible.py>`. + +You could subclass the test and change ``targets`` to a different target. + +You may also change ``sstate_targets`` which would allow you to "pre-cache" some +set of recipes before the test, meaning they are excluded from reproducibility +testing. As a practical example, you could set ``sstate_targets`` to +``core-image-sato``, then setting ``targets`` to ``core-image-sato-sdk`` would +run reproducibility tests only on the targets belonging only to ``core-image-sato-sdk``. diff --git a/poky/documentation/test-manual/yocto-project-compatible.rst b/poky/documentation/test-manual/yocto-project-compatible.rst new file mode 100644 index 000000000..a7897469f --- /dev/null +++ b/poky/documentation/test-manual/yocto-project-compatible.rst @@ -0,0 +1,124 @@ +.. SPDX-License-Identifier: CC-BY-SA-2.0-UK + +************************ +Yocto Project Compatible +************************ + +============ +Introduction +============ + +After the introduction of layers to OpenEmbedded, it quickly became clear +that while some layers were popular and worked well, others developed a +reputation for being "problematic". Those were layers which didn't +interoperate well with others and tended to assume they controlled all +the aspects of the final output. This usually isn't intentional but happens +because such layers are often created by developers with a particular focus +(e.g. a company's :term:`BSP<Board Support Package (BSP)>`) whilst the end +users have a different one (e.g. integrating that +:term:`BSP<Board Support Package (BSP)>` into a product). + +As a result of noticing such patterns and friction between layers, the project +developed the "Yocto Project Compatible" badge program, allowing layers +following the best known practises to be marked as being widely compatible +with other ones. This takes the form of a set of "yes/no" binary answer +questions where layers can declare if they meet the appropriate criteria. +In the second version of the program, a script was added to make validation +easier and clearer, the script is called ``yocto-check-layer`` and is +available in :term:`OpenEmbedded-Core (OE-Core)`. + +See :ref:`dev-manual/common-tasks:making sure your layer is compatible with yocto project` +for details. + +======== +Benefits +======== + +:ref:`overview-manual/yp-intro:the yocto project layer model` is powerful +and flexible: it gives users the ultimate power to change pretty much any +aspect of the system but as with most things, power comes with responsibility. +The Yocto Project would like to see people able to mix and match BSPs with +distro configs or software stacks and be able to merge succesfully. +Over time, the project identified characteristics in layers that allow them +to operate well together. "anti-patterns" were also found, preventing layers +from working well together. + +The intent of the compatibility program is simple: if the layer passes the +compatibility tests, it is considered "well behaved" and should operate +and cooperate well with other compatible layers. + +The benefits of compatibility can be seen from multiple different user and +member perspectives. From a hardware perspective +(a :ref:`overview-manual/concepts:bsp layer`), compatibility means the +hardware can be used in many different products and use cases without +impacting the software stacks being run with it. For a company developing +a product, compatibility gives you a specification / standard you can +require in a contract and then know it will have certain desired +characteristics for interoperability. It also puts constraints on how invasive +the code bases are into the rest of the system, meaning that multiple +different separate hardware support layers can coexist (e.g. for multiple +product lines from different hardware manufacturers). This can also make it +easier for one or more parties to upgrade those system components for security +purposes during the lifecycle of a product. + +================== +Validating a layer +================== + +The badges are available to members of the Yocto Project (as member benefit) +and to open source projects run on a non-commercial basis. However, anyone can +answer the questions and run the script. + +The project encourages all layer maintainers to review the questions and the +output from the script against their layer, as the way some layers are +constructed often has unintended consequences. The questions and the script +are designed to highlight known issues which are often easy to solve. This +makes layers easier to use and therefore more popular. + +It is intended that over time, the tests will evolve as new best known +practices are identified, and as new interoperability issues are found, +unnecessarily restricting layer interoperability. If anyone becomes aware of +either type, please let the project know through the +:yocto_home:`technical calls </public-virtual-meetings/>`, +the :yocto_home:`mailing lists </community/mailing-lists/>` +or through the :oe_wiki:`Technical Steering Committee (TSC) </TSC>`. +The TSC is responsible for the technical criteria used by the program. + +Layers are divided into three types: + +- :ref:`"BSP" or "hardware support"<overview-manual/concepts:bsp layer>` + layers contain support for particular pieces of hardware. This includes + kernel and boot loader configuration, and any recipes for firmware or + kernel modules needed for the hardware. Such layers usually correspond + to a :term:`MACHINE` setting. + +- :ref:`"distro" layers<overview-manual/concepts:distro layer>` defined + as layers providing configuration options and settings such as the + choice of init system, compiler and optimisation options, and + configuration and choices of software components. This would usually + correspond to a :term:`DISTRO` setting. + +- "software" layers are usually recipes. A layer might target a + particular graphical UI or software stack component. + +Here are key best practices the program tries to encourage: + +- A layer should clearly show who maintains it, and who change + submissions and bug reports should be sent to. + +- Where multiple types of functionality are present, the layer should + be internally divided into sublayers to separate these components. + That's because some users may only need one of them and separability + is a key best practice. + +- Adding a layer to a build should not modify that build, unless the + user changes a configuration setting to activate the layer, by selecting + a :term:`MACHINE`, a :term:`DISTRO` or a :term:`DISTRO_FEATURES` setting. + +The project does test the compatibility status of the core project layers on +its :doc:`Autobuilder </test-manual/understand-autobuilder>`. + +The official form to submit compatibility requests with is at +:yocto_home:`/ecosystem/branding/compatible-registration/`. +Applicants can display the badge they get when their application is successful. + |