From 213cb2696d00a85cd48d356cb5131824a302d828 Mon Sep 17 00:00:00 2001 From: Patrick Williams Date: Sat, 7 Aug 2021 19:21:33 -0500 Subject: subtree updates MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit meta-raspberrypi: 8dc3a31088..c7f4c739a3: Khem Raj (5): linux-raspberrypi: Upgrade to 5.10.52 userland: Update to latest master branch raspberrypi-firmware: Update to latest raspberrypi-tools: Update to latest sdcard_image-rpi.bbclass: Fix IMAGE_TYPEDEP override to use new syntax Martin Jansa (4): Convert to new override syntax Manually fix conversion layer.conf: Update to honister userland: package man pages in PN-doc Pierre-Jean Texier (2): kas: local.conf: bump CONF_VERSION variable kas: local.conf: disable prelink poky: 17aabc0127..492205ea83: Alexander Kanavin (17): llvm: update 12.0.0 -> 12.0.1 systemd: update 248.3 -> 249.1 python3-testools: update 2.4.0 -> 2.5.0 libuv: update 1.41.0 -> 1.42.0 gnu-config: update to latest revision vulkan-samples: update to latest revision cmake: update 3.20.5 -> 3.21.0 cmake: update 3.21.0 -> 3.21.1 mtools: update 4.0.32 -> 4.0.34 util-linux: update 2.37 -> 2.37.1 iputils: update 20210202 -> 20210722 freetype: update 2.10.4 -> 2.11.0 devtool: print a warning on upgrades if PREFERRED_VERSION is set rpm: do not RRECOMMEND rpm-build selftest: add core-image-weston to no-gpl3-no-meta-gpl2 image test shadow: update 4.8.1 -> 4.9 local.conf.sample: disable prelink Bernhard Rosenkränzer (1): gcc: update 11.1 -> 11.2 Bruce Ashfield (6): linux-yocto/5.10: update to v5.10.53 linux-yocto/5.13: update to v5.13.5 linux-yocto/5.4: update to v5.4.135 linux-yocto-rt/5.10: update to -rt47 linux-yocto/5.13: enable TYPEC_TCPCI in usbc fragment linux-yocto/5.10: enable TYPEC_TCPCI in usbc fragment Changqing Li (1): archiver.bbclass: fix do_ar_configured failure for kernel Chen Qi (3): zstd: fix CVE_PRODUCT insane.bbclass: fix the file-rdeps QA message for the new override syntax iputils: fix do_configure failure of missing ip command Damian Wrobel (1): rootfs: remove ldconfig auxiliary cache where appropriate Denys Dmytriyenko (4): meta: convert nested overrides leftovers to new syntax convert-overrides.py: handle few more cases of overrides libwpe: remove rpi-specific custom code poky-tiny: drop uclibc override Jon Mason (1): parselogs.py: qemuarm should be qemuarmv5 Joshua Watt (4): mesa: Fix v3d & vc4 dmabuf import bitbake: bitbake: asyncrpc: Catch early SIGTERM libxft: Fix bad PKG value bitbake: contrib: vim: Update for new override syntax Kai Kang (2): u-boot_2021.07: set UBOOT_MACHINE for qemumips and qemumips64 python3-pytest: display correct version info Kevin Hao (2): meta-yocto-bsp: Introduce the v5.13 bbappend meta-yocto-bsp: Bump to the v5.10.55 Khem Raj (10): binutils: Upgrade to 2.37 branch texinfo: Update gnulib to fix build with glibc 2.34 systemd: Fix build on musl stress-ng: Drop defining daddr_t stress-ng: Detemine minimal stack size via sysconf mesa: Define a fallback for DRIDRIVERS libssh2: Fix syntax for using ptest override toaster-managed-mode.json: Correctly specify term with new override syntax distrooverrides.bbclass: Correct override syntax devtool.py: Correct override syntax Lee Chee Yang (1): aspell: fix CVE-2019-25051 Marek Vasut (2): image_types: Restore pre-btrfs-tools 4.14.1 mkfs.btrfs shrink behavior kernel-uboot: Handle gzip and lzo compression options Martin Jansa (6): convert-overrides.py: show processed file and version of this script convert-overrides.py: remove base_dep_prepend and autotools_dep_prepend exception convert-overrides.py: 0.9.1 include '(' as delimiter for shortvars convert-overrides.py: allow specifying multiple target dirs convert-overrides.py: allow dots before override in vars_re and shortvars_re systemd-boot: use ld.bfd as efi-ld even when gold or lld is used in ${LD} Matthias Klein (2): runqemu: Fix typo in error message runqemu: decouple bios and kernel options Matthias Schiffer (3): initscripts: populate-volatile.sh: do not log to tty0 initscripts: populate-volatile.sh: run create_file synchronously initscripts: fix creation order for /var/log with VOLATILE_LOG_DIR=true Michael Halstead (1): releases: update to include 3.3.1 Michael Opdenacker (18): oe-setup-builddir: update YP docs and OE URLs conf-notes.txt: now suggesting to run 'runqemu qemux86-64' test-manual: document LTO related reproducibility bug quick start manual: update "source oe-init-build-env" output dev-manual: fix wrong reference to class documentation/README: improve BitBake manual referencing guidelines manuals: simplify references to BitBake manual manuals: remove explicit BitBake variable references meta-skeleton: add recipe examples from documentation sources bitbake: doc: bitbake-user-manual: fix syntax in example and improve description bitbake: doc: bitbake-user-manual: update bitbake option help bitbake: doc: bitbake-user-manual: grammar fix for the number of "metadata" manuals: initial documentation for CVE management ref-manual: remove example recipe source files profile-manual: document how to build perf manpages on target cve-check: fix comments cve-check: update link to NVD website for CVE details cve-check: improve comment about CVE patch file names Mingli Yu (2): perlcross: not break build if already patched curl: Upgrade to 7.78.0 Nicolas Dechesne (4): yocto-check-layer: improve missed dependencies checklayer: new function get_layer_dependencies() checklayer: rename _find_layer_depends yocto-check-layer: ensure that all layer dependencies are tested too Oleksandr Kravchuk (1): bitbake.conf: change GNOME_MIRROR to new one Patrick Williams (1): pixman: re-disable iwmmxt Paul Barker (4): bitbake: asyncrpc: Fix bad message error in client bitbake: asyncrpc: Set timeout when waiting for reply from server bitbake: parse/ast: Substitute '~' when naming anonymous functions kernel-yocto: Simplify no git repo case in do_kernel_checkout Quentin Schulz (4): bitbake: doc: Makefile: turn warnings into errors by default bitbake: doc: bitbake-user-manual: ref-variables: order alphabetically the glossary sources bitbake: doc: bitbake-user-manual: ref-variables: force glossary output to be alphabetically sorted bitbake: doc: bitbake-user-manual: replace ``FOO`` by :term:`FOO` where possible Richard Purdie (49): Add MAINTAINERS.md file yocto-check-layer: Remove duplicated code libubootenv: Drop default-env RRECOMMENDS bitbake: data_smart: Allow colon in variable expansion regex meta-poky/meta-yocto-bsp: Convert to new override syntax layer.conf: Update to honister autotools/base/icecc: Remove prepend from function names scripts/contrib: Add override conversion script systemtap: Fix headers issue with x86 and 5.13 headers migration-guides: Add start of 3.4 guide with override migration notes common-tasks: Fix conversion error in npm example bitbake: bitbake: Switch to using new override syntax bitbake: doc/lib: Update to use new override syntax containing colons bitbake: doc/lib: Add fixes for issues missed by the automated conversion bitbake: bitbake: Update to version 1.51.1 layer.conf: Override changes mean we're only compatible with honister Convert to new override syntax meta: Manual override fixes local.conf.sample: Bump version so users update their config sanity.conf: Require bitbake 1.51.1 dropbear: Fix incorrect package override for postrm convert-overrides: Allow script to handle patch/diffs sdk: Decouple default install path from built in path sstate: Fix rebuilds when changing layer config populate_sdk_ext: Fix handling of TOOLCHAIN_HOST_TASK in the eSDK case local.conf.sample: Bump version so users update their config poky: Use SDKPATHINSTALL instead of SDKPATH vim: Clarify where RDEPENDS/RRECOMMENDS apply bitbake: data_smart: Fix inactive overide accidental variable value corruption local.conf.sample: Fix missed override conversion license: Exclude COPYING.MIT from pseudo meta: Convert IMAGE_TYPEDEP to use override syntax uboot-extlinux-config: Fix missing override conversion image/image_types: Convert CONVERSION_CMD/COMPRESS_CMD to new override syntax image: Drop COMPRESS_CMD devupstream: Allow support of native class extensions diffoscope: Upgrade 178 -> 179 strace: Upgrade 5.12 -> 5.13 valgrind: Add patches for glibc 2.34 support bitbake: runqueue: Improve multiconfig deferred task issues elfutils: Add patch from upstream for glibc 2.34 ptest fixes bitbake: doc: Fix append/prepend/remove references bitbake: fetch/tests/toaster: Override conversion fixups bitbake: process: Improve traceback error reporting from main loop bitbake: command: Ensure we catch/handle exceptions bitbake: ui/taskexp: Improve startup exception handling bitbake: ui/taskexp: Fix to work with empty build directories oeqa/runtime/cases/ptest: Increase test timeout from 300s to 450s packagedata: Fix after override syntax change Ross Burton (2): glew: fix Makefile race libx11: fix xkb compilation with _EVDEVK symbols Saul Wold (1): MAINTAINERS: Saul will cover devtool and eSDK Stefan Wiehler (1): dev-manual: fix source release example script Stefano Babic (1): mtd-utils: upgrade 2.1.2 -> 2.1.3 Tim Orling (2): python3-hypothesis: upgrade 6.14.3 -> 6.14.5 python3-importlib-metadata: upgrade 4.6.1 -> 4.6.3 Tony Battersby (2): lto.inc: disable LTO for grub gcc: Backport patch to make LTO builds more reproducible Tony Tascioglu (6): ffmpeg: fix-CVE-2020-20446 ffmpeg: fix CVE-2020-20453 ffmpeg: fix CVE-2020-22015 ffmpeg: fix CVE-2020-22021 ffmpeg: fix CVE-2020-22033 and CVE-2020-22019 ffmpeg: fix CVE-2021-33815 Trevor Woerner (1): ffmpeg: add libatomic for armv5 Ulrich Ölmann (2): initramfs-framework: fix whitespace issue initramfs-framework/setup-live: fix shebang Vinay Kumar (1): glibc: Fix CVE-2021-33574 Vivien Didelot (1): init-manager-systemd: define weak dev manager Zqiang (1): python3: use monotonic clock for condvar if possible hongxu (1): createrepo-c: fix createrepo-c failed in nativesdk leimaohui (1): archiver.bbclass: Fix patch error for recipes that inherit dos2unix. wangmy (3): bind: upgrade 9.16.18 -> 9.16.19 i2c-tools: upgrade 4.2 -> 4.3 diffoscope: upgrade 177 -> 178 zangrc (2): python3-dbus: upgrade 1.2.16 -> 1.2.18 python3-pip: upgrade 21.1.3 -> 21.2.1 meta-openembedded: 8fbcfb9f02..3cf2475ea0: Anastasios Kavoukis (1): pm-qa: fix paths for shell scripts Andreas Müller (3): mozjs/0001-Port-build-to-python3.patch: Fix typos in description jack: upgrade 1.19.18 -> 1.19.19 fluidsynth: upgrade 2.2.1 -> 2.2.2 Andrej Valek (1): thrift: upgrade to 0.14.2 Andrew Jeffery (2): python3-gmpy: Add native support python3-ecdsa: Add native support Armin Kuster (2): hiawatha: fix url. wireshark: update to 3.4.7 Ben Brown (1): android-tools: fix install of adb client when TOOLS is overridden Changqing Li (1): apache2: upgrade 2.4.46 -> 2.4.48 Devendra Tewari (1): Suppress eol in functionfs setup scripts (#147) Gianfranco (1): vboxguestdrivers: upgrade 6.1.22 -> 6.1.24 Joe Slater (2): php: move to version 7.4.21 gtksourceview4: work around dependency deficiency Johannes Obermüller (1): evtest: fix timestamps in output Kai Kang (2): python3-blivet: 3.1.4 -> 3.4.0 python3-blivetgui: 2.1.10 -> 2.2.1 Khem Raj (23): netperf: Update to latest netperf: Add systemd unit file packagegroup-meta-oe: Add lmdb packagegroup-meta-oe: Add mbw addcli: check for ns_get16 and ns_get32 fuse: Define closefrom if not available autofs: Fix build with glibc 2.34+ ntp: Do not use PTHREAD_STACK_MIN on glibc ntp: Fix make check mongodb: Upgrade to 4.4.7 vboxguestdrivers: Remove __divmoddi4 patch packagegroup-meta-oe: Add jemalloc apitrace: Exclude from builds with glibc 2.34+ libhugetlbfs: Disable build with glibc 2.34+ fvwm: Package extra files and man pages luajit: Fix override syntax lua: Drop uclibc patch packagegroup-meta-oe: Correct override name and fix syntax recipes: Fix override syntax emacs,libgpiod,cockpit: Fix override syntax in using FILES_${PN} fvwm: Fix build time paths in target perl/python scripts nis: Drop uclibc check in anon python function jemalloc: Fix build on musl Leon Anavi (3): python3-networkx: Upgrade 2.6.1 -> 2.6.2 python3-pysonos: Upgrade 0.0.53 -> 0.0.54 python3-zeroconf: Upgrade 0.33.1 -> 0.33.2 Li Wang (1): openlldp: fix segfault Maksym Sloyko (1): libusbgx: Configure the Devices Used Martin Jansa (5): Convert to new override syntax layer.conf: Update to honister mariadb: manually fix the conversion packagegroup-meta-oe: manually finish override syntax conversion klibc.bbclass, image_types_sparse.bbclass, packagegroup-meta-oe.bb: update the overrides syntax conversion Mingli Yu (4): mariadb: redefine log-error item jemalloc: add new recipe hdf5: improve reproducibility mariadb: Update SRC_URI Nicolas Dechesne (1): mbw: add new recipe Paulo Neves (1): htop: Add ncurses-terminfo-base to RDEPENDS Sakib Sajal (1): lmdb: add recipe Salman Ahmed (2): nginx: upgrade 1.18.0 -> 1.20.1 nginx: upgrade 1.19.6 -> 1.21.1 Tony Battersby (1): net-snmp: fix QA Issue after LDFLAGS change Yi Zhao (3): postfix: upgrade 3.6.1 -> 3.6.2 audit: upgrade 3.0.2 -> 3.0.3 audit: fix compile error for 2.8.5 Zang Ruochen (1): python3-robotframework: upgrade 4.0.3 -> 4.1 wangmy (17): evince: upgrade 40.2 -> 40.4 gnome-backgrounds: upgrade 3.36.0 -> 3.38.0 gnome-desktop3: upgrade 3.36.6 -> 3.38.8 cmark: upgrade 0.30.0 -> 0.30.1 ctags: upgrade 5.9.20210711.0 -> 5.9.20210718.0 libnet-dns-perl: upgrade 1.31 -> 1.32 libtalloc: upgrade 2.3.2 -> 2.3.3 nghttp2: upgrade 1.43.0 -> 1.44.0 bats: upgrade 1.3.0 -> 1.4.1 networkmanager: upgrade 1.32.2 -> 1.32.4 gensio: upgrade 2.2.7 -> 2.2.8 libmbim: upgrade 1.24.8 -> 1.26.0 fetchmail: upgrade 6.4.19 -> 6.4.20 ctags: upgrade 5.9.20210718.0 -> 5.9.20210801.0 libblockdev: upgrade 2.25 -> 2.26 libqmi: upgrade 1.28.6 -> 1.28.8 monit: upgrade 5.28.0 -> 5.28.1 zangrc (15): python3-qrcode: upgrade 7.1 -> 7.2 python3-rdflib: upgrade 5.0.0 -> 6.0.0 python3-simplejson: upgrade 3.17.2 -> 3.17.3 python3-bitstring: upgrade 3.1.7 -> 3.1.9 python3-iso8601: upgrade 0.1.14 -> 0.1.16 python3-gmqtt: upgrade 0.6.9 -> 0.6.10 python3-graphviz: upgrade 0.16 -> 0.17 python3-smbus: upgrade 4.2 -> 4.3 python3-pandas: upgrade 1.3.0 -> 1.3.1 python3-progress: upgrade 1.5 -> 1.6 python3-sentry-sdk: upgrade 1.3.0 -> 1.3.1 python3-socketio: upgrade 5.3.0 -> 5.4.0 python3-tqdm: upgrade 4.61.2 -> 4.62.0 python3-twisted: upgrade 21.2.0 -> 21.7.0 python3-xlsxwriter: upgrade 1.4.4 -> 1.4.5 zhengruoqin (15): live555: upgrade 20210710 -> 20210720 libtest-warnings-perl: upgrade 0.030 -> 0.031 python3-pybind11: upgrade 2.6.2 -> 2.7.0 python3-pymongo: upgrade 3.11.4 -> 3.12.0 python3-sqlalchemy: upgrade 1.4.20 -> 1.4.22 python3-sentry-sdk: upgrade 1.2.0 -> 1.3.0 libcurses-perl: upgrade 1.37 -> 1.38 libdbd-sqlite-perl: upgrade 1.66 -> 1.68 libencode-perl: upgrade 3.10 -> 3.11 python3-bitarray: upgrade 2.2.2 -> 2.2.3 python3-cbor2: upgrade 5.4.0 -> 5.4.1 python3-gast: upgrade 0.5.0 -> 0.5.1 poppler: upgrade 21.07.0 -> 21.08.0 valijson: upgrade 0.4 -> 0.5 xwd: upgrade 1.0.7 -> 1.0.8 meta-security: 152cdb506b..c885d399cd: Armin Kuster (18): suricata.inc: exclude ppc in rust version suricata: Drop 4.1.x its EOL add meta-rust crowdsec: add pkg packagegroup-core-security.bb: fix suricat-ptest inclusion gitlab-ci.yml: streamline builds matrix krill: Add new pkg clamav: fix branch name and update meta-security: Convert to new override syntax meta-tpm: Convert to new override syntax meta-integrity: Convert to new override syntax meta-hardening: Convert to new override syntax meta-security-isafw: Convert to new override syntax meta-parsec: Convert to new override syntax meta-security-compliance: Convert to new override syntax dynamix-layers: Convert to new override syntax kas: Convert to new override syntax packagegroup-core-security.bb: only include suricat-ptest if rust is included Martin Jansa (1): layer.conf: Update to honister Signed-off-by: Patrick Williams Change-Id: Iec7301cf1c43b7cec462dcf88292a8b1b12a5045 --- poky/bitbake/bin/bitbake | 2 +- poky/bitbake/contrib/vim/plugin/newbbappend.vim | 2 +- poky/bitbake/contrib/vim/syntax/bitbake.vim | 11 +- poky/bitbake/doc/Makefile | 2 +- .../bitbake-user-manual-execution.rst | 54 ++--- .../bitbake-user-manual-fetching.rst | 20 +- .../bitbake-user-manual-hello.rst | 14 +- .../bitbake-user-manual-intro.rst | 13 +- .../bitbake-user-manual-metadata.rst | 171 ++++++++-------- .../bitbake-user-manual-ref-variables.rst | 217 +++++++++++---------- poky/bitbake/lib/bb/__init__.py | 2 +- poky/bitbake/lib/bb/asyncrpc/client.py | 11 +- poky/bitbake/lib/bb/asyncrpc/serv.py | 21 ++ poky/bitbake/lib/bb/cache.py | 4 +- poky/bitbake/lib/bb/command.py | 14 +- poky/bitbake/lib/bb/cooker.py | 3 +- poky/bitbake/lib/bb/data_smart.py | 123 ++++++------ poky/bitbake/lib/bb/fetch2/__init__.py | 4 +- poky/bitbake/lib/bb/parse/ast.py | 4 +- poky/bitbake/lib/bb/runqueue.py | 58 +++--- poky/bitbake/lib/bb/server/process.py | 6 +- poky/bitbake/lib/bb/siggen.py | 4 +- poky/bitbake/lib/bb/tests/codeparser.py | 4 +- poky/bitbake/lib/bb/tests/data.py | 95 ++++----- poky/bitbake/lib/bb/tests/parse.py | 18 +- poky/bitbake/lib/bb/ui/taskexp.py | 5 + poky/bitbake/lib/bb/utils.py | 2 +- poky/bitbake/lib/hashserv/tests.py | 54 +++-- poky/bitbake/lib/toaster/orm/models.py | 2 +- poky/bitbake/lib/toaster/toastergui/views.py | 4 +- .../toastermain/management/commands/buildimport.py | 2 +- 31 files changed, 515 insertions(+), 431 deletions(-) (limited to 'poky/bitbake') diff --git a/poky/bitbake/bin/bitbake b/poky/bitbake/bin/bitbake index f0ff2400fc..dc1873af69 100755 --- a/poky/bitbake/bin/bitbake +++ b/poky/bitbake/bin/bitbake @@ -26,7 +26,7 @@ from bb.main import bitbake_main, BitBakeConfigParameters, BBMainException if sys.getfilesystemencoding() != "utf-8": sys.exit("Please use a locale setting which supports UTF-8 (such as LANG=en_US.UTF-8).\nPython can't change the filesystem locale after loading so we need a UTF-8 when Python starts or things won't work.") -__version__ = "1.51.0" +__version__ = "1.51.1" if __name__ == "__main__": if __version__ != bb.__version__: diff --git a/poky/bitbake/contrib/vim/plugin/newbbappend.vim b/poky/bitbake/contrib/vim/plugin/newbbappend.vim index e04174cf62..3f65f79cdc 100644 --- a/poky/bitbake/contrib/vim/plugin/newbbappend.vim +++ b/poky/bitbake/contrib/vim/plugin/newbbappend.vim @@ -20,7 +20,7 @@ fun! NewBBAppendTemplate() set nopaste " New bbappend template - 0 put ='FILESEXTRAPATHS_prepend := \"${THISDIR}/${PN}:\"' + 0 put ='FILESEXTRAPATHS:prepend := \"${THISDIR}/${PN}:\"' 2 if paste == 1 diff --git a/poky/bitbake/contrib/vim/syntax/bitbake.vim b/poky/bitbake/contrib/vim/syntax/bitbake.vim index f964621ae5..d8aa0f1ba1 100644 --- a/poky/bitbake/contrib/vim/syntax/bitbake.vim +++ b/poky/bitbake/contrib/vim/syntax/bitbake.vim @@ -51,9 +51,9 @@ syn region bbString matchgroup=bbQuote start=+'+ skip=+\\$+ end=+'+ syn match bbExport "^export" nextgroup=bbIdentifier skipwhite syn keyword bbExportFlag export contained nextgroup=bbIdentifier skipwhite syn match bbIdentifier "[a-zA-Z0-9\-_\.\/\+]\+" display contained -syn match bbVarDeref "${[a-zA-Z0-9\-_\.\/\+]\+}" contained +syn match bbVarDeref "${[a-zA-Z0-9\-_:\.\/\+]\+}" contained syn match bbVarEq "\(:=\|+=\|=+\|\.=\|=\.\|?=\|??=\|=\)" contained nextgroup=bbVarValue -syn match bbVarDef "^\(export\s*\)\?\([a-zA-Z0-9\-_\.\/\+]\+\(_[${}a-zA-Z0-9\-_\.\/\+]\+\)\?\)\s*\(:=\|+=\|=+\|\.=\|=\.\|?=\|??=\|=\)\@=" contains=bbExportFlag,bbIdentifier,bbVarDeref nextgroup=bbVarEq +syn match bbVarDef "^\(export\s*\)\?\([a-zA-Z0-9\-_\.\/\+][${}a-zA-Z0-9\-_:\.\/\+]*\)\s*\(:=\|+=\|=+\|\.=\|=\.\|?=\|??=\|=\)\@=" contains=bbExportFlag,bbIdentifier,bbOverrideOperator,bbVarDeref nextgroup=bbVarEq syn match bbVarValue ".*$" contained contains=bbString,bbVarDeref,bbVarPyValue syn region bbVarPyValue start=+${@+ skip=+\\$+ end=+}+ contained contains=@python @@ -77,13 +77,15 @@ syn keyword bbOEFunctions do_fetch do_unpack do_patch do_configure do_comp " Generic Functions syn match bbFunction "\h[0-9A-Za-z_\-\.]*" display contained contains=bbOEFunctions +syn keyword bbOverrideOperator append prepend contained + " BitBake shell metadata syn include @shell syntax/sh.vim if exists("b:current_syntax") unlet b:current_syntax endif syn keyword bbShFakeRootFlag fakeroot contained -syn match bbShFuncDef "^\(fakeroot\s*\)\?\([\.0-9A-Za-z_${}\-\.]\+\)\(python\)\@` @@ -822,7 +823,7 @@ does not suit a ``.bbclass`` file. Similar to how BitBake handles :ref:`include `, if the path specified on the require line is a relative path, BitBake -locates the first file it can find within ``BBPATH``. +locates the first file it can find within :term:`BBPATH`. As an example, suppose you have two versions of a recipe (e.g. ``foo_1.2.2.bb`` and ``foo_2.0.0.bb``) where each version contains some @@ -851,7 +852,7 @@ As an example, suppose you needed to inherit a class file called This configuration directive causes the named class to be inherited at the point of the directive during parsing. As with the ``inherit`` directive, the ``.bbclass`` file must be located in a "classes" -subdirectory in one of the directories specified in ``BBPATH``. +subdirectory in one of the directories specified in :term:`BBPATH`. .. note:: @@ -907,7 +908,7 @@ rules. The scripts are executed by ``/bin/sh``, which may not be a bash shell but might be something such as ``dash``. You should not use Bash-specific script (bashisms). -Overrides and override-style operators like ``_append`` and ``_prepend`` +Overrides and override-style operators like ``:append`` and ``:prepend`` can also be applied to shell functions. Most commonly, this application would be used in a ``.bbappend`` file to modify functions in the main recipe. It can also be used to modify functions inherited from classes. @@ -919,7 +920,7 @@ As an example, consider the following:: fn } - fn_prepend() { + fn:prepend() { bbplain second } @@ -927,7 +928,7 @@ As an example, consider the following:: bbplain third } - do_foo_append() { + do_foo:append() { bbplain fourth } @@ -977,7 +978,7 @@ override-style operators to BitBake-style Python functions. As an example, consider the following:: - python do_foo_prepend() { + python do_foo:prepend() { bb.plain("first") } @@ -985,7 +986,7 @@ As an example, consider the following:: bb.plain("second") } - python do_foo_append() { + python do_foo:append() { bb.plain("third") } @@ -1015,7 +1016,7 @@ is an example:: SOMECONDITION = "1" DEPENDS = "${@get_depends(d)}" -This would result in ``DEPENDS`` containing ``dependencywithcond``. +This would result in :term:`DEPENDS` containing ``dependencywithcond``. Here are some things to know about Python functions: @@ -1134,12 +1135,12 @@ equivalent to the following snippet:: values set for the variables within the anonymous functions become available to tasks, which always run after parsing. -Overrides and override-style operators such as "``_append``" are applied +Overrides and override-style operators such as "``:append``" are applied before anonymous functions run. In the following example, ``FOO`` ends up with the value "foo from anonymous":: FOO = "foo" - FOO_append = " from outside" + FOO:append = " from outside" python () { d.setVar("FOO", "foo from anonymous") @@ -1164,7 +1165,7 @@ To understand the benefits of this feature, consider the basic scenario where a class defines a task function and your recipe inherits the class. In this basic scenario, your recipe inherits the task function as defined in the class. If desired, your recipe can add to the start and -end of the function by using the "_prepend" or "_append" operations +end of the function by using the ":prepend" or ":append" operations respectively, or it can redefine the function completely. However, if it redefines the function, there is no means for it to call the class version of the function. ``EXPORT_FUNCTIONS`` provides a mechanism that @@ -1382,7 +1383,7 @@ Sometimes, it is useful to be able to obtain information from the original execution environment. BitBake saves a copy of the original environment into a special variable named :term:`BB_ORIGENV`. -The ``BB_ORIGENV`` variable returns a datastore object that can be +The :term:`BB_ORIGENV` variable returns a datastore object that can be queried using the standard datastore operators such as ``getVar(, False)``. The datastore object is useful, for example, to find the original ``DISPLAY`` variable. Here is an example:: @@ -1467,7 +1468,7 @@ functionality of the task: can result in unpredictable behavior. - Setting the varflag to a value greater than the value used in - the ``BB_NUMBER_THREADS`` variable causes ``number_threads`` to + the :term:`BB_NUMBER_THREADS` variable causes ``number_threads`` to have no effect. - ``[postfuncs]``: List of functions to call after the completion of @@ -1537,7 +1538,7 @@ intent is to make it easy to do things like email notification on build failures. Following is an example event handler that prints the name of the event -and the content of the ``FILE`` variable:: +and the content of the :term:`FILE` variable:: addhandler myclass_eventhandler python myclass_eventhandler() { @@ -1576,7 +1577,7 @@ might have an interest in viewing: - ``bb.event.ConfigParsed()``: Fired when the base configuration; which consists of ``bitbake.conf``, ``base.bbclass`` and any global - ``INHERIT`` statements; has been parsed. You can see multiple such + :term:`INHERIT` statements; has been parsed. You can see multiple such events when each of the workers parse the base configuration or if the server changes configuration and reparses. Any given datastore only has one such event executed against it, however. If @@ -1733,13 +1734,13 @@ Build Dependencies BitBake uses the :term:`DEPENDS` variable to manage build time dependencies. The ``[deptask]`` varflag for tasks signifies -the task of each item listed in ``DEPENDS`` that must complete before +the task of each item listed in :term:`DEPENDS` that must complete before that task can be executed. Here is an example:: do_configure[deptask] = "do_populate_sysroot" In this example, the ``do_populate_sysroot`` task -of each item in ``DEPENDS`` must complete before ``do_configure`` can +of each item in :term:`DEPENDS` must complete before ``do_configure`` can execute. Runtime Dependencies @@ -1748,8 +1749,8 @@ Runtime Dependencies BitBake uses the :term:`PACKAGES`, :term:`RDEPENDS`, and :term:`RRECOMMENDS` variables to manage runtime dependencies. -The ``PACKAGES`` variable lists runtime packages. Each of those packages -can have ``RDEPENDS`` and ``RRECOMMENDS`` runtime dependencies. The +The :term:`PACKAGES` variable lists runtime packages. Each of those packages +can have :term:`RDEPENDS` and :term:`RRECOMMENDS` runtime dependencies. The ``[rdeptask]`` flag for tasks is used to signify the task of each item runtime dependency which must have completed before that task can be executed. :: @@ -1757,9 +1758,9 @@ executed. :: do_package_qa[rdeptask] = "do_packagedata" In the previous -example, the ``do_packagedata`` task of each item in ``RDEPENDS`` must +example, the ``do_packagedata`` task of each item in :term:`RDEPENDS` must have completed before ``do_package_qa`` can execute. -Although ``RDEPENDS`` contains entries from the +Although :term:`RDEPENDS` contains entries from the runtime dependency namespace, BitBake knows how to map them back to the build-time dependency namespace, in which the tasks are defined. @@ -1802,7 +1803,7 @@ Inter-Task Dependencies BitBake uses the ``[depends]`` flag in a more generic form to manage inter-task dependencies. This more generic form allows for inter-dependency checks for specific tasks rather than checks for the -data in ``DEPENDS``. Here is an example:: +data in :term:`DEPENDS`. Here is an example:: do_patch[depends] = "quilt-native:do_populate_sysroot" diff --git a/poky/bitbake/doc/bitbake-user-manual/bitbake-user-manual-ref-variables.rst b/poky/bitbake/doc/bitbake-user-manual/bitbake-user-manual-ref-variables.rst index 2dca52c4a0..6283c2654c 100644 --- a/poky/bitbake/doc/bitbake-user-manual/bitbake-user-manual-ref-variables.rst +++ b/poky/bitbake/doc/bitbake-user-manual/bitbake-user-manual-ref-variables.rst @@ -24,13 +24,14 @@ overview of their function and contents. described here in this glossary. .. glossary:: + :sorted: :term:`ASSUME_PROVIDED` Lists recipe names (:term:`PN` values) BitBake does not attempt to build. Instead, BitBake assumes these recipes have already been built. - In OpenEmbedded-Core, ``ASSUME_PROVIDED`` mostly specifies native + In OpenEmbedded-Core, :term:`ASSUME_PROVIDED` mostly specifies native tools that should not be built. An example is ``git-native``, which when specified allows for the Git binary from the host to be used rather than building ``git-native``. @@ -83,14 +84,14 @@ overview of their function and contents. - Attempts to access networks not in the host list cause a failure. - Using ``BB_ALLOWED_NETWORKS`` in conjunction with + Using :term:`BB_ALLOWED_NETWORKS` in conjunction with :term:`PREMIRRORS` is very useful. Adding the - host you want to use to ``PREMIRRORS`` results in the source code + host you want to use to :term:`PREMIRRORS` results in the source code being fetched from an allowed location and avoids raising an error when a host that is not allowed is in a :term:`SRC_URI` statement. This is because the - fetcher does not attempt to use the host listed in ``SRC_URI`` after - a successful fetch from the ``PREMIRRORS`` occurs. + fetcher does not attempt to use the host listed in :term:`SRC_URI` after + a successful fetch from the :term:`PREMIRRORS` occurs. :term:`BB_CONSOLELOG` Specifies the path to a log file into which BitBake's user interface @@ -177,7 +178,7 @@ overview of their function and contents. issues a warning when the disk space in the ``${SSTATE_DIR}`` directory drops below 1 Gbyte or the number of free inodes drops below 100 Kbytes. Subsequent warnings are issued during intervals as - defined by the ``BB_DISKMON_WARNINTERVAL`` variable. + defined by the :term:`BB_DISKMON_WARNINTERVAL` variable. The second example stops the build after all currently executing tasks complete when the minimum disk space in the ``${TMPDIR}`` @@ -191,14 +192,14 @@ overview of their function and contents. :term:`BB_DISKMON_WARNINTERVAL` Defines the disk space and free inode warning intervals. - If you are going to use the ``BB_DISKMON_WARNINTERVAL`` variable, you + If you are going to use the :term:`BB_DISKMON_WARNINTERVAL` variable, you must also use the :term:`BB_DISKMON_DIRS` variable and define its action as "WARN". During the build, subsequent warnings are issued each time disk space or number of free inodes further reduces by the respective interval. - If you do not provide a ``BB_DISKMON_WARNINTERVAL`` variable and you - do use ``BB_DISKMON_DIRS`` with the "WARN" action, the disk + If you do not provide a :term:`BB_DISKMON_WARNINTERVAL` variable and you + do use :term:`BB_DISKMON_DIRS` with the "WARN" action, the disk monitoring interval defaults to the following: BB_DISKMON_WARNINTERVAL = "50M,5K" @@ -231,23 +232,23 @@ overview of their function and contents. based on the interval occur each time a respective interval is reached beyond the initial warning (i.e. 1 Gbytes and 100 Kbytes). - :term:`BB_ENV_WHITELIST` - Specifies the internal whitelist of variables to allow through from - the external environment into BitBake's datastore. If the value of - this variable is not specified (which is the default), the following - list is used: :term:`BBPATH`, :term:`BB_PRESERVE_ENV`, - :term:`BB_ENV_WHITELIST`, and :term:`BB_ENV_EXTRAWHITE`. + :term:`BB_ENV_EXTRAWHITE` + Specifies an additional set of variables to allow through (whitelist) + from the external environment into BitBake's datastore. This list of + variables are on top of the internal list set in + :term:`BB_ENV_WHITELIST`. .. note:: You must set this variable in the external environment in order for it to work. - :term:`BB_ENV_EXTRAWHITE` - Specifies an additional set of variables to allow through (whitelist) - from the external environment into BitBake's datastore. This list of - variables are on top of the internal list set in - :term:`BB_ENV_WHITELIST`. + :term:`BB_ENV_WHITELIST` + Specifies the internal whitelist of variables to allow through from + the external environment into BitBake's datastore. If the value of + this variable is not specified (which is the default), the following + list is used: :term:`BBPATH`, :term:`BB_PRESERVE_ENV`, + :term:`BB_ENV_WHITELIST`, and :term:`BB_ENV_EXTRAWHITE`. .. note:: @@ -263,7 +264,7 @@ overview of their function and contents. :term:`BB_FILENAME` Contains the filename of the recipe that owns the currently running task. For example, if the ``do_fetch`` task that resides in the - ``my-recipe.bb`` is executing, the ``BB_FILENAME`` variable contains + ``my-recipe.bb`` is executing, the :term:`BB_FILENAME` variable contains "/foo/path/my-recipe.bb". :term:`BB_GENERATE_MIRROR_TARBALLS` @@ -276,18 +277,6 @@ overview of their function and contents. BB_GENERATE_MIRROR_TARBALLS = "1" - :term:`BB_HASHCONFIG_WHITELIST` - Lists variables that are excluded from base configuration checksum, - which is used to determine if the cache can be reused. - - One of the ways BitBake determines whether to re-parse the main - metadata is through checksums of the variables in the datastore of - the base configuration data. There are variables that you typically - want to exclude when checking whether or not to re-parse and thus - rebuild the cache. As an example, you would usually exclude ``TIME`` - and ``DATE`` because these variables are always changing. If you did - not exclude them, BitBake would never reuse the cache. - :term:`BB_HASHBASE_WHITELIST` Lists variables that are excluded from checksum and dependency data. Variables that are excluded can therefore change without affecting @@ -309,6 +298,18 @@ overview of their function and contents. However, the more accurate the data returned, the more efficient the build will be. + :term:`BB_HASHCONFIG_WHITELIST` + Lists variables that are excluded from base configuration checksum, + which is used to determine if the cache can be reused. + + One of the ways BitBake determines whether to re-parse the main + metadata is through checksums of the variables in the datastore of + the base configuration data. There are variables that you typically + want to exclude when checking whether or not to re-parse and thus + rebuild the cache. As an example, you would usually exclude ``TIME`` + and ``DATE`` because these variables are always changing. If you did + not exclude them, BitBake would never reuse the cache. + :term:`BB_HASHSERVE` Specifies the Hash Equivalence server to use. @@ -333,7 +334,7 @@ overview of their function and contents. :term:`BB_LOGFMT` Specifies the name of the log files saved into - ``${``\ :term:`T`\ ``}``. By default, the ``BB_LOGFMT`` + ``${``\ :term:`T`\ ``}``. By default, the :term:`BB_LOGFMT` variable is undefined and the log file names get created using the following form:: @@ -357,15 +358,15 @@ overview of their function and contents. running builds when not connected to the Internet, and when operating in certain kinds of firewall environments. + :term:`BB_NUMBER_PARSE_THREADS` + Sets the number of threads BitBake uses when parsing. By default, the + number of threads is equal to the number of cores on the system. + :term:`BB_NUMBER_THREADS` The maximum number of tasks BitBake should run in parallel at any one time. If your host development system supports multiple cores, a good rule of thumb is to set this variable to twice the number of cores. - :term:`BB_NUMBER_PARSE_THREADS` - Sets the number of threads BitBake uses when parsing. By default, the - number of threads is equal to the number of cores on the system. - :term:`BB_ORIGENV` Contains a copy of the original external environment in which BitBake was run. The copy is taken before any whitelisted variable values are @@ -388,7 +389,7 @@ overview of their function and contents. :term:`BB_RUNFMT` Specifies the name of the executable script files (i.e. run files) saved into ``${``\ :term:`T`\ ``}``. By default, the - ``BB_RUNFMT`` variable is undefined and the run file names get + :term:`BB_RUNFMT` variable is undefined and the run file names get created using the following form:: run.{task}.{pid} @@ -454,7 +455,7 @@ overview of their function and contents. :term:`BB_SRCREV_POLICY` Defines the behavior of the fetcher when it interacts with source control systems and dynamic source revisions. The - ``BB_SRCREV_POLICY`` variable is useful when working without a + :term:`BB_SRCREV_POLICY` variable is useful when working without a network. The variable can be set using one of two policies: @@ -498,7 +499,7 @@ overview of their function and contents. Allows adjustment of a task's Input/Output priority. During Autobuilder testing, random failures can occur for tasks due to I/O starvation. These failures occur during various QEMU runtime - timeouts. You can use the ``BB_TASK_IONICE_LEVEL`` variable to adjust + timeouts. You can use the :term:`BB_TASK_IONICE_LEVEL` variable to adjust the I/O priority of these tasks. .. note:: @@ -572,13 +573,13 @@ overview of their function and contents. .. note:: - Internally, the ``BBCLASSEXTEND`` mechanism generates recipe + Internally, the :term:`BBCLASSEXTEND` mechanism generates recipe variants by rewriting variable values and applying overrides such as ``_class-native``. For example, to generate a native version of a recipe, a :term:`DEPENDS` on "foo" is - rewritten to a ``DEPENDS`` on "foo-native". + rewritten to a :term:`DEPENDS` on "foo-native". - Even when using ``BBCLASSEXTEND``, the recipe is only parsed once. + Even when using :term:`BBCLASSEXTEND`, the recipe is only parsed once. Parsing once adds some limitations. For example, it is not possible to include a different file depending on the variant, since ``include`` statements are processed when the recipe is @@ -614,14 +615,14 @@ overview of their function and contents. - effectively letting you control the precedence for the multiple layers. The precedence established through this variable stands regardless of a recipe's version (:term:`PV` variable). - For example, a layer that has a recipe with a higher ``PV`` value but - for which the ``BBFILE_PRIORITY`` is set to have a lower precedence + For example, a layer that has a recipe with a higher :term:`PV` value but + for which the :term:`BBFILE_PRIORITY` is set to have a lower precedence still has a lower precedence. - A larger value for the ``BBFILE_PRIORITY`` variable results in a + A larger value for the :term:`BBFILE_PRIORITY` variable results in a higher precedence. For example, the value 6 has a higher precedence - than the value 5. If not specified, the ``BBFILE_PRIORITY`` variable - is set based on layer dependencies (see the ``LAYERDEPENDS`` variable + than the value 5. If not specified, the :term:`BBFILE_PRIORITY` variable + is set based on layer dependencies (see the :term:`LAYERDEPENDS` variable for more information. The default priority, if unspecified for a layer with no dependencies, is the lowest defined priority + 1 (or 1 if no priorities are defined). @@ -644,7 +645,7 @@ overview of their function and contents. Activates content depending on presence of identified layers. You identify the layers by the collections that the layers define. - Use the ``BBFILES_DYNAMIC`` variable to avoid ``.bbappend`` files whose + Use the :term:`BBFILES_DYNAMIC` variable to avoid ``.bbappend`` files whose corresponding ``.bb`` file is in a layer that attempts to modify other layers through ``.bbappend`` but does not want to introduce a hard dependency on those other layers. @@ -653,7 +654,7 @@ overview of their function and contents. ``.bb`` files in case a layer is not present. Use this avoid hard dependency on those other layers. - Use the following form for ``BBFILES_DYNAMIC``:: + Use the following form for :term:`BBFILES_DYNAMIC`:: collection_name:filename_pattern @@ -690,7 +691,7 @@ overview of their function and contents. :term:`BBINCLUDELOGS_LINES` If :term:`BBINCLUDELOGS` is set, specifies the maximum number of lines from the task log file to print when - reporting a failed task. If you do not set ``BBINCLUDELOGS_LINES``, + reporting a failed task. If you do not set :term:`BBINCLUDELOGS_LINES`, the entire log is printed. :term:`BBLAYERS` @@ -716,7 +717,7 @@ overview of their function and contents. :term:`BBMASK` Prevents BitBake from processing recipes and recipe append files. - You can use the ``BBMASK`` variable to "hide" these ``.bb`` and + You can use the :term:`BBMASK` variable to "hide" these ``.bb`` and ``.bbappend`` files. BitBake ignores any recipe or recipe append files that match any of the expressions. It is as if BitBake does not see them at all. Consequently, matching files are not parsed or @@ -753,7 +754,7 @@ overview of their function and contents. Enables BitBake to perform multiple configuration builds and lists each separate configuration (multiconfig). You can use this variable to cause BitBake to build multiple targets where each target has a - separate configuration. Define ``BBMULTICONFIG`` in your + separate configuration. Define :term:`BBMULTICONFIG` in your ``conf/local.conf`` configuration file. As an example, the following line specifies three multiconfigs, each @@ -765,7 +766,7 @@ overview of their function and contents. build directory within a directory named ``conf/multiconfig`` (e.g. build_directory\ ``/conf/multiconfig/configA.conf``). - For information on how to use ``BBMULTICONFIG`` in an environment + For information on how to use :term:`BBMULTICONFIG` in an environment that supports building targets with multiple configurations, see the ":ref:`bitbake-user-manual/bitbake-user-manual-intro:executing a multiple configuration build`" section. @@ -776,7 +777,7 @@ overview of their function and contents. variable. If you run BitBake from a directory outside of the build directory, - you must be sure to set ``BBPATH`` to point to the build directory. + you must be sure to set :term:`BBPATH` to point to the build directory. Set the variable as you would any environment variable and then run BitBake:: @@ -823,7 +824,7 @@ overview of their function and contents. The most common usage of this is variable is to set it to "-1" within a recipe for a development version of a piece of software. Using the variable in this way causes the stable version of the recipe to build - by default in the absence of ``PREFERRED_VERSION`` being used to + by default in the absence of :term:`PREFERRED_VERSION` being used to build the development version. .. note:: @@ -836,7 +837,7 @@ overview of their function and contents. Lists a recipe's build-time dependencies (i.e. other recipe files). Consider this simple example for two recipes named "a" and "b" that - produce similarly named packages. In this example, the ``DEPENDS`` + produce similarly named packages. In this example, the :term:`DEPENDS` statement appears in the "a" recipe:: DEPENDS = "b" @@ -854,7 +855,7 @@ overview of their function and contents. :term:`DL_DIR` The central download directory used by the build process to store - downloads. By default, ``DL_DIR`` gets files suitable for mirroring for + downloads. By default, :term:`DL_DIR` gets files suitable for mirroring for everything except Git repositories. If you want tarballs of Git repositories, use the :term:`BB_GENERATE_MIRROR_TARBALLS` variable. @@ -869,14 +870,14 @@ overview of their function and contents. .. note:: - Recipes added to ``EXCLUDE_FROM_WORLD`` may still be built during a world + Recipes added to :term:`EXCLUDE_FROM_WORLD` may still be built during a world build in order to satisfy dependencies of other recipes. Adding a - recipe to ``EXCLUDE_FROM_WORLD`` only ensures that the recipe is not + recipe to :term:`EXCLUDE_FROM_WORLD` only ensures that the recipe is not explicitly added to the list of build targets in a world build. :term:`FAKEROOT` Contains the command to use when running a shell script in a fakeroot - environment. The ``FAKEROOT`` variable is obsolete and has been + environment. The :term:`FAKEROOT` variable is obsolete and has been replaced by the other ``FAKEROOT*`` variables. See these entries in the glossary for more information. @@ -939,9 +940,9 @@ overview of their function and contents. Causes the named class or classes to be inherited globally. Anonymous functions in the class or classes are not executed for the base configuration and in each individual recipe. The OpenEmbedded build - system ignores changes to ``INHERIT`` in individual recipes. + system ignores changes to :term:`INHERIT` in individual recipes. - For more information on ``INHERIT``, see the + For more information on :term:`INHERIT`, see the ":ref:`bitbake-user-manual/bitbake-user-manual-metadata:\`\`inherit\`\` configuration directive`" section. @@ -989,7 +990,7 @@ overview of their function and contents. the build system searches for source code, it first tries the local download directory. If that location fails, the build system tries locations defined by :term:`PREMIRRORS`, the - upstream source, and then locations specified by ``MIRRORS`` in that + upstream source, and then locations specified by :term:`MIRRORS` in that order. :term:`MULTI_PROVIDER_WHITELIST` @@ -1006,12 +1007,12 @@ overview of their function and contents. ``virtual/kernel``, and so forth). :term:`OVERRIDES` - BitBake uses ``OVERRIDES`` to control what variables are overridden + BitBake uses :term:`OVERRIDES` to control what variables are overridden after BitBake parses recipes and configuration files. Following is a simple example that uses an overrides list based on machine architectures: OVERRIDES = "arm:x86:mips:powerpc" You can - find information on how to use ``OVERRIDES`` in the + find information on how to use :term:`OVERRIDES` in the ":ref:`bitbake-user-manual/bitbake-user-manual-metadata:conditional syntax (overrides)`" section. @@ -1025,11 +1026,11 @@ overview of their function and contents. :term:`PACKAGES_DYNAMIC` A promise that your recipe satisfies runtime dependencies for optional modules that are found in other recipes. - ``PACKAGES_DYNAMIC`` does not actually satisfy the dependencies, it + :term:`PACKAGES_DYNAMIC` does not actually satisfy the dependencies, it only states that they should be satisfied. For example, if a hard, runtime dependency (:term:`RDEPENDS`) of another package is satisfied during the build through the - ``PACKAGES_DYNAMIC`` variable, but a package with the module name is + :term:`PACKAGES_DYNAMIC` variable, but a package with the module name is never actually produced, then the other package will be broken. :term:`PE` @@ -1068,8 +1069,8 @@ overview of their function and contents. :term:`PREFERRED_PROVIDERS` Determines which recipe should be given preference for cases where multiple recipes provide the same item. Functionally, - ``PREFERRED_PROVIDERS`` is identical to - :term:`PREFERRED_PROVIDER`. However, the ``PREFERRED_PROVIDERS`` variable + :term:`PREFERRED_PROVIDERS` is identical to + :term:`PREFERRED_PROVIDER`. However, the :term:`PREFERRED_PROVIDERS` variable lets you define preferences for multiple situations using the following form:: @@ -1087,7 +1088,7 @@ overview of their function and contents. select, and you should set :term:`PV` accordingly for precedence. - The ``PREFERRED_VERSION`` variable supports limited wildcard use + The :term:`PREFERRED_VERSION` variable supports limited wildcard use through the "``%``" character. You can use the character to match any number of characters, which can be useful when specifying versions that contain long revision numbers that potentially change. Here are @@ -1110,14 +1111,14 @@ overview of their function and contents. Specifies additional paths from which BitBake gets source code. When the build system searches for source code, it first tries the local download directory. If that location fails, the build system tries - locations defined by ``PREMIRRORS``, the upstream source, and then + locations defined by :term:`PREMIRRORS`, the upstream source, and then locations specified by :term:`MIRRORS` in that order. Typically, you would add a specific server for the build system to attempt before any others by adding something like the following to your configuration:: - PREMIRRORS_prepend = "\ + PREMIRRORS:prepend = "\ git://.*/.* http://www.yoctoproject.org/sources/ \n \ ftp://.*/.* http://www.yoctoproject.org/sources/ \n \ http://.*/.* http://www.yoctoproject.org/sources/ \n \ @@ -1130,25 +1131,25 @@ overview of their function and contents. :term:`PROVIDES` A list of aliases by which a particular recipe can be known. By - default, a recipe's own ``PN`` is implicitly already in its - ``PROVIDES`` list. If a recipe uses ``PROVIDES``, the additional + default, a recipe's own :term:`PN` is implicitly already in its + :term:`PROVIDES` list. If a recipe uses :term:`PROVIDES`, the additional aliases are synonyms for the recipe and can be useful satisfying dependencies of other recipes during the build as specified by - ``DEPENDS``. + :term:`DEPENDS`. - Consider the following example ``PROVIDES`` statement from a recipe + Consider the following example :term:`PROVIDES` statement from a recipe file ``libav_0.8.11.bb``:: PROVIDES += "libpostproc" - The ``PROVIDES`` statement results in the "libav" recipe also being known + The :term:`PROVIDES` statement results in the "libav" recipe also being known as "libpostproc". In addition to providing recipes under alternate names, the - ``PROVIDES`` mechanism is also used to implement virtual targets. A + :term:`PROVIDES` mechanism is also used to implement virtual targets. A virtual target is a name that corresponds to some particular functionality (e.g. a Linux kernel). Recipes that provide the - functionality in question list the virtual target in ``PROVIDES``. + functionality in question list the virtual target in :term:`PROVIDES`. Recipes that depend on the functionality in question can include the virtual target in :term:`DEPENDS` to leave the choice of provider open. @@ -1160,12 +1161,12 @@ overview of their function and contents. :term:`PRSERV_HOST` The network based :term:`PR` service host and port. - Following is an example of how the ``PRSERV_HOST`` variable is set:: + Following is an example of how the :term:`PRSERV_HOST` variable is set:: PRSERV_HOST = "localhost:0" You must set the variable if you want to automatically start a local PR - service. You can set ``PRSERV_HOST`` to other values to use a remote PR + service. You can set :term:`PRSERV_HOST` to other values to use a remote PR service. :term:`PV` @@ -1177,24 +1178,24 @@ overview of their function and contents. a package in this list cannot be found during the build, you will get a build error. - Because the ``RDEPENDS`` variable applies to packages being built, + Because the :term:`RDEPENDS` variable applies to packages being built, you should always use the variable in a form with an attached package name. For example, suppose you are building a development package that depends on the ``perl`` package. In this case, you would use the - following ``RDEPENDS`` statement:: + following :term:`RDEPENDS` statement:: - RDEPENDS_${PN}-dev += "perl" + RDEPENDS:${PN}-dev += "perl" In the example, the development package depends on the ``perl`` package. - Thus, the ``RDEPENDS`` variable has the ``${PN}-dev`` package name as part + Thus, the :term:`RDEPENDS` variable has the ``${PN}-dev`` package name as part of the variable. BitBake supports specifying versioned dependencies. Although the syntax varies depending on the packaging format, BitBake hides these differences from you. Here is the general syntax to specify versions - with the ``RDEPENDS`` variable:: + with the :term:`RDEPENDS` variable:: - RDEPENDS_${PN} = "package (operator version)" + RDEPENDS:${PN} = "package (operator version)" For ``operator``, you can specify the following:: @@ -1207,7 +1208,7 @@ overview of their function and contents. For example, the following sets up a dependency on version 1.2 or greater of the package ``foo``:: - RDEPENDS_${PN} = "foo (>= 1.2)" + RDEPENDS:${PN} = "foo (>= 1.2)" For information on build-time dependencies, see the :term:`DEPENDS` variable. @@ -1218,39 +1219,39 @@ overview of their function and contents. :term:`REQUIRED_VERSION` If there are multiple versions of a recipe available, this variable - determines which version should be given preference. ``REQUIRED_VERSION`` + determines which version should be given preference. :term:`REQUIRED_VERSION` works in exactly the same manner as :term:`PREFERRED_VERSION`, except that if the specified version is not available then an error message is shown and the build fails immediately. - If both ``REQUIRED_VERSION`` and ``PREFERRED_VERSION`` are set for - the same recipe, the ``REQUIRED_VERSION`` value applies. + If both :term:`REQUIRED_VERSION` and :term:`PREFERRED_VERSION` are set for + the same recipe, the :term:`REQUIRED_VERSION` value applies. :term:`RPROVIDES` A list of package name aliases that a package also provides. These aliases are useful for satisfying runtime dependencies of other packages both during the build and on the target (as specified by - ``RDEPENDS``). + :term:`RDEPENDS`). As with all package-controlling variables, you must always use the variable in conjunction with a package name override. Here is an example:: - RPROVIDES_${PN} = "widget-abi-2" + RPROVIDES:${PN} = "widget-abi-2" :term:`RRECOMMENDS` A list of packages that extends the usability of a package being built. The package being built does not depend on this list of packages in order to successfully build, but needs them for the extended usability. To specify runtime dependencies for packages, see - the ``RDEPENDS`` variable. + the :term:`RDEPENDS` variable. BitBake supports specifying versioned recommends. Although the syntax varies depending on the packaging format, BitBake hides these differences from you. Here is the general syntax to specify versions - with the ``RRECOMMENDS`` variable:: + with the :term:`RRECOMMENDS` variable:: - RRECOMMENDS_${PN} = "package (operator version)" + RRECOMMENDS:${PN} = "package (operator version)" For ``operator``, you can specify the following:: @@ -1263,7 +1264,7 @@ overview of their function and contents. For example, the following sets up a recommend on version 1.2 or greater of the package ``foo``:: - RRECOMMENDS_${PN} = "foo (>= 1.2)" + RRECOMMENDS:${PN} = "foo (>= 1.2)" :term:`SECTION` The section in which packages should be categorized. @@ -1272,10 +1273,10 @@ overview of their function and contents. The list of source files - local or remote. This variable tells BitBake which bits to pull for the build and how to pull them. For example, if the recipe or append file needs to fetch a single tarball - from the Internet, the recipe or append file uses a ``SRC_URI`` entry + from the Internet, the recipe or append file uses a :term:`SRC_URI` entry that specifies that tarball. On the other hand, if the recipe or append file needs to fetch a tarball and include a custom file, the - recipe or append file needs an ``SRC_URI`` variable that specifies + recipe or append file needs an :term:`SRC_URI` variable that specifies all those sources. The following list explains the available URI protocols: @@ -1328,8 +1329,8 @@ overview of their function and contents. subdirectory within the archive. - ``name`` : Specifies a name to be used for association with - ``SRC_URI`` checksums when you have more than one file specified - in ``SRC_URI``. + :term:`SRC_URI` checksums when you have more than one file specified + in :term:`SRC_URI`. - ``downloadfilename`` : Specifies the filename used when storing the downloaded file. @@ -1344,7 +1345,7 @@ overview of their function and contents. variable applies only when using Subversion, Git, Mercurial and Bazaar. If you want to build a fixed revision and you want to avoid performing a query on the remote repository every time BitBake parses - your recipe, you should specify a ``SRCREV`` that is a full revision + your recipe, you should specify a :term:`SRCREV` that is a full revision identifier and not just a tag. :term:`SRCREV_FORMAT` @@ -1353,10 +1354,10 @@ overview of their function and contents. :term:`SRC_URI`. The system needs help constructing these values under these - circumstances. Each component in the ``SRC_URI`` is assigned a name - and these are referenced in the ``SRCREV_FORMAT`` variable. Consider + circumstances. Each component in the :term:`SRC_URI` is assigned a name + and these are referenced in the :term:`SRCREV_FORMAT` variable. Consider an example with URLs named "machine" and "meta". In this case, - ``SRCREV_FORMAT`` could look like "machine_meta" and those names + :term:`SRCREV_FORMAT` could look like "machine_meta" and those names would have the SCM versions substituted into each position. Only one ``AUTOINC`` placeholder is added and if needed. And, this placeholder is placed at the start of the returned string. @@ -1368,7 +1369,7 @@ overview of their function and contents. :term:`STAMPCLEAN` Specifies the base path used to create recipe stamp files. Unlike the - :term:`STAMP` variable, ``STAMPCLEAN`` can contain + :term:`STAMP` variable, :term:`STAMPCLEAN` can contain wildcards to match the range of files a clean operation should remove. BitBake uses a clean operation to remove any other stamps it should be removing when creating a new stamp. diff --git a/poky/bitbake/lib/bb/__init__.py b/poky/bitbake/lib/bb/__init__.py index a144bd609a..c1e30697b3 100644 --- a/poky/bitbake/lib/bb/__init__.py +++ b/poky/bitbake/lib/bb/__init__.py @@ -9,7 +9,7 @@ # SPDX-License-Identifier: GPL-2.0-only # -__version__ = "1.51.0" +__version__ = "1.51.1" import sys if sys.version_info < (3, 5, 0): diff --git a/poky/bitbake/lib/bb/asyncrpc/client.py b/poky/bitbake/lib/bb/asyncrpc/client.py index 79919c5be6..3eb4fdde8a 100644 --- a/poky/bitbake/lib/bb/asyncrpc/client.py +++ b/poky/bitbake/lib/bb/asyncrpc/client.py @@ -11,13 +11,14 @@ from . import chunkify, DEFAULT_MAX_CHUNK class AsyncClient(object): - def __init__(self, proto_name, proto_version, logger): + def __init__(self, proto_name, proto_version, logger, timeout=30): self.reader = None self.writer = None self.max_chunk = DEFAULT_MAX_CHUNK self.proto_name = proto_name self.proto_version = proto_version self.logger = logger + self.timeout = timeout async def connect_tcp(self, address, port): async def connect_sock(): @@ -70,14 +71,18 @@ class AsyncClient(object): async def send_message(self, msg): async def get_line(): - line = await self.reader.readline() + try: + line = await asyncio.wait_for(self.reader.readline(), self.timeout) + except asyncio.TimeoutError: + raise ConnectionError("Timed out waiting for server") + if not line: raise ConnectionError("Connection closed") line = line.decode("utf-8") if not line.endswith("\n"): - raise ConnectionError("Bad message %r" % msg) + raise ConnectionError("Bad message %r" % (line)) return line diff --git a/poky/bitbake/lib/bb/asyncrpc/serv.py b/poky/bitbake/lib/bb/asyncrpc/serv.py index ef20cb71df..4084f300df 100644 --- a/poky/bitbake/lib/bb/asyncrpc/serv.py +++ b/poky/bitbake/lib/bb/asyncrpc/serv.py @@ -9,6 +9,7 @@ import os import signal import socket import sys +import multiprocessing from . import chunkify, DEFAULT_MAX_CHUNK @@ -201,12 +202,14 @@ class AsyncServer(object): pass def signal_handler(self): + self.logger.debug("Got exit signal") self.loop.stop() def serve_forever(self): asyncio.set_event_loop(self.loop) try: self.loop.add_signal_handler(signal.SIGTERM, self.signal_handler) + signal.pthread_sigmask(signal.SIG_UNBLOCK, [signal.SIGTERM]) self.run_loop_forever() self.server.close() @@ -221,3 +224,21 @@ class AsyncServer(object): if self._cleanup_socket is not None: self._cleanup_socket() + + def serve_as_process(self, *, prefunc=None, args=()): + def run(): + if prefunc is not None: + prefunc(self, *args) + self.serve_forever() + + # Temporarily block SIGTERM. The server process will inherit this + # block which will ensure it doesn't receive the SIGTERM until the + # handler is ready for it + mask = signal.pthread_sigmask(signal.SIG_BLOCK, [signal.SIGTERM]) + try: + self.process = multiprocessing.Process(target=run) + self.process.start() + + return self.process + finally: + signal.pthread_sigmask(signal.SIG_SETMASK, mask) diff --git a/poky/bitbake/lib/bb/cache.py b/poky/bitbake/lib/bb/cache.py index 27eb271798..73bc6e9665 100644 --- a/poky/bitbake/lib/bb/cache.py +++ b/poky/bitbake/lib/bb/cache.py @@ -53,12 +53,12 @@ class RecipeInfoCommon(object): @classmethod def pkgvar(cls, var, packages, metadata): - return dict((pkg, cls.depvar("%s_%s" % (var, pkg), metadata)) + return dict((pkg, cls.depvar("%s:%s" % (var, pkg), metadata)) for pkg in packages) @classmethod def taskvar(cls, var, tasks, metadata): - return dict((task, cls.getvar("%s_task-%s" % (var, task), metadata)) + return dict((task, cls.getvar("%s:task-%s" % (var, task), metadata)) for task in tasks) @classmethod diff --git a/poky/bitbake/lib/bb/command.py b/poky/bitbake/lib/bb/command.py index f530cf844b..a81dcb1366 100644 --- a/poky/bitbake/lib/bb/command.py +++ b/poky/bitbake/lib/bb/command.py @@ -65,9 +65,17 @@ class Command: # Ensure cooker is ready for commands if command != "updateConfig" and command != "setFeatures": - self.cooker.init_configdata() - if not self.remotedatastores: - self.remotedatastores = bb.remotedata.RemoteDatastores(self.cooker) + try: + self.cooker.init_configdata() + if not self.remotedatastores: + self.remotedatastores = bb.remotedata.RemoteDatastores(self.cooker) + except (Exception, SystemExit) as exc: + import traceback + if isinstance(exc, bb.BBHandledException): + # We need to start returning real exceptions here. Until we do, we can't + # tell if an exception is an instance of bb.BBHandledException + return None, "bb.BBHandledException()\n" + traceback.format_exc() + return None, traceback.format_exc() if hasattr(CommandsSync, command): # Can run synchronous commands straight away diff --git a/poky/bitbake/lib/bb/cooker.py b/poky/bitbake/lib/bb/cooker.py index 39e10e6133..b2d69c28cf 100644 --- a/poky/bitbake/lib/bb/cooker.py +++ b/poky/bitbake/lib/bb/cooker.py @@ -390,8 +390,7 @@ class BBCooker: dbfile = (self.data.getVar("PERSISTENT_DIR") or self.data.getVar("CACHE")) + "/hashserv.db" self.hashservaddr = "unix://%s/hashserve.sock" % self.data.getVar("TOPDIR") self.hashserv = hashserv.create_server(self.hashservaddr, dbfile, sync=False) - self.hashserv.process = multiprocessing.Process(target=self.hashserv.serve_forever) - self.hashserv.process.start() + self.hashserv.serve_as_process() self.data.setVar("BB_HASHSERVE", self.hashservaddr) self.databuilder.origdata.setVar("BB_HASHSERVE", self.hashservaddr) self.databuilder.data.setVar("BB_HASHSERVE", self.hashservaddr) diff --git a/poky/bitbake/lib/bb/data_smart.py b/poky/bitbake/lib/bb/data_smart.py index f48726a348..65528c6ae6 100644 --- a/poky/bitbake/lib/bb/data_smart.py +++ b/poky/bitbake/lib/bb/data_smart.py @@ -26,9 +26,9 @@ from bb.COW import COWDictBase logger = logging.getLogger("BitBake.Data") -__setvar_keyword__ = ["_append", "_prepend", "_remove"] -__setvar_regexp__ = re.compile(r'(?P.*?)(?P_append|_prepend|_remove)(_(?P[^A-Z]*))?$') -__expand_var_regexp__ = re.compile(r"\${[a-zA-Z0-9\-_+./~]+?}") +__setvar_keyword__ = [":append", ":prepend", ":remove"] +__setvar_regexp__ = re.compile(r'(?P.*?)(?P:append|:prepend|:remove)(:(?P[^A-Z]*))?$') +__expand_var_regexp__ = re.compile(r"\${[a-zA-Z0-9\-_+./~:]+?}") __expand_python_regexp__ = re.compile(r"\${@.+?}") __whitespace_split__ = re.compile(r'(\s)') __override_regexp__ = re.compile(r'[a-z0-9]+') @@ -277,7 +277,7 @@ class VariableHistory(object): for (r, override) in d.overridedata[var]: for event in self.variable(r): loginfo = event.copy() - if 'flag' in loginfo and not loginfo['flag'].startswith("_"): + if 'flag' in loginfo and not loginfo['flag'].startswith(("_", ":")): continue loginfo['variable'] = var loginfo['op'] = 'override[%s]:%s' % (override, loginfo['op']) @@ -342,7 +342,7 @@ class VariableHistory(object): for event in history: if 'flag' in event: continue - if event['op'] == '_remove': + if event['op'] == ':remove': continue if isset and event['op'] == 'set?': continue @@ -481,7 +481,15 @@ class DataSmart(MutableMapping): def setVar(self, var, value, **loginfo): #print("var=" + str(var) + " val=" + str(value)) - var = var.replace(":", "_") + + if "_append" in var or "_prepend" in var or "_remove" in var: + info = "%s" % var + if "filename" in loginfo: + info += " file: %s" % loginfo[filename] + if "lineno" in loginfo: + info += " line: %s" % loginfo[lineno] + bb.fatal("Variable %s contains an operation using the old override syntax. Please convert this layer/metadata before attempting to use with a newer bitbake." % info) + self.expand_cache = {} parsing=False if 'parsing' in loginfo: @@ -510,7 +518,7 @@ class DataSmart(MutableMapping): # pay the cookie monster # more cookies for the cookie monster - if '_' in var: + if ':' in var: self._setvar_update_overrides(base, **loginfo) if base in self.overridevars: @@ -521,27 +529,27 @@ class DataSmart(MutableMapping): self._makeShadowCopy(var) if not parsing: - if "_append" in self.dict[var]: - del self.dict[var]["_append"] - if "_prepend" in self.dict[var]: - del self.dict[var]["_prepend"] - if "_remove" in self.dict[var]: - del self.dict[var]["_remove"] + if ":append" in self.dict[var]: + del self.dict[var][":append"] + if ":prepend" in self.dict[var]: + del self.dict[var][":prepend"] + if ":remove" in self.dict[var]: + del self.dict[var][":remove"] if var in self.overridedata: active = [] self.need_overrides() for (r, o) in self.overridedata[var]: if o in self.overridesset: active.append(r) - elif "_" in o: - if set(o.split("_")).issubset(self.overridesset): + elif ":" in o: + if set(o.split(":")).issubset(self.overridesset): active.append(r) for a in active: self.delVar(a) del self.overridedata[var] # more cookies for the cookie monster - if '_' in var: + if ':' in var: self._setvar_update_overrides(var, **loginfo) # setting var @@ -567,8 +575,8 @@ class DataSmart(MutableMapping): def _setvar_update_overrides(self, var, **loginfo): # aka pay the cookie monster - override = var[var.rfind('_')+1:] - shortvar = var[:var.rfind('_')] + override = var[var.rfind(':')+1:] + shortvar = var[:var.rfind(':')] while override and __override_regexp__.match(override): if shortvar not in self.overridedata: self.overridedata[shortvar] = [] @@ -577,9 +585,9 @@ class DataSmart(MutableMapping): self.overridedata[shortvar] = list(self.overridedata[shortvar]) self.overridedata[shortvar].append([var, override]) override = None - if "_" in shortvar: - override = var[shortvar.rfind('_')+1:] - shortvar = var[:shortvar.rfind('_')] + if ":" in shortvar: + override = var[shortvar.rfind(':')+1:] + shortvar = var[:shortvar.rfind(':')] if len(shortvar) == 0: override = None @@ -590,8 +598,6 @@ class DataSmart(MutableMapping): """ Rename the variable key to newkey """ - key = key.replace(":", "_") - newkey = newkey.replace(":", "_") if key == newkey: bb.warn("Calling renameVar with equivalent keys (%s) is invalid" % key) return @@ -620,7 +626,7 @@ class DataSmart(MutableMapping): self.overridedata[newkey].append([v.replace(key, newkey), o]) self.renameVar(v, v.replace(key, newkey)) - if '_' in newkey and val is None: + if ':' in newkey and val is None: self._setvar_update_overrides(newkey, **loginfo) loginfo['variable'] = key @@ -632,15 +638,14 @@ class DataSmart(MutableMapping): def appendVar(self, var, value, **loginfo): loginfo['op'] = 'append' self.varhistory.record(**loginfo) - self.setVar(var + "_append", value, ignore=True, parsing=True) + self.setVar(var + ":append", value, ignore=True, parsing=True) def prependVar(self, var, value, **loginfo): loginfo['op'] = 'prepend' self.varhistory.record(**loginfo) - self.setVar(var + "_prepend", value, ignore=True, parsing=True) + self.setVar(var + ":prepend", value, ignore=True, parsing=True) def delVar(self, var, **loginfo): - var = var.replace(":", "_") self.expand_cache = {} loginfo['detail'] = "" @@ -649,9 +654,9 @@ class DataSmart(MutableMapping): self.dict[var] = {} if var in self.overridedata: del self.overridedata[var] - if '_' in var: - override = var[var.rfind('_')+1:] - shortvar = var[:var.rfind('_')] + if ':' in var: + override = var[var.rfind(':')+1:] + shortvar = var[:var.rfind(':')] while override and override.islower(): try: if shortvar in self.overridedata: @@ -661,14 +666,13 @@ class DataSmart(MutableMapping): except ValueError as e: pass override = None - if "_" in shortvar: - override = var[shortvar.rfind('_')+1:] - shortvar = var[:shortvar.rfind('_')] + if ":" in shortvar: + override = var[shortvar.rfind(':')+1:] + shortvar = var[:shortvar.rfind(':')] if len(shortvar) == 0: override = None def setVarFlag(self, var, flag, value, **loginfo): - var = var.replace(":", "_") self.expand_cache = {} if 'op' not in loginfo: @@ -679,7 +683,7 @@ class DataSmart(MutableMapping): self._makeShadowCopy(var) self.dict[var][flag] = value - if flag == "_defaultval" and '_' in var: + if flag == "_defaultval" and ':' in var: self._setvar_update_overrides(var, **loginfo) if flag == "_defaultval" and var in self.overridevars: self._setvar_update_overridevars(var, value) @@ -692,7 +696,6 @@ class DataSmart(MutableMapping): self.dict["__exportlist"]["_content"].add(var) def getVarFlag(self, var, flag, expand=True, noweakdefault=False, parsing=False, retparser=False): - var = var.replace(":", "_") if flag == "_content": cachename = var else: @@ -712,11 +715,11 @@ class DataSmart(MutableMapping): active = {} self.need_overrides() for (r, o) in overridedata: - # What about double overrides both with "_" in the name? + # FIXME What about double overrides both with "_" in the name? if o in self.overridesset: active[o] = r - elif "_" in o: - if set(o.split("_")).issubset(self.overridesset): + elif ":" in o: + if set(o.split(":")).issubset(self.overridesset): active[o] = r mod = True @@ -724,10 +727,10 @@ class DataSmart(MutableMapping): mod = False for o in self.overrides: for a in active.copy(): - if a.endswith("_" + o): + if a.endswith(":" + o): t = active[a] del active[a] - active[a.replace("_" + o, "")] = t + active[a.replace(":" + o, "")] = t mod = True elif a == o: match = active[a] @@ -746,31 +749,31 @@ class DataSmart(MutableMapping): value = copy.copy(local_var["_defaultval"]) - if flag == "_content" and local_var is not None and "_append" in local_var and not parsing: - if not value: - value = "" + if flag == "_content" and local_var is not None and ":append" in local_var and not parsing: self.need_overrides() - for (r, o) in local_var["_append"]: + for (r, o) in local_var[":append"]: match = True if o: - for o2 in o.split("_"): + for o2 in o.split(":"): if not o2 in self.overrides: match = False if match: + if value is None: + value = "" value = value + r - if flag == "_content" and local_var is not None and "_prepend" in local_var and not parsing: - if not value: - value = "" + if flag == "_content" and local_var is not None and ":prepend" in local_var and not parsing: self.need_overrides() - for (r, o) in local_var["_prepend"]: + for (r, o) in local_var[":prepend"]: match = True if o: - for o2 in o.split("_"): + for o2 in o.split(":"): if not o2 in self.overrides: match = False if match: + if value is None: + value = "" value = r + value parser = None @@ -779,12 +782,12 @@ class DataSmart(MutableMapping): if expand: value = parser.value - if value and flag == "_content" and local_var is not None and "_remove" in local_var and not parsing: + if value and flag == "_content" and local_var is not None and ":remove" in local_var and not parsing: self.need_overrides() - for (r, o) in local_var["_remove"]: + for (r, o) in local_var[":remove"]: match = True if o: - for o2 in o.split("_"): + for o2 in o.split(":"): if not o2 in self.overrides: match = False if match: @@ -820,7 +823,6 @@ class DataSmart(MutableMapping): return value def delVarFlag(self, var, flag, **loginfo): - var = var.replace(":", "_") self.expand_cache = {} local_var, _ = self._findVar(var) @@ -838,7 +840,6 @@ class DataSmart(MutableMapping): del self.dict[var][flag] def appendVarFlag(self, var, flag, value, **loginfo): - var = var.replace(":", "_") loginfo['op'] = 'append' loginfo['flag'] = flag self.varhistory.record(**loginfo) @@ -846,7 +847,6 @@ class DataSmart(MutableMapping): self.setVarFlag(var, flag, newvalue, ignore=True) def prependVarFlag(self, var, flag, value, **loginfo): - var = var.replace(":", "_") loginfo['op'] = 'prepend' loginfo['flag'] = flag self.varhistory.record(**loginfo) @@ -854,7 +854,6 @@ class DataSmart(MutableMapping): self.setVarFlag(var, flag, newvalue, ignore=True) def setVarFlags(self, var, flags, **loginfo): - var = var.replace(":", "_") self.expand_cache = {} infer_caller_details(loginfo) if not var in self.dict: @@ -869,13 +868,12 @@ class DataSmart(MutableMapping): self.dict[var][i] = flags[i] def getVarFlags(self, var, expand = False, internalflags=False): - var = var.replace(":", "_") local_var, _ = self._findVar(var) flags = {} if local_var: for i in local_var: - if i.startswith("_") and not internalflags: + if i.startswith(("_", ":")) and not internalflags: continue flags[i] = local_var[i] if expand and i in expand: @@ -886,7 +884,6 @@ class DataSmart(MutableMapping): def delVarFlags(self, var, **loginfo): - var = var.replace(":", "_") self.expand_cache = {} if not var in self.dict: self._makeShadowCopy(var) @@ -974,8 +971,8 @@ class DataSmart(MutableMapping): for (r, o) in self.overridedata[var]: if o in self.overridesset: overrides.add(var) - elif "_" in o: - if set(o.split("_")).issubset(self.overridesset): + elif ":" in o: + if set(o.split(":")).issubset(self.overridesset): overrides.add(var) for k in keylist(self.dict): diff --git a/poky/bitbake/lib/bb/fetch2/__init__.py b/poky/bitbake/lib/bb/fetch2/__init__.py index 0d49e1da37..ad898680ff 100644 --- a/poky/bitbake/lib/bb/fetch2/__init__.py +++ b/poky/bitbake/lib/bb/fetch2/__init__.py @@ -1146,11 +1146,11 @@ def srcrev_internal_helper(ud, d, name): pn = d.getVar("PN") attempts = [] if name != '' and pn: - attempts.append("SRCREV_%s_pn-%s" % (name, pn)) + attempts.append("SRCREV_%s:pn-%s" % (name, pn)) if name != '': attempts.append("SRCREV_%s" % name) if pn: - attempts.append("SRCREV_pn-%s" % pn) + attempts.append("SRCREV:pn-%s" % pn) attempts.append("SRCREV") for a in attempts: diff --git a/poky/bitbake/lib/bb/parse/ast.py b/poky/bitbake/lib/bb/parse/ast.py index db2bdc35ec..743ea0dfc0 100644 --- a/poky/bitbake/lib/bb/parse/ast.py +++ b/poky/bitbake/lib/bb/parse/ast.py @@ -97,7 +97,6 @@ class DataNode(AstNode): def eval(self, data): groupd = self.groupd key = groupd["var"] - key = key.replace(":", "_") loginfo = { 'variable': key, 'file': self.filename, @@ -146,7 +145,7 @@ class DataNode(AstNode): data.setVar(key, val, parsing=True, **loginfo) class MethodNode(AstNode): - tr_tbl = str.maketrans('/.+-@%&', '_______') + tr_tbl = str.maketrans('/.+-@%&~', '________') def __init__(self, filename, lineno, func_name, body, python, fakeroot): AstNode.__init__(self, filename, lineno) @@ -208,7 +207,6 @@ class ExportFuncsNode(AstNode): def eval(self, data): for func in self.n: - func = func.replace(":", "_") calledfunc = self.classname + "_" + func if data.getVar(func, False) and not data.getVarFlag(func, 'export_func', False): diff --git a/poky/bitbake/lib/bb/runqueue.py b/poky/bitbake/lib/bb/runqueue.py index 6c41fe6d43..25e012125c 100644 --- a/poky/bitbake/lib/bb/runqueue.py +++ b/poky/bitbake/lib/bb/runqueue.py @@ -2443,6 +2443,11 @@ class RunQueueExecute: if update_tasks: self.sqdone = False + for tid in [t[0] for t in update_tasks]: + h = pending_hash_index(tid, self.rqdata) + if h in self.sqdata.hashes and tid != self.sqdata.hashes[h]: + self.sq_deferred[tid] = self.sqdata.hashes[h] + bb.note("Deferring %s after %s" % (tid, self.sqdata.hashes[h])) update_scenequeue_data([t[0] for t in update_tasks], self.sqdata, self.rqdata, self.rq, self.cooker, self.stampcache, self, summary=False) for (tid, harddepfail, origvalid) in update_tasks: @@ -2786,6 +2791,19 @@ def build_scenequeue_data(sqdata, rqdata, rq, cooker, stampcache, sqrq): sqdata.stamppresent = set() sqdata.valid = set() + sqdata.hashes = {} + sqrq.sq_deferred = {} + for mc in sorted(sqdata.multiconfigs): + for tid in sorted(sqdata.sq_revdeps): + if mc_from_tid(tid) != mc: + continue + h = pending_hash_index(tid, rqdata) + if h not in sqdata.hashes: + sqdata.hashes[h] = tid + else: + sqrq.sq_deferred[tid] = sqdata.hashes[h] + bb.note("Deferring %s after %s" % (tid, sqdata.hashes[h])) + update_scenequeue_data(sqdata.sq_revdeps, sqdata, rqdata, rq, cooker, stampcache, sqrq, summary=True) # Compute a list of 'stale' sstate tasks where the current hash does not match the one @@ -2850,32 +2868,20 @@ def update_scenequeue_data(tids, sqdata, rqdata, rq, cooker, stampcache, sqrq, s sqdata.valid |= rq.validate_hashes(tocheck, cooker.data, len(sqdata.stamppresent), False, summary=summary) - sqdata.hashes = {} - sqrq.sq_deferred = {} - for mc in sorted(sqdata.multiconfigs): - for tid in sorted(sqdata.sq_revdeps): - if mc_from_tid(tid) != mc: - continue - if tid in sqdata.stamppresent: - continue - if tid in sqdata.valid: - continue - if tid in sqdata.noexec: - continue - if tid in sqrq.scenequeue_notcovered: - continue - if tid in sqrq.scenequeue_covered: - continue - - h = pending_hash_index(tid, rqdata) - if h not in sqdata.hashes: - if tid in tids: - sqdata.outrightfail.add(tid) - sqdata.hashes[h] = tid - else: - sqrq.sq_deferred[tid] = sqdata.hashes[h] - bb.note("Deferring %s after %s" % (tid, sqdata.hashes[h])) - + for tid in tids: + if tid in sqdata.stamppresent: + continue + if tid in sqdata.valid: + continue + if tid in sqdata.noexec: + continue + if tid in sqrq.scenequeue_covered: + continue + if tid in sqrq.scenequeue_notcovered: + continue + if tid in sqrq.sq_deferred: + continue + sqdata.outrightfail.add(tid) class TaskFailure(Exception): """ diff --git a/poky/bitbake/lib/bb/server/process.py b/poky/bitbake/lib/bb/server/process.py index a0955722e3..6127fd40e6 100644 --- a/poky/bitbake/lib/bb/server/process.py +++ b/poky/bitbake/lib/bb/server/process.py @@ -26,6 +26,7 @@ import errno import re import datetime import pickle +import traceback import bb.server.xmlrpcserver from bb import daemonize from multiprocessing import queues @@ -217,8 +218,9 @@ class ProcessServer(): self.command_channel_reply.send(self.cooker.command.runCommand(command)) serverlog("Command Completed") except Exception as e: - serverlog('Exception in server main event loop running command %s (%s)' % (command, str(e))) - logger.exception('Exception in server main event loop running command %s (%s)' % (command, str(e))) + stack = traceback.format_exc() + serverlog('Exception in server main event loop running command %s (%s)' % (command, stack)) + logger.exception('Exception in server main event loop running command %s (%s)' % (command, stack)) if self.xmlrpc in ready: self.xmlrpc.handle_requests() diff --git a/poky/bitbake/lib/bb/siggen.py b/poky/bitbake/lib/bb/siggen.py index 07692e673c..3f9fe50642 100644 --- a/poky/bitbake/lib/bb/siggen.py +++ b/poky/bitbake/lib/bb/siggen.py @@ -228,7 +228,7 @@ class SignatureGeneratorBasic(SignatureGenerator): # self.dump_sigtask(fn, task, d.getVar("STAMP"), False) for task in taskdeps: - d.setVar("BB_BASEHASH_task-%s" % task, self.basehash[fn + ":" + task]) + d.setVar("BB_BASEHASH:task-%s" % task, self.basehash[fn + ":" + task]) def postparsing_clean_cache(self): # @@ -325,7 +325,7 @@ class SignatureGeneratorBasic(SignatureGenerator): h = hashlib.sha256(data.encode("utf-8")).hexdigest() self.taskhash[tid] = h - #d.setVar("BB_TASKHASH_task-%s" % task, taskhash[task]) + #d.setVar("BB_TASKHASH:task-%s" % task, taskhash[task]) return h def writeout_file_checksum_cache(self): diff --git a/poky/bitbake/lib/bb/tests/codeparser.py b/poky/bitbake/lib/bb/tests/codeparser.py index 826a2d2f6d..f485204791 100644 --- a/poky/bitbake/lib/bb/tests/codeparser.py +++ b/poky/bitbake/lib/bb/tests/codeparser.py @@ -111,9 +111,9 @@ ${D}${libdir}/pkgconfig/*.pc self.assertExecs(set(["sed"])) def test_parameter_expansion_modifiers(self): - # - and + are also valid modifiers for parameter expansion, but are + # -,+ and : are also valid modifiers for parameter expansion, but are # valid characters in bitbake variable names, so are not included here - for i in ('=', ':-', ':=', '?', ':?', ':+', '#', '%', '##', '%%'): + for i in ('=', '?', '#', '%', '##', '%%'): name = "foo%sbar" % i self.parseExpression("${%s}" % name) self.assertNotIn(name, self.references) diff --git a/poky/bitbake/lib/bb/tests/data.py b/poky/bitbake/lib/bb/tests/data.py index 1d4a64b109..e667c7c7d3 100644 --- a/poky/bitbake/lib/bb/tests/data.py +++ b/poky/bitbake/lib/bb/tests/data.py @@ -245,35 +245,35 @@ class TestConcatOverride(unittest.TestCase): def test_prepend(self): self.d.setVar("TEST", "${VAL}") - self.d.setVar("TEST_prepend", "${FOO}:") + self.d.setVar("TEST:prepend", "${FOO}:") self.assertEqual(self.d.getVar("TEST"), "foo:val") def test_append(self): self.d.setVar("TEST", "${VAL}") - self.d.setVar("TEST_append", ":${BAR}") + self.d.setVar("TEST:append", ":${BAR}") self.assertEqual(self.d.getVar("TEST"), "val:bar") def test_multiple_append(self): self.d.setVar("TEST", "${VAL}") - self.d.setVar("TEST_prepend", "${FOO}:") - self.d.setVar("TEST_append", ":val2") - self.d.setVar("TEST_append", ":${BAR}") + self.d.setVar("TEST:prepend", "${FOO}:") + self.d.setVar("TEST:append", ":val2") + self.d.setVar("TEST:append", ":${BAR}") self.assertEqual(self.d.getVar("TEST"), "foo:val:val2:bar") def test_append_unset(self): - self.d.setVar("TEST_prepend", "${FOO}:") - self.d.setVar("TEST_append", ":val2") - self.d.setVar("TEST_append", ":${BAR}") + self.d.setVar("TEST:prepend", "${FOO}:") + self.d.setVar("TEST:append", ":val2") + self.d.setVar("TEST:append", ":${BAR}") self.assertEqual(self.d.getVar("TEST"), "foo::val2:bar") def test_remove(self): self.d.setVar("TEST", "${VAL} ${BAR}") - self.d.setVar("TEST_remove", "val") + self.d.setVar("TEST:remove", "val") self.assertEqual(self.d.getVar("TEST"), " bar") def test_remove_cleared(self): self.d.setVar("TEST", "${VAL} ${BAR}") - self.d.setVar("TEST_remove", "val") + self.d.setVar("TEST:remove", "val") self.d.setVar("TEST", "${VAL} ${BAR}") self.assertEqual(self.d.getVar("TEST"), "val bar") @@ -281,42 +281,42 @@ class TestConcatOverride(unittest.TestCase): # (including that whitespace is preserved) def test_remove_inactive_override(self): self.d.setVar("TEST", "${VAL} ${BAR} 123") - self.d.setVar("TEST_remove_inactiveoverride", "val") + self.d.setVar("TEST:remove:inactiveoverride", "val") self.assertEqual(self.d.getVar("TEST"), "val bar 123") def test_doubleref_remove(self): self.d.setVar("TEST", "${VAL} ${BAR}") - self.d.setVar("TEST_remove", "val") + self.d.setVar("TEST:remove", "val") self.d.setVar("TEST_TEST", "${TEST} ${TEST}") self.assertEqual(self.d.getVar("TEST_TEST"), " bar bar") def test_empty_remove(self): self.d.setVar("TEST", "") - self.d.setVar("TEST_remove", "val") + self.d.setVar("TEST:remove", "val") self.assertEqual(self.d.getVar("TEST"), "") def test_remove_expansion(self): self.d.setVar("BAR", "Z") self.d.setVar("TEST", "${BAR}/X Y") - self.d.setVar("TEST_remove", "${BAR}/X") + self.d.setVar("TEST:remove", "${BAR}/X") self.assertEqual(self.d.getVar("TEST"), " Y") def test_remove_expansion_items(self): self.d.setVar("TEST", "A B C D") self.d.setVar("BAR", "B D") - self.d.setVar("TEST_remove", "${BAR}") + self.d.setVar("TEST:remove", "${BAR}") self.assertEqual(self.d.getVar("TEST"), "A C ") def test_remove_preserve_whitespace(self): # When the removal isn't active, the original value should be preserved self.d.setVar("TEST", " A B") - self.d.setVar("TEST_remove", "C") + self.d.setVar("TEST:remove", "C") self.assertEqual(self.d.getVar("TEST"), " A B") def test_remove_preserve_whitespace2(self): # When the removal is active preserve the whitespace self.d.setVar("TEST", " A B") - self.d.setVar("TEST_remove", "B") + self.d.setVar("TEST:remove", "B") self.assertEqual(self.d.getVar("TEST"), " A ") class TestOverrides(unittest.TestCase): @@ -329,70 +329,70 @@ class TestOverrides(unittest.TestCase): self.assertEqual(self.d.getVar("TEST"), "testvalue") def test_one_override(self): - self.d.setVar("TEST_bar", "testvalue2") + self.d.setVar("TEST:bar", "testvalue2") self.assertEqual(self.d.getVar("TEST"), "testvalue2") def test_one_override_unset(self): - self.d.setVar("TEST2_bar", "testvalue2") + self.d.setVar("TEST2:bar", "testvalue2") self.assertEqual(self.d.getVar("TEST2"), "testvalue2") - self.assertCountEqual(list(self.d.keys()), ['TEST', 'TEST2', 'OVERRIDES', 'TEST2_bar']) + self.assertCountEqual(list(self.d.keys()), ['TEST', 'TEST2', 'OVERRIDES', 'TEST2:bar']) def test_multiple_override(self): - self.d.setVar("TEST_bar", "testvalue2") - self.d.setVar("TEST_local", "testvalue3") - self.d.setVar("TEST_foo", "testvalue4") + self.d.setVar("TEST:bar", "testvalue2") + self.d.setVar("TEST:local", "testvalue3") + self.d.setVar("TEST:foo", "testvalue4") self.assertEqual(self.d.getVar("TEST"), "testvalue3") - self.assertCountEqual(list(self.d.keys()), ['TEST', 'TEST_foo', 'OVERRIDES', 'TEST_bar', 'TEST_local']) + self.assertCountEqual(list(self.d.keys()), ['TEST', 'TEST:foo', 'OVERRIDES', 'TEST:bar', 'TEST:local']) def test_multiple_combined_overrides(self): - self.d.setVar("TEST_local_foo_bar", "testvalue3") + self.d.setVar("TEST:local:foo:bar", "testvalue3") self.assertEqual(self.d.getVar("TEST"), "testvalue3") def test_multiple_overrides_unset(self): - self.d.setVar("TEST2_local_foo_bar", "testvalue3") + self.d.setVar("TEST2:local:foo:bar", "testvalue3") self.assertEqual(self.d.getVar("TEST2"), "testvalue3") def test_keyexpansion_override(self): self.d.setVar("LOCAL", "local") - self.d.setVar("TEST_bar", "testvalue2") - self.d.setVar("TEST_${LOCAL}", "testvalue3") - self.d.setVar("TEST_foo", "testvalue4") + self.d.setVar("TEST:bar", "testvalue2") + self.d.setVar("TEST:${LOCAL}", "testvalue3") + self.d.setVar("TEST:foo", "testvalue4") bb.data.expandKeys(self.d) self.assertEqual(self.d.getVar("TEST"), "testvalue3") def test_rename_override(self): - self.d.setVar("ALTERNATIVE_ncurses-tools_class-target", "a") + self.d.setVar("ALTERNATIVE:ncurses-tools:class-target", "a") self.d.setVar("OVERRIDES", "class-target") - self.d.renameVar("ALTERNATIVE_ncurses-tools", "ALTERNATIVE_lib32-ncurses-tools") - self.assertEqual(self.d.getVar("ALTERNATIVE_lib32-ncurses-tools"), "a") + self.d.renameVar("ALTERNATIVE:ncurses-tools", "ALTERNATIVE:lib32-ncurses-tools") + self.assertEqual(self.d.getVar("ALTERNATIVE:lib32-ncurses-tools"), "a") def test_underscore_override(self): - self.d.setVar("TEST_bar", "testvalue2") - self.d.setVar("TEST_some_val", "testvalue3") - self.d.setVar("TEST_foo", "testvalue4") + self.d.setVar("TEST:bar", "testvalue2") + self.d.setVar("TEST:some_val", "testvalue3") + self.d.setVar("TEST:foo", "testvalue4") self.d.setVar("OVERRIDES", "foo:bar:some_val") self.assertEqual(self.d.getVar("TEST"), "testvalue3") def test_remove_with_override(self): - self.d.setVar("TEST_bar", "testvalue2") - self.d.setVar("TEST_some_val", "testvalue3 testvalue5") - self.d.setVar("TEST_some_val_remove", "testvalue3") - self.d.setVar("TEST_foo", "testvalue4") + self.d.setVar("TEST:bar", "testvalue2") + self.d.setVar("TEST:some_val", "testvalue3 testvalue5") + self.d.setVar("TEST:some_val:remove", "testvalue3") + self.d.setVar("TEST:foo", "testvalue4") self.d.setVar("OVERRIDES", "foo:bar:some_val") self.assertEqual(self.d.getVar("TEST"), " testvalue5") def test_append_and_override_1(self): - self.d.setVar("TEST_append", "testvalue2") - self.d.setVar("TEST_bar", "testvalue3") + self.d.setVar("TEST:append", "testvalue2") + self.d.setVar("TEST:bar", "testvalue3") self.assertEqual(self.d.getVar("TEST"), "testvalue3testvalue2") def test_append_and_override_2(self): - self.d.setVar("TEST_append_bar", "testvalue2") + self.d.setVar("TEST:append:bar", "testvalue2") self.assertEqual(self.d.getVar("TEST"), "testvaluetestvalue2") def test_append_and_override_3(self): - self.d.setVar("TEST_bar_append", "testvalue2") + self.d.setVar("TEST:bar:append", "testvalue2") self.assertEqual(self.d.getVar("TEST"), "testvalue2") # Test an override with _ in it based on a real world OE issue @@ -400,11 +400,16 @@ class TestOverrides(unittest.TestCase): self.d.setVar("TARGET_ARCH", "x86_64") self.d.setVar("PN", "test-${TARGET_ARCH}") self.d.setVar("VERSION", "1") - self.d.setVar("VERSION_pn-test-${TARGET_ARCH}", "2") + self.d.setVar("VERSION:pn-test-${TARGET_ARCH}", "2") self.d.setVar("OVERRIDES", "pn-${PN}") bb.data.expandKeys(self.d) self.assertEqual(self.d.getVar("VERSION"), "2") + def test_append_and_unused_override(self): + # Had a bug where an unused override append could return "" instead of None + self.d.setVar("BAR:append:unusedoverride", "testvalue2") + self.assertEqual(self.d.getVar("BAR"), None) + class TestKeyExpansion(unittest.TestCase): def setUp(self): self.d = bb.data.init() @@ -498,7 +503,7 @@ class TaskHash(unittest.TestCase): d.setVar("VAR", "val") # Adding an inactive removal shouldn't change the hash d.setVar("BAR", "notbar") - d.setVar("MYCOMMAND_remove", "${BAR}") + d.setVar("MYCOMMAND:remove", "${BAR}") nexthash = gettask_bashhash("mytask", d) self.assertEqual(orighash, nexthash) diff --git a/poky/bitbake/lib/bb/tests/parse.py b/poky/bitbake/lib/bb/tests/parse.py index 9e21e18425..4d17f82edc 100644 --- a/poky/bitbake/lib/bb/tests/parse.py +++ b/poky/bitbake/lib/bb/tests/parse.py @@ -98,8 +98,8 @@ exportD = "d" overridetest = """ -RRECOMMENDS_${PN} = "a" -RRECOMMENDS_${PN}_libc = "b" +RRECOMMENDS:${PN} = "a" +RRECOMMENDS:${PN}:libc = "b" OVERRIDES = "libc:${PN}" PN = "gtk+" """ @@ -110,13 +110,13 @@ PN = "gtk+" self.assertEqual(d.getVar("RRECOMMENDS"), "b") bb.data.expandKeys(d) self.assertEqual(d.getVar("RRECOMMENDS"), "b") - d.setVar("RRECOMMENDS_gtk+", "c") + d.setVar("RRECOMMENDS:gtk+", "c") self.assertEqual(d.getVar("RRECOMMENDS"), "c") overridetest2 = """ EXTRA_OECONF = "" -EXTRA_OECONF_class-target = "b" -EXTRA_OECONF_append = " c" +EXTRA_OECONF:class-target = "b" +EXTRA_OECONF:append = " c" """ def test_parse_overrides(self): @@ -128,7 +128,7 @@ EXTRA_OECONF_append = " c" overridetest3 = """ DESCRIPTION = "A" -DESCRIPTION_${PN}-dev = "${DESCRIPTION} B" +DESCRIPTION:${PN}-dev = "${DESCRIPTION} B" PN = "bc" """ @@ -136,15 +136,15 @@ PN = "bc" f = self.parsehelper(self.overridetest3) d = bb.parse.handle(f.name, self.d)[''] bb.data.expandKeys(d) - self.assertEqual(d.getVar("DESCRIPTION_bc-dev"), "A B") + self.assertEqual(d.getVar("DESCRIPTION:bc-dev"), "A B") d.setVar("DESCRIPTION", "E") - d.setVar("DESCRIPTION_bc-dev", "C D") + d.setVar("DESCRIPTION:bc-dev", "C D") d.setVar("OVERRIDES", "bc-dev") self.assertEqual(d.getVar("DESCRIPTION"), "C D") classextend = """ -VAR_var_override1 = "B" +VAR_var:override1 = "B" EXTRA = ":override1" OVERRIDES = "nothing${EXTRA}" diff --git a/poky/bitbake/lib/bb/ui/taskexp.py b/poky/bitbake/lib/bb/ui/taskexp.py index 2b246710ca..c00eaf6638 100644 --- a/poky/bitbake/lib/bb/ui/taskexp.py +++ b/poky/bitbake/lib/bb/ui/taskexp.py @@ -8,6 +8,7 @@ # import sys +import traceback try: import gi @@ -196,6 +197,7 @@ def main(server, eventHandler, params): gtkgui.start() try: + params.updateToServer(server, os.environ.copy()) params.updateFromServer(server) cmdline = params.parseActions() if not cmdline: @@ -218,6 +220,9 @@ def main(server, eventHandler, params): except client.Fault as x: print("XMLRPC Fault getting commandline:\n %s" % x) return + except Exception as e: + print("Exception in startup:\n %s" % traceback.format_exc()) + return if gtkthread.quit.isSet(): return diff --git a/poky/bitbake/lib/bb/utils.py b/poky/bitbake/lib/bb/utils.py index 6ba1d2a376..e6e82d1118 100644 --- a/poky/bitbake/lib/bb/utils.py +++ b/poky/bitbake/lib/bb/utils.py @@ -1178,7 +1178,7 @@ def edit_metadata(meta_lines, variables, varfunc, match_overrides=False): variables: a list of variable names to look for. Functions may also be specified, but must be specified with '()' at the end of the name. Note that the function doesn't have - any intrinsic understanding of _append, _prepend, _remove, + any intrinsic understanding of :append, :prepend, :remove, or overrides, so these are considered as part of the name. These values go into a regular expression, so regular expression syntax is allowed. diff --git a/poky/bitbake/lib/hashserv/tests.py b/poky/bitbake/lib/hashserv/tests.py index e2b762dbf0..e851535c59 100644 --- a/poky/bitbake/lib/hashserv/tests.py +++ b/poky/bitbake/lib/hashserv/tests.py @@ -15,28 +15,32 @@ import tempfile import threading import unittest import socket +import time +import signal -def _run_server(server, idx): - # logging.basicConfig(level=logging.DEBUG, filename='bbhashserv.log', filemode='w', - # format='%(levelname)s %(filename)s:%(lineno)d %(message)s') +def server_prefunc(server, idx): + logging.basicConfig(level=logging.DEBUG, filename='bbhashserv.log', filemode='w', + format='%(levelname)s %(filename)s:%(lineno)d %(message)s') + server.logger.debug("Running server %d" % idx) sys.stdout = open('bbhashserv-%d.log' % idx, 'w') sys.stderr = sys.stdout - server.serve_forever() - class HashEquivalenceTestSetup(object): METHOD = 'TestMethod' server_index = 0 - def start_server(self, dbpath=None, upstream=None, read_only=False): + def start_server(self, dbpath=None, upstream=None, read_only=False, prefunc=server_prefunc): self.server_index += 1 if dbpath is None: dbpath = os.path.join(self.temp_dir.name, "db%d.sqlite" % self.server_index) - def cleanup_thread(thread): - thread.terminate() - thread.join() + def cleanup_server(server): + if server.process.exitcode is not None: + return + + server.process.terminate() + server.process.join() server = create_server(self.get_server_addr(self.server_index), dbpath, @@ -44,9 +48,8 @@ class HashEquivalenceTestSetup(object): read_only=read_only) server.dbpath = dbpath - server.thread = multiprocessing.Process(target=_run_server, args=(server, self.server_index)) - server.thread.start() - self.addCleanup(cleanup_thread, server.thread) + server.serve_as_process(prefunc=prefunc, args=(self.server_index,)) + self.addCleanup(cleanup_server, server) def cleanup_client(client): client.close() @@ -283,6 +286,33 @@ class HashEquivalenceCommonTests(object): self.assertClientGetHash(self.client, taskhash2, None) + def test_slow_server_start(self): + """ + Ensures that the server will exit correctly even if it gets a SIGTERM + before entering the main loop + """ + + event = multiprocessing.Event() + + def prefunc(server, idx): + nonlocal event + server_prefunc(server, idx) + event.wait() + + def do_nothing(signum, frame): + pass + + old_signal = signal.signal(signal.SIGTERM, do_nothing) + self.addCleanup(signal.signal, signal.SIGTERM, old_signal) + + _, server = self.start_server(prefunc=prefunc) + server.process.terminate() + time.sleep(30) + event.set() + server.process.join(300) + self.assertIsNotNone(server.process.exitcode, "Server did not exit in a timely manner!") + + class TestHashEquivalenceUnixServer(HashEquivalenceTestSetup, HashEquivalenceCommonTests, unittest.TestCase): def get_server_addr(self, server_idx): return "unix://" + os.path.join(self.temp_dir.name, 'sock%d' % server_idx) diff --git a/poky/bitbake/lib/toaster/orm/models.py b/poky/bitbake/lib/toaster/orm/models.py index 7f7e922ade..4c94b407d7 100644 --- a/poky/bitbake/lib/toaster/orm/models.py +++ b/poky/bitbake/lib/toaster/orm/models.py @@ -1719,7 +1719,7 @@ class CustomImageRecipe(Recipe): """Generate the contents for the recipe file.""" # If we have no excluded packages we only need to _append if self.excludes_set.count() == 0: - packages_conf = "IMAGE_INSTALL_append = \" " + packages_conf = "IMAGE_INSTALL:append = \" " for pkg in self.appends_set.all(): packages_conf += pkg.name+' ' diff --git a/poky/bitbake/lib/toaster/toastergui/views.py b/poky/bitbake/lib/toaster/toastergui/views.py index 9a5e48e3bb..74f9d56963 100644 --- a/poky/bitbake/lib/toaster/toastergui/views.py +++ b/poky/bitbake/lib/toaster/toastergui/views.py @@ -1708,7 +1708,7 @@ if True: except ProjectVariable.DoesNotExist: pass try: - return_data['image_install_append'] = ProjectVariable.objects.get(project = prj, name = "IMAGE_INSTALL_append").value, + return_data['image_install:append'] = ProjectVariable.objects.get(project = prj, name = "IMAGE_INSTALL:append").value, except ProjectVariable.DoesNotExist: pass try: @@ -1839,7 +1839,7 @@ if True: except ProjectVariable.DoesNotExist: pass try: - context['image_install_append'] = ProjectVariable.objects.get(project = prj, name = "IMAGE_INSTALL_append").value + context['image_install:append'] = ProjectVariable.objects.get(project = prj, name = "IMAGE_INSTALL:append").value context['image_install_append_defined'] = "1" except ProjectVariable.DoesNotExist: pass diff --git a/poky/bitbake/lib/toaster/toastermain/management/commands/buildimport.py b/poky/bitbake/lib/toaster/toastermain/management/commands/buildimport.py index 59da6ff7ac..e25b55e5ab 100644 --- a/poky/bitbake/lib/toaster/toastermain/management/commands/buildimport.py +++ b/poky/bitbake/lib/toaster/toastermain/management/commands/buildimport.py @@ -451,7 +451,7 @@ class Command(BaseCommand): # Catch vars relevant to Toaster (in case no Toaster section) self.update_project_vars(project,'DISTRO') self.update_project_vars(project,'MACHINE') - self.update_project_vars(project,'IMAGE_INSTALL_append') + self.update_project_vars(project,'IMAGE_INSTALL:append') self.update_project_vars(project,'IMAGE_FSTYPES') self.update_project_vars(project,'PACKAGE_CLASSES') # These vars are typically only assigned by Toaster -- cgit v1.2.3