diff options
| author | Patrick Williams <patrick@stwcx.xyz> | 2021-08-08 03:21:33 +0300 |
|---|---|---|
| committer | Patrick Williams <patrick@stwcx.xyz> | 2021-08-11 23:39:32 +0300 |
| commit | 213cb2696d00a85cd48d356cb5131824a302d828 (patch) | |
| tree | bfdf8fcdfef3a88e392ca3acfca6dec4dc836c9f /poky/bitbake | |
| parent | 40d8f44b51f83548f7ca8df062dd7435b784c0a8 (diff) | |
| download | openbmc-213cb2696d00a85cd48d356cb5131824a302d828.tar.xz | |
subtree updates
meta-raspberrypi: 8dc3a31088..c7f4c739a3:
Khem Raj (5):
linux-raspberrypi: Upgrade to 5.10.52
userland: Update to latest master branch
raspberrypi-firmware: Update to latest
raspberrypi-tools: Update to latest
sdcard_image-rpi.bbclass: Fix IMAGE_TYPEDEP override to use new syntax
Martin Jansa (4):
Convert to new override syntax
Manually fix conversion
layer.conf: Update to honister
userland: package man pages in PN-doc
Pierre-Jean Texier (2):
kas: local.conf: bump CONF_VERSION variable
kas: local.conf: disable prelink
poky: 17aabc0127..492205ea83:
Alexander Kanavin (17):
llvm: update 12.0.0 -> 12.0.1
systemd: update 248.3 -> 249.1
python3-testools: update 2.4.0 -> 2.5.0
libuv: update 1.41.0 -> 1.42.0
gnu-config: update to latest revision
vulkan-samples: update to latest revision
cmake: update 3.20.5 -> 3.21.0
cmake: update 3.21.0 -> 3.21.1
mtools: update 4.0.32 -> 4.0.34
util-linux: update 2.37 -> 2.37.1
iputils: update 20210202 -> 20210722
freetype: update 2.10.4 -> 2.11.0
devtool: print a warning on upgrades if PREFERRED_VERSION is set
rpm: do not RRECOMMEND rpm-build
selftest: add core-image-weston to no-gpl3-no-meta-gpl2 image test
shadow: update 4.8.1 -> 4.9
local.conf.sample: disable prelink
Bernhard Rosenkränzer (1):
gcc: update 11.1 -> 11.2
Bruce Ashfield (6):
linux-yocto/5.10: update to v5.10.53
linux-yocto/5.13: update to v5.13.5
linux-yocto/5.4: update to v5.4.135
linux-yocto-rt/5.10: update to -rt47
linux-yocto/5.13: enable TYPEC_TCPCI in usbc fragment
linux-yocto/5.10: enable TYPEC_TCPCI in usbc fragment
Changqing Li (1):
archiver.bbclass: fix do_ar_configured failure for kernel
Chen Qi (3):
zstd: fix CVE_PRODUCT
insane.bbclass: fix the file-rdeps QA message for the new override syntax
iputils: fix do_configure failure of missing ip command
Damian Wrobel (1):
rootfs: remove ldconfig auxiliary cache where appropriate
Denys Dmytriyenko (4):
meta: convert nested overrides leftovers to new syntax
convert-overrides.py: handle few more cases of overrides
libwpe: remove rpi-specific custom code
poky-tiny: drop uclibc override
Jon Mason (1):
parselogs.py: qemuarm should be qemuarmv5
Joshua Watt (4):
mesa: Fix v3d & vc4 dmabuf import
bitbake: bitbake: asyncrpc: Catch early SIGTERM
libxft: Fix bad PKG value
bitbake: contrib: vim: Update for new override syntax
Kai Kang (2):
u-boot_2021.07: set UBOOT_MACHINE for qemumips and qemumips64
python3-pytest: display correct version info
Kevin Hao (2):
meta-yocto-bsp: Introduce the v5.13 bbappend
meta-yocto-bsp: Bump to the v5.10.55
Khem Raj (10):
binutils: Upgrade to 2.37 branch
texinfo: Update gnulib to fix build with glibc 2.34
systemd: Fix build on musl
stress-ng: Drop defining daddr_t
stress-ng: Detemine minimal stack size via sysconf
mesa: Define a fallback for DRIDRIVERS
libssh2: Fix syntax for using ptest override
toaster-managed-mode.json: Correctly specify term with new override syntax
distrooverrides.bbclass: Correct override syntax
devtool.py: Correct override syntax
Lee Chee Yang (1):
aspell: fix CVE-2019-25051
Marek Vasut (2):
image_types: Restore pre-btrfs-tools 4.14.1 mkfs.btrfs shrink behavior
kernel-uboot: Handle gzip and lzo compression options
Martin Jansa (6):
convert-overrides.py: show processed file and version of this script
convert-overrides.py: remove base_dep_prepend and autotools_dep_prepend exception
convert-overrides.py: 0.9.1 include '(' as delimiter for shortvars
convert-overrides.py: allow specifying multiple target dirs
convert-overrides.py: allow dots before override in vars_re and shortvars_re
systemd-boot: use ld.bfd as efi-ld even when gold or lld is used in ${LD}
Matthias Klein (2):
runqemu: Fix typo in error message
runqemu: decouple bios and kernel options
Matthias Schiffer (3):
initscripts: populate-volatile.sh: do not log to tty0
initscripts: populate-volatile.sh: run create_file synchronously
initscripts: fix creation order for /var/log with VOLATILE_LOG_DIR=true
Michael Halstead (1):
releases: update to include 3.3.1
Michael Opdenacker (18):
oe-setup-builddir: update YP docs and OE URLs
conf-notes.txt: now suggesting to run 'runqemu qemux86-64'
test-manual: document LTO related reproducibility bug
quick start manual: update "source oe-init-build-env" output
dev-manual: fix wrong reference to class
documentation/README: improve BitBake manual referencing guidelines
manuals: simplify references to BitBake manual
manuals: remove explicit BitBake variable references
meta-skeleton: add recipe examples from documentation sources
bitbake: doc: bitbake-user-manual: fix syntax in example and improve description
bitbake: doc: bitbake-user-manual: update bitbake option help
bitbake: doc: bitbake-user-manual: grammar fix for the number of "metadata"
manuals: initial documentation for CVE management
ref-manual: remove example recipe source files
profile-manual: document how to build perf manpages on target
cve-check: fix comments
cve-check: update link to NVD website for CVE details
cve-check: improve comment about CVE patch file names
Mingli Yu (2):
perlcross: not break build if already patched
curl: Upgrade to 7.78.0
Nicolas Dechesne (4):
yocto-check-layer: improve missed dependencies
checklayer: new function get_layer_dependencies()
checklayer: rename _find_layer_depends
yocto-check-layer: ensure that all layer dependencies are tested too
Oleksandr Kravchuk (1):
bitbake.conf: change GNOME_MIRROR to new one
Patrick Williams (1):
pixman: re-disable iwmmxt
Paul Barker (4):
bitbake: asyncrpc: Fix bad message error in client
bitbake: asyncrpc: Set timeout when waiting for reply from server
bitbake: parse/ast: Substitute '~' when naming anonymous functions
kernel-yocto: Simplify no git repo case in do_kernel_checkout
Quentin Schulz (4):
bitbake: doc: Makefile: turn warnings into errors by default
bitbake: doc: bitbake-user-manual: ref-variables: order alphabetically the glossary sources
bitbake: doc: bitbake-user-manual: ref-variables: force glossary output to be alphabetically sorted
bitbake: doc: bitbake-user-manual: replace ``FOO`` by :term:`FOO` where possible
Richard Purdie (49):
Add MAINTAINERS.md file
yocto-check-layer: Remove duplicated code
libubootenv: Drop default-env RRECOMMENDS
bitbake: data_smart: Allow colon in variable expansion regex
meta-poky/meta-yocto-bsp: Convert to new override syntax
layer.conf: Update to honister
autotools/base/icecc: Remove prepend from function names
scripts/contrib: Add override conversion script
systemtap: Fix headers issue with x86 and 5.13 headers
migration-guides: Add start of 3.4 guide with override migration notes
common-tasks: Fix conversion error in npm example
bitbake: bitbake: Switch to using new override syntax
bitbake: doc/lib: Update to use new override syntax containing colons
bitbake: doc/lib: Add fixes for issues missed by the automated conversion
bitbake: bitbake: Update to version 1.51.1
layer.conf: Override changes mean we're only compatible with honister
Convert to new override syntax
meta: Manual override fixes
local.conf.sample: Bump version so users update their config
sanity.conf: Require bitbake 1.51.1
dropbear: Fix incorrect package override for postrm
convert-overrides: Allow script to handle patch/diffs
sdk: Decouple default install path from built in path
sstate: Fix rebuilds when changing layer config
populate_sdk_ext: Fix handling of TOOLCHAIN_HOST_TASK in the eSDK case
local.conf.sample: Bump version so users update their config
poky: Use SDKPATHINSTALL instead of SDKPATH
vim: Clarify where RDEPENDS/RRECOMMENDS apply
bitbake: data_smart: Fix inactive overide accidental variable value corruption
local.conf.sample: Fix missed override conversion
license: Exclude COPYING.MIT from pseudo
meta: Convert IMAGE_TYPEDEP to use override syntax
uboot-extlinux-config: Fix missing override conversion
image/image_types: Convert CONVERSION_CMD/COMPRESS_CMD to new override syntax
image: Drop COMPRESS_CMD
devupstream: Allow support of native class extensions
diffoscope: Upgrade 178 -> 179
strace: Upgrade 5.12 -> 5.13
valgrind: Add patches for glibc 2.34 support
bitbake: runqueue: Improve multiconfig deferred task issues
elfutils: Add patch from upstream for glibc 2.34 ptest fixes
bitbake: doc: Fix append/prepend/remove references
bitbake: fetch/tests/toaster: Override conversion fixups
bitbake: process: Improve traceback error reporting from main loop
bitbake: command: Ensure we catch/handle exceptions
bitbake: ui/taskexp: Improve startup exception handling
bitbake: ui/taskexp: Fix to work with empty build directories
oeqa/runtime/cases/ptest: Increase test timeout from 300s to 450s
packagedata: Fix after override syntax change
Ross Burton (2):
glew: fix Makefile race
libx11: fix xkb compilation with _EVDEVK symbols
Saul Wold (1):
MAINTAINERS: Saul will cover devtool and eSDK
Stefan Wiehler (1):
dev-manual: fix source release example script
Stefano Babic (1):
mtd-utils: upgrade 2.1.2 -> 2.1.3
Tim Orling (2):
python3-hypothesis: upgrade 6.14.3 -> 6.14.5
python3-importlib-metadata: upgrade 4.6.1 -> 4.6.3
Tony Battersby (2):
lto.inc: disable LTO for grub
gcc: Backport patch to make LTO builds more reproducible
Tony Tascioglu (6):
ffmpeg: fix-CVE-2020-20446
ffmpeg: fix CVE-2020-20453
ffmpeg: fix CVE-2020-22015
ffmpeg: fix CVE-2020-22021
ffmpeg: fix CVE-2020-22033 and CVE-2020-22019
ffmpeg: fix CVE-2021-33815
Trevor Woerner (1):
ffmpeg: add libatomic for armv5
Ulrich Ölmann (2):
initramfs-framework: fix whitespace issue
initramfs-framework/setup-live: fix shebang
Vinay Kumar (1):
glibc: Fix CVE-2021-33574
Vivien Didelot (1):
init-manager-systemd: define weak dev manager
Zqiang (1):
python3: use monotonic clock for condvar if possible
hongxu (1):
createrepo-c: fix createrepo-c failed in nativesdk
leimaohui (1):
archiver.bbclass: Fix patch error for recipes that inherit dos2unix.
wangmy (3):
bind: upgrade 9.16.18 -> 9.16.19
i2c-tools: upgrade 4.2 -> 4.3
diffoscope: upgrade 177 -> 178
zangrc (2):
python3-dbus: upgrade 1.2.16 -> 1.2.18
python3-pip: upgrade 21.1.3 -> 21.2.1
meta-openembedded: 8fbcfb9f02..3cf2475ea0:
Anastasios Kavoukis (1):
pm-qa: fix paths for shell scripts
Andreas Müller (3):
mozjs/0001-Port-build-to-python3.patch: Fix typos in description
jack: upgrade 1.19.18 -> 1.19.19
fluidsynth: upgrade 2.2.1 -> 2.2.2
Andrej Valek (1):
thrift: upgrade to 0.14.2
Andrew Jeffery (2):
python3-gmpy: Add native support
python3-ecdsa: Add native support
Armin Kuster (2):
hiawatha: fix url.
wireshark: update to 3.4.7
Ben Brown (1):
android-tools: fix install of adb client when TOOLS is overridden
Changqing Li (1):
apache2: upgrade 2.4.46 -> 2.4.48
Devendra Tewari (1):
Suppress eol in functionfs setup scripts (#147)
Gianfranco (1):
vboxguestdrivers: upgrade 6.1.22 -> 6.1.24
Joe Slater (2):
php: move to version 7.4.21
gtksourceview4: work around dependency deficiency
Johannes Obermüller (1):
evtest: fix timestamps in output
Kai Kang (2):
python3-blivet: 3.1.4 -> 3.4.0
python3-blivetgui: 2.1.10 -> 2.2.1
Khem Raj (23):
netperf: Update to latest
netperf: Add systemd unit file
packagegroup-meta-oe: Add lmdb
packagegroup-meta-oe: Add mbw
addcli: check for ns_get16 and ns_get32
fuse: Define closefrom if not available
autofs: Fix build with glibc 2.34+
ntp: Do not use PTHREAD_STACK_MIN on glibc
ntp: Fix make check
mongodb: Upgrade to 4.4.7
vboxguestdrivers: Remove __divmoddi4 patch
packagegroup-meta-oe: Add jemalloc
apitrace: Exclude from builds with glibc 2.34+
libhugetlbfs: Disable build with glibc 2.34+
fvwm: Package extra files and man pages
luajit: Fix override syntax
lua: Drop uclibc patch
packagegroup-meta-oe: Correct override name and fix syntax
recipes: Fix override syntax
emacs,libgpiod,cockpit: Fix override syntax in using FILES_${PN}
fvwm: Fix build time paths in target perl/python scripts
nis: Drop uclibc check in anon python function
jemalloc: Fix build on musl
Leon Anavi (3):
python3-networkx: Upgrade 2.6.1 -> 2.6.2
python3-pysonos: Upgrade 0.0.53 -> 0.0.54
python3-zeroconf: Upgrade 0.33.1 -> 0.33.2
Li Wang (1):
openlldp: fix segfault
Maksym Sloyko (1):
libusbgx: Configure the Devices Used
Martin Jansa (5):
Convert to new override syntax
layer.conf: Update to honister
mariadb: manually fix the conversion
packagegroup-meta-oe: manually finish override syntax conversion
klibc.bbclass, image_types_sparse.bbclass, packagegroup-meta-oe.bb: update the overrides syntax conversion
Mingli Yu (4):
mariadb: redefine log-error item
jemalloc: add new recipe
hdf5: improve reproducibility
mariadb: Update SRC_URI
Nicolas Dechesne (1):
mbw: add new recipe
Paulo Neves (1):
htop: Add ncurses-terminfo-base to RDEPENDS
Sakib Sajal (1):
lmdb: add recipe
Salman Ahmed (2):
nginx: upgrade 1.18.0 -> 1.20.1
nginx: upgrade 1.19.6 -> 1.21.1
Tony Battersby (1):
net-snmp: fix QA Issue after LDFLAGS change
Yi Zhao (3):
postfix: upgrade 3.6.1 -> 3.6.2
audit: upgrade 3.0.2 -> 3.0.3
audit: fix compile error for 2.8.5
Zang Ruochen (1):
python3-robotframework: upgrade 4.0.3 -> 4.1
wangmy (17):
evince: upgrade 40.2 -> 40.4
gnome-backgrounds: upgrade 3.36.0 -> 3.38.0
gnome-desktop3: upgrade 3.36.6 -> 3.38.8
cmark: upgrade 0.30.0 -> 0.30.1
ctags: upgrade 5.9.20210711.0 -> 5.9.20210718.0
libnet-dns-perl: upgrade 1.31 -> 1.32
libtalloc: upgrade 2.3.2 -> 2.3.3
nghttp2: upgrade 1.43.0 -> 1.44.0
bats: upgrade 1.3.0 -> 1.4.1
networkmanager: upgrade 1.32.2 -> 1.32.4
gensio: upgrade 2.2.7 -> 2.2.8
libmbim: upgrade 1.24.8 -> 1.26.0
fetchmail: upgrade 6.4.19 -> 6.4.20
ctags: upgrade 5.9.20210718.0 -> 5.9.20210801.0
libblockdev: upgrade 2.25 -> 2.26
libqmi: upgrade 1.28.6 -> 1.28.8
monit: upgrade 5.28.0 -> 5.28.1
zangrc (15):
python3-qrcode: upgrade 7.1 -> 7.2
python3-rdflib: upgrade 5.0.0 -> 6.0.0
python3-simplejson: upgrade 3.17.2 -> 3.17.3
python3-bitstring: upgrade 3.1.7 -> 3.1.9
python3-iso8601: upgrade 0.1.14 -> 0.1.16
python3-gmqtt: upgrade 0.6.9 -> 0.6.10
python3-graphviz: upgrade 0.16 -> 0.17
python3-smbus: upgrade 4.2 -> 4.3
python3-pandas: upgrade 1.3.0 -> 1.3.1
python3-progress: upgrade 1.5 -> 1.6
python3-sentry-sdk: upgrade 1.3.0 -> 1.3.1
python3-socketio: upgrade 5.3.0 -> 5.4.0
python3-tqdm: upgrade 4.61.2 -> 4.62.0
python3-twisted: upgrade 21.2.0 -> 21.7.0
python3-xlsxwriter: upgrade 1.4.4 -> 1.4.5
zhengruoqin (15):
live555: upgrade 20210710 -> 20210720
libtest-warnings-perl: upgrade 0.030 -> 0.031
python3-pybind11: upgrade 2.6.2 -> 2.7.0
python3-pymongo: upgrade 3.11.4 -> 3.12.0
python3-sqlalchemy: upgrade 1.4.20 -> 1.4.22
python3-sentry-sdk: upgrade 1.2.0 -> 1.3.0
libcurses-perl: upgrade 1.37 -> 1.38
libdbd-sqlite-perl: upgrade 1.66 -> 1.68
libencode-perl: upgrade 3.10 -> 3.11
python3-bitarray: upgrade 2.2.2 -> 2.2.3
python3-cbor2: upgrade 5.4.0 -> 5.4.1
python3-gast: upgrade 0.5.0 -> 0.5.1
poppler: upgrade 21.07.0 -> 21.08.0
valijson: upgrade 0.4 -> 0.5
xwd: upgrade 1.0.7 -> 1.0.8
meta-security: 152cdb506b..c885d399cd:
Armin Kuster (18):
suricata.inc: exclude ppc in rust version
suricata: Drop 4.1.x its EOL
add meta-rust
crowdsec: add pkg
packagegroup-core-security.bb: fix suricat-ptest inclusion
gitlab-ci.yml: streamline builds matrix
krill: Add new pkg
clamav: fix branch name and update
meta-security: Convert to new override syntax
meta-tpm: Convert to new override syntax
meta-integrity: Convert to new override syntax
meta-hardening: Convert to new override syntax
meta-security-isafw: Convert to new override syntax
meta-parsec: Convert to new override syntax
meta-security-compliance: Convert to new override syntax
dynamix-layers: Convert to new override syntax
kas: Convert to new override syntax
packagegroup-core-security.bb: only include suricat-ptest if rust is included
Martin Jansa (1):
layer.conf: Update to honister
Signed-off-by: Patrick Williams <patrick@stwcx.xyz>
Change-Id: Iec7301cf1c43b7cec462dcf88292a8b1b12a5045
Diffstat (limited to 'poky/bitbake')
31 files changed, 515 insertions, 431 deletions
diff --git a/poky/bitbake/bin/bitbake b/poky/bitbake/bin/bitbake index f0ff2400f..dc1873af6 100755 --- a/poky/bitbake/bin/bitbake +++ b/poky/bitbake/bin/bitbake @@ -26,7 +26,7 @@ from bb.main import bitbake_main, BitBakeConfigParameters, BBMainException if sys.getfilesystemencoding() != "utf-8": sys.exit("Please use a locale setting which supports UTF-8 (such as LANG=en_US.UTF-8).\nPython can't change the filesystem locale after loading so we need a UTF-8 when Python starts or things won't work.") -__version__ = "1.51.0" +__version__ = "1.51.1" if __name__ == "__main__": if __version__ != bb.__version__: diff --git a/poky/bitbake/contrib/vim/plugin/newbbappend.vim b/poky/bitbake/contrib/vim/plugin/newbbappend.vim index e04174cf6..3f65f79cd 100644 --- a/poky/bitbake/contrib/vim/plugin/newbbappend.vim +++ b/poky/bitbake/contrib/vim/plugin/newbbappend.vim @@ -20,7 +20,7 @@ fun! NewBBAppendTemplate() set nopaste " New bbappend template - 0 put ='FILESEXTRAPATHS_prepend := \"${THISDIR}/${PN}:\"' + 0 put ='FILESEXTRAPATHS:prepend := \"${THISDIR}/${PN}:\"' 2 if paste == 1 diff --git a/poky/bitbake/contrib/vim/syntax/bitbake.vim b/poky/bitbake/contrib/vim/syntax/bitbake.vim index f964621ae..d8aa0f1ba 100644 --- a/poky/bitbake/contrib/vim/syntax/bitbake.vim +++ b/poky/bitbake/contrib/vim/syntax/bitbake.vim @@ -51,9 +51,9 @@ syn region bbString matchgroup=bbQuote start=+'+ skip=+\\$+ end=+'+ syn match bbExport "^export" nextgroup=bbIdentifier skipwhite syn keyword bbExportFlag export contained nextgroup=bbIdentifier skipwhite syn match bbIdentifier "[a-zA-Z0-9\-_\.\/\+]\+" display contained -syn match bbVarDeref "${[a-zA-Z0-9\-_\.\/\+]\+}" contained +syn match bbVarDeref "${[a-zA-Z0-9\-_:\.\/\+]\+}" contained syn match bbVarEq "\(:=\|+=\|=+\|\.=\|=\.\|?=\|??=\|=\)" contained nextgroup=bbVarValue -syn match bbVarDef "^\(export\s*\)\?\([a-zA-Z0-9\-_\.\/\+]\+\(_[${}a-zA-Z0-9\-_\.\/\+]\+\)\?\)\s*\(:=\|+=\|=+\|\.=\|=\.\|?=\|??=\|=\)\@=" contains=bbExportFlag,bbIdentifier,bbVarDeref nextgroup=bbVarEq +syn match bbVarDef "^\(export\s*\)\?\([a-zA-Z0-9\-_\.\/\+][${}a-zA-Z0-9\-_:\.\/\+]*\)\s*\(:=\|+=\|=+\|\.=\|=\.\|?=\|??=\|=\)\@=" contains=bbExportFlag,bbIdentifier,bbOverrideOperator,bbVarDeref nextgroup=bbVarEq syn match bbVarValue ".*$" contained contains=bbString,bbVarDeref,bbVarPyValue syn region bbVarPyValue start=+${@+ skip=+\\$+ end=+}+ contained contains=@python @@ -77,13 +77,15 @@ syn keyword bbOEFunctions do_fetch do_unpack do_patch do_configure do_comp " Generic Functions syn match bbFunction "\h[0-9A-Za-z_\-\.]*" display contained contains=bbOEFunctions +syn keyword bbOverrideOperator append prepend contained + " BitBake shell metadata syn include @shell syntax/sh.vim if exists("b:current_syntax") unlet b:current_syntax endif syn keyword bbShFakeRootFlag fakeroot contained -syn match bbShFuncDef "^\(fakeroot\s*\)\?\([\.0-9A-Za-z_${}\-\.]\+\)\(python\)\@<!\(\s*()\s*\)\({\)\@=" contains=bbShFakeRootFlag,bbFunction,bbVarDeref,bbDelimiter nextgroup=bbShFuncRegion skipwhite +syn match bbShFuncDef "^\(fakeroot\s*\)\?\([\.0-9A-Za-z_:${}\-\.]\+\)\(python\)\@<!\(\s*()\s*\)\({\)\@=" contains=bbShFakeRootFlag,bbFunction,bbOverrideOperator,bbVarDeref,bbDelimiter nextgroup=bbShFuncRegion skipwhite syn region bbShFuncRegion matchgroup=bbDelimiter start="{\s*$" end="^}\s*$" contained contains=@shell " Python value inside shell functions @@ -91,7 +93,7 @@ syn region shDeref start=+${@+ skip=+\\$+ excludenl end=+}+ contained co " BitBake python metadata syn keyword bbPyFlag python contained -syn match bbPyFuncDef "^\(fakeroot\s*\)\?\(python\)\(\s\+[0-9A-Za-z_${}\-\.]\+\)\?\(\s*()\s*\)\({\)\@=" contains=bbShFakeRootFlag,bbPyFlag,bbFunction,bbVarDeref,bbDelimiter nextgroup=bbPyFuncRegion skipwhite +syn match bbPyFuncDef "^\(fakeroot\s*\)\?\(python\)\(\s\+[0-9A-Za-z_:${}\-\.]\+\)\?\(\s*()\s*\)\({\)\@=" contains=bbShFakeRootFlag,bbPyFlag,bbFunction,bbOverrideOperator,bbVarDeref,bbDelimiter nextgroup=bbPyFuncRegion skipwhite syn region bbPyFuncRegion matchgroup=bbDelimiter start="{\s*$" end="^}\s*$" contained contains=@python " BitBake 'def'd python functions @@ -122,5 +124,6 @@ hi def link bbStatement Statement hi def link bbStatementRest Identifier hi def link bbOEFunctions Special hi def link bbVarPyValue PreProc +hi def link bbOverrideOperator Operator let b:current_syntax = "bb" diff --git a/poky/bitbake/doc/Makefile b/poky/bitbake/doc/Makefile index d40f390e2..996f01b7d 100644 --- a/poky/bitbake/doc/Makefile +++ b/poky/bitbake/doc/Makefile @@ -3,7 +3,7 @@ # You can set these variables from the command line, and also # from the environment for the first two. -SPHINXOPTS ?= -j auto +SPHINXOPTS ?= -W --keep-going -j auto SPHINXBUILD ?= sphinx-build SOURCEDIR = . BUILDDIR = _build diff --git a/poky/bitbake/doc/bitbake-user-manual/bitbake-user-manual-execution.rst b/poky/bitbake/doc/bitbake-user-manual/bitbake-user-manual-execution.rst index 84d65fa9c..a6ef90db1 100644 --- a/poky/bitbake/doc/bitbake-user-manual/bitbake-user-manual-execution.rst +++ b/poky/bitbake/doc/bitbake-user-manual/bitbake-user-manual-execution.rst @@ -40,7 +40,7 @@ the BitBake command and its options, see ":ref:`The BitBake Command the number of processors, which takes into account hyper-threading. Thus, a quad-core build host with hyper-threading most likely shows eight processors, which is the value you would then assign to - ``BB_NUMBER_THREADS``. + :term:`BB_NUMBER_THREADS`. A possibly simpler solution is that some Linux distributions (e.g. Debian and Ubuntu) provide the ``ncpus`` command. @@ -65,13 +65,13 @@ data itself is of various types: The ``layer.conf`` files are used to construct key variables such as :term:`BBPATH` and :term:`BBFILES`. -``BBPATH`` is used to search for configuration and class files under the -``conf`` and ``classes`` directories, respectively. ``BBFILES`` is used +:term:`BBPATH` is used to search for configuration and class files under the +``conf`` and ``classes`` directories, respectively. :term:`BBFILES` is used to locate both recipe and recipe append files (``.bb`` and ``.bbappend``). If there is no ``bblayers.conf`` file, it is assumed the -user has set the ``BBPATH`` and ``BBFILES`` directly in the environment. +user has set the :term:`BBPATH` and :term:`BBFILES` directly in the environment. -Next, the ``bitbake.conf`` file is located using the ``BBPATH`` variable +Next, the ``bitbake.conf`` file is located using the :term:`BBPATH` variable that was just constructed. The ``bitbake.conf`` file may also include other configuration files using the ``include`` or ``require`` directives. @@ -104,7 +104,7 @@ BitBake first searches the current working directory for an optional contain a :term:`BBLAYERS` variable that is a space-delimited list of 'layer' directories. Recall that if BitBake cannot find a ``bblayers.conf`` file, then it is assumed the user has -set the ``BBPATH`` and ``BBFILES`` variables directly in the +set the :term:`BBPATH` and :term:`BBFILES` variables directly in the environment. For each directory (layer) in this list, a ``conf/layer.conf`` file is @@ -114,7 +114,7 @@ files automatically set up :term:`BBPATH` and other variables correctly for a given build directory. BitBake then expects to find the ``conf/bitbake.conf`` file somewhere in -the user-specified ``BBPATH``. That configuration file generally has +the user-specified :term:`BBPATH`. That configuration file generally has include directives to pull in any other metadata such as files specific to the architecture, the machine, the local environment, and so forth. @@ -135,7 +135,7 @@ The ``base.bbclass`` file is always included. Other classes that are specified in the configuration using the :term:`INHERIT` variable are also included. BitBake searches for class files in a ``classes`` subdirectory under the paths -in ``BBPATH`` in the same way as configuration files. +in :term:`BBPATH` in the same way as configuration files. A good way to get an idea of the configuration files and the class files used in your execution environment is to run the following BitBake @@ -184,13 +184,13 @@ Locating and Parsing Recipes During the configuration phase, BitBake will have set :term:`BBFILES`. BitBake now uses it to construct a list of recipes to parse, along with any append files (``.bbappend``) to -apply. ``BBFILES`` is a space-separated list of available files and +apply. :term:`BBFILES` is a space-separated list of available files and supports wildcards. An example would be:: BBFILES = "/path/to/bbfiles/*.bb /path/to/appends/*.bbappend" BitBake parses each -recipe and append file located with ``BBFILES`` and stores the values of +recipe and append file located with :term:`BBFILES` and stores the values of various variables into the datastore. .. note:: @@ -201,7 +201,7 @@ For each file, a fresh copy of the base configuration is made, then the recipe is parsed line by line. Any inherit statements cause BitBake to find and then parse class files (``.bbclass``) using :term:`BBPATH` as the search path. Finally, BitBake -parses in order any append files found in ``BBFILES``. +parses in order any append files found in :term:`BBFILES`. One common convention is to use the recipe filename to define pieces of metadata. For example, in ``bitbake.conf`` the recipe name and version @@ -212,7 +212,7 @@ are used to set the variables :term:`PN` and PV = "${@bb.parse.BBHandler.vars_from_file(d.getVar('FILE', False),d)[1] or '1.0'}" In this example, a recipe called "something_1.2.3.bb" would set -``PN`` to "something" and ``PV`` to "1.2.3". +:term:`PN` to "something" and :term:`PV` to "1.2.3". By the time parsing is complete for a recipe, BitBake has a list of tasks that the recipe defines and a set of data consisting of keys and @@ -260,21 +260,21 @@ Providers Assuming BitBake has been instructed to execute a target and that all the recipe files have been parsed, BitBake starts to figure out how to -build the target. BitBake looks through the ``PROVIDES`` list for each -of the recipes. A ``PROVIDES`` list is the list of names by which the -recipe can be known. Each recipe's ``PROVIDES`` list is created +build the target. BitBake looks through the :term:`PROVIDES` list for each +of the recipes. A :term:`PROVIDES` list is the list of names by which the +recipe can be known. Each recipe's :term:`PROVIDES` list is created implicitly through the recipe's :term:`PN` variable and explicitly through the recipe's :term:`PROVIDES` variable, which is optional. -When a recipe uses ``PROVIDES``, that recipe's functionality can be -found under an alternative name or names other than the implicit ``PN`` +When a recipe uses :term:`PROVIDES`, that recipe's functionality can be +found under an alternative name or names other than the implicit :term:`PN` name. As an example, suppose a recipe named ``keyboard_1.0.bb`` contained the following:: PROVIDES += "fullkeyboard" -The ``PROVIDES`` +The :term:`PROVIDES` list for this recipe becomes "keyboard", which is implicit, and "fullkeyboard", which is explicit. Consequently, the functionality found in ``keyboard_1.0.bb`` can be found under two different names. @@ -284,12 +284,12 @@ in ``keyboard_1.0.bb`` can be found under two different names. Preferences =========== -The ``PROVIDES`` list is only part of the solution for figuring out a +The :term:`PROVIDES` list is only part of the solution for figuring out a target's recipes. Because targets might have multiple providers, BitBake needs to prioritize providers by determining provider preferences. A common example in which a target has multiple providers is -"virtual/kernel", which is on the ``PROVIDES`` list for each kernel +"virtual/kernel", which is on the :term:`PROVIDES` list for each kernel recipe. Each machine often selects the best kernel provider by using a line similar to the following in the machine configuration file:: @@ -309,10 +309,10 @@ specify a particular version. You can influence the order by using the :term:`DEFAULT_PREFERENCE` variable. By default, files have a preference of "0". Setting -``DEFAULT_PREFERENCE`` to "-1" makes the recipe unlikely to be used -unless it is explicitly referenced. Setting ``DEFAULT_PREFERENCE`` to -"1" makes it likely the recipe is used. ``PREFERRED_VERSION`` overrides -any ``DEFAULT_PREFERENCE`` setting. ``DEFAULT_PREFERENCE`` is often used +:term:`DEFAULT_PREFERENCE` to "-1" makes the recipe unlikely to be used +unless it is explicitly referenced. Setting :term:`DEFAULT_PREFERENCE` to +"1" makes it likely the recipe is used. :term:`PREFERRED_VERSION` overrides +any :term:`DEFAULT_PREFERENCE` setting. :term:`DEFAULT_PREFERENCE` is often used to mark newer and more experimental recipe versions until they have undergone sufficient testing to be considered stable. @@ -394,7 +394,7 @@ ready to run, those tasks have all their dependencies met, and the thread threshold has not been exceeded. It is worth noting that you can greatly speed up the build time by -properly setting the ``BB_NUMBER_THREADS`` variable. +properly setting the :term:`BB_NUMBER_THREADS` variable. As each task completes, a timestamp is written to the directory specified by the :term:`STAMP` variable. On subsequent @@ -561,7 +561,7 @@ behavior is unchanged from previous versions. ``OE-Core`` uses the BB_SIGNATURE_HANDLER ?= "OEBasicHash" -The "OEBasicHash" ``BB_SIGNATURE_HANDLER`` is the same as the "OEBasic" +The "OEBasicHash" :term:`BB_SIGNATURE_HANDLER` is the same as the "OEBasic" version but adds the task hash to the stamp files. This results in any metadata change that changes the task hash, automatically causing the task to be run again. This removes the need to bump @@ -581,7 +581,7 @@ the build. This information includes: - ``BBHASHDEPS_``\ *filename:taskname*: The task dependencies for each task. -- ``BB_TASKHASH``: The hash of the currently running task. +- :term:`BB_TASKHASH`: The hash of the currently running task. It is worth noting that BitBake's "-S" option lets you debug BitBake's processing of signatures. The options passed to -S allow different diff --git a/poky/bitbake/doc/bitbake-user-manual/bitbake-user-manual-fetching.rst b/poky/bitbake/doc/bitbake-user-manual/bitbake-user-manual-fetching.rst index bd1fb4fc7..593de61f2 100644 --- a/poky/bitbake/doc/bitbake-user-manual/bitbake-user-manual-fetching.rst +++ b/poky/bitbake/doc/bitbake-user-manual/bitbake-user-manual-fetching.rst @@ -51,7 +51,7 @@ This code unpacks the downloaded files to the specified by ``WORKDIR``. examine the OpenEmbedded class file ``base.bbclass`` . -The ``SRC_URI`` and ``WORKDIR`` variables are not hardcoded into the +The :term:`SRC_URI` and ``WORKDIR`` variables are not hardcoded into the fetcher, since those fetcher methods can be (and are) called with different variable names. In OpenEmbedded for example, the shared state (sstate) code uses the fetch module to fetch the sstate files. @@ -64,14 +64,14 @@ URLs by looking for source files in a specific search order: :term:`PREMIRRORS` variable. - *Source URI:* If pre-mirrors fail, BitBake uses the original URL (e.g - from ``SRC_URI``). + from :term:`SRC_URI`). - *Mirror Sites:* If fetch failures occur, BitBake next uses mirror locations as defined by the :term:`MIRRORS` variable. For each URL passed to the fetcher, the fetcher calls the submodule that handles that particular URL type. This behavior can be the source of -some confusion when you are providing URLs for the ``SRC_URI`` variable. +some confusion when you are providing URLs for the :term:`SRC_URI` variable. Consider the following two URLs:: http://git.yoctoproject.org/git/poky;protocol=git @@ -110,14 +110,14 @@ which is specified by the :term:`DL_DIR` variable. File integrity is of key importance for reproducing builds. For non-local archive downloads, the fetcher code can verify SHA-256 and MD5 checksums to ensure the archives have been downloaded correctly. You can -specify these checksums by using the ``SRC_URI`` variable with the +specify these checksums by using the :term:`SRC_URI` variable with the appropriate varflags as follows:: SRC_URI[md5sum] = "value" SRC_URI[sha256sum] = "value" You can also specify the checksums as -parameters on the ``SRC_URI`` as shown below:: +parameters on the :term:`SRC_URI` as shown below:: SRC_URI = "http://example.com/foobar.tar.bz2;md5sum=4a8e0f237e961fd7785d19d07fdb994d" @@ -129,7 +129,7 @@ shows how you name the URIs:: SRC_URI[foo.md5sum] = 4a8e0f237e961fd7785d19d07fdb994d After a file has been downloaded and -has had its checksum checked, a ".done" stamp is placed in ``DL_DIR``. +has had its checksum checked, a ".done" stamp is placed in :term:`DL_DIR`. BitBake uses this stamp during subsequent builds to avoid downloading or comparing a checksum for the file again. @@ -438,7 +438,7 @@ Here are some example URLs:: .. note:: Specifying passwords directly in ``git://`` urls is not supported. - There are several reasons: ``SRC_URI`` is often written out to logs and + There are several reasons: :term:`SRC_URI` is often written out to logs and other places, and that could easily leak passwords; it is also all too easy to share metadata without removing passwords. SSH keys, ``~/.netrc`` and ``~/.ssh/config`` files can be used as alternatives. @@ -487,7 +487,7 @@ To use this fetcher, make sure your recipe has proper The fetcher uses the ``rcleartool`` or ``cleartool`` remote client, depending on which one is available. -Following are options for the ``SRC_URI`` statement: +Following are options for the :term:`SRC_URI` statement: - *vob*: The name, which must include the prepending "/" character, of the ClearCase VOB. This option is required. @@ -549,7 +549,7 @@ password if you do not wish to keep those values in a recipe itself. If you choose not to use ``P4CONFIG``, or to explicitly set variables that ``P4CONFIG`` can contain, you can specify the ``P4PORT`` value, which is the server's URL and port number, and you can specify a username and -password directly in your recipe within ``SRC_URI``. +password directly in your recipe within :term:`SRC_URI`. Here is an example that relies on ``P4CONFIG`` to specify the server URL and port, username, and password, and fetches the Head Revision:: @@ -680,4 +680,4 @@ submodules. However, you might find the code helpful and readable. Auto Revisions ============== -We need to document ``AUTOREV`` and ``SRCREV_FORMAT`` here. +We need to document ``AUTOREV`` and :term:`SRCREV_FORMAT` here. diff --git a/poky/bitbake/doc/bitbake-user-manual/bitbake-user-manual-hello.rst b/poky/bitbake/doc/bitbake-user-manual/bitbake-user-manual-hello.rst index a9c33709a..c5a4ce60b 100644 --- a/poky/bitbake/doc/bitbake-user-manual/bitbake-user-manual-hello.rst +++ b/poky/bitbake/doc/bitbake-user-manual/bitbake-user-manual-hello.rst @@ -145,23 +145,23 @@ Following is the complete "Hello World" example. The majority of this output is specific to environment variables that are not directly relevant to BitBake. However, the very first - message regarding the ``BBPATH`` variable and the + message regarding the :term:`BBPATH` variable and the ``conf/bblayers.conf`` file is relevant. When you run BitBake, it begins looking for metadata files. The :term:`BBPATH` variable is what tells BitBake where - to look for those files. ``BBPATH`` is not set and you need to set - it. Without ``BBPATH``, BitBake cannot find any configuration files + to look for those files. :term:`BBPATH` is not set and you need to set + it. Without :term:`BBPATH`, BitBake cannot find any configuration files (``.conf``) or recipe files (``.bb``) at all. BitBake also cannot find the ``bitbake.conf`` file. -#. **Setting BBPATH:** For this example, you can set ``BBPATH`` in +#. **Setting BBPATH:** For this example, you can set :term:`BBPATH` in the same manner that you set ``PATH`` earlier in the appendix. You should realize, though, that it is much more flexible to set the - ``BBPATH`` variable up in a configuration file for each project. + :term:`BBPATH` variable up in a configuration file for each project. From your shell, enter the following commands to set and export the - ``BBPATH`` variable:: + :term:`BBPATH` variable:: $ BBPATH="projectdirectory" $ export BBPATH @@ -175,7 +175,7 @@ Following is the complete "Hello World" example. ("~") character as BitBake does not expand that character as the shell would. -#. **Run BitBake:** Now that you have ``BBPATH`` defined, run the +#. **Run BitBake:** Now that you have :term:`BBPATH` defined, run the ``bitbake`` command again:: $ bitbake diff --git a/poky/bitbake/doc/bitbake-user-manual/bitbake-user-manual-intro.rst b/poky/bitbake/doc/bitbake-user-manual/bitbake-user-manual-intro.rst index b3cea61ff..76c8e3dac 100644 --- a/poky/bitbake/doc/bitbake-user-manual/bitbake-user-manual-intro.rst +++ b/poky/bitbake/doc/bitbake-user-manual/bitbake-user-manual-intro.rst @@ -27,7 +27,7 @@ Linux software stacks using a task-oriented approach. Conceptually, BitBake is similar to GNU Make in some regards but has significant differences: -- BitBake executes tasks according to provided metadata that builds up +- BitBake executes tasks according to the provided metadata that builds up the tasks. Metadata is stored in recipe (``.bb``) and related recipe "append" (``.bbappend``) files, configuration (``.conf``) and underlying include (``.inc``) files, and in class (``.bbclass``) @@ -417,8 +417,8 @@ Following is the usage and syntax for BitBake:: -l DEBUG_DOMAINS, --log-domains=DEBUG_DOMAINS Show debug logging for the specified logging domains -P, --profile Profile the command and save reports. - -u UI, --ui=UI The user interface to use (knotty, ncurses or taskexp - - default knotty). + -u UI, --ui=UI The user interface to use (knotty, ncurses, taskexp or + teamcity - default knotty). --token=XMLRPCTOKEN Specify the connection token to be used when connecting to a remote server. --revisions-changed Set the exit code depending on whether upstream @@ -433,6 +433,9 @@ Following is the usage and syntax for BitBake:: Environment variable BB_SERVER_TIMEOUT. --no-setscene Do not run any setscene tasks. sstate will be ignored and everything needed, built. + --skip-setscene Skip setscene tasks if they would be executed. Tasks + previously restored from sstate will be kept, unlike + --no-setscene --setscene-only Only run setscene tasks, don't run any real tasks. --remote-server=REMOTE_SERVER Connect to the specified server. @@ -537,7 +540,7 @@ current working directory: To stop depending on common depends, use the "-I" depend option and BitBake omits them from the graph. Leaving this information out can produce more readable graphs. This way, you can remove from the graph -``DEPENDS`` from inherited classes such as ``base.bbclass``. +:term:`DEPENDS` from inherited classes such as ``base.bbclass``. Here are two examples that create dependency graphs. The second example omits depends common in OpenEmbedded from the graph:: @@ -564,7 +567,7 @@ for two separate targets: .. image:: figures/bb_multiconfig_files.png :align: center -The reason for this required file hierarchy is because the ``BBPATH`` +The reason for this required file hierarchy is because the :term:`BBPATH` variable is not constructed until the layers are parsed. Consequently, using the configuration file as a pre-configuration file is not possible unless it is located in the current working directory. diff --git a/poky/bitbake/doc/bitbake-user-manual/bitbake-user-manual-metadata.rst b/poky/bitbake/doc/bitbake-user-manual/bitbake-user-manual-metadata.rst index 20c330e6a..b0494d080 100644 --- a/poky/bitbake/doc/bitbake-user-manual/bitbake-user-manual-metadata.rst +++ b/poky/bitbake/doc/bitbake-user-manual/bitbake-user-manual-metadata.rst @@ -91,9 +91,10 @@ level changes: Variables that are exported to the environment are preceded by the string "export" in the command's output. -- For recipe changes, use the following:: +- To find changes to a given variable in a specific recipe, use the + following:: - $ bitbake recipe -e \| grep VARIABLE=" + $ bitbake recipename -e | grep VARIABLENAME=\" This command checks to see if the variable actually makes it into a specific recipe. @@ -225,7 +226,7 @@ immediately, rather than when the variable is actually used:: C := "${C}append" In this example, ``A`` contains "test 123", even though the final value -of ``T`` is "456". The variable ``B`` will end up containing "456 +of :term:`T` is "456". The variable :term:`B` will end up containing "456 cvalappend". This is because references to undefined variables are preserved as is during (immediate)expansion. This is in contrast to GNU Make, where undefined variables expand to nothing. The variable ``C`` @@ -248,7 +249,7 @@ examples:: C = "cval" C =+ "test" -The variable ``B`` contains "bval additionaldata" and ``C`` contains "test +The variable :term:`B` contains "bval additionaldata" and ``C`` contains "test cval". .. _appending-and-prepending-without-spaces: @@ -267,7 +268,7 @@ examples:: C = "cval" C =. "test" -The variable ``B`` contains "bvaladditionaldata" and ``C`` contains +The variable :term:`B` contains "bvaladditionaldata" and ``C`` contains "testcval". Appending and Prepending (Override Style Syntax) @@ -281,13 +282,13 @@ operators in that their effects are applied at variable expansion time rather than being immediately applied. Here are some examples:: B = "bval" - B_append = " additional data" + B:append = " additional data" C = "cval" - C_prepend = "additional data " + C:prepend = "additional data " D = "dval" - D_append = "additional data" + D:append = "additional data" -The variable ``B`` +The variable :term:`B` becomes "bval additional data" and ``C`` becomes "additional data cval". The variable ``D`` becomes "dvaladditional data". @@ -312,10 +313,10 @@ When you use this syntax, BitBake expects one or more strings. Surrounding spaces and spacing are preserved. Here is an example:: FOO = "123 456 789 123456 123 456 123 456" - FOO_remove = "123" - FOO_remove = "456" + FOO:remove = "123" + FOO:remove = "456" FOO2 = " abc def ghi abcdef abc def abc def def" - FOO2_remove = "\ + FOO2:remove = "\ def \ abc \ ghi \ @@ -324,14 +325,14 @@ Surrounding spaces and spacing are preserved. Here is an example:: The variable ``FOO`` becomes " 789 123456 " and ``FOO2`` becomes " abcdef ". -Like "_append" and "_prepend", "_remove" is applied at variable +Like ":append" and ":prepend", ":remove" is applied at variable expansion time. Override Style Operation Advantages ----------------------------------- -An advantage of the override style operations "_append", "_prepend", and -"_remove" as compared to the "+=" and "=+" operators is that the +An advantage of the override style operations ":append", ":prepend", and +":remove" as compared to the "+=" and "=+" operators is that the override style operators provide guaranteed operations. For example, consider a class ``foo.bbclass`` that needs to add the value "val" to the variable ``FOO``, and a recipe that uses ``foo.bbclass`` as follows:: @@ -346,18 +347,18 @@ not what is desired:: FOO += "val" If, on the other hand, ``foo.bbclass`` -uses the "_append" operator, then the final value of ``FOO`` will be +uses the ":append" operator, then the final value of ``FOO`` will be "initial val", as intended:: - FOO_append = " val" + FOO:append = " val" .. note:: - It is never necessary to use "+=" together with "_append". The following + It is never necessary to use "+=" together with ":append". The following sequence of assignments appends "barbaz" to FOO:: - FOO_append = "bar" - FOO_append = "baz" + FOO:append = "bar" + FOO:append = "baz" The only effect of changing the second assignment in the previous @@ -378,8 +379,8 @@ You can find more out about variable flags in general in the You can define, append, and prepend values to variable flags. All the standard syntax operations previously mentioned work for variable flags -except for override style syntax (i.e. "_prepend", "_append", and -"_remove"). +except for override style syntax (i.e. ":prepend", ":append", and +":remove"). Here are some examples showing how to set variable flags:: @@ -496,14 +497,14 @@ Conditional Syntax (Overrides) BitBake uses :term:`OVERRIDES` to control what variables are overridden after BitBake parses recipes and configuration -files. This section describes how you can use ``OVERRIDES`` as +files. This section describes how you can use :term:`OVERRIDES` as conditional metadata, talks about key expansion in relationship to -``OVERRIDES``, and provides some examples to help with understanding. +:term:`OVERRIDES`, and provides some examples to help with understanding. Conditional Metadata -------------------- -You can use ``OVERRIDES`` to conditionally select a specific version of +You can use :term:`OVERRIDES` to conditionally select a specific version of a variable and to conditionally append or prepend the value of a variable. @@ -513,10 +514,10 @@ variable. underscores are not permitted in override names as they are used to separate overrides from each other and from the variable name. -- *Selecting a Variable:* The ``OVERRIDES`` variable is a +- *Selecting a Variable:* The :term:`OVERRIDES` variable is a colon-character-separated list that contains items for which you want to satisfy conditions. Thus, if you have a variable that is - conditional on "arm", and "arm" is in ``OVERRIDES``, then the + conditional on "arm", and "arm" is in :term:`OVERRIDES`, then the "arm"-specific version of the variable is used rather than the non-conditional version. Here is an example:: @@ -525,7 +526,7 @@ variable. TEST_os = "osspecific" TEST_nooverride = "othercondvalue" - In this example, the ``OVERRIDES`` + In this example, the :term:`OVERRIDES` variable lists three overrides: "architecture", "os", and "machine". The variable ``TEST`` by itself has a default value of "default". You select the os-specific version of the ``TEST`` variable by appending @@ -538,36 +539,36 @@ variable. that value based on the architecture of the build:: KBRANCH = "standard/base" - KBRANCH_qemuarm = "standard/arm-versatile-926ejs" - KBRANCH_qemumips = "standard/mti-malta32" - KBRANCH_qemuppc = "standard/qemuppc" - KBRANCH_qemux86 = "standard/common-pc/base" - KBRANCH_qemux86-64 = "standard/common-pc-64/base" - KBRANCH_qemumips64 = "standard/mti-malta64" + KBRANCH:qemuarm = "standard/arm-versatile-926ejs" + KBRANCH:qemumips = "standard/mti-malta32" + KBRANCH:qemuppc = "standard/qemuppc" + KBRANCH:qemux86 = "standard/common-pc/base" + KBRANCH:qemux86-64 = "standard/common-pc-64/base" + KBRANCH:qemumips64 = "standard/mti-malta64" - *Appending and Prepending:* BitBake also supports append and prepend operations to variable values based on whether a specific item is - listed in ``OVERRIDES``. Here is an example:: + listed in :term:`OVERRIDES`. Here is an example:: DEPENDS = "glibc ncurses" OVERRIDES = "machine:local" - DEPENDS_append_machine = "libmad" + DEPENDS:append:machine = "libmad" - In this example, ``DEPENDS`` becomes "glibc ncurses libmad". + In this example, :term:`DEPENDS` becomes "glibc ncurses libmad". Again, using an OpenEmbedded metadata-based kernel recipe file as an example, the following lines will conditionally append to the ``KERNEL_FEATURES`` variable based on the architecture:: - KERNEL_FEATURES_append = " ${KERNEL_EXTRA_FEATURES}" - KERNEL_FEATURES_append_qemux86=" cfg/sound.scc cfg/paravirt_kvm.scc" - KERNEL_FEATURES_append_qemux86-64=" cfg/sound.scc cfg/paravirt_kvm.scc" + KERNEL_FEATURES:append = " ${KERNEL_EXTRA_FEATURES}" + KERNEL_FEATURES:append:qemux86=" cfg/sound.scc cfg/paravirt_kvm.scc" + KERNEL_FEATURES:append:qemux86-64=" cfg/sound.scc cfg/paravirt_kvm.scc" - *Setting a Variable for a Single Task:* BitBake supports setting a variable just for the duration of a single task. Here is an example:: FOO_task-configure = "val 1" - FOO_task-compile = "val 2" + FOO:task-compile = "val 2" In the previous example, ``FOO`` has the value "val 1" while the @@ -580,9 +581,9 @@ variable. ``do_compile`` task. You can also use this syntax with other combinations (e.g. - "``_prepend``") as shown in the following example:: + "``:prepend``") as shown in the following example:: - EXTRA_OEMAKE_prepend_task-compile = "${PARALLEL_MAKE} " + EXTRA_OEMAKE:prepend:task-compile = "${PARALLEL_MAKE} " Key Expansion ------------- @@ -612,33 +613,33 @@ users. There is often confusion concerning the order in which overrides and various "append" operators take effect. Recall that an append or prepend -operation using "_append" and "_prepend" does not result in an immediate +operation using ":append" and ":prepend" does not result in an immediate assignment as would "+=", ".=", "=+", or "=.". Consider the following example:: OVERRIDES = "foo" A = "Z" - A_foo_append = "X" + A:foo:append = "X" For this case, ``A`` is unconditionally set to "Z" and "X" is unconditionally and -immediately appended to the variable ``A_foo``. Because overrides have -not been applied yet, ``A_foo`` is set to "X" due to the append and +immediately appended to the variable ``A:foo``. Because overrides have +not been applied yet, ``A:foo`` is set to "X" due to the append and ``A`` simply equals "Z". Applying overrides, however, changes things. Since "foo" is listed in -``OVERRIDES``, the conditional variable ``A`` is replaced with the "foo" -version, which is equal to "X". So effectively, ``A_foo`` replaces +:term:`OVERRIDES`, the conditional variable ``A`` is replaced with the "foo" +version, which is equal to "X". So effectively, ``A:foo`` replaces ``A``. This next example changes the order of the override and the append:: OVERRIDES = "foo" A = "Z" - A_append_foo = "X" + A:append:foo = "X" For this case, before -overrides are handled, ``A`` is set to "Z" and ``A_append_foo`` is set +overrides are handled, ``A`` is set to "Z" and ``A:append:foo`` is set to "X". Once the override for "foo" is applied, however, ``A`` gets appended with "X". Consequently, ``A`` becomes "ZX". Notice that spaces are not appended. @@ -648,21 +649,21 @@ back as in the first example:: OVERRIDES = "foo" A = "Y" - A_foo_append = "Z" - A_foo_append = "X" + A:foo:append = "Z" + A:foo:append = "X" For this case, before any overrides are resolved, ``A`` is set to "Y" using an immediate assignment. After this immediate -assignment, ``A_foo`` is set to "Z", and then further appended with "X" +assignment, ``A:foo`` is set to "Z", and then further appended with "X" leaving the variable set to "ZX". Finally, applying the override for "foo" results in the conditional variable ``A`` becoming "ZX" (i.e. -``A`` is replaced with ``A_foo``). +``A`` is replaced with ``A:foo``). This final example mixes in some varying operators:: A = "1" - A_append = "2" - A_append = "3" + A:append = "2" + A:append = "3" A += "4" A .= "5" @@ -670,7 +671,7 @@ For this case, the type of append operators are affecting the order of assignments as BitBake passes through the code multiple times. Initially, ``A`` is set to "1 45" because of the three statements that use immediate operators. After -these assignments are made, BitBake applies the "_append" operations. +these assignments are made, BitBake applies the ":append" operations. Those operations result in ``A`` becoming "1 4523". Sharing Functionality @@ -686,7 +687,7 @@ share the task. This section presents the mechanisms BitBake provides to allow you to share functionality between recipes. Specifically, the mechanisms -include ``include``, ``inherit``, ``INHERIT``, and ``require`` +include ``include``, ``inherit``, :term:`INHERIT`, and ``require`` directives. Locating Include and Class Files @@ -702,7 +703,7 @@ current directory for ``include`` and ``require`` directives. In order for include and class files to be found by BitBake, they need to be located in a "classes" subdirectory that can be found in -``BBPATH``. +:term:`BBPATH`. ``inherit`` Directive --------------------- @@ -725,7 +726,7 @@ functionality for using Autotools that could be shared across recipes:: inherit autotools In this case, BitBake would search for the directory -``classes/autotools.bbclass`` in ``BBPATH``. +``classes/autotools.bbclass`` in :term:`BBPATH`. .. note:: @@ -752,7 +753,7 @@ parsed. One way to achieve a conditional inherit in this case is to use overrides:: VARIABLE = "" - VARIABLE_someoverride = "myclass" + VARIABLE:someoverride = "myclass" Another method is by using anonymous Python. Here is an example:: @@ -780,7 +781,7 @@ BitBake understands the ``include`` directive. This directive causes BitBake to parse whatever file you specify, and to insert that file at that location. The directive is much like its equivalent in Make except that if the path specified on the include line is a relative path, -BitBake locates the first file it can find within ``BBPATH``. +BitBake locates the first file it can find within :term:`BBPATH`. The include directive is a more generic method of including functionality as compared to the :ref:`inherit <bitbake-user-manual/bitbake-user-manual-metadata:\`\`inherit\`\` directive>` @@ -822,7 +823,7 @@ does not suit a ``.bbclass`` file. Similar to how BitBake handles :ref:`include <bitbake-user-manual/bitbake-user-manual-metadata:\`\`include\`\` directive>`, if the path specified on the require line is a relative path, BitBake -locates the first file it can find within ``BBPATH``. +locates the first file it can find within :term:`BBPATH`. As an example, suppose you have two versions of a recipe (e.g. ``foo_1.2.2.bb`` and ``foo_2.0.0.bb``) where each version contains some @@ -851,7 +852,7 @@ As an example, suppose you needed to inherit a class file called This configuration directive causes the named class to be inherited at the point of the directive during parsing. As with the ``inherit`` directive, the ``.bbclass`` file must be located in a "classes" -subdirectory in one of the directories specified in ``BBPATH``. +subdirectory in one of the directories specified in :term:`BBPATH`. .. note:: @@ -907,7 +908,7 @@ rules. The scripts are executed by ``/bin/sh``, which may not be a bash shell but might be something such as ``dash``. You should not use Bash-specific script (bashisms). -Overrides and override-style operators like ``_append`` and ``_prepend`` +Overrides and override-style operators like ``:append`` and ``:prepend`` can also be applied to shell functions. Most commonly, this application would be used in a ``.bbappend`` file to modify functions in the main recipe. It can also be used to modify functions inherited from classes. @@ -919,7 +920,7 @@ As an example, consider the following:: fn } - fn_prepend() { + fn:prepend() { bbplain second } @@ -927,7 +928,7 @@ As an example, consider the following:: bbplain third } - do_foo_append() { + do_foo:append() { bbplain fourth } @@ -977,7 +978,7 @@ override-style operators to BitBake-style Python functions. As an example, consider the following:: - python do_foo_prepend() { + python do_foo:prepend() { bb.plain("first") } @@ -985,7 +986,7 @@ As an example, consider the following:: bb.plain("second") } - python do_foo_append() { + python do_foo:append() { bb.plain("third") } @@ -1015,7 +1016,7 @@ is an example:: SOMECONDITION = "1" DEPENDS = "${@get_depends(d)}" -This would result in ``DEPENDS`` containing ``dependencywithcond``. +This would result in :term:`DEPENDS` containing ``dependencywithcond``. Here are some things to know about Python functions: @@ -1134,12 +1135,12 @@ equivalent to the following snippet:: values set for the variables within the anonymous functions become available to tasks, which always run after parsing. -Overrides and override-style operators such as "``_append``" are applied +Overrides and override-style operators such as "``:append``" are applied before anonymous functions run. In the following example, ``FOO`` ends up with the value "foo from anonymous":: FOO = "foo" - FOO_append = " from outside" + FOO:append = " from outside" python () { d.setVar("FOO", "foo from anonymous") @@ -1164,7 +1165,7 @@ To understand the benefits of this feature, consider the basic scenario where a class defines a task function and your recipe inherits the class. In this basic scenario, your recipe inherits the task function as defined in the class. If desired, your recipe can add to the start and -end of the function by using the "_prepend" or "_append" operations +end of the function by using the ":prepend" or ":append" operations respectively, or it can redefine the function completely. However, if it redefines the function, there is no means for it to call the class version of the function. ``EXPORT_FUNCTIONS`` provides a mechanism that @@ -1382,7 +1383,7 @@ Sometimes, it is useful to be able to obtain information from the original execution environment. BitBake saves a copy of the original environment into a special variable named :term:`BB_ORIGENV`. -The ``BB_ORIGENV`` variable returns a datastore object that can be +The :term:`BB_ORIGENV` variable returns a datastore object that can be queried using the standard datastore operators such as ``getVar(, False)``. The datastore object is useful, for example, to find the original ``DISPLAY`` variable. Here is an example:: @@ -1467,7 +1468,7 @@ functionality of the task: can result in unpredictable behavior. - Setting the varflag to a value greater than the value used in - the ``BB_NUMBER_THREADS`` variable causes ``number_threads`` to + the :term:`BB_NUMBER_THREADS` variable causes ``number_threads`` to have no effect. - ``[postfuncs]``: List of functions to call after the completion of @@ -1537,7 +1538,7 @@ intent is to make it easy to do things like email notification on build failures. Following is an example event handler that prints the name of the event -and the content of the ``FILE`` variable:: +and the content of the :term:`FILE` variable:: addhandler myclass_eventhandler python myclass_eventhandler() { @@ -1576,7 +1577,7 @@ might have an interest in viewing: - ``bb.event.ConfigParsed()``: Fired when the base configuration; which consists of ``bitbake.conf``, ``base.bbclass`` and any global - ``INHERIT`` statements; has been parsed. You can see multiple such + :term:`INHERIT` statements; has been parsed. You can see multiple such events when each of the workers parse the base configuration or if the server changes configuration and reparses. Any given datastore only has one such event executed against it, however. If @@ -1733,13 +1734,13 @@ Build Dependencies BitBake uses the :term:`DEPENDS` variable to manage build time dependencies. The ``[deptask]`` varflag for tasks signifies -the task of each item listed in ``DEPENDS`` that must complete before +the task of each item listed in :term:`DEPENDS` that must complete before that task can be executed. Here is an example:: do_configure[deptask] = "do_populate_sysroot" In this example, the ``do_populate_sysroot`` task -of each item in ``DEPENDS`` must complete before ``do_configure`` can +of each item in :term:`DEPENDS` must complete before ``do_configure`` can execute. Runtime Dependencies @@ -1748,8 +1749,8 @@ Runtime Dependencies BitBake uses the :term:`PACKAGES`, :term:`RDEPENDS`, and :term:`RRECOMMENDS` variables to manage runtime dependencies. -The ``PACKAGES`` variable lists runtime packages. Each of those packages -can have ``RDEPENDS`` and ``RRECOMMENDS`` runtime dependencies. The +The :term:`PACKAGES` variable lists runtime packages. Each of those packages +can have :term:`RDEPENDS` and :term:`RRECOMMENDS` runtime dependencies. The ``[rdeptask]`` flag for tasks is used to signify the task of each item runtime dependency which must have completed before that task can be executed. :: @@ -1757,9 +1758,9 @@ executed. :: do_package_qa[rdeptask] = "do_packagedata" In the previous -example, the ``do_packagedata`` task of each item in ``RDEPENDS`` must +example, the ``do_packagedata`` task of each item in :term:`RDEPENDS` must have completed before ``do_package_qa`` can execute. -Although ``RDEPENDS`` contains entries from the +Although :term:`RDEPENDS` contains entries from the runtime dependency namespace, BitBake knows how to map them back to the build-time dependency namespace, in which the tasks are defined. @@ -1802,7 +1803,7 @@ Inter-Task Dependencies BitBake uses the ``[depends]`` flag in a more generic form to manage inter-task dependencies. This more generic form allows for inter-dependency checks for specific tasks rather than checks for the -data in ``DEPENDS``. Here is an example:: +data in :term:`DEPENDS`. Here is an example:: do_patch[depends] = "quilt-native:do_populate_sysroot" diff --git a/poky/bitbake/doc/bitbake-user-manual/bitbake-user-manual-ref-variables.rst b/poky/bitbake/doc/bitbake-user-manual/bitbake-user-manual-ref-variables.rst index 2dca52c4a..6283c2654 100644 --- a/poky/bitbake/doc/bitbake-user-manual/bitbake-user-manual-ref-variables.rst +++ b/poky/bitbake/doc/bitbake-user-manual/bitbake-user-manual-ref-variables.rst @@ -24,13 +24,14 @@ overview of their function and contents. described here in this glossary. .. glossary:: + :sorted: :term:`ASSUME_PROVIDED` Lists recipe names (:term:`PN` values) BitBake does not attempt to build. Instead, BitBake assumes these recipes have already been built. - In OpenEmbedded-Core, ``ASSUME_PROVIDED`` mostly specifies native + In OpenEmbedded-Core, :term:`ASSUME_PROVIDED` mostly specifies native tools that should not be built. An example is ``git-native``, which when specified allows for the Git binary from the host to be used rather than building ``git-native``. @@ -83,14 +84,14 @@ overview of their function and contents. - Attempts to access networks not in the host list cause a failure. - Using ``BB_ALLOWED_NETWORKS`` in conjunction with + Using :term:`BB_ALLOWED_NETWORKS` in conjunction with :term:`PREMIRRORS` is very useful. Adding the - host you want to use to ``PREMIRRORS`` results in the source code + host you want to use to :term:`PREMIRRORS` results in the source code being fetched from an allowed location and avoids raising an error when a host that is not allowed is in a :term:`SRC_URI` statement. This is because the - fetcher does not attempt to use the host listed in ``SRC_URI`` after - a successful fetch from the ``PREMIRRORS`` occurs. + fetcher does not attempt to use the host listed in :term:`SRC_URI` after + a successful fetch from the :term:`PREMIRRORS` occurs. :term:`BB_CONSOLELOG` Specifies the path to a log file into which BitBake's user interface @@ -177,7 +178,7 @@ overview of their function and contents. issues a warning when the disk space in the ``${SSTATE_DIR}`` directory drops below 1 Gbyte or the number of free inodes drops below 100 Kbytes. Subsequent warnings are issued during intervals as - defined by the ``BB_DISKMON_WARNINTERVAL`` variable. + defined by the :term:`BB_DISKMON_WARNINTERVAL` variable. The second example stops the build after all currently executing tasks complete when the minimum disk space in the ``${TMPDIR}`` @@ -191,14 +192,14 @@ overview of their function and contents. :term:`BB_DISKMON_WARNINTERVAL` Defines the disk space and free inode warning intervals. - If you are going to use the ``BB_DISKMON_WARNINTERVAL`` variable, you + If you are going to use the :term:`BB_DISKMON_WARNINTERVAL` variable, you must also use the :term:`BB_DISKMON_DIRS` variable and define its action as "WARN". During the build, subsequent warnings are issued each time disk space or number of free inodes further reduces by the respective interval. - If you do not provide a ``BB_DISKMON_WARNINTERVAL`` variable and you - do use ``BB_DISKMON_DIRS`` with the "WARN" action, the disk + If you do not provide a :term:`BB_DISKMON_WARNINTERVAL` variable and you + do use :term:`BB_DISKMON_DIRS` with the "WARN" action, the disk monitoring interval defaults to the following: BB_DISKMON_WARNINTERVAL = "50M,5K" @@ -231,23 +232,23 @@ overview of their function and contents. based on the interval occur each time a respective interval is reached beyond the initial warning (i.e. 1 Gbytes and 100 Kbytes). - :term:`BB_ENV_WHITELIST` - Specifies the internal whitelist of variables to allow through from - the external environment into BitBake's datastore. If the value of - this variable is not specified (which is the default), the following - list is used: :term:`BBPATH`, :term:`BB_PRESERVE_ENV`, - :term:`BB_ENV_WHITELIST`, and :term:`BB_ENV_EXTRAWHITE`. + :term:`BB_ENV_EXTRAWHITE` + Specifies an additional set of variables to allow through (whitelist) + from the external environment into BitBake's datastore. This list of + variables are on top of the internal list set in + :term:`BB_ENV_WHITELIST`. .. note:: You must set this variable in the external environment in order for it to work. - :term:`BB_ENV_EXTRAWHITE` - Specifies an additional set of variables to allow through (whitelist) - from the external environment into BitBake's datastore. This list of - variables are on top of the internal list set in - :term:`BB_ENV_WHITELIST`. + :term:`BB_ENV_WHITELIST` + Specifies the internal whitelist of variables to allow through from + the external environment into BitBake's datastore. If the value of + this variable is not specified (which is the default), the following + list is used: :term:`BBPATH`, :term:`BB_PRESERVE_ENV`, + :term:`BB_ENV_WHITELIST`, and :term:`BB_ENV_EXTRAWHITE`. .. note:: @@ -263,7 +264,7 @@ overview of their function and contents. :term:`BB_FILENAME` Contains the filename of the recipe that owns the currently running task. For example, if the ``do_fetch`` task that resides in the - ``my-recipe.bb`` is executing, the ``BB_FILENAME`` variable contains + ``my-recipe.bb`` is executing, the :term:`BB_FILENAME` variable contains "/foo/path/my-recipe.bb". :term:`BB_GENERATE_MIRROR_TARBALLS` @@ -276,18 +277,6 @@ overview of their function and contents. BB_GENERATE_MIRROR_TARBALLS = "1" - :term:`BB_HASHCONFIG_WHITELIST` - Lists variables that are excluded from base configuration checksum, - which is used to determine if the cache can be reused. - - One of the ways BitBake determines whether to re-parse the main - metadata is through checksums of the variables in the datastore of - the base configuration data. There are variables that you typically - want to exclude when checking whether or not to re-parse and thus - rebuild the cache. As an example, you would usually exclude ``TIME`` - and ``DATE`` because these variables are always changing. If you did - not exclude them, BitBake would never reuse the cache. - :term:`BB_HASHBASE_WHITELIST` Lists variables that are excluded from checksum and dependency data. Variables that are excluded can therefore change without affecting @@ -309,6 +298,18 @@ overview of their function and contents. However, the more accurate the data returned, the more efficient the build will be. + :term:`BB_HASHCONFIG_WHITELIST` + Lists variables that are excluded from base configuration checksum, + which is used to determine if the cache can be reused. + + One of the ways BitBake determines whether to re-parse the main + metadata is through checksums of the variables in the datastore of + the base configuration data. There are variables that you typically + want to exclude when checking whether or not to re-parse and thus + rebuild the cache. As an example, you would usually exclude ``TIME`` + and ``DATE`` because these variables are always changing. If you did + not exclude them, BitBake would never reuse the cache. + :term:`BB_HASHSERVE` Specifies the Hash Equivalence server to use. @@ -333,7 +334,7 @@ overview of their function and contents. :term:`BB_LOGFMT` Specifies the name of the log files saved into - ``${``\ :term:`T`\ ``}``. By default, the ``BB_LOGFMT`` + ``${``\ :term:`T`\ ``}``. By default, the :term:`BB_LOGFMT` variable is undefined and the log file names get created using the following form:: @@ -357,15 +358,15 @@ overview of their function and contents. running builds when not connected to the Internet, and when operating in certain kinds of firewall environments. + :term:`BB_NUMBER_PARSE_THREADS` + Sets the number of threads BitBake uses when parsing. By default, the + number of threads is equal to the number of cores on the system. + :term:`BB_NUMBER_THREADS` The maximum number of tasks BitBake should run in parallel at any one time. If your host development system supports multiple cores, a good rule of thumb is to set this variable to twice the number of cores. - :term:`BB_NUMBER_PARSE_THREADS` - Sets the number of threads BitBake uses when parsing. By default, the - number of threads is equal to the number of cores on the system. - :term:`BB_ORIGENV` Contains a copy of the original external environment in which BitBake was run. The copy is taken before any whitelisted variable values are @@ -388,7 +389,7 @@ overview of their function and contents. :term:`BB_RUNFMT` Specifies the name of the executable script files (i.e. run files) saved into ``${``\ :term:`T`\ ``}``. By default, the - ``BB_RUNFMT`` variable is undefined and the run file names get + :term:`BB_RUNFMT` variable is undefined and the run file names get created using the following form:: run.{task}.{pid} @@ -454,7 +455,7 @@ overview of their function and contents. :term:`BB_SRCREV_POLICY` Defines the behavior of the fetcher when it interacts with source control systems and dynamic source revisions. The - ``BB_SRCREV_POLICY`` variable is useful when working without a + :term:`BB_SRCREV_POLICY` variable is useful when working without a network. The variable can be set using one of two policies: @@ -498,7 +499,7 @@ overview of their function and contents. Allows adjustment of a task's Input/Output priority. During Autobuilder testing, random failures can occur for tasks due to I/O starvation. These failures occur during various QEMU runtime - timeouts. You can use the ``BB_TASK_IONICE_LEVEL`` variable to adjust + timeouts. You can use the :term:`BB_TASK_IONICE_LEVEL` variable to adjust the I/O priority of these tasks. .. note:: @@ -572,13 +573,13 @@ overview of their function and contents. .. note:: - Internally, the ``BBCLASSEXTEND`` mechanism generates recipe + Internally, the :term:`BBCLASSEXTEND` mechanism generates recipe variants by rewriting variable values and applying overrides such as ``_class-native``. For example, to generate a native version of a recipe, a :term:`DEPENDS` on "foo" is - rewritten to a ``DEPENDS`` on "foo-native". + rewritten to a :term:`DEPENDS` on "foo-native". - Even when using ``BBCLASSEXTEND``, the recipe is only parsed once. + Even when using :term:`BBCLASSEXTEND`, the recipe is only parsed once. Parsing once adds some limitations. For example, it is not possible to include a different file depending on the variant, since ``include`` statements are processed when the recipe is @@ -614,14 +615,14 @@ overview of their function and contents. - effectively letting you control the precedence for the multiple layers. The precedence established through this variable stands regardless of a recipe's version (:term:`PV` variable). - For example, a layer that has a recipe with a higher ``PV`` value but - for which the ``BBFILE_PRIORITY`` is set to have a lower precedence + For example, a layer that has a recipe with a higher :term:`PV` value but + for which the :term:`BBFILE_PRIORITY` is set to have a lower precedence still has a lower precedence. - A larger value for the ``BBFILE_PRIORITY`` variable results in a + A larger value for the :term:`BBFILE_PRIORITY` variable results in a higher precedence. For example, the value 6 has a higher precedence - than the value 5. If not specified, the ``BBFILE_PRIORITY`` variable - is set based on layer dependencies (see the ``LAYERDEPENDS`` variable + than the value 5. If not specified, the :term:`BBFILE_PRIORITY` variable + is set based on layer dependencies (see the :term:`LAYERDEPENDS` variable for more information. The default priority, if unspecified for a layer with no dependencies, is the lowest defined priority + 1 (or 1 if no priorities are defined). @@ -644,7 +645,7 @@ overview of their function and contents. Activates content depending on presence of identified layers. You identify the layers by the collections that the layers define. - Use the ``BBFILES_DYNAMIC`` variable to avoid ``.bbappend`` files whose + Use the :term:`BBFILES_DYNAMIC` variable to avoid ``.bbappend`` files whose corresponding ``.bb`` file is in a layer that attempts to modify other layers through ``.bbappend`` but does not want to introduce a hard dependency on those other layers. @@ -653,7 +654,7 @@ overview of their function and contents. ``.bb`` files in case a layer is not present. Use this avoid hard dependency on those other layers. - Use the following form for ``BBFILES_DYNAMIC``:: + Use the following form for :term:`BBFILES_DYNAMIC`:: collection_name:filename_pattern @@ -690,7 +691,7 @@ overview of their function and contents. :term:`BBINCLUDELOGS_LINES` If :term:`BBINCLUDELOGS` is set, specifies the maximum number of lines from the task log file to print when - reporting a failed task. If you do not set ``BBINCLUDELOGS_LINES``, + reporting a failed task. If you do not set :term:`BBINCLUDELOGS_LINES`, the entire log is printed. :term:`BBLAYERS` @@ -716,7 +717,7 @@ overview of their function and contents. :term:`BBMASK` Prevents BitBake from processing recipes and recipe append files. - You can use the ``BBMASK`` variable to "hide" these ``.bb`` and + You can use the :term:`BBMASK` variable to "hide" these ``.bb`` and ``.bbappend`` files. BitBake ignores any recipe or recipe append files that match any of the expressions. It is as if BitBake does not see them at all. Consequently, matching files are not parsed or @@ -753,7 +754,7 @@ overview of their function and contents. Enables BitBake to perform multiple configuration builds and lists each separate configuration (multiconfig). You can use this variable to cause BitBake to build multiple targets where each target has a - separate configuration. Define ``BBMULTICONFIG`` in your + separate configuration. Define :term:`BBMULTICONFIG` in your ``conf/local.conf`` configuration file. As an example, the following line specifies three multiconfigs, each @@ -765,7 +766,7 @@ overview of their function and contents. build directory within a directory named ``conf/multiconfig`` (e.g. build_directory\ ``/conf/multiconfig/configA.conf``). - For information on how to use ``BBMULTICONFIG`` in an environment + For information on how to use :term:`BBMULTICONFIG` in an environment that supports building targets with multiple configurations, see the ":ref:`bitbake-user-manual/bitbake-user-manual-intro:executing a multiple configuration build`" section. @@ -776,7 +777,7 @@ overview of their function and contents. variable. If you run BitBake from a directory outside of the build directory, - you must be sure to set ``BBPATH`` to point to the build directory. + you must be sure to set :term:`BBPATH` to point to the build directory. Set the variable as you would any environment variable and then run BitBake:: @@ -823,7 +824,7 @@ overview of their function and contents. The most common usage of this is variable is to set it to "-1" within a recipe for a development version of a piece of software. Using the variable in this way causes the stable version of the recipe to build - by default in the absence of ``PREFERRED_VERSION`` being used to + by default in the absence of :term:`PREFERRED_VERSION` being used to build the development version. .. note:: @@ -836,7 +837,7 @@ overview of their function and contents. Lists a recipe's build-time dependencies (i.e. other recipe files). Consider this simple example for two recipes named "a" and "b" that - produce similarly named packages. In this example, the ``DEPENDS`` + produce similarly named packages. In this example, the :term:`DEPENDS` statement appears in the "a" recipe:: DEPENDS = "b" @@ -854,7 +855,7 @@ overview of their function and contents. :term:`DL_DIR` The central download directory used by the build process to store - downloads. By default, ``DL_DIR`` gets files suitable for mirroring for + downloads. By default, :term:`DL_DIR` gets files suitable for mirroring for everything except Git repositories. If you want tarballs of Git repositories, use the :term:`BB_GENERATE_MIRROR_TARBALLS` variable. @@ -869,14 +870,14 @@ overview of their function and contents. .. note:: - Recipes added to ``EXCLUDE_FROM_WORLD`` may still be built during a world + Recipes added to :term:`EXCLUDE_FROM_WORLD` may still be built during a world build in order to satisfy dependencies of other recipes. Adding a - recipe to ``EXCLUDE_FROM_WORLD`` only ensures that the recipe is not + recipe to :term:`EXCLUDE_FROM_WORLD` only ensures that the recipe is not explicitly added to the list of build targets in a world build. :term:`FAKEROOT` Contains the command to use when running a shell script in a fakeroot - environment. The ``FAKEROOT`` variable is obsolete and has been + environment. The :term:`FAKEROOT` variable is obsolete and has been replaced by the other ``FAKEROOT*`` variables. See these entries in the glossary for more information. @@ -939,9 +940,9 @@ overview of their function and contents. Causes the named class or classes to be inherited globally. Anonymous functions in the class or classes are not executed for the base configuration and in each individual recipe. The OpenEmbedded build - system ignores changes to ``INHERIT`` in individual recipes. + system ignores changes to :term:`INHERIT` in individual recipes. - For more information on ``INHERIT``, see the + For more information on :term:`INHERIT`, see the ":ref:`bitbake-user-manual/bitbake-user-manual-metadata:\`\`inherit\`\` configuration directive`" section. @@ -989,7 +990,7 @@ overview of their function and contents. the build system searches for source code, it first tries the local download directory. If that location fails, the build system tries locations defined by :term:`PREMIRRORS`, the - upstream source, and then locations specified by ``MIRRORS`` in that + upstream source, and then locations specified by :term:`MIRRORS` in that order. :term:`MULTI_PROVIDER_WHITELIST` @@ -1006,12 +1007,12 @@ overview of their function and contents. ``virtual/kernel``, and so forth). :term:`OVERRIDES` - BitBake uses ``OVERRIDES`` to control what variables are overridden + BitBake uses :term:`OVERRIDES` to control what variables are overridden after BitBake parses recipes and configuration files. Following is a simple example that uses an overrides list based on machine architectures: OVERRIDES = "arm:x86:mips:powerpc" You can - find information on how to use ``OVERRIDES`` in the + find information on how to use :term:`OVERRIDES` in the ":ref:`bitbake-user-manual/bitbake-user-manual-metadata:conditional syntax (overrides)`" section. @@ -1025,11 +1026,11 @@ overview of their function and contents. :term:`PACKAGES_DYNAMIC` A promise that your recipe satisfies runtime dependencies for optional modules that are found in other recipes. - ``PACKAGES_DYNAMIC`` does not actually satisfy the dependencies, it + :term:`PACKAGES_DYNAMIC` does not actually satisfy the dependencies, it only states that they should be satisfied. For example, if a hard, runtime dependency (:term:`RDEPENDS`) of another package is satisfied during the build through the - ``PACKAGES_DYNAMIC`` variable, but a package with the module name is + :term:`PACKAGES_DYNAMIC` variable, but a package with the module name is never actually produced, then the other package will be broken. :term:`PE` @@ -1068,8 +1069,8 @@ overview of their function and contents. :term:`PREFERRED_PROVIDERS` Determines which recipe should be given preference for cases where multiple recipes provide the same item. Functionally, - ``PREFERRED_PROVIDERS`` is identical to - :term:`PREFERRED_PROVIDER`. However, the ``PREFERRED_PROVIDERS`` variable + :term:`PREFERRED_PROVIDERS` is identical to + :term:`PREFERRED_PROVIDER`. However, the :term:`PREFERRED_PROVIDERS` variable lets you define preferences for multiple situations using the following form:: @@ -1087,7 +1088,7 @@ overview of their function and contents. select, and you should set :term:`PV` accordingly for precedence. - The ``PREFERRED_VERSION`` variable supports limited wildcard use + The :term:`PREFERRED_VERSION` variable supports limited wildcard use through the "``%``" character. You can use the character to match any number of characters, which can be useful when specifying versions that contain long revision numbers that potentially change. Here are @@ -1110,14 +1111,14 @@ overview of their function and contents. Specifies additional paths from which BitBake gets source code. When the build system searches for source code, it first tries the local download directory. If that location fails, the build system tries - locations defined by ``PREMIRRORS``, the upstream source, and then + locations defined by :term:`PREMIRRORS`, the upstream source, and then locations specified by :term:`MIRRORS` in that order. Typically, you would add a specific server for the build system to attempt before any others by adding something like the following to your configuration:: - PREMIRRORS_prepend = "\ + PREMIRRORS:prepend = "\ git://.*/.* http://www.yoctoproject.org/sources/ \n \ ftp://.*/.* http://www.yoctoproject.org/sources/ \n \ http://.*/.* http://www.yoctoproject.org/sources/ \n \ @@ -1130,25 +1131,25 @@ overview of their function and contents. :term:`PROVIDES` A list of aliases by which a particular recipe can be known. By - default, a recipe's own ``PN`` is implicitly already in its - ``PROVIDES`` list. If a recipe uses ``PROVIDES``, the additional + default, a recipe's own :term:`PN` is implicitly already in its + :term:`PROVIDES` list. If a recipe uses :term:`PROVIDES`, the additional aliases are synonyms for the recipe and can be useful satisfying dependencies of other recipes during the build as specified by - ``DEPENDS``. + :term:`DEPENDS`. - Consider the following example ``PROVIDES`` statement from a recipe + Consider the following example :term:`PROVIDES` statement from a recipe file ``libav_0.8.11.bb``:: PROVIDES += "libpostproc" - The ``PROVIDES`` statement results in the "libav" recipe also being known + The :term:`PROVIDES` statement results in the "libav" recipe also being known as "libpostproc". In addition to providing recipes under alternate names, the - ``PROVIDES`` mechanism is also used to implement virtual targets. A + :term:`PROVIDES` mechanism is also used to implement virtual targets. A virtual target is a name that corresponds to some particular functionality (e.g. a Linux kernel). Recipes that provide the - functionality in question list the virtual target in ``PROVIDES``. + functionality in question list the virtual target in :term:`PROVIDES`. Recipes that depend on the functionality in question can include the virtual target in :term:`DEPENDS` to leave the choice of provider open. @@ -1160,12 +1161,12 @@ overview of their function and contents. :term:`PRSERV_HOST` The network based :term:`PR` service host and port. - Following is an example of how the ``PRSERV_HOST`` variable is set:: + Following is an example of how the :term:`PRSERV_HOST` variable is set:: PRSERV_HOST = "localhost:0" You must set the variable if you want to automatically start a local PR - service. You can set ``PRSERV_HOST`` to other values to use a remote PR + service. You can set :term:`PRSERV_HOST` to other values to use a remote PR service. :term:`PV` @@ -1177,24 +1178,24 @@ overview of their function and contents. a package in this list cannot be found during the build, you will get a build error. - Because the ``RDEPENDS`` variable applies to packages being built, + Because the :term:`RDEPENDS` variable applies to packages being built, you should always use the variable in a form with an attached package name. For example, suppose you are building a development package that depends on the ``perl`` package. In this case, you would use the - following ``RDEPENDS`` statement:: + following :term:`RDEPENDS` statement:: - RDEPENDS_${PN}-dev += "perl" + RDEPENDS:${PN}-dev += "perl" In the example, the development package depends on the ``perl`` package. - Thus, the ``RDEPENDS`` variable has the ``${PN}-dev`` package name as part + Thus, the :term:`RDEPENDS` variable has the ``${PN}-dev`` package name as part of the variable. BitBake supports specifying versioned dependencies. Although the syntax varies depending on the packaging format, BitBake hides these differences from you. Here is the general syntax to specify versions - with the ``RDEPENDS`` variable:: + with the :term:`RDEPENDS` variable:: - RDEPENDS_${PN} = "package (operator version)" + RDEPENDS:${PN} = "package (operator version)" For ``operator``, you can specify the following:: @@ -1207,7 +1208,7 @@ overview of their function and contents. For example, the following sets up a dependency on version 1.2 or greater of the package ``foo``:: - RDEPENDS_${PN} = "foo (>= 1.2)" + RDEPENDS:${PN} = "foo (>= 1.2)" For information on build-time dependencies, see the :term:`DEPENDS` variable. @@ -1218,39 +1219,39 @@ overview of their function and contents. :term:`REQUIRED_VERSION` If there are multiple versions of a recipe available, this variable - determines which version should be given preference. ``REQUIRED_VERSION`` + determines which version should be given preference. :term:`REQUIRED_VERSION` works in exactly the same manner as :term:`PREFERRED_VERSION`, except that if the specified version is not available then an error message is shown and the build fails immediately. - If both ``REQUIRED_VERSION`` and ``PREFERRED_VERSION`` are set for - the same recipe, the ``REQUIRED_VERSION`` value applies. + If both :term:`REQUIRED_VERSION` and :term:`PREFERRED_VERSION` are set for + the same recipe, the :term:`REQUIRED_VERSION` value applies. :term:`RPROVIDES` A list of package name aliases that a package also provides. These aliases are useful for satisfying runtime dependencies of other packages both during the build and on the target (as specified by - ``RDEPENDS``). + :term:`RDEPENDS`). As with all package-controlling variables, you must always use the variable in conjunction with a package name override. Here is an example:: - RPROVIDES_${PN} = "widget-abi-2" + RPROVIDES:${PN} = "widget-abi-2" :term:`RRECOMMENDS` A list of packages that extends the usability of a package being built. The package being built does not depend on this list of packages in order to successfully build, but needs them for the extended usability. To specify runtime dependencies for packages, see - the ``RDEPENDS`` variable. + the :term:`RDEPENDS` variable. BitBake supports specifying versioned recommends. Although the syntax varies depending on the packaging format, BitBake hides these differences from you. Here is the general syntax to specify versions - with the ``RRECOMMENDS`` variable:: + with the :term:`RRECOMMENDS` variable:: - RRECOMMENDS_${PN} = "package (operator version)" + RRECOMMENDS:${PN} = "package (operator version)" For ``operator``, you can specify the following:: @@ -1263,7 +1264,7 @@ overview of their function and contents. For example, the following sets up a recommend on version 1.2 or greater of the package ``foo``:: - RRECOMMENDS_${PN} = "foo (>= 1.2)" + RRECOMMENDS:${PN} = "foo (>= 1.2)" :term:`SECTION` The section in which packages should be categorized. @@ -1272,10 +1273,10 @@ overview of their function and contents. The list of source files - local or remote. This variable tells BitBake which bits to pull for the build and how to pull them. For example, if the recipe or append file needs to fetch a single tarball - from the Internet, the recipe or append file uses a ``SRC_URI`` entry + from the Internet, the recipe or append file uses a :term:`SRC_URI` entry that specifies that tarball. On the other hand, if the recipe or append file needs to fetch a tarball and include a custom file, the - recipe or append file needs an ``SRC_URI`` variable that specifies + recipe or append file needs an :term:`SRC_URI` variable that specifies all those sources. The following list explains the available URI protocols: @@ -1328,8 +1329,8 @@ overview of their function and contents. subdirectory within the archive. - ``name`` : Specifies a name to be used for association with - ``SRC_URI`` checksums when you have more than one file specified - in ``SRC_URI``. + :term:`SRC_URI` checksums when you have more than one file specified + in :term:`SRC_URI`. - ``downloadfilename`` : Specifies the filename used when storing the downloaded file. @@ -1344,7 +1345,7 @@ overview of their function and contents. variable applies only when using Subversion, Git, Mercurial and Bazaar. If you want to build a fixed revision and you want to avoid performing a query on the remote repository every time BitBake parses - your recipe, you should specify a ``SRCREV`` that is a full revision + your recipe, you should specify a :term:`SRCREV` that is a full revision identifier and not just a tag. :term:`SRCREV_FORMAT` @@ -1353,10 +1354,10 @@ overview of their function and contents. :term:`SRC_URI`. The system needs help constructing these values under these - circumstances. Each component in the ``SRC_URI`` is assigned a name - and these are referenced in the ``SRCREV_FORMAT`` variable. Consider + circumstances. Each component in the :term:`SRC_URI` is assigned a name + and these are referenced in the :term:`SRCREV_FORMAT` variable. Consider an example with URLs named "machine" and "meta". In this case, - ``SRCREV_FORMAT`` could look like "machine_meta" and those names + :term:`SRCREV_FORMAT` could look like "machine_meta" and those names would have the SCM versions substituted into each position. Only one ``AUTOINC`` placeholder is added and if needed. And, this placeholder is placed at the start of the returned string. @@ -1368,7 +1369,7 @@ overview of their function and contents. :term:`STAMPCLEAN` Specifies the base path used to create recipe stamp files. Unlike the - :term:`STAMP` variable, ``STAMPCLEAN`` can contain + :term:`STAMP` variable, :term:`STAMPCLEAN` can contain wildcards to match the range of files a clean operation should remove. BitBake uses a clean operation to remove any other stamps it should be removing when creating a new stamp. diff --git a/poky/bitbake/lib/bb/__init__.py b/poky/bitbake/lib/bb/__init__.py index a144bd609..c1e30697b 100644 --- a/poky/bitbake/lib/bb/__init__.py +++ b/poky/bitbake/lib/bb/__init__.py @@ -9,7 +9,7 @@ # SPDX-License-Identifier: GPL-2.0-only # -__version__ = "1.51.0" +__version__ = "1.51.1" import sys if sys.version_info < (3, 5, 0): diff --git a/poky/bitbake/lib/bb/asyncrpc/client.py b/poky/bitbake/lib/bb/asyncrpc/client.py index 79919c5be..3eb4fdde8 100644 --- a/poky/bitbake/lib/bb/asyncrpc/client.py +++ b/poky/bitbake/lib/bb/asyncrpc/client.py @@ -11,13 +11,14 @@ from . import chunkify, DEFAULT_MAX_CHUNK class AsyncClient(object): - def __init__(self, proto_name, proto_version, logger): + def __init__(self, proto_name, proto_version, logger, timeout=30): self.reader = None self.writer = None self.max_chunk = DEFAULT_MAX_CHUNK self.proto_name = proto_name self.proto_version = proto_version self.logger = logger + self.timeout = timeout async def connect_tcp(self, address, port): async def connect_sock(): @@ -70,14 +71,18 @@ class AsyncClient(object): async def send_message(self, msg): async def get_line(): - line = await self.reader.readline() + try: + line = await asyncio.wait_for(self.reader.readline(), self.timeout) + except asyncio.TimeoutError: + raise ConnectionError("Timed out waiting for server") + if not line: raise ConnectionError("Connection closed") line = line.decode("utf-8") if not line.endswith("\n"): - raise ConnectionError("Bad message %r" % msg) + raise ConnectionError("Bad message %r" % (line)) return line diff --git a/poky/bitbake/lib/bb/asyncrpc/serv.py b/poky/bitbake/lib/bb/asyncrpc/serv.py index ef20cb71d..4084f300d 100644 --- a/poky/bitbake/lib/bb/asyncrpc/serv.py +++ b/poky/bitbake/lib/bb/asyncrpc/serv.py @@ -9,6 +9,7 @@ import os import signal import socket import sys +import multiprocessing from . import chunkify, DEFAULT_MAX_CHUNK @@ -201,12 +202,14 @@ class AsyncServer(object): pass def signal_handler(self): + self.logger.debug("Got exit signal") self.loop.stop() def serve_forever(self): asyncio.set_event_loop(self.loop) try: self.loop.add_signal_handler(signal.SIGTERM, self.signal_handler) + signal.pthread_sigmask(signal.SIG_UNBLOCK, [signal.SIGTERM]) self.run_loop_forever() self.server.close() @@ -221,3 +224,21 @@ class AsyncServer(object): if self._cleanup_socket is not None: self._cleanup_socket() + + def serve_as_process(self, *, prefunc=None, args=()): + def run(): + if prefunc is not None: + prefunc(self, *args) + self.serve_forever() + + # Temporarily block SIGTERM. The server process will inherit this + # block which will ensure it doesn't receive the SIGTERM until the + # handler is ready for it + mask = signal.pthread_sigmask(signal.SIG_BLOCK, [signal.SIGTERM]) + try: + self.process = multiprocessing.Process(target=run) + self.process.start() + + return self.process + finally: + signal.pthread_sigmask(signal.SIG_SETMASK, mask) diff --git a/poky/bitbake/lib/bb/cache.py b/poky/bitbake/lib/bb/cache.py index 27eb27179..73bc6e966 100644 --- a/poky/bitbake/lib/bb/cache.py +++ b/poky/bitbake/lib/bb/cache.py @@ -53,12 +53,12 @@ class RecipeInfoCommon(object): @classmethod def pkgvar(cls, var, packages, metadata): - return dict((pkg, cls.depvar("%s_%s" % (var, pkg), metadata)) + return dict((pkg, cls.depvar("%s:%s" % (var, pkg), metadata)) for pkg in packages) @classmethod def taskvar(cls, var, tasks, metadata): - return dict((task, cls.getvar("%s_task-%s" % (var, task), metadata)) + return dict((task, cls.getvar("%s:task-%s" % (var, task), metadata)) for task in tasks) @classmethod diff --git a/poky/bitbake/lib/bb/command.py b/poky/bitbake/lib/bb/command.py index f530cf844..a81dcb136 100644 --- a/poky/bitbake/lib/bb/command.py +++ b/poky/bitbake/lib/bb/command.py @@ -65,9 +65,17 @@ class Command: # Ensure cooker is ready for commands if command != "updateConfig" and command != "setFeatures": - self.cooker.init_configdata() - if not self.remotedatastores: - self.remotedatastores = bb.remotedata.RemoteDatastores(self.cooker) + try: + self.cooker.init_configdata() + if not self.remotedatastores: + self.remotedatastores = bb.remotedata.RemoteDatastores(self.cooker) + except (Exception, SystemExit) as exc: + import traceback + if isinstance(exc, bb.BBHandledException): + # We need to start returning real exceptions here. Until we do, we can't + # tell if an exception is an instance of bb.BBHandledException + return None, "bb.BBHandledException()\n" + traceback.format_exc() + return None, traceback.format_exc() if hasattr(CommandsSync, command): # Can run synchronous commands straight away diff --git a/poky/bitbake/lib/bb/cooker.py b/poky/bitbake/lib/bb/cooker.py index 39e10e613..b2d69c28c 100644 --- a/poky/bitbake/lib/bb/cooker.py +++ b/poky/bitbake/lib/bb/cooker.py @@ -390,8 +390,7 @@ class BBCooker: dbfile = (self.data.getVar("PERSISTENT_DIR") or self.data.getVar("CACHE")) + "/hashserv.db" self.hashservaddr = "unix://%s/hashserve.sock" % self.data.getVar("TOPDIR") self.hashserv = hashserv.create_server(self.hashservaddr, dbfile, sync=False) - self.hashserv.process = multiprocessing.Process(target=self.hashserv.serve_forever) - self.hashserv.process.start() + self.hashserv.serve_as_process() self.data.setVar("BB_HASHSERVE", self.hashservaddr) self.databuilder.origdata.setVar("BB_HASHSERVE", self.hashservaddr) self.databuilder.data.setVar("BB_HASHSERVE", self.hashservaddr) diff --git a/poky/bitbake/lib/bb/data_smart.py b/poky/bitbake/lib/bb/data_smart.py index f48726a34..65528c6ae 100644 --- a/poky/bitbake/lib/bb/data_smart.py +++ b/poky/bitbake/lib/bb/data_smart.py @@ -26,9 +26,9 @@ from bb.COW import COWDictBase logger = logging.getLogger("BitBake.Data") -__setvar_keyword__ = ["_append", "_prepend", "_remove"] -__setvar_regexp__ = re.compile(r'(?P<base>.*?)(?P<keyword>_append|_prepend|_remove)(_(?P<add>[^A-Z]*))?$') -__expand_var_regexp__ = re.compile(r"\${[a-zA-Z0-9\-_+./~]+?}") +__setvar_keyword__ = [":append", ":prepend", ":remove"] +__setvar_regexp__ = re.compile(r'(?P<base>.*?)(?P<keyword>:append|:prepend|:remove)(:(?P<add>[^A-Z]*))?$') +__expand_var_regexp__ = re.compile(r"\${[a-zA-Z0-9\-_+./~:]+?}") __expand_python_regexp__ = re.compile(r"\${@.+?}") __whitespace_split__ = re.compile(r'(\s)') __override_regexp__ = re.compile(r'[a-z0-9]+') @@ -277,7 +277,7 @@ class VariableHistory(object): for (r, override) in d.overridedata[var]: for event in self.variable(r): loginfo = event.copy() - if 'flag' in loginfo and not loginfo['flag'].startswith("_"): + if 'flag' in loginfo and not loginfo['flag'].startswith(("_", ":")): continue loginfo['variable'] = var loginfo['op'] = 'override[%s]:%s' % (override, loginfo['op']) @@ -342,7 +342,7 @@ class VariableHistory(object): for event in history: if 'flag' in event: continue - if event['op'] == '_remove': + if event['op'] == ':remove': continue if isset and event['op'] == 'set?': continue @@ -481,7 +481,15 @@ class DataSmart(MutableMapping): def setVar(self, var, value, **loginfo): #print("var=" + str(var) + " val=" + str(value)) - var = var.replace(":", "_") + + if "_append" in var or "_prepend" in var or "_remove" in var: + info = "%s" % var + if "filename" in loginfo: + info += " file: %s" % loginfo[filename] + if "lineno" in loginfo: + info += " line: %s" % loginfo[lineno] + bb.fatal("Variable %s contains an operation using the old override syntax. Please convert this layer/metadata before attempting to use with a newer bitbake." % info) + self.expand_cache = {} parsing=False if 'parsing' in loginfo: @@ -510,7 +518,7 @@ class DataSmart(MutableMapping): # pay the cookie monster # more cookies for the cookie monster - if '_' in var: + if ':' in var: self._setvar_update_overrides(base, **loginfo) if base in self.overridevars: @@ -521,27 +529,27 @@ class DataSmart(MutableMapping): self._makeShadowCopy(var) if not parsing: - if "_append" in self.dict[var]: - del self.dict[var]["_append"] - if "_prepend" in self.dict[var]: - del self.dict[var]["_prepend"] - if "_remove" in self.dict[var]: - del self.dict[var]["_remove"] + if ":append" in self.dict[var]: + del self.dict[var][":append"] + if ":prepend" in self.dict[var]: + del self.dict[var][":prepend"] + if ":remove" in self.dict[var]: + del self.dict[var][":remove"] if var in self.overridedata: active = [] self.need_overrides() for (r, o) in self.overridedata[var]: if o in self.overridesset: active.append(r) - elif "_" in o: - if set(o.split("_")).issubset(self.overridesset): + elif ":" in o: + if set(o.split(":")).issubset(self.overridesset): active.append(r) for a in active: self.delVar(a) del self.overridedata[var] # more cookies for the cookie monster - if '_' in var: + if ':' in var: self._setvar_update_overrides(var, **loginfo) # setting var @@ -567,8 +575,8 @@ class DataSmart(MutableMapping): def _setvar_update_overrides(self, var, **loginfo): # aka pay the cookie monster - override = var[var.rfind('_')+1:] - shortvar = var[:var.rfind('_')] + override = var[var.rfind(':')+1:] + shortvar = var[:var.rfind(':')] while override and __override_regexp__.match(override): if shortvar not in self.overridedata: self.overridedata[shortvar] = [] @@ -577,9 +585,9 @@ class DataSmart(MutableMapping): self.overridedata[shortvar] = list(self.overridedata[shortvar]) self.overridedata[shortvar].append([var, override]) override = None - if "_" in shortvar: - override = var[shortvar.rfind('_')+1:] - shortvar = var[:shortvar.rfind('_')] + if ":" in shortvar: + override = var[shortvar.rfind(':')+1:] + shortvar = var[:shortvar.rfind(':')] if len(shortvar) == 0: override = None @@ -590,8 +598,6 @@ class DataSmart(MutableMapping): """ Rename the variable key to newkey """ - key = key.replace(":", "_") - newkey = newkey.replace(":", "_") if key == newkey: bb.warn("Calling renameVar with equivalent keys (%s) is invalid" % key) return @@ -620,7 +626,7 @@ class DataSmart(MutableMapping): self.overridedata[newkey].append([v.replace(key, newkey), o]) self.renameVar(v, v.replace(key, newkey)) - if '_' in newkey and val is None: + if ':' in newkey and val is None: self._setvar_update_overrides(newkey, **loginfo) loginfo['variable'] = key @@ -632,15 +638,14 @@ class DataSmart(MutableMapping): def appendVar(self, var, value, **loginfo): loginfo['op'] = 'append' self.varhistory.record(**loginfo) - self.setVar(var + "_append", value, ignore=True, parsing=True) + self.setVar(var + ":append", value, ignore=True, parsing=True) def prependVar(self, var, value, **loginfo): loginfo['op'] = 'prepend' self.varhistory.record(**loginfo) - self.setVar(var + "_prepend", value, ignore=True, parsing=True) + self.setVar(var + ":prepend", value, ignore=True, parsing=True) def delVar(self, var, **loginfo): - var = var.replace(":", "_") self.expand_cache = {} loginfo['detail'] = "" @@ -649,9 +654,9 @@ class DataSmart(MutableMapping): self.dict[var] = {} if var in self.overridedata: del self.overridedata[var] - if '_' in var: - override = var[var.rfind('_')+1:] - shortvar = var[:var.rfind('_')] + if ':' in var: + override = var[var.rfind(':')+1:] + shortvar = var[:var.rfind(':')] while override and override.islower(): try: if shortvar in self.overridedata: @@ -661,14 +666,13 @@ class DataSmart(MutableMapping): except ValueError as e: pass override = None - if "_" in shortvar: - override = var[shortvar.rfind('_')+1:] - shortvar = var[:shortvar.rfind('_')] + if ":" in shortvar: + override = var[shortvar.rfind(':')+1:] + shortvar = var[:shortvar.rfind(':')] if len(shortvar) == 0: override = None def setVarFlag(self, var, flag, value, **loginfo): - var = var.replace(":", "_") self.expand_cache = {} if 'op' not in loginfo: @@ -679,7 +683,7 @@ class DataSmart(MutableMapping): self._makeShadowCopy(var) self.dict[var][flag] = value - if flag == "_defaultval" and '_' in var: + if flag == "_defaultval" and ':' in var: self._setvar_update_overrides(var, **loginfo) if flag == "_defaultval" and var in self.overridevars: self._setvar_update_overridevars(var, value) @@ -692,7 +696,6 @@ class DataSmart(MutableMapping): self.dict["__exportlist"]["_content"].add(var) def getVarFlag(self, var, flag, expand=True, noweakdefault=False, parsing=False, retparser=False): - var = var.replace(":", "_") if flag == "_content": cachename = var else: @@ -712,11 +715,11 @@ class DataSmart(MutableMapping): active = {} self.need_overrides() for (r, o) in overridedata: - # What about double overrides both with "_" in the name? + # FIXME What about double overrides both with "_" in the name? if o in self.overridesset: active[o] = r - elif "_" in o: - if set(o.split("_")).issubset(self.overridesset): + elif ":" in o: + if set(o.split(":")).issubset(self.overridesset): active[o] = r mod = True @@ -724,10 +727,10 @@ class DataSmart(MutableMapping): mod = False for o in self.overrides: for a in active.copy(): - if a.endswith("_" + o): + if a.endswith(":" + o): t = active[a] del active[a] - active[a.replace("_" + o, "")] = t + active[a.replace(":" + o, "")] = t mod = True elif a == o: match = active[a] @@ -746,31 +749,31 @@ class DataSmart(MutableMapping): value = copy.copy(local_var["_defaultval"]) - if flag == "_content" and local_var is not None and "_append" in local_var and not parsing: - if not value: - value = "" + if flag == "_content" and local_var is not None and ":append" in local_var and not parsing: self.need_overrides() - for (r, o) in local_var["_append"]: + for (r, o) in local_var[":append"]: match = True if o: - for o2 in o.split("_"): + for o2 in o.split(":"): if not o2 in self.overrides: match = False if match: + if value is None: + value = "" value = value + r - if flag == "_content" and local_var is not None and "_prepend" in local_var and not parsing: - if not value: - value = "" + if flag == "_content" and local_var is not None and ":prepend" in local_var and not parsing: self.need_overrides() - for (r, o) in local_var["_prepend"]: + for (r, o) in local_var[":prepend"]: match = True if o: - for o2 in o.split("_"): + for o2 in o.split(":"): if not o2 in self.overrides: match = False if match: + if value is None: + value = "" value = r + value parser = None @@ -779,12 +782,12 @@ class DataSmart(MutableMapping): if expand: value = parser.value - if value and flag == "_content" and local_var is not None and "_remove" in local_var and not parsing: + if value and flag == "_content" and local_var is not None and ":remove" in local_var and not parsing: self.need_overrides() - for (r, o) in local_var["_remove"]: + for (r, o) in local_var[":remove"]: match = True if o: - for o2 in o.split("_"): + for o2 in o.split(":"): if not o2 in self.overrides: match = False if match: @@ -820,7 +823,6 @@ class DataSmart(MutableMapping): return value def delVarFlag(self, var, flag, **loginfo): - var = var.replace(":", "_") self.expand_cache = {} local_var, _ = self._findVar(var) @@ -838,7 +840,6 @@ class DataSmart(MutableMapping): del self.dict[var][flag] def appendVarFlag(self, var, flag, value, **loginfo): - var = var.replace(":", "_") loginfo['op'] = 'append' loginfo['flag'] = flag self.varhistory.record(**loginfo) @@ -846,7 +847,6 @@ class DataSmart(MutableMapping): self.setVarFlag(var, flag, newvalue, ignore=True) def prependVarFlag(self, var, flag, value, **loginfo): - var = var.replace(":", "_") loginfo['op'] = 'prepend' loginfo['flag'] = flag self.varhistory.record(**loginfo) @@ -854,7 +854,6 @@ class DataSmart(MutableMapping): self.setVarFlag(var, flag, newvalue, ignore=True) def setVarFlags(self, var, flags, **loginfo): - var = var.replace(":", "_") self.expand_cache = {} infer_caller_details(loginfo) if not var in self.dict: @@ -869,13 +868,12 @@ class DataSmart(MutableMapping): self.dict[var][i] = flags[i] def getVarFlags(self, var, expand = False, internalflags=False): - var = var.replace(":", "_") local_var, _ = self._findVar(var) flags = {} if local_var: for i in local_var: - if i.startswith("_") and not internalflags: + if i.startswith(("_", ":")) and not internalflags: continue flags[i] = local_var[i] if expand and i in expand: @@ -886,7 +884,6 @@ class DataSmart(MutableMapping): def delVarFlags(self, var, **loginfo): - var = var.replace(":", "_") self.expand_cache = {} if not var in self.dict: self._makeShadowCopy(var) @@ -974,8 +971,8 @@ class DataSmart(MutableMapping): for (r, o) in self.overridedata[var]: if o in self.overridesset: overrides.add(var) - elif "_" in o: - if set(o.split("_")).issubset(self.overridesset): + elif ":" in o: + if set(o.split(":")).issubset(self.overridesset): overrides.add(var) for k in keylist(self.dict): diff --git a/poky/bitbake/lib/bb/fetch2/__init__.py b/poky/bitbake/lib/bb/fetch2/__init__.py index 0d49e1da3..ad898680f 100644 --- a/poky/bitbake/lib/bb/fetch2/__init__.py +++ b/poky/bitbake/lib/bb/fetch2/__init__.py @@ -1146,11 +1146,11 @@ def srcrev_internal_helper(ud, d, name): pn = d.getVar("PN") attempts = [] if name != '' and pn: - attempts.append("SRCREV_%s_pn-%s" % (name, pn)) + attempts.append("SRCREV_%s:pn-%s" % (name, pn)) if name != '': attempts.append("SRCREV_%s" % name) if pn: - attempts.append("SRCREV_pn-%s" % pn) + attempts.append("SRCREV:pn-%s" % pn) attempts.append("SRCREV") for a in attempts: diff --git a/poky/bitbake/lib/bb/parse/ast.py b/poky/bitbake/lib/bb/parse/ast.py index db2bdc35e..743ea0dfc 100644 --- a/poky/bitbake/lib/bb/parse/ast.py +++ b/poky/bitbake/lib/bb/parse/ast.py @@ -97,7 +97,6 @@ class DataNode(AstNode): def eval(self, data): groupd = self.groupd key = groupd["var"] - key = key.replace(":", "_") loginfo = { 'variable': key, 'file': self.filename, @@ -146,7 +145,7 @@ class DataNode(AstNode): data.setVar(key, val, parsing=True, **loginfo) class MethodNode(AstNode): - tr_tbl = str.maketrans('/.+-@%&', '_______') + tr_tbl = str.maketrans('/.+-@%&~', '________') def __init__(self, filename, lineno, func_name, body, python, fakeroot): AstNode.__init__(self, filename, lineno) @@ -208,7 +207,6 @@ class ExportFuncsNode(AstNode): def eval(self, data): for func in self.n: - func = func.replace(":", "_") calledfunc = self.classname + "_" + func if data.getVar(func, False) and not data.getVarFlag(func, 'export_func', False): diff --git a/poky/bitbake/lib/bb/runqueue.py b/poky/bitbake/lib/bb/runqueue.py index 6c41fe6d4..25e012125 100644 --- a/poky/bitbake/lib/bb/runqueue.py +++ b/poky/bitbake/lib/bb/runqueue.py @@ -2443,6 +2443,11 @@ class RunQueueExecute: if update_tasks: self.sqdone = False + for tid in [t[0] for t in update_tasks]: + h = pending_hash_index(tid, self.rqdata) + if h in self.sqdata.hashes and tid != self.sqdata.hashes[h]: + self.sq_deferred[tid] = self.sqdata.hashes[h] + bb.note("Deferring %s after %s" % (tid, self.sqdata.hashes[h])) update_scenequeue_data([t[0] for t in update_tasks], self.sqdata, self.rqdata, self.rq, self.cooker, self.stampcache, self, summary=False) for (tid, harddepfail, origvalid) in update_tasks: @@ -2786,6 +2791,19 @@ def build_scenequeue_data(sqdata, rqdata, rq, cooker, stampcache, sqrq): sqdata.stamppresent = set() sqdata.valid = set() + sqdata.hashes = {} + sqrq.sq_deferred = {} + for mc in sorted(sqdata.multiconfigs): + for tid in sorted(sqdata.sq_revdeps): + if mc_from_tid(tid) != mc: + continue + h = pending_hash_index(tid, rqdata) + if h not in sqdata.hashes: + sqdata.hashes[h] = tid + else: + sqrq.sq_deferred[tid] = sqdata.hashes[h] + bb.note("Deferring %s after %s" % (tid, sqdata.hashes[h])) + update_scenequeue_data(sqdata.sq_revdeps, sqdata, rqdata, rq, cooker, stampcache, sqrq, summary=True) # Compute a list of 'stale' sstate tasks where the current hash does not match the one @@ -2850,32 +2868,20 @@ def update_scenequeue_data(tids, sqdata, rqdata, rq, cooker, stampcache, sqrq, s sqdata.valid |= rq.validate_hashes(tocheck, cooker.data, len(sqdata.stamppresent), False, summary=summary) - sqdata.hashes = {} - sqrq.sq_deferred = {} - for mc in sorted(sqdata.multiconfigs): - for tid in sorted(sqdata.sq_revdeps): - if mc_from_tid(tid) != mc: - continue - if tid in sqdata.stamppresent: - continue - if tid in sqdata.valid: - continue - if tid in sqdata.noexec: - continue - if tid in sqrq.scenequeue_notcovered: - continue - if tid in sqrq.scenequeue_covered: - continue - - h = pending_hash_index(tid, rqdata) - if h not in sqdata.hashes: - if tid in tids: - sqdata.outrightfail.add(tid) - sqdata.hashes[h] = tid - else: - sqrq.sq_deferred[tid] = sqdata.hashes[h] - bb.note("Deferring %s after %s" % (tid, sqdata.hashes[h])) - + for tid in tids: + if tid in sqdata.stamppresent: + continue + if tid in sqdata.valid: + continue + if tid in sqdata.noexec: + continue + if tid in sqrq.scenequeue_covered: + continue + if tid in sqrq.scenequeue_notcovered: + continue + if tid in sqrq.sq_deferred: + continue + sqdata.outrightfail.add(tid) class TaskFailure(Exception): """ diff --git a/poky/bitbake/lib/bb/server/process.py b/poky/bitbake/lib/bb/server/process.py index a0955722e..6127fd40e 100644 --- a/poky/bitbake/lib/bb/server/process.py +++ b/poky/bitbake/lib/bb/server/process.py @@ -26,6 +26,7 @@ import errno import re import datetime import pickle +import traceback import bb.server.xmlrpcserver from bb import daemonize from multiprocessing import queues @@ -217,8 +218,9 @@ class ProcessServer(): self.command_channel_reply.send(self.cooker.command.runCommand(command)) serverlog("Command Completed") except Exception as e: - serverlog('Exception in server main event loop running command %s (%s)' % (command, str(e))) - logger.exception('Exception in server main event loop running command %s (%s)' % (command, str(e))) + stack = traceback.format_exc() + serverlog('Exception in server main event loop running command %s (%s)' % (command, stack)) + logger.exception('Exception in server main event loop running command %s (%s)' % (command, stack)) if self.xmlrpc in ready: self.xmlrpc.handle_requests() diff --git a/poky/bitbake/lib/bb/siggen.py b/poky/bitbake/lib/bb/siggen.py index 07692e673..3f9fe5064 100644 --- a/poky/bitbake/lib/bb/siggen.py +++ b/poky/bitbake/lib/bb/siggen.py @@ -228,7 +228,7 @@ class SignatureGeneratorBasic(SignatureGenerator): # self.dump_sigtask(fn, task, d.getVar("STAMP"), False) for task in taskdeps: - d.setVar("BB_BASEHASH_task-%s" % task, self.basehash[fn + ":" + task]) + d.setVar("BB_BASEHASH:task-%s" % task, self.basehash[fn + ":" + task]) def postparsing_clean_cache(self): # @@ -325,7 +325,7 @@ class SignatureGeneratorBasic(SignatureGenerator): h = hashlib.sha256(data.encode("utf-8")).hexdigest() self.taskhash[tid] = h - #d.setVar("BB_TASKHASH_task-%s" % task, taskhash[task]) + #d.setVar("BB_TASKHASH:task-%s" % task, taskhash[task]) return h def writeout_file_checksum_cache(self): diff --git a/poky/bitbake/lib/bb/tests/codeparser.py b/poky/bitbake/lib/bb/tests/codeparser.py index 826a2d2f6..f48520479 100644 --- a/poky/bitbake/lib/bb/tests/codeparser.py +++ b/poky/bitbake/lib/bb/tests/codeparser.py @@ -111,9 +111,9 @@ ${D}${libdir}/pkgconfig/*.pc self.assertExecs(set(["sed"])) def test_parameter_expansion_modifiers(self): - # - and + are also valid modifiers for parameter expansion, but are + # -,+ and : are also valid modifiers for parameter expansion, but are # valid characters in bitbake variable names, so are not included here - for i in ('=', ':-', ':=', '?', ':?', ':+', '#', '%', '##', '%%'): + for i in ('=', '?', '#', '%', '##', '%%'): name = "foo%sbar" % i self.parseExpression("${%s}" % name) self.assertNotIn(name, self.references) diff --git a/poky/bitbake/lib/bb/tests/data.py b/poky/bitbake/lib/bb/tests/data.py index 1d4a64b10..e667c7c7d 100644 --- a/poky/bitbake/lib/bb/tests/data.py +++ b/poky/bitbake/lib/bb/tests/data.py @@ -245,35 +245,35 @@ class TestConcatOverride(unittest.TestCase): def test_prepend(self): self.d.setVar("TEST", "${VAL}") - self.d.setVar("TEST_prepend", "${FOO}:") + self.d.setVar("TEST:prepend", "${FOO}:") self.assertEqual(self.d.getVar("TEST"), "foo:val") def test_append(self): self.d.setVar("TEST", "${VAL}") - self.d.setVar("TEST_append", ":${BAR}") + self.d.setVar("TEST:append", ":${BAR}") self.assertEqual(self.d.getVar("TEST"), "val:bar") def test_multiple_append(self): self.d.setVar("TEST", "${VAL}") - self.d.setVar("TEST_prepend", "${FOO}:") - self.d.setVar("TEST_append", ":val2") - self.d.setVar("TEST_append", ":${BAR}") + self.d.setVar("TEST:prepend", "${FOO}:") + self.d.setVar("TEST:append", ":val2") + self.d.setVar("TEST:append", ":${BAR}") self.assertEqual(self.d.getVar("TEST"), "foo:val:val2:bar") def test_append_unset(self): - self.d.setVar("TEST_prepend", "${FOO}:") - self.d.setVar("TEST_append", ":val2") - self.d.setVar("TEST_append", ":${BAR}") + self.d.setVar("TEST:prepend", "${FOO}:") + self.d.setVar("TEST:append", ":val2") + self.d.setVar("TEST:append", ":${BAR}") self.assertEqual(self.d.getVar("TEST"), "foo::val2:bar") def test_remove(self): self.d.setVar("TEST", "${VAL} ${BAR}") - self.d.setVar("TEST_remove", "val") + self.d.setVar("TEST:remove", "val") self.assertEqual(self.d.getVar("TEST"), " bar") def test_remove_cleared(self): self.d.setVar("TEST", "${VAL} ${BAR}") - self.d.setVar("TEST_remove", "val") + self.d.setVar("TEST:remove", "val") self.d.setVar("TEST", "${VAL} ${BAR}") self.assertEqual(self.d.getVar("TEST"), "val bar") @@ -281,42 +281,42 @@ class TestConcatOverride(unittest.TestCase): # (including that whitespace is preserved) def test_remove_inactive_override(self): self.d.setVar("TEST", "${VAL} ${BAR} 123") - self.d.setVar("TEST_remove_inactiveoverride", "val") + self.d.setVar("TEST:remove:inactiveoverride", "val") self.assertEqual(self.d.getVar("TEST"), "val bar 123") def test_doubleref_remove(self): self.d.setVar("TEST", "${VAL} ${BAR}") - self.d.setVar("TEST_remove", "val") + self.d.setVar("TEST:remove", "val") self.d.setVar("TEST_TEST", "${TEST} ${TEST}") self.assertEqual(self.d.getVar("TEST_TEST"), " bar bar") def test_empty_remove(self): self.d.setVar("TEST", "") - self.d.setVar("TEST_remove", "val") + self.d.setVar("TEST:remove", "val") self.assertEqual(self.d.getVar("TEST"), "") def test_remove_expansion(self): self.d.setVar("BAR", "Z") self.d.setVar("TEST", "${BAR}/X Y") - self.d.setVar("TEST_remove", "${BAR}/X") + self.d.setVar("TEST:remove", "${BAR}/X") self.assertEqual(self.d.getVar("TEST"), " Y") def test_remove_expansion_items(self): self.d.setVar("TEST", "A B C D") self.d.setVar("BAR", "B D") - self.d.setVar("TEST_remove", "${BAR}") + self.d.setVar("TEST:remove", "${BAR}") self.assertEqual(self.d.getVar("TEST"), "A C ") def test_remove_preserve_whitespace(self): # When the removal isn't active, the original value should be preserved self.d.setVar("TEST", " A B") - self.d.setVar("TEST_remove", "C") + self.d.setVar("TEST:remove", "C") self.assertEqual(self.d.getVar("TEST"), " A B") def test_remove_preserve_whitespace2(self): # When the removal is active preserve the whitespace self.d.setVar("TEST", " A B") - self.d.setVar("TEST_remove", "B") + self.d.setVar("TEST:remove", "B") self.assertEqual(self.d.getVar("TEST"), " A ") class TestOverrides(unittest.TestCase): @@ -329,70 +329,70 @@ class TestOverrides(unittest.TestCase): self.assertEqual(self.d.getVar("TEST"), "testvalue") def test_one_override(self): - self.d.setVar("TEST_bar", "testvalue2") + self.d.setVar("TEST:bar", "testvalue2") self.assertEqual(self.d.getVar("TEST"), "testvalue2") def test_one_override_unset(self): - self.d.setVar("TEST2_bar", "testvalue2") + self.d.setVar("TEST2:bar", "testvalue2") self.assertEqual(self.d.getVar("TEST2"), "testvalue2") - self.assertCountEqual(list(self.d.keys()), ['TEST', 'TEST2', 'OVERRIDES', 'TEST2_bar']) + self.assertCountEqual(list(self.d.keys()), ['TEST', 'TEST2', 'OVERRIDES', 'TEST2:bar']) def test_multiple_override(self): - self.d.setVar("TEST_bar", "testvalue2") - self.d.setVar("TEST_local", "testvalue3") - self.d.setVar("TEST_foo", "testvalue4") + self.d.setVar("TEST:bar", "testvalue2") + self.d.setVar("TEST:local", "testvalue3") + self.d.setVar("TEST:foo", "testvalue4") self.assertEqual(self.d.getVar("TEST"), "testvalue3") - self.assertCountEqual(list(self.d.keys()), ['TEST', 'TEST_foo', 'OVERRIDES', 'TEST_bar', 'TEST_local']) + self.assertCountEqual(list(self.d.keys()), ['TEST', 'TEST:foo', 'OVERRIDES', 'TEST:bar', 'TEST:local']) def test_multiple_combined_overrides(self): - self.d.setVar("TEST_local_foo_bar", "testvalue3") + self.d.setVar("TEST:local:foo:bar", "testvalue3") self.assertEqual(self.d.getVar("TEST"), "testvalue3") def test_multiple_overrides_unset(self): - self.d.setVar("TEST2_local_foo_bar", "testvalue3") + self.d.setVar("TEST2:local:foo:bar", "testvalue3") self.assertEqual(self.d.getVar("TEST2"), "testvalue3") def test_keyexpansion_override(self): self.d.setVar("LOCAL", "local") - self.d.setVar("TEST_bar", "testvalue2") - self.d.setVar("TEST_${LOCAL}", "testvalue3") - self.d.setVar("TEST_foo", "testvalue4") + self.d.setVar("TEST:bar", "testvalue2") + self.d.setVar("TEST:${LOCAL}", "testvalue3") + self.d.setVar("TEST:foo", "testvalue4") bb.data.expandKeys(self.d) self.assertEqual(self.d.getVar("TEST"), "testvalue3") def test_rename_override(self): - self.d.setVar("ALTERNATIVE_ncurses-tools_class-target", "a") + self.d.setVar("ALTERNATIVE:ncurses-tools:class-target", "a") self.d.setVar("OVERRIDES", "class-target") - self.d.renameVar("ALTERNATIVE_ncurses-tools", "ALTERNATIVE_lib32-ncurses-tools") - self.assertEqual(self.d.getVar("ALTERNATIVE_lib32-ncurses-tools"), "a") + self.d.renameVar("ALTERNATIVE:ncurses-tools", "ALTERNATIVE:lib32-ncurses-tools") + self.assertEqual(self.d.getVar("ALTERNATIVE:lib32-ncurses-tools"), "a") def test_underscore_override(self): - self.d.setVar("TEST_bar", "testvalue2") - self.d.setVar("TEST_some_val", "testvalue3") - self.d.setVar("TEST_foo", "testvalue4") + self.d.setVar("TEST:bar", "testvalue2") + self.d.setVar("TEST:some_val", "testvalue3") + self.d.setVar("TEST:foo", "testvalue4") self.d.setVar("OVERRIDES", "foo:bar:some_val") self.assertEqual(self.d.getVar("TEST"), "testvalue3") def test_remove_with_override(self): - self.d.setVar("TEST_bar", "testvalue2") - self.d.setVar("TEST_some_val", "testvalue3 testvalue5") - self.d.setVar("TEST_some_val_remove", "testvalue3") - self.d.setVar("TEST_foo", "testvalue4") + self.d.setVar("TEST:bar", "testvalue2") + self.d.setVar("TEST:some_val", "testvalue3 testvalue5") + self.d.setVar("TEST:some_val:remove", "testvalue3") + self.d.setVar("TEST:foo", "testvalue4") self.d.setVar("OVERRIDES", "foo:bar:some_val") self.assertEqual(self.d.getVar("TEST"), " testvalue5") def test_append_and_override_1(self): - self.d.setVar("TEST_append", "testvalue2") - self.d.setVar("TEST_bar", "testvalue3") + self.d.setVar("TEST:append", "testvalue2") + self.d.setVar("TEST:bar", "testvalue3") self.assertEqual(self.d.getVar("TEST"), "testvalue3testvalue2") def test_append_and_override_2(self): - self.d.setVar("TEST_append_bar", "testvalue2") + self.d.setVar("TEST:append:bar", "testvalue2") self.assertEqual(self.d.getVar("TEST"), "testvaluetestvalue2") def test_append_and_override_3(self): - self.d.setVar("TEST_bar_append", "testvalue2") + self.d.setVar("TEST:bar:append", "testvalue2") self.assertEqual(self.d.getVar("TEST"), "testvalue2") # Test an override with _<numeric> in it based on a real world OE issue @@ -400,11 +400,16 @@ class TestOverrides(unittest.TestCase): self.d.setVar("TARGET_ARCH", "x86_64") self.d.setVar("PN", "test-${TARGET_ARCH}") self.d.setVar("VERSION", "1") - self.d.setVar("VERSION_pn-test-${TARGET_ARCH}", "2") + self.d.setVar("VERSION:pn-test-${TARGET_ARCH}", "2") self.d.setVar("OVERRIDES", "pn-${PN}") bb.data.expandKeys(self.d) self.assertEqual(self.d.getVar("VERSION"), "2") + def test_append_and_unused_override(self): + # Had a bug where an unused override append could return "" instead of None + self.d.setVar("BAR:append:unusedoverride", "testvalue2") + self.assertEqual(self.d.getVar("BAR"), None) + class TestKeyExpansion(unittest.TestCase): def setUp(self): self.d = bb.data.init() @@ -498,7 +503,7 @@ class TaskHash(unittest.TestCase): d.setVar("VAR", "val") # Adding an inactive removal shouldn't change the hash d.setVar("BAR", "notbar") - d.setVar("MYCOMMAND_remove", "${BAR}") + d.setVar("MYCOMMAND:remove", "${BAR}") nexthash = gettask_bashhash("mytask", d) self.assertEqual(orighash, nexthash) diff --git a/poky/bitbake/lib/bb/tests/parse.py b/poky/bitbake/lib/bb/tests/parse.py index 9e21e1842..4d17f82ed 100644 --- a/poky/bitbake/lib/bb/tests/parse.py +++ b/poky/bitbake/lib/bb/tests/parse.py @@ -98,8 +98,8 @@ exportD = "d" overridetest = """ -RRECOMMENDS_${PN} = "a" -RRECOMMENDS_${PN}_libc = "b" +RRECOMMENDS:${PN} = "a" +RRECOMMENDS:${PN}:libc = "b" OVERRIDES = "libc:${PN}" PN = "gtk+" """ @@ -110,13 +110,13 @@ PN = "gtk+" self.assertEqual(d.getVar("RRECOMMENDS"), "b") bb.data.expandKeys(d) self.assertEqual(d.getVar("RRECOMMENDS"), "b") - d.setVar("RRECOMMENDS_gtk+", "c") + d.setVar("RRECOMMENDS:gtk+", "c") self.assertEqual(d.getVar("RRECOMMENDS"), "c") overridetest2 = """ EXTRA_OECONF = "" -EXTRA_OECONF_class-target = "b" -EXTRA_OECONF_append = " c" +EXTRA_OECONF:class-target = "b" +EXTRA_OECONF:append = " c" """ def test_parse_overrides(self): @@ -128,7 +128,7 @@ EXTRA_OECONF_append = " c" overridetest3 = """ DESCRIPTION = "A" -DESCRIPTION_${PN}-dev = "${DESCRIPTION} B" +DESCRIPTION:${PN}-dev = "${DESCRIPTION} B" PN = "bc" """ @@ -136,15 +136,15 @@ PN = "bc" f = self.parsehelper(self.overridetest3) d = bb.parse.handle(f.name, self.d)[''] bb.data.expandKeys(d) - self.assertEqual(d.getVar("DESCRIPTION_bc-dev"), "A B") + self.assertEqual(d.getVar("DESCRIPTION:bc-dev"), "A B") d.setVar("DESCRIPTION", "E") - d.setVar("DESCRIPTION_bc-dev", "C D") + d.setVar("DESCRIPTION:bc-dev", "C D") d.setVar("OVERRIDES", "bc-dev") self.assertEqual(d.getVar("DESCRIPTION"), "C D") classextend = """ -VAR_var_override1 = "B" +VAR_var:override1 = "B" EXTRA = ":override1" OVERRIDES = "nothing${EXTRA}" diff --git a/poky/bitbake/lib/bb/ui/taskexp.py b/poky/bitbake/lib/bb/ui/taskexp.py index 2b246710c..c00eaf663 100644 --- a/poky/bitbake/lib/bb/ui/taskexp.py +++ b/poky/bitbake/lib/bb/ui/taskexp.py @@ -8,6 +8,7 @@ # import sys +import traceback try: import gi @@ -196,6 +197,7 @@ def main(server, eventHandler, params): gtkgui.start() try: + params.updateToServer(server, os.environ.copy()) params.updateFromServer(server) cmdline = params.parseActions() if not cmdline: @@ -218,6 +220,9 @@ def main(server, eventHandler, params): except client.Fault as x: print("XMLRPC Fault getting commandline:\n %s" % x) return + except Exception as e: + print("Exception in startup:\n %s" % traceback.format_exc()) + return if gtkthread.quit.isSet(): return diff --git a/poky/bitbake/lib/bb/utils.py b/poky/bitbake/lib/bb/utils.py index 6ba1d2a37..e6e82d111 100644 --- a/poky/bitbake/lib/bb/utils.py +++ b/poky/bitbake/lib/bb/utils.py @@ -1178,7 +1178,7 @@ def edit_metadata(meta_lines, variables, varfunc, match_overrides=False): variables: a list of variable names to look for. Functions may also be specified, but must be specified with '()' at the end of the name. Note that the function doesn't have - any intrinsic understanding of _append, _prepend, _remove, + any intrinsic understanding of :append, :prepend, :remove, or overrides, so these are considered as part of the name. These values go into a regular expression, so regular expression syntax is allowed. diff --git a/poky/bitbake/lib/hashserv/tests.py b/poky/bitbake/lib/hashserv/tests.py index e2b762dbf..e851535c5 100644 --- a/poky/bitbake/lib/hashserv/tests.py +++ b/poky/bitbake/lib/hashserv/tests.py @@ -15,28 +15,32 @@ import tempfile import threading import unittest import socket +import time +import signal -def _run_server(server, idx): - # logging.basicConfig(level=logging.DEBUG, filename='bbhashserv.log', filemode='w', - # format='%(levelname)s %(filename)s:%(lineno)d %(message)s') +def server_prefunc(server, idx): + logging.basicConfig(level=logging.DEBUG, filename='bbhashserv.log', filemode='w', + format='%(levelname)s %(filename)s:%(lineno)d %(message)s') + server.logger.debug("Running server %d" % idx) sys.stdout = open('bbhashserv-%d.log' % idx, 'w') sys.stderr = sys.stdout - server.serve_forever() - class HashEquivalenceTestSetup(object): METHOD = 'TestMethod' server_index = 0 - def start_server(self, dbpath=None, upstream=None, read_only=False): + def start_server(self, dbpath=None, upstream=None, read_only=False, prefunc=server_prefunc): self.server_index += 1 if dbpath is None: dbpath = os.path.join(self.temp_dir.name, "db%d.sqlite" % self.server_index) - def cleanup_thread(thread): - thread.terminate() - thread.join() + def cleanup_server(server): + if server.process.exitcode is not None: + return + + server.process.terminate() + server.process.join() server = create_server(self.get_server_addr(self.server_index), dbpath, @@ -44,9 +48,8 @@ class HashEquivalenceTestSetup(object): read_only=read_only) server.dbpath = dbpath - server.thread = multiprocessing.Process(target=_run_server, args=(server, self.server_index)) - server.thread.start() - self.addCleanup(cleanup_thread, server.thread) + server.serve_as_process(prefunc=prefunc, args=(self.server_index,)) + self.addCleanup(cleanup_server, server) def cleanup_client(client): client.close() @@ -283,6 +286,33 @@ class HashEquivalenceCommonTests(object): self.assertClientGetHash(self.client, taskhash2, None) + def test_slow_server_start(self): + """ + Ensures that the server will exit correctly even if it gets a SIGTERM + before entering the main loop + """ + + event = multiprocessing.Event() + + def prefunc(server, idx): + nonlocal event + server_prefunc(server, idx) + event.wait() + + def do_nothing(signum, frame): + pass + + old_signal = signal.signal(signal.SIGTERM, do_nothing) + self.addCleanup(signal.signal, signal.SIGTERM, old_signal) + + _, server = self.start_server(prefunc=prefunc) + server.process.terminate() + time.sleep(30) + event.set() + server.process.join(300) + self.assertIsNotNone(server.process.exitcode, "Server did not exit in a timely manner!") + + class TestHashEquivalenceUnixServer(HashEquivalenceTestSetup, HashEquivalenceCommonTests, unittest.TestCase): def get_server_addr(self, server_idx): return "unix://" + os.path.join(self.temp_dir.name, 'sock%d' % server_idx) diff --git a/poky/bitbake/lib/toaster/orm/models.py b/poky/bitbake/lib/toaster/orm/models.py index 7f7e922ad..4c94b407d 100644 --- a/poky/bitbake/lib/toaster/orm/models.py +++ b/poky/bitbake/lib/toaster/orm/models.py @@ -1719,7 +1719,7 @@ class CustomImageRecipe(Recipe): """Generate the contents for the recipe file.""" # If we have no excluded packages we only need to _append if self.excludes_set.count() == 0: - packages_conf = "IMAGE_INSTALL_append = \" " + packages_conf = "IMAGE_INSTALL:append = \" " for pkg in self.appends_set.all(): packages_conf += pkg.name+' ' diff --git a/poky/bitbake/lib/toaster/toastergui/views.py b/poky/bitbake/lib/toaster/toastergui/views.py index 9a5e48e3b..74f9d5696 100644 --- a/poky/bitbake/lib/toaster/toastergui/views.py +++ b/poky/bitbake/lib/toaster/toastergui/views.py @@ -1708,7 +1708,7 @@ if True: except ProjectVariable.DoesNotExist: pass try: - return_data['image_install_append'] = ProjectVariable.objects.get(project = prj, name = "IMAGE_INSTALL_append").value, + return_data['image_install:append'] = ProjectVariable.objects.get(project = prj, name = "IMAGE_INSTALL:append").value, except ProjectVariable.DoesNotExist: pass try: @@ -1839,7 +1839,7 @@ if True: except ProjectVariable.DoesNotExist: pass try: - context['image_install_append'] = ProjectVariable.objects.get(project = prj, name = "IMAGE_INSTALL_append").value + context['image_install:append'] = ProjectVariable.objects.get(project = prj, name = "IMAGE_INSTALL:append").value context['image_install_append_defined'] = "1" except ProjectVariable.DoesNotExist: pass diff --git a/poky/bitbake/lib/toaster/toastermain/management/commands/buildimport.py b/poky/bitbake/lib/toaster/toastermain/management/commands/buildimport.py index 59da6ff7a..e25b55e5a 100644 --- a/poky/bitbake/lib/toaster/toastermain/management/commands/buildimport.py +++ b/poky/bitbake/lib/toaster/toastermain/management/commands/buildimport.py @@ -451,7 +451,7 @@ class Command(BaseCommand): # Catch vars relevant to Toaster (in case no Toaster section) self.update_project_vars(project,'DISTRO') self.update_project_vars(project,'MACHINE') - self.update_project_vars(project,'IMAGE_INSTALL_append') + self.update_project_vars(project,'IMAGE_INSTALL:append') self.update_project_vars(project,'IMAGE_FSTYPES') self.update_project_vars(project,'PACKAGE_CLASSES') # These vars are typically only assigned by Toaster |