diff options
author | Andrew Geissler <geissonator@yahoo.com> | 2022-06-14 14:47:25 +0300 |
---|---|---|
committer | Andrew Geissler <andrew@geissonator.com> | 2022-06-17 20:13:53 +0300 |
commit | 78b727985e7571e0b196561e44427690f04d57d9 (patch) | |
tree | 6342b2dc624db0c1f7dde6e6f2a2a4d327405b83 /poky/bitbake | |
parent | 9036b819caacfa5ad9abd327b6127f0a19132eac (diff) | |
download | openbmc-78b727985e7571e0b196561e44427690f04d57d9.tar.xz |
subtree updates
meta-openembedded: a9e6d16e66..11df15765c:
Adrian Fiergolski (1):
python3-matplotlib: add missing dependency
Adrian Freihofer (6):
conntrack-tools: fix postinst script
networkmanager: improve dependency handling
networkmanager: simplify selective installation
networkmanager: use nftables by default
networkmanager: udpate to 1.38.0
modemmanager: update to 1.18.8
Armin Kuster (2):
mariadb: update to 10.7.4
mariadb: Fix i386 Clang builds
Bartosz Golaszewski (2):
python3-uinput: new package
python3-speedtest-cli: fix RDEPENDS
Changqing Li (1):
redis: upgrade 7.0-rc3 -> 7.0.0
Denys Dmytriyenko (1):
devmem2: the source and patches moved to github repo
Enrico Scholz (1):
nodejs-oe-cache-native: initial checkin
Jiaqing Zhao (1):
openldap: Remove unnecessary use-urandom.patch
Kai Kang (2):
libportal: add distro features check
graphviz: rrecommends on liberation-fonts
Khem Raj (5):
ubi-utils-klibc: Disable lzo compression by default
unattended-upgrades: Disable auto-detecting modules
sdbus-c++: Link with libatomic for rv32
sdbus-c++-libsystemd: Fix patch fuzz
python3-uinput: Fix build on 32bit arches using 64bit times_t
Luca Boccassi (1):
dbus-broker: update 29 -> 31
Marcel Ziswiler (1):
libavtp: add recipe for audio video transport protocol (avtp)
Markus Volk (6):
jack: allow to build native/nativesdk
pipewire: reduce native/nativesdk dependencies; add backport patch
p8platform: unbreak do_populate_sdk
pavucontrol: update; fix build for wayland only
gnome-disk-utility: fix build for wayland only
unblock some recipes for wayland
Martin Jansa (1):
mm-common: package the files from ${PN} in ${PN}-dev and use allarch
Ming Liu (1):
bluealsa: uprev to 4.0.0
Nikhil R (1):
duktape: Add ptest
Peter Marko (1):
libgpiod: move test dependencies to ptest package
Ross Burton (8):
python3-cppy: fix inherits and DEPENDS
python3-setuptools-scm-git-archive: add new recipe
python3-traitlets: upgrade to 5.2.1
python3-pathspec: add new recipe
python3-hatchling: add new recipe and build class
python3-editables: add new recipe
python3-setuptools-declarative-requirements: add new recipe
lzop: add (from oe-core)
Samuli Piippo (2):
flite: add recipe
libtomcrypt: add recipe
Thomas Perrot (1):
nbd: update 3.20 -> 3.24
Vyacheslav Yurkov (4):
packagegroup-meta-filesystems: fix build issue
overlayfs-progs: add new recipe
overlayfs-tools: add new recipe
xfstests: add new recipe
Wang Mingyu (38):
babeld: upgrade 1.12 -> 1.12.1
ctags: upgrade 5.9.20220508.0 -> 5.9.20220515.0
libbpf: upgrade 0.7.0 -> 0.8.0
evtest: upgrade 1.34 -> 1.35
nbdkit: upgrade 1.31.5 -> 1.31.7
smarty: upgrade 4.1.0 -> 4.1.1
thingsboard-gateway: upgrade 2.9 -> 3.1
opencl-headers: upgrade 2022.01.04 -> 2022.05.18
python3-robotframework: upgrade 5.0 -> 5.0.1
python3-watchdog: upgrade 2.1.7 -> 2.1.8
python3-web3: upgrade 5.29.0 -> 5.29.1
python3-xmlschema: upgrade 1.10.0 -> 1.11.0
python3-sqlalchemy: upgrade 1.4.35 -> 1.4.36
python3-yappi: upgrade 1.3.3 -> 1.3.5
apitrace: upgrade 11.0 -> 11.1
ctags: upgrade 5.9.20220515.0 -> 5.9.20220529.0
gedit: upgrade 42.0 -> 42.1
hidapi: upgrade 0.11.2 -> 0.12.0
libbytesize: upgrade 2.6 -> 2.7
libdvdread: upgrade 6.1.2 -> 6.1.3
links: upgrade 2.26 -> 2.27
libxmlb: upgrade 0.3.8 -> 0.3.9
ser2net: upgrade 4.3.5 -> 4.3.6
python3-awesomeversion: upgrade 22.5.1 -> 22.5.2
htop: upgrade 3.2.0 -> 3.2.1
hwdata: upgrade 0.359 -> 0.360
libnet-dns-perl: upgrade 1.33 -> 1.34
tinyproxy: upgrade 1.11.0 -> 1.11.1
function2: upgrade 4.2.0 -> 4.2.1
openvpn: upgrade 2.5.6 -> 2.5.7
poppler: upgrade 22.05.0 -> 22.06.0
sshfs-fuse: upgrade 3.7.2 -> 3.7.3
tgt: upgrade 1.0.82 -> 1.0.83
tracker: upgrade 3.3.0 -> 3.3.1
unbound: upgrade 1.15.0 -> 1.16.0
zabbix: upgrade 6.0.4 -> 6.0.5
botan: upgrade 2.19.1 -> 2.19.2
evolution-data-server: upgrade 3.44.1 -> 3.44.2
Wolfgang Meyer (1):
fbida: remove bash from RDEPENDS
Xu Huan (17):
python3-pint: upgrade 0.19.1 -> 0.19.2
python3-pylint: upgrade 2.13.7 -> 2.13.9
python3-redis: upgrade 4.2.2 -> 4.3.1
python3-werkzeug: upgrade 2.1.1 -> 2.1.2
python3-zeroconf: upgrade 0.38.4 -> 0.38.6
python3-sentry-sdk: upgrade 1.5.10 -> 1.5.12
python3-astroid: upgrade 2.11.3 -> 2.11.5
python3-cachetools: upgrade 5.0.0 -> 5.1.0
python3-imageio: upgrade 2.19.1 -> 2.19.2
python3-asyncinotify: upgrade 2.0.2 -> 2.0.3
python3-croniter: upgrade 1.3.4 -> 1.3.5
python3-google-api-core: upgrade 2.7.3 -> 2.8.0
python3-flask-socketio: upgrade 5.1.2 -> 5.2.0
python3-h5py: upgrade 3.6.0 -> 3.7.0
python3-lz4: upgrade 4.0.0 -> 4.0.1
python3-mypy: upgrade 0.950 -> 0.960
python3-pyscaffold: upgrade 4.2.1 -> 4.2.2
zhengrq.fnst (10):
python3-google-api-python-client: upgrade 2.45.0 -> 2.48.0
python3-grpcio-tools: upgrade 1.46.0 -> 1.46.3
python3-openpyxl: upgrade 3.0.9 -> 3.0.10
python3-paramiko: upgrade 2.10.4 -> 2.11.0
python3-humanize: upgrade 4.0.0 -> 4.1.0
python3-pychromecast: upgrade 12.1.1 -> 12.1.2
python3-cachetools: upgrade 5.1.0 -> 5.2.0
python3-google-api-python-client: upgrade 2.48.0 -> 2.49.0
python3-googleapis-common-protos: upgrade 1.56.1 -> 1.56.2
python3-imageio: upgrade 2.19.2 -> 2.19.3
zhengruoqin (6):
python3-bitarray: upgrade 2.5.0 -> 2.5.1
python3-eventlet: upgrade 0.33.0 -> 0.33.1
python3-googleapis-common-protos: upgrade 1.56.0 -> 1.56.1
python3-imageio: upgrade 2.18.0 -> 2.19.1
python3-pyjwt: upgrade 2.3.0 -> 2.4.0
python3-wrapt: upgrade 1.14.0 -> 1.14.1
poky: 13d70e57f8..ee0d001b81:
Alex Stewart (1):
opkg: upgrade to version 0.6.0
Alexander Kanavin (23):
bash: submit patch upstream
valgrind: submit arm patches upstream
apt: fix upstream version check
zip/unzip: mark all submittable patches as Inactive-Upstream
less: mark upstream version as unknown
wayland: exclude pre-releases from version check
mesa-demos: update 8.4.0 -> 8.5.0
seatd: update 0.6.4 -> 0.7.0
systemd: update 250.5 -> 251.2
btrfs-tools: update 5.16.2 -> 5.18
llvm: update 14.0.3 -> 14.0.4
python3-psutil: update 5.9.0 -> 5.9.1
tiff: update 4.3.0 -> 4.4.0
pulseaudio: update 15.0 -> 16.0
alsa-utils-scripts: merge into alsa-utils
alsa-utils: update 1.2.6 -> 1.2.7
ovmf: update 202202 -> 202205
cmake: update 3.23.1 -> 3.23.2
ltp: upgrade 20220121 -> 20220527
perl: update 5.34.1 -> 5.36.0
perl: drop perltoc regeneration
perl: clean prior to build
perl: enable _GNU_SOURCE define via d_gnulibc
Bruce Ashfield (7):
linux-yocto/5.15: bpf: explicitly disable unpriv eBPF by default
linux-yocto/5.15: update to v5.15.43
linux-yocto/5.10: update to v5.10.118
linux-yocto/5.15: Enable MDIO bus config
linux-yocto/5.15: cfg/xen: Move x86 configs to separate file
linux-yocto/5.15: update to v5.15.44
linux-yocto/5.10: update to v5.10.119
Chen Qi (1):
libsdl2: add back xvm and xinerama options
Daiane Angolini (1):
python3-pip: Fix RDEPENDS after the update
Davide Gardenal (2):
efivar: add musl libc compatibility
baremetal-image: fix broken symlink in do_rootfs
Dmitry Baryshkov (2):
go.bbclass: fix path to linker in native Go builds
linux-firmware: add support for building snapshots
Ernst Sjöstrand (2):
cve-check: Add helper for symlink handling
cve-check: Only include installed packages for rootfs manifest
He Zhe (1):
lttng-modules: Fix build failure for 5.10.119+ and 5.15.44+ kernel
Jack Mitchell (1):
meson.bbclass: add cython binary to cross/native toolchain config
Jeremy Puhlman (1):
gcc: depend on zstd-native
Jiaqing Zhao (1):
systemd: Correct 0001-pass-correct-parameters-to-getdents64.patch
Joerg Vehlow (1):
libseccomp: Add missing files for ptests
Jose Quaresma (1):
archiver: use bb.note instead of echo
Kai Kang (1):
xxhash: fix build with gcc 12
Marcel Ziswiler (2):
alsa-plugins: fix libavtp vs. avtp packageconfig
gstreamer1.0-plugins-bad: add libavtp packageconfig
Markus Volk (1):
gcr: build with gtk+3 for wayland
Marta Rybczynska (4):
cve-check: move update_symlinks to a library
cve-check: write empty fragment files in the text mode
cve-check: fix return type in check_cves
cve-update-db-native: make it possible to disable database updates
Martin Jansa (9):
makedevs: Don't use COPYING.patch just to add license file into ${S}
insane.bbclass: make sure to close .patch files
staging.bbclass: process direct dependencies in deterministic order
patch.py: make sure that patches/series file exists before quilt pop
lttng-modules: fix shell syntax
buildhistory.bbclass: fix shell syntax when using dash
rootfs.py: close kernel_abi_ver_file
ltp: use bfd even when gold is used with ld-is-gold
systemd: Fix build without utmp
Michael Opdenacker (1):
migration guides: release notes for 4.0.1
Mikko Rapeli (1):
bitbake: event.py: ignore exceptions from stdout and sterr operations in atexit
Ming Liu (1):
udev-extraconf: let automount base directory configurable
Mingli Yu (4):
perl: Fix build with gcc-12
ccache: Fix build with gcc-12
oescripts: change compare logic in OEListPackageconfigTests
python3-cryptography: remove test_x509.py
Naveen Saini (1):
pciutils: avoid lspci conflict with busybox
Pavel Zhukov (6):
bitbake.conf: Make TCLIBC and TCMODE lazy assigned
bitbake: fetch2: Honour BB_FETCH_PREMIRRORONLY option
bitbake: Add tests to cover BB_FETCH_PREMIRRORONLY functionality
dbus: Specify runstatedir configure option
bitbake: tests/fetch: Drop unnecessary duplicated function
bitbake: tests/fetch: Add tests for premirror using real project
Peter Kjellerstedt (2):
libseccomp: Correct LIC_FILES_CHKSUM
license.bbclass: Bound beginline and endline in copy_license_files()
Quentin Schulz (2):
docs: set_versions.py: remove honister from active releases list
docs: set_versions.py: check for first latest release tag
Rasmus Villemoes (2):
vim: put xxd in its own package
e2fsprogs: add alternatives handling of lsattr as well
Ricardo Salveti (1):
gnu-efi: enable for riscv64
Richard Purdie (51):
cve-extra-exclusions: Add kernel CVEs
lzo: Add further info to a patch and mark as Inactive-Upstream
python3: Remove problematic paths from sysroot files
python3: Ensure stale empty python module directories don't break the build
Revert "qemu.inc: Remove empty egg-info directories before running meson"
Revert "meson.bblcass: Remove empty egg-info directories before running meson"
vim: Upgrade 8.2.4912 -> 8.2.5034 to fix 9 CVEs
tiff: Add jbig PACKAGECONFIG and clarify CVE-2022-1210
libxslt: Mark CVE-2022-29824 as not applying
oeqa/imagefeatures: Replace lzo with zst
oeqa/imagefeatures: Disable squashfs-lzo
cve-check: Allow warnings to be disabled
openssl: Backport fix for ptest cert expiry
bitbake: runqueue: Fix unihash cache mismatch issues
bitbake: cache/siggen: Add unihash cache copy function
bitbake: bitbake: Bump to version 2.0.1
populate_sdk_ext: Fix race condition on bb_unihashes.dat
gcc-cross-canadian: Add nativesdk-zstd dependency
glib-2.0: upgrade 2.72.1 -> 2.72.2
dnf: upgrade 4.12.0 -> 4.13.0
python3-dtschema: upgrade 2022.4 -> 2022.5
python3-sphinx: upgrade 4.5.0 -> 5.0.0
python3-pip: upgrade 22.1.1 -> 22.1.2
alsa-lib: upgrade 1.2.6.1 -> 1.2.7
sysklogd: upgrade 2.3.0 -> 2.4.0
libxkbcommon: upgrade 1.4.0 -> 1.4.1
piglit: upgrade to latest revision
sysstat: upgrade 12.4.5 -> 12.6.0
harfbuzz: upgrade 4.2.1 -> 4.3.0
gtk+3: upgrade 3.24.33 -> 3.24.34
xwayland: upgrade 22.1.1 -> 22.1.2
alsa-ucm-conf: upgrade 1.2.6.3 -> 1.2.7
gnutls: upgrade 3.7.5 -> 3.7.6
webkitgtk: upgrade 2.36.1 -> 2.36.3
diffoscope: upgrade 212 -> 215
populate_sdk_ext: Fix second bb_unihashes reference
sanity: Switch to make 4.0 as a minimum version
perl: Add dependency on make-native to avoid race issues
glibc: Drop make-native dependency
bitbake: fetch/wget: Move files into place atomically
bitbake: server/process: Avoid risk of exception deadlocks
bitbake: server/process: Remove daemonic thread usage
bitbake: server/process: Avoid tracebacks at exit
uboot-sign: Fix potential index error issues
selftest/multiconfig: Test that multiconfigs in separate layers works
bitbake: cooker: Drop sre_constants usage
classes/buildcfg: Move git/layer revision code into new OE module buildcfg
lib/buildcfg: Share common clean/dirty layer function
buildcfg: Drop unused svn revision function
base/buildhistory/image-buildinfo: Use common buildcfg function
image-buildinfo: Improve and extend to SDK coverage too
Robert Yang (1):
systemd: Set RebootWatchdogSec to 60s as watchdog
Ross Burton (8):
python3-pluggy: add BBCLASSEXTEND for native/nativesdk
btrfs-tools: add a PACKAGECONFIG for lzo
tiff: mark CVE-2022-1622 and CVE-2022-1623 as invalid
packagegroup-self-hosted: remove lzo
libarchive: disable LZO by default
squashfs-tools: disable LZO by default
lzop: remove recipe from oe-core
setuptools3: clean up class
Rusty Howell (1):
oe-depends-dot: Handle new format for task-depends.dot
Sean Anderson (1):
rootfs.py: find .ko.zst kernel modules
Stefan Wiehler (1):
kernel-yocto.bbclass: Reset to exiting on non-zero return code at end of task
Tobias Schmidl (2):
oeqa/selftest/wic.py: Repaired test_qemu()
wic/plugins/images/direct: Allow changes in fstab on rootfs
Vyacheslav Yurkov (2):
files: rootfs-postcommands: move helper commands to script
files: respect overlayfs owner from lower layer
Xiaobing Luo (1):
devtool: Fix _copy_file() TypeError
Zach Welch (2):
test-manual/intro: reorder bitbake-selftest steps
test-manual/intro: bitbake-selftest needs bitbake
leimaohui (1):
gnutls: Added fips option.
wangmy (30):
bind: upgrade 9.18.2 -> 9.18.3
ccache: upgrade 4.6 -> 4.6.1
init-system-helpers: upgrade 1.62 -> 1.63
ninja: upgrade 1.10.2 -> 1.11.0
python3-certifi: upgrade 2021.10.8 -> 2022.5.18.1
python3-cython: upgrade 0.29.28 -> 0.29.30
python3-hypothesis: upgrade 6.46.4 -> 6.46.7
python3-importlib-metadata: upgrade 4.11.3 -> 4.11.4
python3-magic: upgrade 0.4.25 -> 0.4.26
python3-pip: upgrade 22.1 -> 22.1.1
python3-setuptools: upgrade 62.3.1 -> 62.3.2
python3-hypothesis: upgrade 6.46.7 -> 6.46.9
python3-semantic-version: upgrade 2.9.0 -> 2.10.0
python3-webcolors: upgrade 1.11.1 -> 1.12
python3-pytest-subtests: upgrade 0.7.0 -> 0.8.0
asciidoc: upgrade 10.1.4 -> 10.2.0
cups: upgrade 2.4.1 -> 2.4.2
iproute2: upgrade 5.17.0 -> 5.18.0
iw: upgrade 5.16 -> 5.19
logrotate: upgrade 3.19.0 -> 3.20.1
dpkg: upgrade 1.21.7 -> 1.21.8
repo: upgrade 2.25 -> 2.26
iso-codes: upgrade 4.9.0 -> 4.10.0
lttng-ust: upgrade 2.13.2 -> 2.13.3
meson: upgrade 0.62.1 -> 0.62.2
mtools: upgrade 4.0.39 -> 4.0.40
nettle: upgrade 3.7.3 -> 3.8
kbd: upgrade 2.4.0 -> 2.5.0
python3-hypothesis: upgrade 6.46.9 -> 6.46.11
xkeyboard-config: upgrade 2.35.1 -> 2.36
meta-security: 7628a3e90b..8c6fe006a1:
Armin Kuster (18):
swtpm: enable seccomp if DISTRO is enabled
security-tpm2-image: add swtpm
swtpm: enable gnutls
oeqa/swtpm: add swtpm runtime
oeqa/tpm2: fix and cleanup tests
tpm2-pkcs11: we really need the symlinks
smack-test: switch to python3
oeqa/smack: consolidate classes
checksec: update 2.6.0
chkrootkit: update SRC_URI
packagegroup-core-security: add arpwatch and chkrootkit to pkg grp
layer.conf: Post release codename changes
README: Update for dynamic layers
arpwatch: riscv not supported
packagegroup-core-security: drop arpwatch for riscv from pkg grp
chkrootkit: Fix missing includes for musl
arpwatch: update to 3.3
packagegroup-core-security: don't include aprwatch for musl
Signed-off-by: Andrew Geissler <geissonator@yahoo.com>
Change-Id: Ic83db16445cf0a1286685f11d378e1e3e9b794c3
Diffstat (limited to 'poky/bitbake')
-rwxr-xr-x | poky/bitbake/bin/bitbake | 2 | ||||
-rw-r--r-- | poky/bitbake/lib/bb/__init__.py | 2 | ||||
-rw-r--r-- | poky/bitbake/lib/bb/cache.py | 9 | ||||
-rw-r--r-- | poky/bitbake/lib/bb/cooker.py | 5 | ||||
-rw-r--r-- | poky/bitbake/lib/bb/event.py | 10 | ||||
-rw-r--r-- | poky/bitbake/lib/bb/fetch2/git.py | 13 | ||||
-rw-r--r-- | poky/bitbake/lib/bb/fetch2/wget.py | 11 | ||||
-rw-r--r-- | poky/bitbake/lib/bb/runqueue.py | 3 | ||||
-rw-r--r-- | poky/bitbake/lib/bb/server/process.py | 37 | ||||
-rw-r--r-- | poky/bitbake/lib/bb/siggen.py | 6 | ||||
-rw-r--r-- | poky/bitbake/lib/bb/tests/fetch.py | 93 |
11 files changed, 157 insertions, 34 deletions
diff --git a/poky/bitbake/bin/bitbake b/poky/bitbake/bin/bitbake index 042c91807d..b56f6207c6 100755 --- a/poky/bitbake/bin/bitbake +++ b/poky/bitbake/bin/bitbake @@ -28,7 +28,7 @@ from bb.main import bitbake_main, BitBakeConfigParameters, BBMainException if sys.getfilesystemencoding() != "utf-8": sys.exit("Please use a locale setting which supports UTF-8 (such as LANG=en_US.UTF-8).\nPython can't change the filesystem locale after loading so we need a UTF-8 when Python starts or things won't work.") -__version__ = "2.0.0" +__version__ = "2.0.1" if __name__ == "__main__": if __version__ != bb.__version__: diff --git a/poky/bitbake/lib/bb/__init__.py b/poky/bitbake/lib/bb/__init__.py index b8333bdb81..6b470aa195 100644 --- a/poky/bitbake/lib/bb/__init__.py +++ b/poky/bitbake/lib/bb/__init__.py @@ -9,7 +9,7 @@ # SPDX-License-Identifier: GPL-2.0-only # -__version__ = "2.0.0" +__version__ = "2.0.1" import sys if sys.version_info < (3, 6, 0): diff --git a/poky/bitbake/lib/bb/cache.py b/poky/bitbake/lib/bb/cache.py index 92e9a3ced7..988c596c39 100644 --- a/poky/bitbake/lib/bb/cache.py +++ b/poky/bitbake/lib/bb/cache.py @@ -24,6 +24,7 @@ from collections.abc import Mapping import bb.utils from bb import PrefixLoggerAdapter import re +import shutil logger = logging.getLogger("BitBake.Cache") @@ -998,3 +999,11 @@ class SimpleCache(object): p.dump([data, self.cacheversion]) bb.utils.unlockfile(glf) + + def copyfile(self, target): + if not self.cachefile: + return + + glf = bb.utils.lockfile(self.cachefile + ".lock") + shutil.copy(self.cachefile, target) + bb.utils.unlockfile(glf) diff --git a/poky/bitbake/lib/bb/cooker.py b/poky/bitbake/lib/bb/cooker.py index 6da9291f9c..2adf4d297d 100644 --- a/poky/bitbake/lib/bb/cooker.py +++ b/poky/bitbake/lib/bb/cooker.py @@ -13,7 +13,6 @@ import sys, os, glob, os.path, re, time import itertools import logging import multiprocessing -import sre_constants import threading from io import StringIO, UnsupportedOperation from contextlib import closing @@ -1907,7 +1906,7 @@ class CookerCollectFiles(object): try: re.compile(mask) bbmasks.append(mask) - except sre_constants.error: + except re.error: collectlog.critical("BBMASK contains an invalid regular expression, ignoring: %s" % mask) # Then validate the combined regular expressions. This should never @@ -1915,7 +1914,7 @@ class CookerCollectFiles(object): bbmask = "|".join(bbmasks) try: bbmask_compiled = re.compile(bbmask) - except sre_constants.error: + except re.error: collectlog.critical("BBMASK is not a valid regular expression, ignoring: %s" % bbmask) bbmask = None diff --git a/poky/bitbake/lib/bb/event.py b/poky/bitbake/lib/bb/event.py index df020551e3..97668601a1 100644 --- a/poky/bitbake/lib/bb/event.py +++ b/poky/bitbake/lib/bb/event.py @@ -132,8 +132,14 @@ def print_ui_queue(): if not _uiready: from bb.msg import BBLogFormatter # Flush any existing buffered content - sys.stdout.flush() - sys.stderr.flush() + try: + sys.stdout.flush() + except: + pass + try: + sys.stderr.flush() + except: + pass stdout = logging.StreamHandler(sys.stdout) stderr = logging.StreamHandler(sys.stderr) formatter = BBLogFormatter("%(levelname)s: %(message)s") diff --git a/poky/bitbake/lib/bb/fetch2/git.py b/poky/bitbake/lib/bb/fetch2/git.py index 23f8c0da8f..07b3d9a7ff 100644 --- a/poky/bitbake/lib/bb/fetch2/git.py +++ b/poky/bitbake/lib/bb/fetch2/git.py @@ -353,10 +353,15 @@ class Git(FetchMethod): if ud.shallow and os.path.exists(ud.fullshallow) and self.need_update(ud, d): ud.localpath = ud.fullshallow return - elif os.path.exists(ud.fullmirror) and not os.path.exists(ud.clonedir): - bb.utils.mkdirhier(ud.clonedir) - runfetchcmd("tar -xzf %s" % ud.fullmirror, d, workdir=ud.clonedir) - + elif os.path.exists(ud.fullmirror) and self.need_update(ud, d): + if not os.path.exists(ud.clonedir): + bb.utils.mkdirhier(ud.clonedir) + runfetchcmd("tar -xzf %s" % ud.fullmirror, d, workdir=ud.clonedir) + else: + tmpdir = tempfile.mkdtemp(dir=d.getVar('DL_DIR')) + runfetchcmd("tar -xzf %s" % ud.fullmirror, d, workdir=tmpdir) + fetch_cmd = "LANG=C %s fetch -f --progress %s " % (ud.basecmd, shlex.quote(tmpdir)) + runfetchcmd(fetch_cmd, d, workdir=ud.clonedir) repourl = self._get_repo_url(ud) # If the repo still doesn't exist, fallback to cloning it diff --git a/poky/bitbake/lib/bb/fetch2/wget.py b/poky/bitbake/lib/bb/fetch2/wget.py index b3a3de571a..b2b542e1dc 100644 --- a/poky/bitbake/lib/bb/fetch2/wget.py +++ b/poky/bitbake/lib/bb/fetch2/wget.py @@ -106,10 +106,9 @@ class Wget(FetchMethod): fetchcmd = self.basecmd - if 'downloadfilename' in ud.parm: - localpath = os.path.join(d.getVar("DL_DIR"), ud.localfile) - bb.utils.mkdirhier(os.path.dirname(localpath)) - fetchcmd += " -O %s" % shlex.quote(localpath) + localpath = os.path.join(d.getVar("DL_DIR"), ud.localfile) + ".tmp" + bb.utils.mkdirhier(os.path.dirname(localpath)) + fetchcmd += " -O %s" % shlex.quote(localpath) if ud.user and ud.pswd: fetchcmd += " --auth-no-challenge" @@ -133,6 +132,10 @@ class Wget(FetchMethod): self._runwget(ud, d, fetchcmd, False) + # Remove the ".tmp" and move the file into position atomically + # Our lock prevents multiple writers but mirroring code may grab incomplete files + os.rename(localpath, localpath[:-4]) + # Sanity check since wget can pretend it succeed when it didn't # Also, this used to happen if sourceforge sent us to the mirror page if not os.path.exists(ud.localpath): diff --git a/poky/bitbake/lib/bb/runqueue.py b/poky/bitbake/lib/bb/runqueue.py index f34f1568e2..1e47fe70ef 100644 --- a/poky/bitbake/lib/bb/runqueue.py +++ b/poky/bitbake/lib/bb/runqueue.py @@ -2299,6 +2299,9 @@ class RunQueueExecute: self.rqdata.runtaskentries[hashtid].unihash = unihash bb.parse.siggen.set_unihash(hashtid, unihash) toprocess.add(hashtid) + if torehash: + # Need to save after set_unihash above + bb.parse.siggen.save_unitaskhashes() # Work out all tasks which depend upon these total = set() diff --git a/poky/bitbake/lib/bb/server/process.py b/poky/bitbake/lib/bb/server/process.py index 613956f30f..43790b6631 100644 --- a/poky/bitbake/lib/bb/server/process.py +++ b/poky/bitbake/lib/bb/server/process.py @@ -437,6 +437,7 @@ class BitBakeProcessServerConnection(object): self.socket_connection = sock def terminate(self): + self.events.close() self.socket_connection.close() self.connection.connection.close() self.connection.recv.close() @@ -662,23 +663,18 @@ class BBUIEventQueue: self.reader = ConnectionReader(readfd) self.t = threading.Thread() - self.t.daemon = True self.t.run = self.startCallbackHandler self.t.start() def getEvent(self): - self.eventQueueLock.acquire() - - if len(self.eventQueue) == 0: - self.eventQueueLock.release() - return None - - item = self.eventQueue.pop(0) + with self.eventQueueLock: + if len(self.eventQueue) == 0: + return None - if len(self.eventQueue) == 0: - self.eventQueueNotify.clear() + item = self.eventQueue.pop(0) + if len(self.eventQueue) == 0: + self.eventQueueNotify.clear() - self.eventQueueLock.release() return item def waitEvent(self, delay): @@ -686,10 +682,9 @@ class BBUIEventQueue: return self.getEvent() def queue_event(self, event): - self.eventQueueLock.acquire() - self.eventQueue.append(event) - self.eventQueueNotify.set() - self.eventQueueLock.release() + with self.eventQueueLock: + self.eventQueue.append(event) + self.eventQueueNotify.set() def send_event(self, event): self.queue_event(pickle.loads(event)) @@ -698,13 +693,17 @@ class BBUIEventQueue: bb.utils.set_process_name("UIEventQueue") while True: try: - self.reader.wait() - event = self.reader.get() - self.queue_event(event) - except EOFError: + ready = self.reader.wait(0.25) + if ready: + event = self.reader.get() + self.queue_event(event) + except (EOFError, OSError, TypeError): # Easiest way to exit is to close the file descriptor to cause an exit break + + def close(self): self.reader.close() + self.t.join() class ConnectionReader(object): diff --git a/poky/bitbake/lib/bb/siggen.py b/poky/bitbake/lib/bb/siggen.py index 08eca7860e..3f3d6df54d 100644 --- a/poky/bitbake/lib/bb/siggen.py +++ b/poky/bitbake/lib/bb/siggen.py @@ -120,6 +120,9 @@ class SignatureGenerator(object): def save_unitaskhashes(self): return + def copy_unitaskhashes(self, targetdir): + return + def set_setscene_tasks(self, setscene_tasks): return @@ -358,6 +361,9 @@ class SignatureGeneratorBasic(SignatureGenerator): def save_unitaskhashes(self): self.unihash_cache.save(self.unitaskhashes) + def copy_unitaskhashes(self, targetdir): + self.unihash_cache.copyfile(targetdir) + def dump_sigtask(self, fn, task, stampbase, runtime): tid = fn + ":" + task diff --git a/poky/bitbake/lib/bb/tests/fetch.py b/poky/bitbake/lib/bb/tests/fetch.py index 1152e89c0d..622c46a05d 100644 --- a/poky/bitbake/lib/bb/tests/fetch.py +++ b/poky/bitbake/lib/bb/tests/fetch.py @@ -2802,3 +2802,96 @@ class GitSharedTest(FetcherTest): fetcher.unpack(self.unpackdir) alt = os.path.join(self.unpackdir, 'git/.git/objects/info/alternates') self.assertFalse(os.path.exists(alt)) + + +class FetchPremirroronlyLocalTest(FetcherTest): + + def setUp(self): + super(FetchPremirroronlyLocalTest, self).setUp() + self.mirrordir = os.path.join(self.tempdir, "mirrors") + os.mkdir(self.mirrordir) + self.reponame = "bitbake" + self.gitdir = os.path.join(self.tempdir, "git", self.reponame) + self.recipe_url = "git://git.fake.repo/bitbake" + self.d.setVar("BB_FETCH_PREMIRRORONLY", "1") + self.d.setVar("BB_NO_NETWORK", "1") + self.d.setVar("PREMIRRORS", self.recipe_url + " " + "file://{}".format(self.mirrordir) + " \n") + + def make_git_repo(self): + self.mirrorname = "git2_git.fake.repo.bitbake.tar.gz" + recipeurl = "git:/git.fake.repo/bitbake" + os.makedirs(self.gitdir) + self.git("init", self.gitdir) + for i in range(0): + self.git_new_commit() + bb.process.run('tar -czvf {} .'.format(os.path.join(self.mirrordir, self.mirrorname)), cwd = self.gitdir) + + def git_new_commit(self): + import random + testfilename = "bibake-fetch.test" + os.unlink(os.path.join(self.mirrordir, self.mirrorname)) + with open(os.path.join(self.gitdir, testfilename), "w") as testfile: + testfile.write("Useless random data {}".format(random.random())) + self.git("add {}".format(testfilename), self.gitdir) + self.git("commit -a -m \"This random commit {}. I'm useless.\"".format(random.random()), self.gitdir) + bb.process.run('tar -czvf {} .'.format(os.path.join(self.mirrordir, self.mirrorname)), cwd = self.gitdir) + return self.git("rev-parse HEAD", self.gitdir).strip() + + def test_mirror_commit_nonexistent(self): + self.make_git_repo() + self.d.setVar("SRCREV", "0"*40) + fetcher = bb.fetch.Fetch([self.recipe_url], self.d) + with self.assertRaises(bb.fetch2.NetworkAccess): + fetcher.download() + + def test_mirror_commit_exists(self): + self.make_git_repo() + self.d.setVar("SRCREV", self.git_new_commit()) + fetcher = bb.fetch.Fetch([self.recipe_url], self.d) + fetcher.download() + fetcher.unpack(self.unpackdir) + + def test_mirror_tarball_nonexistent(self): + self.d.setVar("SRCREV", "0"*40) + fetcher = bb.fetch.Fetch([self.recipe_url], self.d) + with self.assertRaises(bb.fetch2.NetworkAccess): + fetcher.download() + +class FetchPremirroronlyNetworkTest(FetcherTest): + + def setUp(self): + super(FetchPremirroronlyNetworkTest, self).setUp() + self.mirrordir = os.path.join(self.tempdir, "mirrors") + os.mkdir(self.mirrordir) + self.reponame = "fstests" + self.clonedir = os.path.join(self.tempdir, "git") + self.gitdir = os.path.join(self.tempdir, "git", "{}.git".format(self.reponame)) + self.recipe_url = "git://git.yoctoproject.org/fstests" + self.d.setVar("BB_FETCH_PREMIRRORONLY", "1") + self.d.setVar("BB_NO_NETWORK", "0") + self.d.setVar("PREMIRRORS", self.recipe_url + " " + "file://{}".format(self.mirrordir) + " \n") + + def make_git_repo(self): + import shutil + self.mirrorname = "git2_git.yoctoproject.org.fstests.tar.gz" + os.makedirs(self.clonedir) + self.git("clone --bare --shallow-since=\"01.01.2013\" {}".format(self.recipe_url), self.clonedir) + bb.process.run('tar -czvf {} .'.format(os.path.join(self.mirrordir, self.mirrorname)), cwd = self.gitdir) + shutil.rmtree(self.clonedir) + + @skipIfNoNetwork() + def test_mirror_tarball_updated(self): + self.make_git_repo() + ## Upstream commit is in the mirror + self.d.setVar("SRCREV", "49d65d53c2bf558ae6e9185af0f3af7b79d255ec") + fetcher = bb.fetch.Fetch([self.recipe_url], self.d) + fetcher.download() + + @skipIfNoNetwork() + def test_mirror_tarball_outdated(self): + self.make_git_repo() + ## Upstream commit not in the mirror + self.d.setVar("SRCREV", "15413486df1f5a5b5af699b6f3ba5f0984e52a9f") + fetcher = bb.fetch.Fetch([self.recipe_url], self.d) + with self.assertRaises(bb.fetch2.NetworkAccess): + fetcher.download() |