diff options
author | Andrew Geissler <geissonator@yahoo.com> | 2020-06-27 08:28:28 +0300 |
---|---|---|
committer | Andrew Geissler <geissonator@yahoo.com> | 2020-06-27 08:32:13 +0300 |
commit | d25ed3241ddffad58c7a52e45e388e6c48d5123a (patch) | |
tree | b097477c5b9204689d35c06f5761b1767093b338 /poky/meta/lib/oe | |
parent | c87764fefff10735006a31fab72d76c243a3eb40 (diff) | |
download | openbmc-d25ed3241ddffad58c7a52e45e388e6c48d5123a.tar.xz |
poky: subtree update:26ae42ded7..5951cbcabe
Alex Kiernan (1):
recipetool: Fix list concatenation when using edit
Alexander Kanavin (4):
apr-util: make gdbm optional
gobject-introspection: add a patch to fix a build race
icu: merge .inc into main recipe
icu: make filtered data generation optional, serial and off by default
Alexandru N. Onea (3):
bitbake: perforce: add basic progress handler for perforce
bitbake: perforce: add local path handling SRC_URI options
bitbake: bitbake-user-manual: update perforce fetcher docs
Andreas M?ller (1):
meson.bbclass: avoid unexpected operating-system names
Andreas Müller (6):
boost: Add upstream patch to fix build on depending projects
libinput: upgrade 1.15.5 -> 1.15.6
sqlite3: upgrade 3.32.2 -> 3.32.3
desktop-file-utils: upgrade 0.24 -> 0.26
file: upgrade 5.38 -> 5.39
ffmpeg: upgrade 4.2.3 -> 4.3
Andrej Valek (1):
oeqa/runtime/cases/ptest: Make output content path absolute
Andrew Geissler (1):
meson: backport library ordering fix
Armin Kuster (1):
libuv: move from meta-oe to core for bind update
Arthur She (1):
igt-gpu-tools: add new package
Changqing Li (1):
mime.bbclass: fix post install scriptlet error
Chen Qi (1):
systemd-serialgetty: do not use BindsTo
Daniel McGregor (3):
sign_rpm.bbclass: ignore thread count
systemd-conf: Accept MTU from DHCP
buildhistory-collect-srcrevs: sort directories
He Zhe (1):
ltp: Fix copy_file_rang02 for 32-bit arches
Hongxu Jia (1):
libmodulemd: switch branch master -> main
Jacob Kroon (5):
bitbake: lib/bb/utils.py: Do not preserve TERM in the environment
bitbake: bitbake-user-manual: Remove TERM from BB_HASHBASE_WHITELIST example
bitbake.conf: Remove TERM from default BB_HASHBASE_WHITELIST
grub: Remove native version of grub-efi
distro_alias: Remove unused grub-efi distro aliases
Jens Rehsack (1):
u-boot: avoid blind merging all *.cfg
Joe Slater (1):
systemd: fix CVE-2020-13776
Joshua Watt (5):
sstatesig: Account for all dataCaches being passed
bitbake: bitbake: cache: Fix error message with bad multiconfig
wic: Fix error message when reporting invalid offset
classes/archiver: Create patched archive before configuring
bitbake: cache: Bump cache version
Konrad Weihmann (3):
oeqa/runtime: Add OERequirePackage decorator
bitbake: cookerdata: Add BBFILES_DYNAMIC inverse mode
bitbake: bitbake-user-manual: Add BBFILES_DYNAMIC
Mark Morton (2):
New source files and Makefile update for Test Manual
test-manual: Fixed codeblock formatting
Martin Jansa (1):
net-tools: backport a patch from upstream to use the same ifconfig format as debian/ubuntu
Mingli Yu (3):
python3: add the rdepends for python3-misc
python3: add rdepends for python3-idle
python3-dbusmock: add the missing rdepends
Otavio Salvador (2):
systemd: Sync systemd-serialgetty@.service with upstream
mtd-utils: Fix return value of ubiformat
Ovidiu Panait (2):
dbus-test: Remove EXTRA_OECONF_X configs
dbus,dbus-test: Move common parts to dbus.inc
Paul Barker (2):
bitbake: fetch2/gitsm: Mark srcrev as fetched once all submodules are processed
bitbake: fetch2/gitsm: Make need_update() process submodules
Paul Eggleton (5):
graph-tool: switch to argparse
graph-tool: add filter subcommand
dpkg-native: rebase and reinstate fix for "tar: file changed as we read it"
shadow-sysroot: drop unused SRC_URI checksums
devtool: fix typo
Peter Kjellerstedt (1):
relocatable.bbclass: Avoid an exception if an empty pkgconfig dir exist
Pierre-Jean Texier (3):
diffoscope: upgrade 146 -> 147
ell: upgrade 0.31 -> 0.32
curl: upgrade 7.70.0 -> 7.71.0
Rasmus Villemoes (1):
curl: add debug info
Richard Purdie (15):
buildhistory: Add simplistic file move detection
bitbake: bin/bitbake: Update to next series release version
perl: Fix host specific modules problems
sanity.conf: Require bitbake 1.47.0 as the minimum version
patchelf: Upgrade 0.10 -> 0.11
test-manual: Add SPDX license headers
Makefile: Drop obsolete edison/denzil branch conditionals
bitbake: tests/fetch: Switch from git.infradead.org to a YP mirror
pseudo: Fix attr errors due to incorrect library resolution issues
oeqa/selftest/runcmd: Add better debug for thread count mismatch failures
oeqa/utils/command: Improve stdin handling in runCmd
vulkan-headers: Fix upstream branch deletion issue
recipes: Fix Upstream-Status Accepted -> Backport
scripts/install-buildtools: Update to 3.2 M1 buildtools
scripts/install-buildtools: Handle new format checksum files
Robert P. J. Day (1):
python: use official "pypi.org" URLs for HOMEPAGE
Ross Burton (8):
install-buildtools: fail if an error occurs
install-buildtools: remove hardcoded x86-64 architecture
install-buildtools: add option to disable checksum validation
common-licenses: add BSD-2-Clause-Patent
gstreamer1.0-plugins-bad: add support for vdpau
go-binary-native: add binary Go to bootstrap
tcmode-default: use go-binary-native by default
go-native: merge bb/inc and add comment
Ryan Rowe (1):
python3: fix PGO for non-reproducible biniaries
Sakib Sajal (1):
qemu: uprev v4.2.0 -> v5.0.0
Samuli Piippo (2):
cmake: allow chainloading of the toolchain file
perl: use relative paths in the perl wrapper
Steve Sakoman (1):
buildtools-tarball: export OPENSSL_CONF in environment setup
Tanu Kaskinen (1):
pulseaudio: remove unnecessary libltdl copying
Trevor Gamblin (1):
python3-setuptools: patch entrypoints for faster initialization
Tuomas Salokanto (1):
recipetool: create: fix SRCBRANCH not being passed to params
Valentin Longchamp (2):
tools-profile: disable valgrind for powerpc soft-float
valgrind: disable it for powerpc soft-float
Wang Mingyu (5):
powertop: upgrade 2.12 -> 2.13
man-db: upgrade 2.9.2 -> 2.9.3
valgrind: upgrade 3.16.0 -> 3.16.1
man-pages: upgrade 5.06 -> 5.07
harfbuzz: upgrade 2.6.7 -> 2.6.8
Yi Zhao (2):
iptables: fix invalid symbolic link for ip6tables-apply
iptables: split iptables-apply to its own package
Yongxin Liu (1):
linux-firmware: add ice for Intel E800 series driver
Yuki Hoshino (1):
sysvinit-inittab: Add support for tty devices with 10 or more number.
akuster (9):
bind: update to 9.11.19
adt-manual: Add SPDX license headers
bsp-guide: Add SPDX license headers
brief-yoctoprojectsqa: Add SPDX license headers
dev-manual: Add SPDX License headers
kernel-dev: Add SPDX license headers
profile-manual: Add SPDX licence headers
sdk-manual: Add SPDX license headers
toaster-manaul: Add SPDX license headers
haiqing (1):
libpam: Remove option 'obscure' from common-password
hongxu (1):
kmod: add nativesdk support
zangrc (1):
ethtool:upgrade 5.6 -> 5.7
Signed-off-by: Andrew Geissler <geissonator@yahoo.com>
Change-Id: I1190ca17297b1167286cfc06033e8485396c7cce
Diffstat (limited to 'poky/meta/lib/oe')
-rw-r--r-- | poky/meta/lib/oe/buildhistory_analysis.py | 65 | ||||
-rw-r--r-- | poky/meta/lib/oe/sstatesig.py | 79 |
2 files changed, 104 insertions, 40 deletions
diff --git a/poky/meta/lib/oe/buildhistory_analysis.py b/poky/meta/lib/oe/buildhistory_analysis.py index 5b28774c9..2d6fa1779 100644 --- a/poky/meta/lib/oe/buildhistory_analysis.py +++ b/poky/meta/lib/oe/buildhistory_analysis.py @@ -213,6 +213,7 @@ class FileChange: changetype_perms = 'P' changetype_ownergroup = 'O' changetype_link = 'L' + changetype_move = 'M' def __init__(self, path, changetype, oldvalue = None, newvalue = None): self.path = path @@ -251,10 +252,11 @@ class FileChange: return '%s changed owner/group from %s to %s' % (self.path, self.oldvalue, self.newvalue) elif self.changetype == self.changetype_link: return '%s changed symlink target from %s to %s' % (self.path, self.oldvalue, self.newvalue) + elif self.changetype == self.changetype_move: + return '%s moved to %s' % (self.path, self.oldvalue) else: return '%s changed (unknown)' % self.path - def blob_to_dict(blob): alines = [line for line in blob.data_stream.read().decode('utf-8').splitlines()] adict = {} @@ -281,11 +283,14 @@ def file_list_to_dict(lines): adict[path] = splitv[0:3] return adict +numeric_removal = str.maketrans('0123456789', 'XXXXXXXXXX') def compare_file_lists(alines, blines, compare_ownership=True): adict = file_list_to_dict(alines) bdict = file_list_to_dict(blines) filechanges = [] + additions = [] + removals = [] for path, splitv in adict.items(): newsplitv = bdict.pop(path, None) if newsplitv: @@ -318,11 +323,65 @@ def compare_file_lists(alines, blines, compare_ownership=True): if oldvalue != newvalue: filechanges.append(FileChange(path, FileChange.changetype_link, oldvalue, newvalue)) else: - filechanges.append(FileChange(path, FileChange.changetype_remove)) + removals.append(path) # Whatever is left over has been added for path in bdict: - filechanges.append(FileChange(path, FileChange.changetype_add)) + additions.append(path) + + # Rather than print additions and removals, its nicer to print file 'moves' + # where names or paths are similar. + revmap_remove = {} + for removal in removals: + translated = removal.translate(numeric_removal) + if translated not in revmap_remove: + revmap_remove[translated] = [] + revmap_remove[translated].append(removal) + + # + # We want to detect renames of large trees of files like + # /lib/modules/5.4.40-yocto-standard to /lib/modules/5.4.43-yocto-standard + # + renames = {} + for addition in additions.copy(): + if addition not in additions: + continue + translated = addition.translate(numeric_removal) + if translated in revmap_remove: + if len(revmap_remove[translated]) != 1: + continue + removal = revmap_remove[translated][0] + commondir = addition.split("/") + commondir2 = removal.split("/") + idx = None + for i in range(len(commondir)): + if commondir[i] != commondir2[i]: + idx = i + break + commondir = "/".join(commondir[:i+1]) + commondir2 = "/".join(commondir2[:i+1]) + # If the common parent is in one dict and not the other its likely a rename + # so iterate through those files and process as such + if commondir2 not in bdict and commondir not in adict: + if commondir not in renames: + renames[commondir] = commondir2 + for addition2 in additions.copy(): + if addition2.startswith(commondir): + removal2 = addition2.replace(commondir, commondir2) + if removal2 in removals: + additions.remove(addition2) + removals.remove(removal2) + continue + filechanges.append(FileChange(removal, FileChange.changetype_move, addition)) + additions.remove(addition) + removals.remove(removal) + for rename in renames: + filechanges.append(FileChange(renames[rename], FileChange.changetype_move, rename)) + + for addition in additions: + filechanges.append(FileChange(addition, FileChange.changetype_add)) + for removal in removals: + filechanges.append(FileChange(removal, FileChange.changetype_remove)) return filechanges diff --git a/poky/meta/lib/oe/sstatesig.py b/poky/meta/lib/oe/sstatesig.py index d24e3738a..21ae0a765 100644 --- a/poky/meta/lib/oe/sstatesig.py +++ b/poky/meta/lib/oe/sstatesig.py @@ -2,9 +2,10 @@ # SPDX-License-Identifier: GPL-2.0-only # import bb.siggen +import bb.runqueue import oe -def sstate_rundepfilter(siggen, fn, recipename, task, dep, depname, dataCache): +def sstate_rundepfilter(siggen, fn, recipename, task, dep, depname, dataCaches): # Return True if we should keep the dependency, False to drop it def isNative(x): return x.endswith("-native") @@ -12,23 +13,26 @@ def sstate_rundepfilter(siggen, fn, recipename, task, dep, depname, dataCache): return "-cross-" in x def isNativeSDK(x): return x.startswith("nativesdk-") - def isKernel(fn): - inherits = " ".join(dataCache.inherits[fn]) + def isKernel(mc, fn): + inherits = " ".join(dataCaches[mc].inherits[fn]) return inherits.find("/module-base.bbclass") != -1 or inherits.find("/linux-kernel-base.bbclass") != -1 - def isPackageGroup(fn): - inherits = " ".join(dataCache.inherits[fn]) + def isPackageGroup(mc, fn): + inherits = " ".join(dataCaches[mc].inherits[fn]) return "/packagegroup.bbclass" in inherits - def isAllArch(fn): - inherits = " ".join(dataCache.inherits[fn]) + def isAllArch(mc, fn): + inherits = " ".join(dataCaches[mc].inherits[fn]) return "/allarch.bbclass" in inherits - def isImage(fn): - return "/image.bbclass" in " ".join(dataCache.inherits[fn]) - - # (Almost) always include our own inter-task dependencies. - # The exception is the special do_kernel_configme->do_unpack_and_patch - # dependency from archiver.bbclass. - if recipename == depname: - if task == "do_kernel_configme" and dep.endswith(".do_unpack_and_patch"): + def isImage(mc, fn): + return "/image.bbclass" in " ".join(dataCaches[mc].inherits[fn]) + + depmc, _, deptaskname, depmcfn = bb.runqueue.split_tid_mcfn(dep) + mc, _ = bb.runqueue.split_mc(fn) + + # (Almost) always include our own inter-task dependencies (unless it comes + # from a mcdepends). The exception is the special + # do_kernel_configme->do_unpack_and_patch dependency from archiver.bbclass. + if recipename == depname and depmc == mc: + if task == "do_kernel_configme" and deptaskname == "do_unpack_and_patch": return False return True @@ -47,11 +51,11 @@ def sstate_rundepfilter(siggen, fn, recipename, task, dep, depname, dataCache): # Only target packages beyond here # allarch packagegroups are assumed to have well behaved names which don't change between architecures/tunes - if isPackageGroup(fn) and isAllArch(fn) and not isNative(depname): + if isPackageGroup(mc, fn) and isAllArch(mc, fn) and not isNative(depname): return False # Exclude well defined machine specific configurations which don't change ABI - if depname in siggen.abisaferecipes and not isImage(fn): + if depname in siggen.abisaferecipes and not isImage(mc, fn): return False # Kernel modules are well namespaced. We don't want to depend on the kernel's checksum @@ -59,10 +63,9 @@ def sstate_rundepfilter(siggen, fn, recipename, task, dep, depname, dataCache): # is machine specific. # Therefore if we're not a kernel or a module recipe (inheriting the kernel classes) # and we reccomend a kernel-module, we exclude the dependency. - depfn = dep.rsplit(":", 1)[0] - if dataCache and isKernel(depfn) and not isKernel(fn): - for pkg in dataCache.runrecs[fn]: - if " ".join(dataCache.runrecs[fn][pkg]).find("kernel-module-") != -1: + if dataCaches and isKernel(depmc, depmcfn) and not isKernel(mc, fn): + for pkg in dataCaches[mc].runrecs[fn]: + if " ".join(dataCaches[mc].runrecs[fn][pkg]).find("kernel-module-") != -1: return False # Default to keep dependencies @@ -87,10 +90,12 @@ class SignatureGeneratorOEBasic(bb.siggen.SignatureGeneratorBasic): self.abisaferecipes = (data.getVar("SIGGEN_EXCLUDERECIPES_ABISAFE") or "").split() self.saferecipedeps = (data.getVar("SIGGEN_EXCLUDE_SAFE_RECIPE_DEPS") or "").split() pass - def rundep_check(self, fn, recipename, task, dep, depname, dataCache = None): - return sstate_rundepfilter(self, fn, recipename, task, dep, depname, dataCache) + def rundep_check(self, fn, recipename, task, dep, depname, dataCaches = None): + return sstate_rundepfilter(self, fn, recipename, task, dep, depname, dataCaches) class SignatureGeneratorOEBasicHashMixIn(object): + supports_multiconfig_datacaches = True + def init_rundepcheck(self, data): self.abisaferecipes = (data.getVar("SIGGEN_EXCLUDERECIPES_ABISAFE") or "").split() self.saferecipedeps = (data.getVar("SIGGEN_EXCLUDE_SAFE_RECIPE_DEPS") or "").split() @@ -126,8 +131,8 @@ class SignatureGeneratorOEBasicHashMixIn(object): newsafedeps.append(a1 + "->" + a2) self.saferecipedeps = newsafedeps - def rundep_check(self, fn, recipename, task, dep, depname, dataCache = None): - return sstate_rundepfilter(self, fn, recipename, task, dep, depname, dataCache) + def rundep_check(self, fn, recipename, task, dep, depname, dataCaches = None): + return sstate_rundepfilter(self, fn, recipename, task, dep, depname, dataCaches) def get_taskdata(self): return (self.lockedpnmap, self.lockedhashfn, self.lockedhashes) + super().get_taskdata() @@ -142,41 +147,41 @@ class SignatureGeneratorOEBasicHashMixIn(object): self.dump_lockedsigs(sigfile) return super(bb.siggen.SignatureGeneratorBasicHash, self).dump_sigs(dataCache, options) - def prep_taskhash(self, tid, deps, dataCache): - super().prep_taskhash(tid, deps, dataCache) + def prep_taskhash(self, tid, deps, dataCaches): + super().prep_taskhash(tid, deps, dataCaches) if hasattr(self, "extramethod"): - (_, _, _, fn) = bb.runqueue.split_tid_mcfn(tid) - inherits = " ".join(dataCache.inherits[fn]) + (mc, _, _, fn) = bb.runqueue.split_tid_mcfn(tid) + inherits = " ".join(dataCaches[mc].inherits[fn]) if inherits.find("/native.bbclass") != -1 or inherits.find("/cross.bbclass") != -1: self.extramethod[tid] = ":" + self.buildarch - def get_taskhash(self, tid, deps, dataCache): + def get_taskhash(self, tid, deps, dataCaches): if tid in self.lockedhashes: if self.lockedhashes[tid]: return self.lockedhashes[tid] else: - return super().get_taskhash(tid, deps, dataCache) + return super().get_taskhash(tid, deps, dataCaches) - # get_taskhash will call get_unihash internally in the parent class, we + # get_taskhash will call get_unihash internally in the parent class, we # need to disable our filter of it whilst this runs else # incorrect hashes can be calculated. self._internal = True - h = super().get_taskhash(tid, deps, dataCache) + h = super().get_taskhash(tid, deps, dataCaches) self._internal = False (mc, _, task, fn) = bb.runqueue.split_tid_mcfn(tid) - recipename = dataCache.pkg_fn[fn] + recipename = dataCaches[mc].pkg_fn[fn] self.lockedpnmap[fn] = recipename - self.lockedhashfn[fn] = dataCache.hashfn[fn] + self.lockedhashfn[fn] = dataCaches[mc].hashfn[fn] unlocked = False if recipename in self.unlockedrecipes: unlocked = True else: def recipename_from_dep(dep): - fn = bb.runqueue.fn_from_tid(dep) - return dataCache.pkg_fn[fn] + (depmc, _, _, depfn) = bb.runqueue.split_tid_mcfn(dep) + return dataCaches[depmc].pkg_fn[depfn] # If any unlocked recipe is in the direct dependencies then the # current recipe should be unlocked as well. |