poky: subtree update:26ae42ded7..5951cbcabe

Alex Kiernan (1):
      recipetool: Fix list concatenation when using edit

Alexander Kanavin (4):
      apr-util: make gdbm optional
      gobject-introspection: add a patch to fix a build race
      icu: merge .inc into main recipe
      icu: make filtered data generation optional, serial and off by default

Alexandru N. Onea (3):
      bitbake: perforce: add basic progress handler for perforce
      bitbake: perforce: add local path handling SRC_URI options
      bitbake: bitbake-user-manual: update perforce fetcher docs

Andreas M?ller (1):
      meson.bbclass: avoid unexpected operating-system names

Andreas Müller (6):
      boost: Add upstream patch to fix build on depending projects
      libinput: upgrade 1.15.5 -> 1.15.6
      sqlite3: upgrade 3.32.2 -> 3.32.3
      desktop-file-utils: upgrade 0.24 -> 0.26
      file: upgrade 5.38 -> 5.39
      ffmpeg: upgrade 4.2.3 -> 4.3

Andrej Valek (1):
      oeqa/runtime/cases/ptest: Make output content path absolute

Andrew Geissler (1):
      meson: backport library ordering fix

Armin Kuster (1):
      libuv: move from meta-oe to core for bind update

Arthur She (1):
      igt-gpu-tools: add new package

Changqing Li (1):
      mime.bbclass: fix post install scriptlet error

Chen Qi (1):
      systemd-serialgetty: do not use BindsTo

Daniel McGregor (3):
      sign_rpm.bbclass: ignore thread count
      systemd-conf: Accept MTU from DHCP
      buildhistory-collect-srcrevs: sort directories

He Zhe (1):
      ltp: Fix copy_file_rang02 for 32-bit arches

Hongxu Jia (1):
      libmodulemd: switch branch master -> main

Jacob Kroon (5):
      bitbake: lib/bb/utils.py: Do not preserve TERM in the environment
      bitbake: bitbake-user-manual: Remove TERM from BB_HASHBASE_WHITELIST example
      bitbake.conf: Remove TERM from default BB_HASHBASE_WHITELIST
      grub: Remove native version of grub-efi
      distro_alias: Remove unused grub-efi distro aliases

Jens Rehsack (1):
      u-boot: avoid blind merging all *.cfg

Joe Slater (1):
      systemd: fix CVE-2020-13776

Joshua Watt (5):
      sstatesig: Account for all dataCaches being passed
      bitbake: bitbake: cache: Fix error message with bad multiconfig
      wic: Fix error message when reporting invalid offset
      classes/archiver: Create patched archive before configuring
      bitbake: cache: Bump cache version

Konrad Weihmann (3):
      oeqa/runtime: Add OERequirePackage decorator
      bitbake: cookerdata: Add BBFILES_DYNAMIC inverse mode
      bitbake: bitbake-user-manual: Add BBFILES_DYNAMIC

Mark Morton (2):
      New source files and Makefile update for Test Manual
      test-manual: Fixed codeblock formatting

Martin Jansa (1):
      net-tools: backport a patch from upstream to use the same ifconfig format as debian/ubuntu

Mingli Yu (3):
      python3: add the rdepends for python3-misc
      python3: add rdepends for python3-idle
      python3-dbusmock: add the missing rdepends

Otavio Salvador (2):
      systemd: Sync systemd-serialgetty@.service with upstream
      mtd-utils: Fix return value of ubiformat

Ovidiu Panait (2):
      dbus-test: Remove EXTRA_OECONF_X configs
      dbus,dbus-test: Move common parts to dbus.inc

Paul Barker (2):
      bitbake: fetch2/gitsm: Mark srcrev as fetched once all submodules are processed
      bitbake: fetch2/gitsm: Make need_update() process submodules

Paul Eggleton (5):
      graph-tool: switch to argparse
      graph-tool: add filter subcommand
      dpkg-native: rebase and reinstate fix for "tar: file changed as we read it"
      shadow-sysroot: drop unused SRC_URI checksums
      devtool: fix typo

Peter Kjellerstedt (1):
      relocatable.bbclass: Avoid an exception if an empty pkgconfig dir exist

Pierre-Jean Texier (3):
      diffoscope: upgrade 146 -> 147
      ell: upgrade 0.31 -> 0.32
      curl: upgrade 7.70.0 -> 7.71.0

Rasmus Villemoes (1):
      curl: add debug info

Richard Purdie (15):
      buildhistory: Add simplistic file move detection
      bitbake: bin/bitbake: Update to next series release version
      perl: Fix host specific modules problems
      sanity.conf: Require bitbake 1.47.0 as the minimum version
      patchelf: Upgrade 0.10 -> 0.11
      test-manual: Add SPDX license headers
      Makefile: Drop obsolete edison/denzil branch conditionals
      bitbake: tests/fetch: Switch from git.infradead.org to a YP mirror
      pseudo: Fix attr errors due to incorrect library resolution issues
      oeqa/selftest/runcmd: Add better debug for thread count mismatch failures
      oeqa/utils/command: Improve stdin handling in runCmd
      vulkan-headers: Fix upstream branch deletion issue
      recipes: Fix Upstream-Status Accepted -> Backport
      scripts/install-buildtools: Update to 3.2 M1 buildtools
      scripts/install-buildtools: Handle new format checksum files

Robert P. J. Day (1):
      python: use official "pypi.org" URLs for HOMEPAGE

Ross Burton (8):
      install-buildtools: fail if an error occurs
      install-buildtools: remove hardcoded x86-64 architecture
      install-buildtools: add option to disable checksum validation
      common-licenses: add BSD-2-Clause-Patent
      gstreamer1.0-plugins-bad: add support for vdpau
      go-binary-native: add binary Go to bootstrap
      tcmode-default: use go-binary-native by default
      go-native: merge bb/inc and add comment

Ryan Rowe (1):
      python3: fix PGO for non-reproducible biniaries

Sakib Sajal (1):
      qemu: uprev v4.2.0 -> v5.0.0

Samuli Piippo (2):
      cmake: allow chainloading of the toolchain file
      perl: use relative paths in the perl wrapper

Steve Sakoman (1):
      buildtools-tarball: export OPENSSL_CONF in environment setup

Tanu Kaskinen (1):
      pulseaudio: remove unnecessary libltdl copying

Trevor Gamblin (1):
      python3-setuptools: patch entrypoints for faster initialization

Tuomas Salokanto (1):
      recipetool: create: fix SRCBRANCH not being passed to params

Valentin Longchamp (2):
      tools-profile: disable valgrind for powerpc soft-float
      valgrind: disable it for powerpc soft-float

Wang Mingyu (5):
      powertop: upgrade 2.12 -> 2.13
      man-db: upgrade 2.9.2 -> 2.9.3
      valgrind: upgrade 3.16.0 -> 3.16.1
      man-pages: upgrade 5.06 -> 5.07
      harfbuzz: upgrade 2.6.7 -> 2.6.8

Yi Zhao (2):
      iptables: fix invalid symbolic link for ip6tables-apply
      iptables: split iptables-apply to its own package

Yongxin Liu (1):
      linux-firmware: add ice for Intel E800 series driver

Yuki Hoshino (1):
      sysvinit-inittab: Add support for tty devices with 10 or more number.

akuster (9):
      bind: update to 9.11.19
      adt-manual: Add SPDX license headers
      bsp-guide: Add SPDX license headers
      brief-yoctoprojectsqa: Add SPDX license headers
      dev-manual: Add SPDX License headers
      kernel-dev: Add SPDX license headers
      profile-manual: Add SPDX licence headers
      sdk-manual: Add SPDX license headers
      toaster-manaul: Add SPDX license headers

haiqing (1):
      libpam: Remove option 'obscure' from common-password

hongxu (1):
      kmod: add nativesdk support

zangrc (1):
      ethtool:upgrade 5.6 -> 5.7

Signed-off-by: Andrew Geissler <geissonator@yahoo.com>
Change-Id: I1190ca17297b1167286cfc06033e8485396c7cce

2 files changed, 104 insertions, 40 deletions

diff --git a/poky/meta/lib/oe/buildhistory_analysis.py b/poky/meta/lib/oe/buildhistory_analysis.py
index 5b28774c9..2d6fa1779 100644
--- a/poky/meta/lib/oe/buildhistory_analysis.py
+++ b/poky/meta/lib/oe/buildhistory_analysis.py
@@ -213,6 +213,7 @@ class FileChange:
     changetype_perms = 'P'
     changetype_ownergroup = 'O'
     changetype_link = 'L'
+    changetype_move = 'M'
 
     def __init__(self, path, changetype, oldvalue = None, newvalue = None):
         self.path = path
@@ -251,10 +252,11 @@ class FileChange:
             return '%s changed owner/group from %s to %s' % (self.path, self.oldvalue, self.newvalue)
         elif self.changetype == self.changetype_link:
             return '%s changed symlink target from %s to %s' % (self.path, self.oldvalue, self.newvalue)
+        elif self.changetype == self.changetype_move:
+            return '%s moved to %s' % (self.path, self.oldvalue)
         else:
             return '%s changed (unknown)' % self.path
 
-
 def blob_to_dict(blob):
     alines = [line for line in blob.data_stream.read().decode('utf-8').splitlines()]
     adict = {}
@@ -281,11 +283,14 @@ def file_list_to_dict(lines):
             adict[path] = splitv[0:3]
     return adict
 
+numeric_removal = str.maketrans('0123456789', 'XXXXXXXXXX')
 
 def compare_file_lists(alines, blines, compare_ownership=True):
     adict = file_list_to_dict(alines)
     bdict = file_list_to_dict(blines)
     filechanges = []
+    additions = []
+    removals = []
     for path, splitv in adict.items():
         newsplitv = bdict.pop(path, None)
         if newsplitv:
@@ -318,11 +323,65 @@ def compare_file_lists(alines, blines, compare_ownership=True):
                 if oldvalue != newvalue:
                     filechanges.append(FileChange(path, FileChange.changetype_link, oldvalue, newvalue))
         else:
-            filechanges.append(FileChange(path, FileChange.changetype_remove))
+            removals.append(path)
 
     # Whatever is left over has been added
     for path in bdict:
-        filechanges.append(FileChange(path, FileChange.changetype_add))
+        additions.append(path)
+
+    # Rather than print additions and removals, its nicer to print file 'moves'
+    # where names or paths are similar.
+    revmap_remove = {}
+    for removal in removals:
+        translated = removal.translate(numeric_removal)
+        if translated not in revmap_remove:
+            revmap_remove[translated] = []
+        revmap_remove[translated].append(removal)
+
+    #
+    # We want to detect renames of large trees of files like
+    # /lib/modules/5.4.40-yocto-standard to /lib/modules/5.4.43-yocto-standard
+    #
+    renames = {}
+    for addition in additions.copy():
+        if addition not in additions:
+            continue
+        translated = addition.translate(numeric_removal)
+        if translated in revmap_remove:
+            if len(revmap_remove[translated]) != 1:
+                continue
+            removal = revmap_remove[translated][0]
+            commondir = addition.split("/")
+            commondir2 = removal.split("/")
+            idx = None
+            for i in range(len(commondir)):
+                if commondir[i] != commondir2[i]:
+                    idx = i
+                    break
+            commondir = "/".join(commondir[:i+1])
+            commondir2 = "/".join(commondir2[:i+1])
+            # If the common parent is in one dict and not the other its likely a rename
+            # so iterate through those files and process as such
+            if commondir2 not in bdict and commondir not in adict:
+                if commondir not in renames:
+                    renames[commondir] = commondir2
+                    for addition2 in additions.copy():
+                        if addition2.startswith(commondir):
+                            removal2 = addition2.replace(commondir, commondir2)
+                            if removal2 in removals:
+                                additions.remove(addition2)
+                                removals.remove(removal2)
+                    continue
+            filechanges.append(FileChange(removal, FileChange.changetype_move, addition))
+            additions.remove(addition)
+            removals.remove(removal)
+    for rename in renames:
+        filechanges.append(FileChange(renames[rename], FileChange.changetype_move, rename))
+
+    for addition in additions:
+        filechanges.append(FileChange(addition, FileChange.changetype_add))
+    for removal in removals:
+        filechanges.append(FileChange(removal, FileChange.changetype_remove))
 
     return filechanges
 
diff --git a/poky/meta/lib/oe/sstatesig.py b/poky/meta/lib/oe/sstatesig.py
index d24e3738a..21ae0a765 100644
--- a/poky/meta/lib/oe/sstatesig.py
+++ b/poky/meta/lib/oe/sstatesig.py
@@ -2,9 +2,10 @@
 # SPDX-License-Identifier: GPL-2.0-only
 #
 import bb.siggen
+import bb.runqueue
 import oe
 
-def sstate_rundepfilter(siggen, fn, recipename, task, dep, depname, dataCache):
+def sstate_rundepfilter(siggen, fn, recipename, task, dep, depname, dataCaches):
     # Return True if we should keep the dependency, False to drop it
     def isNative(x):
         return x.endswith("-native")
@@ -12,23 +13,26 @@ def sstate_rundepfilter(siggen, fn, recipename, task, dep, depname, dataCache):
         return "-cross-" in x
     def isNativeSDK(x):
         return x.startswith("nativesdk-")
-    def isKernel(fn):
-        inherits = " ".join(dataCache.inherits[fn])
+    def isKernel(mc, fn):
+        inherits = " ".join(dataCaches[mc].inherits[fn])
         return inherits.find("/module-base.bbclass") != -1 or inherits.find("/linux-kernel-base.bbclass") != -1
-    def isPackageGroup(fn):
-        inherits = " ".join(dataCache.inherits[fn])
+    def isPackageGroup(mc, fn):
+        inherits = " ".join(dataCaches[mc].inherits[fn])
         return "/packagegroup.bbclass" in inherits
-    def isAllArch(fn):
-        inherits = " ".join(dataCache.inherits[fn])
+    def isAllArch(mc, fn):
+        inherits = " ".join(dataCaches[mc].inherits[fn])
         return "/allarch.bbclass" in inherits
-    def isImage(fn):
-        return "/image.bbclass" in " ".join(dataCache.inherits[fn])
-
-    # (Almost) always include our own inter-task dependencies.
-    # The exception is the special do_kernel_configme->do_unpack_and_patch
-    # dependency from archiver.bbclass.
-    if recipename == depname:
-        if task == "do_kernel_configme" and dep.endswith(".do_unpack_and_patch"):
+    def isImage(mc, fn):
+        return "/image.bbclass" in " ".join(dataCaches[mc].inherits[fn])
+
+    depmc, _, deptaskname, depmcfn = bb.runqueue.split_tid_mcfn(dep)
+    mc, _ = bb.runqueue.split_mc(fn)
+
+    # (Almost) always include our own inter-task dependencies (unless it comes
+    # from a mcdepends). The exception is the special
+    # do_kernel_configme->do_unpack_and_patch dependency from archiver.bbclass.
+    if recipename == depname and depmc == mc:
+        if task == "do_kernel_configme" and deptaskname == "do_unpack_and_patch":
             return False
         return True
 
@@ -47,11 +51,11 @@ def sstate_rundepfilter(siggen, fn, recipename, task, dep, depname, dataCache):
     # Only target packages beyond here
 
     # allarch packagegroups are assumed to have well behaved names which don't change between architecures/tunes
-    if isPackageGroup(fn) and isAllArch(fn) and not isNative(depname):
+    if isPackageGroup(mc, fn) and isAllArch(mc, fn) and not isNative(depname):
         return False
 
     # Exclude well defined machine specific configurations which don't change ABI
-    if depname in siggen.abisaferecipes and not isImage(fn):
+    if depname in siggen.abisaferecipes and not isImage(mc, fn):
         return False
 
     # Kernel modules are well namespaced. We don't want to depend on the kernel's checksum
@@ -59,10 +63,9 @@ def sstate_rundepfilter(siggen, fn, recipename, task, dep, depname, dataCache):
     # is machine specific.
     # Therefore if we're not a kernel or a module recipe (inheriting the kernel classes)
     # and we reccomend a kernel-module, we exclude the dependency.
-    depfn = dep.rsplit(":", 1)[0]
-    if dataCache and isKernel(depfn) and not isKernel(fn):
-        for pkg in dataCache.runrecs[fn]:
-            if " ".join(dataCache.runrecs[fn][pkg]).find("kernel-module-") != -1:
+    if dataCaches and isKernel(depmc, depmcfn) and not isKernel(mc, fn):
+        for pkg in dataCaches[mc].runrecs[fn]:
+            if " ".join(dataCaches[mc].runrecs[fn][pkg]).find("kernel-module-") != -1:
                 return False
 
     # Default to keep dependencies
@@ -87,10 +90,12 @@ class SignatureGeneratorOEBasic(bb.siggen.SignatureGeneratorBasic):
         self.abisaferecipes = (data.getVar("SIGGEN_EXCLUDERECIPES_ABISAFE") or "").split()
         self.saferecipedeps = (data.getVar("SIGGEN_EXCLUDE_SAFE_RECIPE_DEPS") or "").split()
         pass
-    def rundep_check(self, fn, recipename, task, dep, depname, dataCache = None):
-        return sstate_rundepfilter(self, fn, recipename, task, dep, depname, dataCache)
+    def rundep_check(self, fn, recipename, task, dep, depname, dataCaches = None):
+        return sstate_rundepfilter(self, fn, recipename, task, dep, depname, dataCaches)
 
 class SignatureGeneratorOEBasicHashMixIn(object):
+    supports_multiconfig_datacaches = True
+
     def init_rundepcheck(self, data):
         self.abisaferecipes = (data.getVar("SIGGEN_EXCLUDERECIPES_ABISAFE") or "").split()
         self.saferecipedeps = (data.getVar("SIGGEN_EXCLUDE_SAFE_RECIPE_DEPS") or "").split()
@@ -126,8 +131,8 @@ class SignatureGeneratorOEBasicHashMixIn(object):
             newsafedeps.append(a1 + "->" + a2)
         self.saferecipedeps = newsafedeps
 
-    def rundep_check(self, fn, recipename, task, dep, depname, dataCache = None):
-        return sstate_rundepfilter(self, fn, recipename, task, dep, depname, dataCache)
+    def rundep_check(self, fn, recipename, task, dep, depname, dataCaches = None):
+        return sstate_rundepfilter(self, fn, recipename, task, dep, depname, dataCaches)
 
     def get_taskdata(self):
         return (self.lockedpnmap, self.lockedhashfn, self.lockedhashes) + super().get_taskdata()
@@ -142,41 +147,41 @@ class SignatureGeneratorOEBasicHashMixIn(object):
         self.dump_lockedsigs(sigfile)
         return super(bb.siggen.SignatureGeneratorBasicHash, self).dump_sigs(dataCache, options)
 
-    def prep_taskhash(self, tid, deps, dataCache):
-        super().prep_taskhash(tid, deps, dataCache)
+    def prep_taskhash(self, tid, deps, dataCaches):
+        super().prep_taskhash(tid, deps, dataCaches)
         if hasattr(self, "extramethod"):
-            (_, _, _, fn) = bb.runqueue.split_tid_mcfn(tid)
-            inherits = " ".join(dataCache.inherits[fn])    
+            (mc, _, _, fn) = bb.runqueue.split_tid_mcfn(tid)
+            inherits = " ".join(dataCaches[mc].inherits[fn])
             if inherits.find("/native.bbclass") != -1 or inherits.find("/cross.bbclass") != -1:
                 self.extramethod[tid] = ":" + self.buildarch
 
-    def get_taskhash(self, tid, deps, dataCache):
+    def get_taskhash(self, tid, deps, dataCaches):
         if tid in self.lockedhashes:
             if self.lockedhashes[tid]:
                 return self.lockedhashes[tid]
             else:
-                return super().get_taskhash(tid, deps, dataCache)
+                return super().get_taskhash(tid, deps, dataCaches)
 
-        # get_taskhash will call get_unihash internally in the parent class, we 
+        # get_taskhash will call get_unihash internally in the parent class, we
         # need to disable our filter of it whilst this runs else
         # incorrect hashes can be calculated.
         self._internal = True
-        h = super().get_taskhash(tid, deps, dataCache)
+        h = super().get_taskhash(tid, deps, dataCaches)
         self._internal = False
 
         (mc, _, task, fn) = bb.runqueue.split_tid_mcfn(tid)
 
-        recipename = dataCache.pkg_fn[fn]
+        recipename = dataCaches[mc].pkg_fn[fn]
         self.lockedpnmap[fn] = recipename
-        self.lockedhashfn[fn] = dataCache.hashfn[fn]
+        self.lockedhashfn[fn] = dataCaches[mc].hashfn[fn]
 
         unlocked = False
         if recipename in self.unlockedrecipes:
             unlocked = True
         else:
             def recipename_from_dep(dep):
-                fn = bb.runqueue.fn_from_tid(dep)
-                return dataCache.pkg_fn[fn]
+                (depmc, _, _, depfn) = bb.runqueue.split_tid_mcfn(dep)
+                return dataCaches[depmc].pkg_fn[depfn]
 
             # If any unlocked recipe is in the direct dependencies then the
             # current recipe should be unlocked as well.