poky: subtree update:81f9e815d3..03d4d9d68f

Adrian Bunk (1):
      json-c: Don't --enable-rdrand

Alessio Igor Bogani (2):
      wic: Using the right rootfs size during prepare_rootfs
      rootfs-postcommands: Avoid use of an hard-coded value

Alexander Kanavin (1):
      binutils: drop UPSTREAM_VERSION_UNKNOWN

Alexandre Bard (1):
      systemd: Expose resolv-conf alternative only when resolved is built

Andre McCurdy (1):
      ffmpeg: enable more verbose build logs

André Draszik (4):
      ruby: drop long-merged CVE patches
      ruby: configure mis-detects isnan/isinf on musl
      ruby: fix non-IPv6 support
      packagegroup: fix a comment regarding PACKAGE_ARCH

Bruce Ashfield (6):
      linux-yocto/5.2: update to v5.2.13
      linux-yocto/4.19: update to v4.19.72
      linux-yocto/5.2: update to v5.2.14
      linux-yocto/5.2: update to v5.2.16
      linux-yocto/5.2: update to v5.2.17
      yocto-bsps: update to v5.2.17

Böszörményi Zoltán via Openembedded-core (1):
      classes/image-live.bbclass: Don't hardcode cpio.gz

Changqing Li (2):
      devtool.py: change to do clean before remove-layer
      devtool.py: fix buildclean test

Chen Qi (1):
      systemd: fix NFS regression

Dan Tran (1):
      unzip: Fix CVE-2019-13232

David Reyna (2):
      bitbake: toaster: issues in import layer when clicking 'add layer'
      bitbake: toaster: improve warnings when adding dependency to packages

Diego Rondini (2):
      initramfs-framework: fix var name
      initramfs-framework: support PARTLABEL option

Douglas Royds (1):
      icecc: Don't use icecc when INHIBIT_DEFAULT_DEPS is set

He Zhe (1):
      ltp: Fix hang of cve test cases

Heiko Schocher (1):
      kernel.fitimage.bbclass: remove ramdisk_ctype

Jacob Kroon (1):
      bitbake: tests/data: Test combinations of _append together with override

Joe Slater (1):
      bash-completion: add image feature

Jonathan Marler (1):
      package: Multiple shlib_providers for the same file should error

Joshua Watt (8):
      classes/reproducible_build: Move SDE deploy to another directory
      oeqa: Test multiconfig parsing
      bitbake: cookerdata: Add mc conffiles hashes to cache hash
      bitbake: hashserve: Add missing import
      bitbake: siggen: Fix attribute error when hashserver fails
      bitbake: hashserv: Don't daemonize server process
      local.conf.sample: Add Hash Equivalence
      classes/reproducible_build: Create SDE destination

Khem Raj (7):
      musl: Fix riscv64 CAS functions
      qemuriscv: Do not blacklist clang anymore
      sdk: Install nativesdk locales for all TCLIBC variants
      strace: Upgrade to 5.3
      packagegroups: All groups are not allarch
      musl: Fix __riscv_mc* containers to match glibc
      core-image-sato-sdk-ptest: Remove valgrind ptests for riscv

Konrad Scherer (1):
      gen-lockedsig-cache: Replace glob lookup with hash to filename lookup

Lei Maohui (1):
      bluez5: update patch to fix do_patch error when PATCHTOOL = "patch".

Li Zhou (1):
      shadow: use relaxed usernames for all

Limeng (1):
      u-boot: add CVE patches for u-boot

Nathan Rossi (2):
      oeqa/core/utils/concurrencytest.py: Handle exceptions and details
      oeqa/core/case.py: Encode binary data of log

Niclas Svensson (1):
      devtool: finish: Keep patches ordered when updating bbappend

Otavio Salvador (1):
      mesa: Add freedreno PACKAGECONFIG option

Peter Kjellerstedt (3):
      systemd: Make it build with hwdb disabled
      devtool: finish: Add suppport for the --no-clean option
      lib/oe/lsb: Make sure the distro ID is always lowercased

Randy MacLeod (1):
      ffmpeg: update from 4.2 to 4.2.1

Richard Purdie (17):
      Revert "meta-extsdk: Either an sstate task is a proper task or it isn't"
      sstatesig: Fix hash equivlanency locked signature issues
      oeqa/selftest/signing: Fix for hash equivlance server
      lib/sstatesig: Fix class inheritance problems
      populate_sdk_ext: Fix for hash equiv
      bitbake: runqueue: Fix task migration problems
      bitbake: siggen: Ensure setscenetasks list is available to worker context
      bitbake: runqueue: Change task migration behaviour for rerunning setscene tasks
      bitbake: siggen/runqueue: Fix signature mismatch issues
      bitbake: siggen: Avoid writing misleading sigdata files
      bitbake: runqueue: Save unihashes more frequently
      bitbake: runqueue: Small performance optimisation
      bitbake: siggen: Remove full path from unitaskhashes keys
      bitbake: tests/runqueue: Fix hashserve shutdown race
      base: Improve module import error message
      sanity.conf: Bump minimum bitbake version
      bitbake: bitbake: Bump verison 1.43.1 -> 1.43.2

Robert Yang (6):
      cases/bbtests.py: test_bitbake_g(): Check base-files rather than busybox
      expect: Fix configure error for nativesdk
      net-tools: Fix installed-vs-shipped for nativesdk
      expect: Fix buffer overflow error when build in long path
      apr: Check for libtoolize rather than libtool
      lttng-ust: Fix for --enable-python-agent

Ross Burton (12):
      oeqa/selftest/reproducible: test ipkgs too
      distcc: clean up the UI install logic
      distcc: use --enable-tcp-insecure instead of --make-me-a-botnet
      distcc: split into client and server packages
      json-c: clean up recipe
      json-c: use GitHub for upstream release checking
      bitbake: fetch2/git: refactor check for git-lfs command
      bitbake: tests/fetch: add test case for git-lfs handling
      python3: move runpy to core
      pango: fix the failing testiter test case
      opkg: remove redundant systemd inherit
      lttng-ust: update patch Signed-off-by

Trevor Gamblin (5):
      python3-subunit: ensure runtime dependencies are present
      python3-pip: ensure pickle is installed
      lighttpd: remove fam as a PACKAGECONFIG option
      tiff: fix CVE-2019-14973
      opkg: remove pathfinder PACKAGECONFIG option

Wang Quanyang (1):
      kexec-tools: fix arm kexec failure for __NR_kexec_file_load

Yi Zhao (1):
      python: add tk-lib as runtime dependency for python-tkinter

Change-Id: I0570125d49f7e4bc3bbf70508cbfd7e10bdbc032
Signed-off-by: Brad Bishop <bradleyb@fuzziesquirrel.com>

1 files changed, 64 insertions, 45 deletions

diff --git a/poky/bitbake/lib/bb/runqueue.py b/poky/bitbake/lib/bb/runqueue.py
index d9a67a3167..18049436fd 100644
--- a/poky/bitbake/lib/bb/runqueue.py
+++ b/poky/bitbake/lib/bb/runqueue.py
@@ -73,7 +73,7 @@ def build_tid(mc, fn, taskname):
 def pending_hash_index(tid, rqdata):
     (mc, fn, taskname, taskfn) = split_tid_mcfn(tid)
     pn = rqdata.dataCaches[mc].pkg_fn[taskfn]
-    h = rqdata.runtaskentries[tid].hash
+    h = rqdata.runtaskentries[tid].unihash
     return pn + ":" + "taskname" + h
 
 class RunQueueStats:
@@ -207,6 +207,8 @@ class RunQueueScheduler(object):
 
     def newbuildable(self, task):
         self.buildable.add(task)
+        # Once tasks are running we don't need to worry about them again
+        self.buildable.difference_update(self.rq.runq_running)
 
     def removebuildable(self, task):
         self.buildable.remove(task)
@@ -1162,6 +1164,8 @@ class RunQueueData:
 
         self.init_progress_reporter.next_stage()
 
+        bb.parse.siggen.set_setscene_tasks(self.runq_setscene_tids)
+
         # Iterate over the task list and call into the siggen code
         dealtwith = set()
         todeal = set(self.runtaskentries)
@@ -1173,7 +1177,6 @@ class RunQueueData:
                     self.prepare_task_hash(tid)
 
         bb.parse.siggen.writeout_file_checksum_cache()
-        bb.parse.siggen.set_setscene_tasks(self.runq_setscene_tids)
 
         #self.dump_data()
         return len(self.runtaskentries)
@@ -1442,6 +1445,7 @@ class RunQueue:
                 self.state = runQueueComplete
             else:
                 self.state = runQueueSceneInit
+                bb.parse.siggen.save_unitaskhashes()
 
         if self.state is runQueueSceneInit:
             self.rqdata.init_progress_reporter.next_stage()
@@ -2299,11 +2303,12 @@ class RunQueueExecute:
         for tid in changed:
             if tid not in self.rqdata.runq_setscene_tids:
                 continue
-            valid = self.rq.validate_hashes(set([tid]), self.cooker.data, None, False)
-            if not valid:
-                continue
             if tid in self.runq_running:
                 continue
+            if tid in self.scenequeue_covered:
+                # Potentially risky, should we report this hash as a match?
+                logger.info("Already covered setscene for %s so ignoring rehash" % (tid))
+                continue
             if tid not in self.pending_migrations:
                 self.pending_migrations.add(tid)
 
@@ -2358,6 +2363,7 @@ class RunQueueExecute:
 
             logger.info("Setscene task %s now valid and being rerun" % tid)
             self.sqdone = False
+            update_scenequeue_data([tid], self.sqdata, self.rqdata, self.rq, self.cooker, self.stampcache, self)
 
         if changed:
             self.holdoff_need_update = True
@@ -2674,64 +2680,77 @@ def build_scenequeue_data(sqdata, rqdata, rq, cooker, stampcache, sqrq):
 
     rqdata.init_progress_reporter.next_stage()
 
-    multiconfigs = set()
+    sqdata.multiconfigs = set()
     for tid in sqdata.sq_revdeps:
-        multiconfigs.add(mc_from_tid(tid))
+        sqdata.multiconfigs.add(mc_from_tid(tid))
         if len(sqdata.sq_revdeps[tid]) == 0:
             sqrq.sq_buildable.add(tid)
 
     rqdata.init_progress_reporter.finish()
 
-    if rq.hashvalidate:
-        noexec = []
-        stamppresent = []
-        tocheck = set()
+    sqdata.noexec = set()
+    sqdata.stamppresent = set()
+    sqdata.valid = set()
 
-        for tid in sorted(sqdata.sq_revdeps):
-            (mc, fn, taskname, taskfn) = split_tid_mcfn(tid)
+    update_scenequeue_data(sqdata.sq_revdeps, sqdata, rqdata, rq, cooker, stampcache, sqrq)
 
-            taskdep = rqdata.dataCaches[mc].task_deps[taskfn]
+def update_scenequeue_data(tids, sqdata, rqdata, rq, cooker, stampcache, sqrq):
 
-            if 'noexec' in taskdep and taskname in taskdep['noexec']:
-                noexec.append(tid)
-                sqrq.sq_task_skip(tid)
-                bb.build.make_stamp(taskname + "_setscene", rqdata.dataCaches[mc], taskfn)
-                continue
+    tocheck = set()
 
-            if rq.check_stamp_task(tid, taskname + "_setscene", cache=stampcache):
-                logger.debug(2, 'Setscene stamp current for task %s', tid)
-                stamppresent.append(tid)
-                sqrq.sq_task_skip(tid)
-                continue
+    for tid in sorted(tids):
+        if tid in sqdata.stamppresent:
+            sqdata.stamppresent.remove(tid)
+        if tid in sqdata.valid:
+            sqdata.valid.remove(tid)
 
-            if rq.check_stamp_task(tid, taskname, recurse = True, cache=stampcache):
-                logger.debug(2, 'Normal stamp current for task %s', tid)
-                stamppresent.append(tid)
-                sqrq.sq_task_skip(tid)
-                continue
+        (mc, fn, taskname, taskfn) = split_tid_mcfn(tid)
 
-            tocheck.add(tid)
+        taskdep = rqdata.dataCaches[mc].task_deps[taskfn]
+
+        if 'noexec' in taskdep and taskname in taskdep['noexec']:
+            sqdata.noexec.add(tid)
+            sqrq.sq_task_skip(tid)
+            bb.build.make_stamp(taskname + "_setscene", rqdata.dataCaches[mc], taskfn)
+            continue
+
+        if rq.check_stamp_task(tid, taskname + "_setscene", cache=stampcache):
+            logger.debug(2, 'Setscene stamp current for task %s', tid)
+            sqdata.stamppresent.add(tid)
+            sqrq.sq_task_skip(tid)
+            continue
+
+        if rq.check_stamp_task(tid, taskname, recurse = True, cache=stampcache):
+            logger.debug(2, 'Normal stamp current for task %s', tid)
+            sqdata.stamppresent.add(tid)
+            sqrq.sq_task_skip(tid)
+            continue
 
-        valid = rq.validate_hashes(tocheck, cooker.data, len(stamppresent), False)
+        tocheck.add(tid)
 
-        valid_new = stamppresent
-        for v in valid:
-            valid_new.append(v)
+    sqdata.valid |= rq.validate_hashes(tocheck, cooker.data, len(sqdata.stamppresent), False)
 
-        hashes = {}
-        for mc in sorted(multiconfigs):
-          for tid in sorted(sqdata.sq_revdeps):
+    sqdata.hashes = {}
+    for mc in sorted(sqdata.multiconfigs):
+        for tid in sorted(sqdata.sq_revdeps):
             if mc_from_tid(tid) != mc:
                 continue
-            if tid not in valid_new and tid not in noexec and tid not in sqrq.scenequeue_notcovered:
-                sqdata.outrightfail.add(tid)
+            if tid in sqdata.stamppresent:
+                continue
+            if tid in sqdata.valid:
+                continue
+            if tid in sqdata.noexec:
+                continue
+            if tid in sqrq.scenequeue_notcovered:
+                continue
+            sqdata.outrightfail.add(tid)
 
-                h = pending_hash_index(tid, rqdata)
-                if h not in hashes:
-                    hashes[h] = tid
-                else:
-                    sqrq.sq_deferred[tid] = hashes[h]
-                    bb.warn("Deferring %s after %s" % (tid, hashes[h]))
+            h = pending_hash_index(tid, rqdata)
+            if h not in sqdata.hashes:
+                sqdata.hashes[h] = tid
+            else:
+                sqrq.sq_deferred[tid] = sqdata.hashes[h]
+                bb.warn("Deferring %s after %s" % (tid, sqdata.hashes[h]))
 
 
 class TaskFailure(Exception):