poky: refresh thud: b904775c2b..7c76c5d78b

Update poky to thud HEAD.

Adam Trhon (1):
      icecc-env: don't raise error when icecc not installed

Alexander Kanavin (1):
      openssl10: update to 1.0.2q

Armin Kuster (1):
      perl: add testdepends for ssh

Bruce Ashfield (2):
      linux-yocto/4.18: update to v4.18.26
      linux-yocto/4.18: update to v4.18.27

Changqing Li (1):
      checklayer: generate locked-sigs.inc under builddir

Dan Dedrick (2):
      devtool: remove duplicate overrides
      devtool: improve git repo checks before check_commits logic

Daniel Ammann (1):
      ref-manual: Typo found and fixed.

Douglas Royds (2):
      openssl ptest: Strip build host paths from configdata.pm
      openssl: Strip perl version from installed ptest configdata.pm file

Dustin Bain (1):
      busybox: update to 1.29.3

Jan Kiszka (1):
      oe-git-proxy: Avoid resolving NO_PROXY against local files

Jens Rehsack (1):
      avahi: avoid depending on skipped package

Jonas Bonn (1):
      keymaps: tighten package write dependency

Kai Kang (1):
      selftest/wic: update test case test_qemu

Khem Raj (3):
      openssl10: Fix mutliple include assumptions for bn.h in opensslconf.h
      send-error-report: Use https instead of http protocol
      multilib_header_wrapper.h: Use #pragma once

Leonardo Augusto (1):
      scripts/lib/wic/engine: Fix cp's target path for ext* filesystems

Liu Haitao (1):
      iw: fix parsing of WEP keys

Mingli Yu (1):
      logrotate.py: restore /etc/logrotate.d/wtmp

Otavio Salvador (1):
      linux-firmware: Bump to 710963f revision

Ovidiu Panait (1):
      ghostscript: Fix CVE-2019-6116

Peter Kjellerstedt (1):
      libaio: Extend to native

Richard Purdie (23):
      package: Add pkg_postinst_ontarget to PACKAGEVARS
      oeqa/runtime/ptest: Avoid traceback for tests with no section
      oeqa/utils/logparser: Simplify ptest log parsing code
      oeqa/logparser: Further simplification/clarification
      oeqa/logparser: Reform the ptest results parser
      oeqa/utils/logparser: Add in support for duration, exitcode and logs by section
      oeqa/logparser: Improve results handling
      oeqa/logparser: Various misc cleanups
      oeqa/runtime/ptest: Ensure OOM errors are logged
      scripts/contrib/build-perf-test-wrapper.sh: Improve interaction with autobuilder automation
      scripts/contrib/build-perf-test.sh: Remove it
      oe-build-perf-report: Allow branch without hostname
      oe-build-perf-report: Allow commits from different branches
      oe-build-perf-report: Improve branch comparision handling
      oe-build-perf-report: Fix missing buildstats comparisions
      wic/engine: Fix missing parted autobuilder failures
      lib/buildstats: Improve error message
      scripts/oe-git-archive: Separate out functionality to library function
      oe-build-perf-report/gitarchive: Move common useful functions to library
      bitbake: runqueue: Fix dependency loop analysis 'hangs'
      bitbake: runqueue: Filter out multiconfig dependencies from BB_TASKDEPDATA
      bitbake: siggen: Fix multiconfig corner case
      bitbake: cooker: Tweak multiconfig dependency resolution

Robert Yang (5):
      bluez5: Fix a race issue for tools
      yocto-check-layer-wrapper: Fix path for oe-init-build-env
      checklayer: Avoid adding the layer if it is already present
      runqemu: Let qemuparams override default settings
      runqemu: Make QB_MEM easier to set

Ross Burton (3):
      e2fsprogs: fix file system generation with large files
      linux-firmware: recommend split up packages
      linux-firmware: split out liquidio firmware

Scott Rifenbark (2):
      poky.ent: Updated "meta-intel" version to "10.1"
      overview-manual, mega-manual: Updated Package Feeds diagram

Serhey Popovych (1):
      openssl: Skip assembler optimized code for powerpc64 with musl

William Bourque (1):
      wic/engine.py: Load paths from PATH environment variable

Xulin Sun (1):
      openssl: fix multilib file install conflicts

Zheng Ruoqin (1):
      mdadm: add init and service scripts

Change-Id: Ib14c2fb69d25d84aa3d4bf0a6715bba57d1eb900
Signed-off-by: Andrew Geissler <geissonator@yahoo.com>

4 files changed, 41 insertions, 27 deletions

diff --git a/poky/scripts/lib/buildstats.py b/poky/scripts/lib/buildstats.py
index d9aadf3cb..f7db3eaf9 100644
--- a/poky/scripts/lib/buildstats.py
+++ b/poky/scripts/lib/buildstats.py
@@ -263,7 +263,7 @@ class BuildStats(dict):
         """Aggregate other buildstats into this"""
         if set(self.keys()) != set(buildstats.keys()):
             raise ValueError("Refusing to aggregate buildstats, set of "
-                             "recipes is different")
+                             "recipes is different: %s" % (set(self.keys()) ^ set(buildstats.keys())))
         for pkg, data in buildstats.items():
             self[pkg].aggregate(data)
 
diff --git a/poky/scripts/lib/checklayer/__init__.py b/poky/scripts/lib/checklayer/__init__.py
index 778804184..670f0eea3 100644
--- a/poky/scripts/lib/checklayer/__init__.py
+++ b/poky/scripts/lib/checklayer/__init__.py
@@ -196,38 +196,36 @@ def add_layer_dependencies(bblayersconf, layer, layers, logger):
     if layer_depends is None:
         return False
     else:
-        # Don't add a layer that is already present.
-        added = set()
-        output = check_command('Getting existing layers failed.', 'bitbake-layers show-layers').decode('utf-8')
-        for layer, path, pri in re.findall(r'^(\S+) +([^\n]*?) +(\d+)$', output, re.MULTILINE):
-            added.add(path)
-
-        for layer_depend in layer_depends:
-            name = layer_depend['name']
-            path = layer_depend['path']
+        add_layers(bblayersconf, layer_depends, logger)
+
+    return True
+
+def add_layers(bblayersconf, layers, logger):
+    # Don't add a layer that is already present.
+    added = set()
+    output = check_command('Getting existing layers failed.', 'bitbake-layers show-layers').decode('utf-8')
+    for layer, path, pri in re.findall(r'^(\S+) +([^\n]*?) +(\d+)$', output, re.MULTILINE):
+        added.add(path)
+
+    with open(bblayersconf, 'a+') as f:
+        for layer in layers:
+            logger.info('Adding layer %s' % layer['name'])
+            name = layer['name']
+            path = layer['path']
             if path in added:
-                continue
+                logger.info('%s is already in %s' % (name, bblayersconf))
             else:
                 added.add(path)
-            logger.info('Adding layer dependency %s' % name)
-            with open(bblayersconf, 'a+') as f:
                 f.write("\nBBLAYERS += \"%s\"\n" % path)
     return True
 
-def add_layer(bblayersconf, layer, layers, logger):
-    logger.info('Adding layer %s' % layer['name'])
-    with open(bblayersconf, 'a+') as f:
-        f.write("\nBBLAYERS += \"%s\"\n" % layer['path'])
-
-    return True
-
-def check_command(error_msg, cmd):
+def check_command(error_msg, cmd, cwd=None):
     '''
     Run a command under a shell, capture stdout and stderr in a single stream,
     throw an error when command returns non-zero exit code. Returns the output.
     '''
 
-    p = subprocess.Popen(cmd, shell=True, stdout=subprocess.PIPE, stderr=subprocess.STDOUT)
+    p = subprocess.Popen(cmd, shell=True, stdout=subprocess.PIPE, stderr=subprocess.STDOUT, cwd=cwd)
     output, _ = p.communicate()
     if p.returncode:
         msg = "%s\nCommand: %s\nOutput:\n%s" % (error_msg, cmd, output.decode('utf-8'))
@@ -257,7 +255,7 @@ def get_signatures(builddir, failsafe=False, machine=None):
         os.unlink(sigs_file)
     try:
         check_command('Generating signatures failed. This might be due to some parse error and/or general layer incompatibilities.',
-                      cmd)
+                      cmd, builddir)
     except RuntimeError as ex:
         if failsafe and os.path.exists(sigs_file):
             # Ignore the error here. Most likely some recipes active
diff --git a/poky/scripts/lib/devtool/standard.py b/poky/scripts/lib/devtool/standard.py
index d14b7a654..b7d4d47df 100644
--- a/poky/scripts/lib/devtool/standard.py
+++ b/poky/scripts/lib/devtool/standard.py
@@ -509,6 +509,11 @@ def _extract_source(srctree, keep_temp, devbranch, sync, config, basepath, works
             if not 'flag' in event:
                 if event['op'].startswith(('_append[', '_prepend[')):
                     extra_overrides.append(event['op'].split('[')[1].split(']')[0])
+        # We want to remove duplicate overrides. If a recipe had multiple
+        # SRC_URI_override += values it would cause mulitple instances of
+        # overrides. This doesn't play nicely with things like creating a
+        # branch for every instance of DEVTOOL_EXTRA_OVERRIDES.
+        extra_overrides = list(set(extra_overrides))
         if extra_overrides:
             logger.info('SRC_URI contains some conditional appends/prepends - will create branches to represent these')
 
@@ -769,9 +774,13 @@ def modify(args, config, basepath, workspace):
             check_commits = True
         else:
             if os.path.exists(os.path.join(srctree, '.git')):
-                # Check if it's a tree previously extracted by us
+                # Check if it's a tree previously extracted by us. This is done
+                # by ensuring that devtool-base and args.branch (devtool) exist.
+                # The check_commits logic will cause an exception if either one
+                # of these doesn't exist
                 try:
                     (stdout, _) = bb.process.run('git branch --contains devtool-base', cwd=srctree)
+                    bb.process.run('git rev-parse %s' % args.branch, cwd=srctree)
                 except bb.process.ExecutionError:
                     stdout = ''
                 if stdout:
diff --git a/poky/scripts/lib/wic/engine.py b/poky/scripts/lib/wic/engine.py
index 4662c665c..ea600d285 100644
--- a/poky/scripts/lib/wic/engine.py
+++ b/poky/scripts/lib/wic/engine.py
@@ -245,9 +245,16 @@ class Disk:
         self._ptable_format = None
 
         # find parted
-        self.paths = "/bin:/usr/bin:/usr/sbin:/sbin/"
+        # read paths from $PATH environment variable
+        # if it fails, use hardcoded paths
+        pathlist = "/bin:/usr/bin:/usr/sbin:/sbin/"
+        try:
+            self.paths = os.environ['PATH'] + ":" + pathlist
+        except KeyError:
+            self.paths = pathlist
+
         if native_sysroot:
-            for path in self.paths.split(':'):
+            for path in pathlist.split(':'):
                 self.paths = "%s%s:%s" % (native_sysroot, path, self.paths)
 
         self.parted = find_executable("parted", self.paths)
@@ -331,7 +338,7 @@ class Disk:
     def copy(self, src, pnum, path):
         """Copy partition image into wic image."""
         if self.partitions[pnum].fstype.startswith('ext'):
-            cmd = "echo -e 'cd {}\nwrite {} {}' | {} -w {}".\
+            cmd = "printf 'cd {}\nwrite {} {}' | {} -w {}".\
                       format(path, src, os.path.basename(src),
                              self.debugfs, self._get_part_image(pnum))
         else: # fat