poky: subtree update:c67f57c09e..c6bc20857c

Adrian Freihofer (2):
      oe-publish-sdk: fix layers init via ssh
      oe-publish-sdk: add --keep-orig option

Alexander Kanavin (68):
      meta-selftest: correct the virgl test for 5.8 kernels
      bison: upgrade 3.6.4 -> 3.7.1
      util-linux: upgrade 2.35.2 -> 2.36
      python3-numpy: upgrade 1.19.0 -> 1.19.1
      python3-setuptools: upgrade 49.3.1 -> 49.6.0
      rsync: upgrade 3.2.2 -> 3.2.3
      util-linux: merge .inc into .bb
      acpica: upgrade 20200528 -> 20200717
      asciidoc: upgrade 9.0.1 -> 9.0.2
      cryptodev: upgrade 1.10 -> 1.11
      diffoscope: upgrade 153 -> 156
      epiphany: upgrade 3.36.3 -> 3.36.4
      font-alias: upgrade 1.0.3 -> 1.0.4
      gtk+3: upgrade 3.24.21 -> 3.24.22
      libcheck: upgrade 0.15.0 -> 0.15.2
      libinput: upgrade 1.16.0 -> 1.16.1
      libpipeline: upgrade 1.5.2 -> 1.5.3
      libx11: upgrade 1.6.9 -> 1.6.11
      linux-firmware: upgrade 20200619 -> 20200721
      man-pages: upgrade 5.07 -> 5.08
      mc: upgrade 4.8.24 -> 4.8.25
      mesa: upgrade 20.1.4 -> 20.1.5
      piglit: upgrade to latest revision
      re2c: upgrade 2.0 -> 2.0.2
      sysstat: upgrade 12.2.2 -> 12.4.0
      vala: upgrade 0.48.7 -> 0.48.9
      bootchart2: update 0.14.8 -> 0.14.9
      harfbuzz: convert to meson, enable gobject introspection
      pango: update 1.44.7 -> 1.46.0
      boost: update 1.73.0 -> 1.74.0
      xev: update 1.2.3 -> 1.2.4
      wpebackend-fdo: update 1.6.1 -> 1.7.1
      gpgme: update 1.13.1 -> 1.14.0
      libpsl: update 0.21.0 -> 0.21.1.
      gettext: update 0.20.2 -> 0.21
      cmake: update 3.17.3 -> 3.18.1
      linux-firmware: update 20200721 -> 20200817
      meson: update 0.55.0 -> 0.55.1
      systemd-boot: bump version to 246.2
      json-glib: inherit upstream-version-is-even
      packagegroup-core-device-devel: remove
      oeqa/x32lib: rework to use readelf from the host
      oeqa/multilib: rework to use readelf from the host
      oeqa/multilib: un-skip the connman test
      poky.conf: do not install packagegroup-core-device-devel into qemu images
      glib-2.0: update 2.64.4 -> 2.64.5
      cmake: upgrade 3.18.1 -> 3.18.2
      libxcrypt: upgrade 4.4.16 -> 4.4.17
      debianutils: upgrade 4.11 -> 4.11.1
      enchant2: upgrade 2.2.8 -> 2.2.9
      harfbuzz: upgrade 2.7.1 -> 2.7.2
      libmpc: upgrade 1.1.0 -> 1.2.0
      librepo: upgrade 1.12.0 -> 1.12.1
      libuv: upgrade 1.38.1 -> 1.39.0
      msmtp: upgrade 1.8.11 -> 1.8.12
      ninja: upgrade 1.10.0 -> 1.10.1
      p11-kit: upgrade 0.23.20 -> 0.23.21
      pango: upgrade 1.46.0 -> 1.46.1
      re2c: upgrade 2.0.2 -> 2.0.3
      resolvconf: upgrade 1.82 -> 1.83
      stress-ng: upgrade 0.11.18 -> 0.11.19
      gnu-config: update to latest revision
      nasm: update 2.15.03 -> 2.15.05
      libva-utils: fix upstream version check
      gnupg: update 2.2.21 -> 2.2.22
      libx11: update 1.6.11 -> 1.6.12
      mesa: update 20.1.5 -> 20.1.6
      xserver-xorg: update 1.20.8 -> 1.20.9

Andrey Zhizhikin (1):
      insane: check for missing update-alternatives inherit

Anibal Limon (1):
      recipes-kernel: linux-firmware add qcom-venus-{5.2,5.4} packages

Aníbal Limón (1):
      recipes-graphics/xorg-xserver: Add patch to fix segfault when probe

Armin Kuster (2):
      bind: update to 9.11.22 ESV
      core-image-sato: qemumips use 512 mem

Bruce Ashfield (30):
      linux-yocto/5.4: update to v5.4.59
      linux-yocto/5.8: update to v5.8.2
      yocto-bsp: update to v5.4.56
      yocto-bsp: update to v5.4.58
      qemu: bump default reference kernel to v5.8
      linux-yocto/5.8: fix perf and virtio_scsi warnings
      linux-yocto-rt/5.8: fix lttng-modules build
      linux-yocto/5.8: selftests/bpf: Prevent runqslower from racing on building bpftool
      linux-yocto/5.8: disable CONFIG_NFS_DISABLE_UDP_SUPPORT
      poky: set preferred version for linux-yocto to be v5.8
      poky-tiny: set preferred version to 5.8
      poky: add preferred version for linux-yocto-rt
      linux-yocto/5.8: update to v5.8.3
      linux-yocto/5.4: update to v5.4.60
      kernel: config cleanups for 5.8+
      linux-yocto/5.4: update to v5.4.61
      linux-yocto/5.8: update to v5.8.4
      linux-yocto/5.8: disable IKHEADERS in default builds
      kernel-yocto: allow promotion of configuration warnings to errors
      kernel-yocto: checksum all modifications to available kernel fragments directories
      lttng-modules/devupstream: bump to latest 2.12 commits
      linux-yocto-dev: bump to v5.9+
      linux-yocto/5.8: update to v5.8.5
      kernel-devsrc: account for HOSTCC and HOSTCXX
      linux-yocto/config: netfilter: Enable nat for ipv4 and ipv6
      linux-yocto/5.8: update to v5.8.8
      linux-yocto/5.4: update to v5.4.64
      linux-yocto/config: configuration warning cleanup
      linux-yocto/5.8: update to v5.8.9
      linux-yocto/5.4: update to v5.4.65

Changhyeok Bae (2):
      iw: upgrade 5.4 -> 5.8
      iputils: upgrade s20190709 -> s20200821

Chris Laplante (12):
      bitbake: compat.py: remove file since it no longer actually implements anything
      bitbake: COW: formatting
      bitbake: COW: migrate test suite into tests/cow
      cve-update-db-native: add progress handler
      cve-check/cve-update-db-native: use lockfile to fix usage under multiconfig
      cve-update-db-native: use context manager for cve_f
      cve-check: avoid FileNotFoundError if no do_cve_check task has run
      bitbake: utils: process_profilelog: use context manager
      bitbake: utils: fix UnboundLocalError when _print_exception raises
      cve-update-db-native: be less magical about checking whether the cve-check class is enabled
      cve-update-db-native: move -journal checking into do_fetch
      cve-update-db-native: remove unused variable

Christophe GUIBOUT (1):
      initramfs-framework: support kernel cmdline with double quotes

Denys Dmytriyenko (2):
      weston: upgrade 8.0.0 -> 9.0.0
      cryptodev: bump 1 commit past 1.11 to fix 5.9-rc1+

Diego Sueiro (2):
      license_image.bbclass: Create symlink to the image license manifest dir
      license_image.bbclass: Fix symlink to the image license manifest dir creation

Douglas Royds (1):
      tcmode-default: Drop gcc-cross-initial, gcc-crosssdk-initial references

Frazer Clews (1):
      bitbake: lib: fix most undefined code picked up by pylint

Geoff Parker (1):
      systemd-serialgetty: Replace sed quoting using ' with " to allow var expansion

Jacob Kroon (1):
      gcc10: Don't default back to -fcommon

Jean-Francois Dagenais (1):
      bitbake: siggen: clean_basepath: remove recipe full path when virtual:xyz present

Jens Rehsack (1):
      lttng-modules: backport patches from 2.12.x to fix 5.4.64+ and 5.8.9+ builds

Joe Slater (1):
      pseudo: fix renaming to self

Jon Mason (4):
      cortex-m0plus.inc: change file permissions
      tune-cortexa55.inc: clean-up ARMv8.2a uses
      tune-cortexa57-cortexa53.inc: add CRC and set march
      tune-cortexa*: Cleanups

Joshua Watt (8):
      wic: Add 512 Byte alignment to --offset
      oeqa: runtime_tests: Extra GPG debugging
      oeqa: sdk: Capture stderr output
      oeqa: reproducible: Fix test not producing diffs
      diffoscope: upgrade 156 -> 158
      bitbake: bitbake: Add parsing torture test
      bitbake: cooker: Block SIGINT in worker processes
      sphinx: dev-manual: Clarify that virtual providers do not apply to runtime dependencies

Kai Kang (1):
      dhcpcd: 9.1.4 -> 9.2.0

Kevin Hao (1):
      meta-yocto-bsp: Bump to the v5.8 kernel

Khairul Rohaizzat Jamaluddin (1):
      wic/bootimg-efi: IMAGE_EFI_BOOT_FILES variable added to separate bootimg-efi and bootimg-partition

Khem Raj (24):
      gcc-cross-canadian: Install gcc/g++ wrappers for musl
      uninative: Upgrade to 2.9
      packagegroup-core-tools-profile: Disable lttng-modules for riscv64
      lttng-modules: Disable on riscv64
      kexec-tools: Fix build with -fno-common on ppc
      lttng-tools: Do not build for riscv64
      util-linux: Allow update alternatives for additional apps
      lttng-tools: lttng-ust works on riscv64
      json-glib: Backport a build fix with clang
      rpcbind: Use update-alternatives for rpcinfo
      go: Upgrade to 1.15 major release
      weston-init: Redefine weston service and add socket activation option
      musl: Upgrade to latest master
      libucontext: Recognise riscv32 architecture
      linuxloader.bbclass: Define riscv32 ldso for musl
      populate_sdk_ext: Do not assume local.conf will always exist
      weston: plane_add_prop() calls break musl atomic modesetting
      weston-init: Enable RDP screen share
      weston-init: Do not use fbdev backend
      weston-init: Select drm/fbdev backends for qemu machines
      oeqa/weston: Fix tests to run with systemd
      core-image-weston: Bump qemu memory to 512M
      go: Update to 1.15.2 minor release
      bind: Inherit update-alternatives

Mark Hatle (6):
      package_tar.bbclass: Sync to the other package_* classes
      kernel.bbclass: Remove do_install[prefunc] no longer needed
      buildhistory.bbclass: Rework to use read_subpackage_metadata
      kernel.bbclass: Move away from calling package_get_auto_pr
      package.bbclass: hash equivalency and pr service
      bitbake: process.py: Handle SystemExit exception to eliminate backtrace

Mark Morton (1):
      sphinx: test-manual code block, link, and format update

Martin Jansa (7):
      devtool: expand SRC_URI when guessing recipe update mode
      image-artifact-names: introduce new bbclass and move some variables into it
      kernel.bbclass: use bash variables like imageType, base_name without {}
      kernel.bbclass: eliminate (initramfs_)symlink_name variables
      kernel.bbclass: use camelCase notation for bash variables in do_deploy
      *-initramfs: don't use .rootfs IMAGE_NAME_SUFFIX
      bitbake.conf: use ${TCMODE}-${TCLIBC} directory for CACHE

Matt Madison (1):
      image.bbclass: fix REPRODUCIBLE_TIMESTAMP_ROOTFS reference

Michael Gloff (2):
      sysvinit rc: Use PSPLASH_FIFO_DIR for progress fifo
      sysvinit: Remove ${B} assignment

Michael Tretter (1):
      devtool: deploy-target: Fix size calculation for hard links

Ming Liu (2):
      systemd: split systemd specific udev rules into its own package
      libubootenv: inherit uboot-config

Mingli Yu (3):
      qemu: always define unknown_lock_type
      qemu: override DEBUG_BUILD
      bison: remove the parallel build patch

Naveen Saini (1):
      lib/oe/recipeutils.py: add support for BBFILES_DYNAMIC

Nicolas Dechesne (73):
      linux-libc-headers: kernel headers are installed in STAGING_KERNEL_BUILDDIR
      bitbake: sphinx: add initial build infrastructure
      bitbake: sphinx: initial sphinx support
      bitbake: sphinx: bitbake-user-manual: use builtin sphinx glossary
      bitbake: sphinx: switch to readthedocs theme
      bitbake: sphinx: override theme CSS
      bitbake: sphinx: fixup for links
      bitbake: sphinx: fix links inside notes
      bitbake: sphinx: fixes all remaining warnings
      bitbake: sphinx: Makefile.sphinx: add clean and publish targets
      bitbake: sphinx: tweak html output a bit
      bitbake: sphinx: add SPDX headers
      bitbake: sphinx: index: move the boilerplate at the end of the page
      bitbake: sphinx: conf: enable extlinks extension
      bitbake: sphinx: add releases page
      bitbake: sphinx: bitbake-user-manual: insert additional blank line after title
      bitbake: sphinx: last manual round of fixes/improvements
      bitbake: sphinx: update style for important, caution and warnings
      bitbake: sphinx: remove leading '/'
      bitbake: sphinx: theme_override: properly set font for verbatim text
      bitbake: bitbake-user-manual: fix bad links
      sphinx: add initial build infrastructure
      sphinx: initial sphinx support
      sphinx: ref-variables: use builtin sphinx glossary
      sphinx: overview-manual: add figures
      sphinx: switch to readthedocs theme
      sphinx: Add SPDX license headers
      sphinx: add CSS theme override
      sphinx: bsp-guide: add figures
      sphinx: add Yocto project logo
      sphinx: conf: update copyright
      sphinx: conf: add substitutions/global variables
      sphinx: add boilerplate file
      sphinx: add boilerplate to manuals
      sphinx: ref-manual: add revision history table
      sphinx: add a general index
      sphinx: conf.py: enable sphinx.ext.autosectionlabel
      sphinx: ref-manual: use builtin glossary for the Terms section
      sphinx: fix internal links
      sphinx: ref-manual: fix typo
      sphinx: fix custom term links
      sphinx: manual updates for some links
      sphinx: dev-manual add figures
      sphinx: kernel-dev: add figures
      sphinx: profile-manual: add figures
      sphinx: fix up bold text for informalexample container
      sphinx: ref-manual: add figures
      sphinx: sdk-manual: add figures
      sphinx: test-manual: add figures
      sphinx: toaster-manual: add figures
      sphinx: add links for Yocto project website
      sphinx: fix links when the link text should be displayed
      sphinx: add links to terms in the BitBake glossary
      sphinx: add links to section in the Bitbake manual
      sphinx: setup extlink for docs.yoctoproject.org
      sphinx: enable intersphinx extension
      sphinx: insert blank below between title and toc
      sphinx: fix up terms related to kernel-fitimage
      sphinx: conf: a few rendering tweaks
      sphinx: makefile: add publish target
      sphinx: conf: include CSS/JS files, the proper way
      sphinx: convert 'what I wish I'd known'
      sphinx: convert 'transitioning to a custom environment'
      sphinx: ref-manual: fix heading for oe-init-build-env
      sphinx: brief-yoctoprojectqs: fix up all remaining rendering issues
      sphinx: Makefile.sphinx improvements
      sphinx: convert bsp-guide
      sphinx: remove leading '/'
      sphinx: update style for important, caution and warnings
      sphinx: profile-manual: convert profile-manual
      sphinx: theme_override: properly set font for verbatim text
      sphinx: theme_override: add tying-it-together admonition
      sphinx: conf: exclude adt-manual/*.rst

Oleksandr Kravchuk (1):
      ell: update to 0.33

Ovidiu Panait (1):
      libxml2: Fix CVE-2020-24977

Peter A. Bigot (2):
      bluez5: fix builds that require ell support
      timezone: include leap second data in tzdata-core

Peter Bergin (1):
      systemd: avoid failing if no udev rules provided

Pierre-Jean Texier (2):
      libubootenv: upgrade 0.3 -> 0.3.1
      diffoscope: upgrade 158 -> 160

Quentin Schulz (16):
      sphinx: brief-yoctoprojectqs: remove redundant welcome
      sphinx: brief-yoctoprojectqs: fix ambiguous note for cyclone5 example
      sphinx: brief-yoctoprojectqs: add missing boilerplate
      sphinx: overview-manual: add link to AUH how-to section
      sphinx: overview-manual: fix bitbake basic explanation
      sphinx: brief-yoctoprojectqs: add note on branch consistency between layers
      sphinx: what-i-wish-id-known: update "don't be fooled by doc search results"
      sphinx: overview-manual: remove highlight in bold section
      sphinx: replace special quotes with single and double quotes
      sphinx: fix incorrect indentations
      sphinx: brief-yoctoprojectqs: put other distros note after Ubuntu-specific packages
      sphinx: fix a few typos or missing/too many words
      sphinx: "highlight" some variables, tasks or files
      sphinx: fix or add missing links and remove mention of Eclipse workflow
      ref-manual: examples: hello-autotools: upgrade to 2.10
      ref-manual: examples: libxpm: add relative path to .inc

Rahul Kumar (1):
      systemd-serialgetty: Fix sed expression quoting

Rasmus Villemoes (1):
      kernel.bbclass: run do_symlink_kernsrc before do_patch

Richard Purdie (74):
      nativesdk-sdk-provides-dummy: Add /bin/sh
      bitbake: fetch2/wget: Remove buffering parameter
      bitbake: cooker: Ensure parse_quit thread is closed down
      bitbake: cooker: Explictly shut down the sync thread
      bitbake: fetch2: Drop cups.org from wget status checks
      bitbake: build/msg: Cleanup verbose option handling
      bitbake: cooker/cookerdata/main: Improve loglevel handling
      bitbake: cookerdata: Ensure UI options are updated to the server
      bitbake: cooker/cookerdata: Ensure UI event log is updated from commandline
      bitbake: cooker: Defer configuration init to after UI connection
      bitbake: server/process: Move the socket code to server process only
      bitbake: main/server/process: Drop configuration object passing
      bitbake: cooker: Ensure BB_ORIGENV is updated by changes to configuration.env
      bitbake: server/process: Log extra threads at exit
      bitbake: server/process: Add bitbake-server and exec() a new server process
      bitbake: runqueue: Don't use sys.argv
      bitbake: cooker: Ensure cooker's enviroment is updated on updateConfig
      connman-gnome/matchbox-desktop: Remove file:// globbing
      selftest/recipetool: Drop globbing SRC_URI test, no longer supported
      local.conf.sample: Document memory resident bitbake
      bitbake: fetch2: Drop globbing supprt in file:// SRC_URIs
      bitbake: server/process: Use sys.executable for bitbake-server
      bitbake: process: Avoid bb.utils.timeout
      bitbake: utils: Drop broken timeout function
      bitbake: server/process: Fix typo in code causing tracebacks
      oeqa/selftest: Apply patch to fix cpio build with -fno-common
      runqemu: Show an error for conflicting graphics options
      lttng: Move platform logic to dedicated inc file
      patchelf: upgrade 0.11 -> 0.12
      build-appliance/packagegroup-core-base-utils: Replace dhcp-client/dhcp-server with dhcpcd/kea
      selftest/prservice: Improve test failure message
      iputils: Adapt ${PN}-tftpd package dependency to PACKAGECONFIG
      bitbake: process/knotty: Improve early exception handling
      bitbake: cooker/cookerdata: Use BBHandledException, not sys.exit()
      bitbake: cookerdata: Fix exception raise statements
      bitbake: process: Avoid printing binary strings for leftover processes
      bitbake: server/process: Ensure logging is flushed
      bitbake: server/process: Don't show tracebacks if the lockfile is removed
      bitbake: cooker: Ensure parser replacement calls parser final_cleanup
      bitbake: cooker: Assign a name to the sync thread to aid debugging
      bitbake: server/process: Ensure we don't keep looping if some other server is started
      bitbake: server/process: Prefix the log data with pid/time information
      bitbake: server/process: Note when commands complete in logs
      bitbake: cooker: Ensure parser is cleaned up
      runqemu: Add a hook to allow it to renice
      bitbake: cooker: Avoid parser deadlocks
      bitbake: cooker: Ensure parser worker signal handlers are default
      selftest/signing: Ensure build path relocation is safe
      oeqa/concurrencytest: Improve builddir path manipulations
      bitbake: cooker/command: Fix disconnection handling
      bitbake: tinfoil: Ensure sockets don't leak even when exceptions occur
      bitbake: tests/fetch: Move away from problematic freedesktop.org urls
      bitbake: sphinx: Enhance the sphinx experience/nagivation with:
      bitbake: sphinx: theme_override: Use bold for emphasis text
      Revert "qemu: always define unknown_lock_type"
      Revert "core-image-sato: qemumips use 512 mem"
      sphinx: Organize top level docs
      sphinx: releases.rst: Add index/links to docs for previous releases
      sphinx: boilerplate.rst: Drop versions notes as we have better navigation now
      sphinx: boilerplate.rst: Sphinx puts the copyright elsewhere
      sphinx: history: Move revision history to its own section
      sphinx: manuals: Move boilerplate after toctree
      sphinx: Add support for multiple docs version
      sphinx: index.rst: Fix links
      sphinx: ref-system-requirements: Improve formatting of the notes sections, merging them
      sphinx: ref-manual links fixes and many other cleanups to import
      sphinx: dev-manual: Various URL, code block and other fixes to imported data
      sphinx: sdk-manual: Various URL, code block and other fixes to imported data
      sphinx: kernel-dev: Various URL, code block and other fixes to imported data
      sphinx: theme_override: Use bold for emphasis text
      sphinx: ref-tasks: Add populate_sdk_ext task definition
      sphinx: ref-manual/migration: Split each release into its own file
      sphinx: overview-manual: Various URL, code block and other fixes to imported data
      build-appliance-image: Update to master head revision

Robert Yang (3):
      bitbake: cooker.py: Save prioritized BBFILES to BBFILES_PRIORITIZED
      bitbake: utils.py: get_file_layer(): Exit the loop when file is matched
      bitbake: utils.py: get_file_layer(): Improve performance

Ross Burton (25):
      package.bbclass: explode the RPROVIDES so we don't think the versions are provides
      elfutils: silence a new QA warning
      insane: improve gnu-hash-style warning
      gdk-pixbuf: add tests PACKAGECONFIG
      debianutils: change SRC_URI to use snapshot.debian.org
      insane: only load real files as ELF
      autoconf: consolidate SRC_URI
      autoconf: consolidate DEPENDS
      kea: no need to depend on kea-native
      kea: don't use PACKAGECONFIG inappropriately
      kea: bump to 1.7.10
      help2man: rewrite recipe
      local.conf.sample.extended: remove help2man reference
      curl: add vendors to CVE_PRODUCT to exclude false positives
      harfbuzz: update patch status
      harfbuzz: fix a build race around hb-version.h
      cmake: whitelist CVE-2016-10642
      ncurses: remove config.cache
      qemu: fix CVE-2020-14364
      cve-update-db-native: remove unused import
      cve-update-db-native: add more logging when fetching
      cve-update-db-native: use fetch task
      alsa-plugins: improve .la removal
      sato-screenshot: improve .la removal
      buildhistory-diff: use BUILDDIR to know where buildhistory is

Saul Wold (1):
      gnupg: uprev 2.2.22 -> 2.2.23

Stacy Gaikovaia (2):
      bison: uprev from 3.7.1 to 3.7.2
      valgrind: fix memcheck vgtests remove fullpath-after flags

Steve Sakoman (1):
      xinput-calibrator: change SRC_URI to branch with libinput support

Sumit Garg (1):
      insane: fix gnu-hash-style check

TeohJayShen (1):
      oeqa/runtime: add test for matchbox-terminal

Tim Orling (1):
      sphinx: toaster-manual: fix vars, links, code blocks

Vijai Kumar K (2):
      image_types_wic: Add ASSUME_PROVIDED to WICVARS
      wic: misc: Add /bin to the list of searchpaths

Yanfei Xu (1):
      kernel-yocto: only replace leading -I in include paths

Yi Zhao (1):
      glib-networking: add ptest

Zhixiong Chi (1):
      gnutls: CVE-2020-24659

akuster (8):
      log4cplus: move meta-oe pkg to core
      kea: Move from meta-networking
      maintainers.inc: Add me as kea & log4plus maintainer.
      dhcpcd: Move from meta-network as OE-Core needs a client
      maintainers.inc: Add me as dhcpcd maintainer
      dhcp: remove from core
      bind: Add 9.16.x
      bind: 9.11 remove

hongxu (1):
      sysstat: fix installed-vs-shipped QA Issue in systemd

zangrc (4):
      libcap:upgrade 2.42 -> 2.43
      libcap-ng:upgrade 0.7.10 -> 0.7.11
      libgpg-error:upgrade 1.38 -> 1.39
      at-spi2-core:upgrade 2.36.0 -> 2.36.1

Signed-off-by: Andrew Geissler <geissonator@yahoo.com>
Change-Id: I5542f5eea751a2641342e945725fd687cd74bebe

1 files changed, 159 insertions, 100 deletions

diff --git a/poky/bitbake/lib/bb/server/process.py b/poky/bitbake/lib/bb/server/process.py
index 65e1eab52..c7cb34f0c 100644
--- a/poky/bitbake/lib/bb/server/process.py
+++ b/poky/bitbake/lib/bb/server/process.py
@@ -25,6 +25,7 @@ import subprocess
 import errno
 import re
 import datetime
+import pickle
 import bb.server.xmlrpcserver
 from bb import daemonize
 from multiprocessing import queues
@@ -34,11 +35,15 @@ logger = logging.getLogger('BitBake')
 class ProcessTimeout(SystemExit):
     pass
 
+def serverlog(msg):
+    print(str(os.getpid()) + " " +  datetime.datetime.now().strftime('%H:%M:%S.%f') + " " + msg)
+    sys.stdout.flush()
+
 class ProcessServer():
     profile_filename = "profile.log"
     profile_processed_filename = "profile.log.processed"
 
-    def __init__(self, lock, sock, sockname, server_timeout, xmlrpcinterface):
+    def __init__(self, lock, lockname, sock, sockname, server_timeout, xmlrpcinterface):
         self.command_channel = False
         self.command_channel_reply = False
         self.quit = False
@@ -54,10 +59,12 @@ class ProcessServer():
         self._idlefuns = {}
 
         self.bitbake_lock = lock
+        self.bitbake_lock_name = lockname
         self.sock = sock
         self.sockname = sockname
 
         self.server_timeout = server_timeout
+        self.timeout = self.server_timeout
         self.xmlrpcinterface = xmlrpcinterface
 
     def register_idle_function(self, function, data):
@@ -70,22 +77,7 @@ class ProcessServer():
         if self.xmlrpcinterface[0]:
             self.xmlrpc = bb.server.xmlrpcserver.BitBakeXMLRPCServer(self.xmlrpcinterface, self.cooker, self)
 
-            print("Bitbake XMLRPC server address: %s, server port: %s" % (self.xmlrpc.host, self.xmlrpc.port))
-
-        heartbeat_event = self.cooker.data.getVar('BB_HEARTBEAT_EVENT')
-        if heartbeat_event:
-            try:
-                self.heartbeat_seconds = float(heartbeat_event)
-            except:
-                bb.warn('Ignoring invalid BB_HEARTBEAT_EVENT=%s, must be a float specifying seconds.' % heartbeat_event)
-
-        self.timeout = self.server_timeout or self.cooker.data.getVar('BB_SERVER_TIMEOUT')
-        try:
-            if self.timeout:
-                self.timeout = float(self.timeout)
-        except:
-            bb.warn('Ignoring invalid BB_SERVER_TIMEOUT=%s, must be a float specifying seconds.' % self.timeout)
-
+            serverlog("Bitbake XMLRPC server address: %s, server port: %s" % (self.xmlrpc.host, self.xmlrpc.port))
 
         try:
             self.bitbake_lock.seek(0)
@@ -96,7 +88,7 @@ class ProcessServer():
                 self.bitbake_lock.write("%s\n" % (os.getpid()))
             self.bitbake_lock.flush()
         except Exception as e:
-            print("Error writing to lock file: %s" % str(e))
+            serverlog("Error writing to lock file: %s" % str(e))
             pass
 
         if self.cooker.configuration.profile:
@@ -110,7 +102,7 @@ class ProcessServer():
 
             prof.dump_stats("profile.log")
             bb.utils.process_profilelog("profile.log")
-            print("Raw profiling information saved to profile.log and processed statistics to profile.log.processed")
+            serverlog("Raw profiling information saved to profile.log and processed statistics to profile.log.processed")
 
         else:
             ret = self.main()
@@ -129,10 +121,11 @@ class ProcessServer():
         fds = [self.sock]
         if self.xmlrpc:
             fds.append(self.xmlrpc)
-        print("Entering server connection loop")
+        seendata = False
+        serverlog("Entering server connection loop")
 
         def disconnect_client(self, fds):
-            print("Disconnecting Client")
+            serverlog("Disconnecting Client")
             if self.controllersock:
                 fds.remove(self.controllersock)
                 self.controllersock.close()
@@ -150,12 +143,12 @@ class ProcessServer():
                 self.haveui = False
             ready = select.select(fds,[],[],0)[0]
             if newconnections:
-                print("Starting new client")
+                serverlog("Starting new client")
                 conn = newconnections.pop(-1)
                 fds.append(conn)
                 self.controllersock = conn
             elif self.timeout is None and not ready:
-                print("No timeout, exiting.")
+                serverlog("No timeout, exiting.")
                 self.quit = True
 
         self.lastui = time.time()
@@ -164,17 +157,17 @@ class ProcessServer():
                 while select.select([self.sock],[],[],0)[0]:
                     controllersock, address = self.sock.accept()
                     if self.controllersock:
-                        print("Queuing %s (%s)" % (str(ready), str(newconnections)))
+                        serverlog("Queuing %s (%s)" % (str(ready), str(newconnections)))
                         newconnections.append(controllersock)
                     else:
-                        print("Accepting %s (%s)" % (str(ready), str(newconnections)))
+                        serverlog("Accepting %s (%s)" % (str(ready), str(newconnections)))
                         self.controllersock = controllersock
                         fds.append(controllersock)
             if self.controllersock in ready:
                 try:
-                    print("Processing Client")
+                    serverlog("Processing Client")
                     ui_fds = recvfds(self.controllersock, 3)
-                    print("Connecting Client")
+                    serverlog("Connecting Client")
 
                     # Where to write events to
                     writer = ConnectionWriter(ui_fds[0])
@@ -198,14 +191,14 @@ class ProcessServer():
 
             if not self.timeout == -1.0 and not self.haveui and self.timeout and \
                     (self.lastui + self.timeout) < time.time():
-                print("Server timeout, exiting.")
+                serverlog("Server timeout, exiting.")
                 self.quit = True
 
             # If we don't see a UI connection within maxuiwait, its unlikely we're going to see
             # one. We have had issue with processes hanging indefinitely so timing out UI-less
             # servers is useful.
             if not self.hadanyui and not self.xmlrpc and not self.timeout and (self.lastui + self.maxuiwait) < time.time():
-                print("No UI connection within max timeout, exiting to avoid infinite loop.")
+                serverlog("No UI connection within max timeout, exiting to avoid infinite loop.")
                 self.quit = True
 
             if self.command_channel in ready:
@@ -220,17 +213,37 @@ class ProcessServer():
                     self.quit = True
                     continue
                 try:
-                    print("Running command %s" % command)
+                    serverlog("Running command %s" % command)
                     self.command_channel_reply.send(self.cooker.command.runCommand(command))
+                    serverlog("Command Completed")
                 except Exception as e:
                    logger.exception('Exception in server main event loop running command %s (%s)' % (command, str(e)))
 
             if self.xmlrpc in ready:
                 self.xmlrpc.handle_requests()
 
+            if not seendata and hasattr(self.cooker, "data"):
+                heartbeat_event = self.cooker.data.getVar('BB_HEARTBEAT_EVENT')
+                if heartbeat_event:
+                    try:
+                        self.heartbeat_seconds = float(heartbeat_event)
+                    except:
+                        bb.warn('Ignoring invalid BB_HEARTBEAT_EVENT=%s, must be a float specifying seconds.' % heartbeat_event)
+
+                self.timeout = self.server_timeout or self.cooker.data.getVar('BB_SERVER_TIMEOUT')
+                try:
+                    if self.timeout:
+                        self.timeout = float(self.timeout)
+                except:
+                    bb.warn('Ignoring invalid BB_SERVER_TIMEOUT=%s, must be a float specifying seconds.' % self.timeout)
+                seendata = True
+
             ready = self.idle_commands(.1, fds)
 
-        print("Exiting")
+        if len(threading.enumerate()) != 1:
+            serverlog("More than one thread left?: " + str(threading.enumerate()))
+
+        serverlog("Exiting")
         # Remove the socket file so we don't get any more connections to avoid races
         try:
             os.unlink(self.sockname)
@@ -253,39 +266,67 @@ class ProcessServer():
 
         # Finally release the lockfile but warn about other processes holding it open
         lock = self.bitbake_lock
-        lockfile = lock.name
+        lockfile = self.bitbake_lock_name
+
+        def get_lock_contents(lockfile):
+            try:
+                with open(lockfile, "r") as f:
+                    return f.readlines()
+            except FileNotFoundError:
+                return None
+
+        lockcontents = get_lock_contents(lockfile)
+        serverlog("Original lockfile contents: " + str(lockcontents))
+
         lock.close()
         lock = None
 
         while not lock:
-            with bb.utils.timeout(3):
-                lock = bb.utils.lockfile(lockfile, shared=False, retry=False, block=True)
-                if lock:
-                    # We hold the lock so we can remove the file (hide stale pid data)
-                    # via unlockfile.
-                    bb.utils.unlockfile(lock)
-                    return
-
+            i = 0
+            lock = None
+            while not lock and i < 30:
+                lock = bb.utils.lockfile(lockfile, shared=False, retry=False, block=False)
                 if not lock:
-                    # Some systems may not have lsof available
-                    procs = None
+                    newlockcontents = get_lock_contents(lockfile)
+                    if newlockcontents != lockcontents:
+                        # A new server was started, the lockfile contents changed, we can exit
+                        serverlog("Lockfile now contains different contents, exiting: " + str(newlockcontents))
+                        return
+                    time.sleep(0.1)
+                i += 1
+            if lock:
+                # We hold the lock so we can remove the file (hide stale pid data)
+                # via unlockfile.
+                bb.utils.unlockfile(lock)
+                serverlog("Exiting as we could obtain the lock")
+                return
+
+            if not lock:
+                # Some systems may not have lsof available
+                procs = None
+                try:
+                    procs = subprocess.check_output(["lsof", '-w', lockfile], stderr=subprocess.STDOUT)
+                except subprocess.CalledProcessError:
+                    # File was deleted?
+                    continue
+                except OSError as e:
+                    if e.errno != errno.ENOENT:
+                        raise
+                if procs is None:
+                    # Fall back to fuser if lsof is unavailable
                     try:
-                        procs = subprocess.check_output(["lsof", '-w', lockfile], stderr=subprocess.STDOUT)
+                        procs = subprocess.check_output(["fuser", '-v', lockfile], stderr=subprocess.STDOUT)
+                    except subprocess.CalledProcessError:
+                        # File was deleted?
+                        continue
                     except OSError as e:
                         if e.errno != errno.ENOENT:
                             raise
-                    if procs is None:
-                        # Fall back to fuser if lsof is unavailable
-                        try:
-                            procs = subprocess.check_output(["fuser", '-v', lockfile], stderr=subprocess.STDOUT)
-                        except OSError as e:
-                            if e.errno != errno.ENOENT:
-                                raise
-
-                    msg = "Delaying shutdown due to active processes which appear to be holding bitbake.lock"
-                    if procs:
-                        msg += ":\n%s" % str(procs)
-                    print(msg)
+
+                msg = "Delaying shutdown due to active processes which appear to be holding bitbake.lock"
+                if procs:
+                    msg += ":\n%s" % str(procs.decode("utf-8"))
+                serverlog(msg)
 
     def idle_commands(self, delay, fds=None):
         nextsleep = delay
@@ -323,8 +364,9 @@ class ProcessServer():
             self.next_heartbeat += self.heartbeat_seconds
             if self.next_heartbeat <= now:
                 self.next_heartbeat = now + self.heartbeat_seconds
-            heartbeat = bb.event.HeartbeatEvent(now)
-            bb.event.fire(heartbeat, self.cooker.data)
+            if hasattr(self.cooker, "data"):
+                heartbeat = bb.event.HeartbeatEvent(now)
+                bb.event.fire(heartbeat, self.cooker.data)
         if nextsleep and now + nextsleep > self.next_heartbeat:
             # Shorten timeout so that we we wake up in time for
             # the heartbeat.
@@ -353,7 +395,12 @@ class ServerCommunicator():
             logger.info("No reply from server in 30s")
             if not self.recv.poll(30):
                 raise ProcessTimeout("Timeout while waiting for a reply from the bitbake server (60s)")
-        return self.recv.get()
+        ret, exc = self.recv.get()
+        # Should probably turn all exceptions in exc back into exceptions?
+        # For now, at least handle BBHandledException
+        if exc and ("BBHandledException" in exc or "SystemExit" in exc):
+            raise bb.BBHandledException()
+        return ret, exc
 
     def updateFeatureSet(self, featureset):
         _, error = self.runCommand(["setFeatures", featureset])
@@ -386,39 +433,26 @@ class BitBakeProcessServerConnection(object):
         self.connection.recv.close()
         return
 
+start_log_format = '--- Starting bitbake server pid %s at %s ---'
+start_log_datetime_format = '%Y-%m-%d %H:%M:%S.%f'
+
 class BitBakeServer(object):
-    start_log_format = '--- Starting bitbake server pid %s at %s ---'
-    start_log_datetime_format = '%Y-%m-%d %H:%M:%S.%f'
 
-    def __init__(self, lock, sockname, configuration, featureset):
+    def __init__(self, lock, sockname, featureset, server_timeout, xmlrpcinterface):
 
-        self.configuration = configuration
+        self.server_timeout = server_timeout
+        self.xmlrpcinterface = xmlrpcinterface
         self.featureset = featureset
         self.sockname = sockname
         self.bitbake_lock = lock
         self.readypipe, self.readypipein = os.pipe()
 
-        # Create server control socket
-        if os.path.exists(sockname):
-            os.unlink(sockname)
-
         # Place the log in the builddirectory alongside the lock file
         logfile = os.path.join(os.path.dirname(self.bitbake_lock.name), "bitbake-cookerdaemon.log")
+        self.logfile = logfile
 
-        self.sock = socket.socket(socket.AF_UNIX, socket.SOCK_STREAM)
-        # AF_UNIX has path length issues so chdir here to workaround
-        cwd = os.getcwd()
-        try:
-            os.chdir(os.path.dirname(sockname))
-            self.sock.bind(os.path.basename(sockname))
-        finally:
-            os.chdir(cwd)
-        self.sock.listen(1)
-
-        os.set_inheritable(self.sock.fileno(), True)
         startdatetime = datetime.datetime.now()
         bb.daemonize.createDaemon(self._startServer, logfile)
-        self.sock.close()
         self.bitbake_lock.close()
         os.close(self.readypipein)
 
@@ -437,7 +471,7 @@ class BitBakeServer(object):
             ready.close()
             bb.error("Unable to start bitbake server (%s)" % str(r))
             if os.path.exists(logfile):
-                logstart_re = re.compile(self.start_log_format % ('([0-9]+)', '([0-9-]+ [0-9:.]+)'))
+                logstart_re = re.compile(start_log_format % ('([0-9]+)', '([0-9-]+ [0-9:.]+)'))
                 started = False
                 lines = []
                 lastlines = []
@@ -447,9 +481,9 @@ class BitBakeServer(object):
                             lines.append(line)
                         else:
                             lastlines.append(line)
-                            res = logstart_re.match(line.rstrip())
+                            res = logstart_re.search(line.rstrip())
                             if res:
-                                ldatetime = datetime.datetime.strptime(res.group(2), self.start_log_datetime_format)
+                                ldatetime = datetime.datetime.strptime(res.group(2), start_log_datetime_format)
                                 if ldatetime >= startdatetime:
                                     started = True
                                     lines.append(line)
@@ -470,28 +504,53 @@ class BitBakeServer(object):
         ready.close()
 
     def _startServer(self):
-        print(self.start_log_format % (os.getpid(), datetime.datetime.now().strftime(self.start_log_datetime_format)))
-        sys.stdout.flush()
+        os.close(self.readypipe)
+        os.set_inheritable(self.bitbake_lock.fileno(), True)
+        os.set_inheritable(self.readypipein, True)
+        serverscript = os.path.realpath(os.path.dirname(__file__) + "/../../../bin/bitbake-server")
+        os.execl(sys.executable, "bitbake-server", serverscript, "decafbad", str(self.bitbake_lock.fileno()), str(self.readypipein), self.logfile, self.bitbake_lock.name, self.sockname,  str(self.server_timeout), str(self.xmlrpcinterface[0]), str(self.xmlrpcinterface[1]))
 
-        try:
-            server = ProcessServer(self.bitbake_lock, self.sock, self.sockname, self.configuration.server_timeout, self.configuration.xmlrpcinterface)
-            os.close(self.readypipe)
-            writer = ConnectionWriter(self.readypipein)
-            try:
-                self.cooker = bb.cooker.BBCooker(self.configuration, self.featureset, server.register_idle_function)
-            except bb.BBHandledException:
-                return None
-            writer.send("r")
-            writer.close()
-            server.cooker = self.cooker
-            print("Started bitbake server pid %d" % os.getpid())
-            sys.stdout.flush()
+def execServer(lockfd, readypipeinfd, lockname, sockname, server_timeout, xmlrpcinterface):
+
+    import bb.cookerdata
+    import bb.cooker
+
+    serverlog(start_log_format % (os.getpid(), datetime.datetime.now().strftime(start_log_datetime_format)))
+
+    try:
+        bitbake_lock = os.fdopen(lockfd, "w")
 
-            server.run()
+        # Create server control socket
+        if os.path.exists(sockname):
+            os.unlink(sockname)
+
+        sock = socket.socket(socket.AF_UNIX, socket.SOCK_STREAM)
+        # AF_UNIX has path length issues so chdir here to workaround
+        cwd = os.getcwd()
+        try:
+            os.chdir(os.path.dirname(sockname))
+            sock.bind(os.path.basename(sockname))
         finally:
-            # Flush any ,essages/errors to the logfile before exit
-            sys.stdout.flush()
-            sys.stderr.flush()
+            os.chdir(cwd)
+        sock.listen(1)
+
+        server = ProcessServer(bitbake_lock, lockname, sock, sockname, server_timeout, xmlrpcinterface)
+        writer = ConnectionWriter(readypipeinfd)
+        try:
+            featureset = []
+            cooker = bb.cooker.BBCooker(featureset, server.register_idle_function)
+        except bb.BBHandledException:
+            return None
+        writer.send("r")
+        writer.close()
+        server.cooker = cooker
+        serverlog("Started bitbake server pid %d" % os.getpid())
+
+        server.run()
+    finally:
+        # Flush any ,essages/errors to the logfile before exit
+        sys.stdout.flush()
+        sys.stderr.flush()
 
 def connectProcessServer(sockname, featureset):
     # Connect to socket