subtree updates

poky: 94dfcaff64..359e1cb62f:
  Alexander Kanavin (76):
        tcf-agent: fetching over git:// no longer works
        lighttpd: convert from autotools to meson
        libxcrypt: upgrade 4.4.23 -> 4.4.25
        python3-cython: upgrade 0.29.23 -> 0.29.24
        python3-numpy: upgrade 1.21.0 -> 1.21.2
        systemd: upgrade 249.1 -> 249.3
        xeyes: upgrade 1.1.2 -> 1.2.0
        btrfs-tools: update 5.13 -> 5.13.1
        diffutils: update 3.7 -> 3.8
        mc: update 4.8.26 - > 4.8.27
        libsdl2: update 2.0.14 -> 2.0.16
        vulkan-samples: update to latest revision
        pulseaudio: update 14.2 -> 15.0
        libjitterentropy: update 3.0.2 -> 3.1.0
        usbutils: upgrade 013 -> 014
        inetutils: upgrade 2.0 -> 2.1
        mobile-broadband-provider-info: upgrade 20201225 -> 20210805
        glib-networking: upgrade 2.68.1 -> 2.68.2
        e2fsprogs: upgrade 1.46.2 -> 1.46.4
        help2man: upgrade 1.48.3 -> 1.48.4
        libedit: upgrade 20210522-3.1 -> 20210714-3.1
        log4cplus: upgrade 2.0.6 -> 2.0.7
        mtools: upgrade 4.0.34 -> 4.0.35
        patchelf: upgrade 0.12 -> 0.13
        pkgconf: upgrade 1.7.4 -> 1.8.0
        python3-git: upgrade 3.1.18 -> 3.1.20
        python3-pip: upgrade 21.2.1 -> 21.2.4
        python3-pygments: upgrade 2.9.0 -> 2.10.0
        python3-setuptools: upgrade 57.1.0 -> 57.4.0
        squashfs-tools: upgrade 4.4 -> 4.5
        acpica: upgrade 20210331 -> 20210730
        libidn2: upgrade 2.3.1 -> 2.3.2
        stress-ng: upgrade 0.12.12 -> 0.13.00
        sudo: upgrade 1.9.7p1 -> 1.9.7p2
        epiphany: upgrade 40.2 -> 40.3
        libgudev: upgrade 236 -> 237
        libjpeg-turbo: upgrade 2.1.0 -> 2.1.1
        libepoxy: upgrade 1.5.8 -> 1.5.9
        pango: upgrade 1.48.7 -> 1.48.9
        mesa: upgrade 21.1.5 -> 21.2.1
        libinput: upgrade 1.18.0 -> 1.18.1
        libxfont2: upgrade 2.0.4 -> 2.0.5
        libxft: upgrade 2.3.3 -> 2.3.4
        xserver-xorg: upgrade 1.20.12 -> 1.20.13
        linux-firmware: upgrade 20210511 -> 20210818
        wireless-regdb: upgrade 2021.04.21 -> 2021.07.14
        libwebp: upgrade 1.2.0 -> 1.2.1
        webkitgtk: upgrade 2.32.2 -> 2.32.3
        boost: upgrade 1.76.0 -> 1.77.0
        diffoscope: upgrade 179 -> 181
        enchant2: upgrade 2.3.0 -> 2.3.1
        re2c: upgrade 2.1.1 -> 2.2
        rng-tools: upgrade 6.13 -> 6.14
        kea: backport a patch to fix build errors exposed by latest update batch
        qemu: add a hint on how to enable CPU render nodes when a suitable GPU is absent
        mc: fix reproducibility
        libjitterentropy: remove contaminated hashequiv entry
        binutils: drop target flex/bison from build dependencies
        gnu-efi: update 3.0.13 -> 3.0.14
        glib-2.0: upgrade 2.68.3 -> 2.68.4
        util-linux: upgrade 2.37.1 -> 2.37.2
        ccache: upgrade 4.3 -> 4.4
        git: upgrade 2.32.0 -> 2.33.0
        openssh: upgrade 8.6p1 -> 8.7p1
        ell: upgrade 0.42 -> 0.43
        python3-mako: upgrade 1.1.4 -> 1.1.5
        vala: upgrade 0.52.4 -> 0.52.5
        libnsl2: upgrade 1.3.0 -> 2.0.0
        gi-docgen: upgrade 2021.6 -> 2021.7
        json-glib: upgrade 1.6.2 -> 1.6.4
        bind: upgrade 9.16.19 -> 9.16.20
        harfbuzz: upgrade 2.8.2 -> 2.9.0
        qemurunner.py: print output from runqemu/qemu-system in stop()
        qemurunner.py: handle getOutput() having nothing to read
        rust: fix upstream version checks
        mesa: enable crocus driver for older intel graphics

  Andreas Müller (2):
        mesa: upgrade 21.1.5 -> 21.1.7
        binutils: Apply upstream patch to fix 'too many open files' on qtwebengine

  Andrej Valek (2):
        busybox: 1.33.1 -> 1.34.0
        vim: add option to disable NLS support

  Andres Beltran (2):
        buildhistory: Add output file listing package information
        buildhistory: Label packages providing per-file dependencies in depends.dot

  Andrey Zhizhikin (2):
        lttng-modules: do not search in non-existing folder during install
        nativesdk-packagegroup-sdk-host: add perl integer module

  Armin Kuster (2):
        lz4: Security Fix for CVE-2021-3520
        lz4: remove rest of ptest artifacts

  Bruce Ashfield (21):
        linux-yocto/5.13: update to v5.13.7
        linux-yocto/5.4: update to v5.4.137
        linux-yocto/5.10: update to v5.10.55
        linux-yocto/5.4: update to v5.4.139
        linux-yocto/5.10: update to v5.10.57
        linux-yocto/5.13: update to v5.13.9
        linux-yocto/5.4: remove recipes
        conf/machine: bump qemu preferred versions to 5.13
        linux-yocto-dev: bump to v5.14+
        lttng-modules: update to 2.13.0
        kernel-devsrc: 5.14+ updates
        kernel-devsrc: fix 5.14+ objtool compilation
        poky/poky-tiny: set default kernel to 5.13
        poky: set default kernel to 5.13
        yocto-bsp: drop 5.4 bbappend
        poky-alt: switch default kernel to 5.10
        linux-yocto/5.13: update to v5.13.11
        linux-yocto/5.10: update to v5.10.59
        linux-yocto/5.13: update to v5.13.12
        linux-yocto/5.10: update to v5.10.60
        parselogs.py: ignore intermittent CD/DVDROM identification failure

  Chen Qi (1):
        package_rpm/update-alternatives: fix package's provides

  Daniel Gomez (2):
        wic: Add --no-fstab-update part option
        oeqa: wic: Add tests for --no-fstab-update

  Denys Dmytriyenko (1):
        grep: upgrade 3.6 -> 3.7

  Enrico Scholz (1):
        bitbake: fetch2/wget: fix 'no_proxy' handling

  Hongxu Jia (2):
        nativesdk-pseudo: Fix to work with glibc 2.34 systems
        glibc: fix create thread failed in unprivileged process

  Hsia-Jun Li (1):
        lib/oe/elf: Add Android OS to machine_dict

  Jon Mason (8):
        arch-armv8m-main: missing space
        conf/machine: move tune files to architecture directories
        yocto-bsp: update machine confs with new tune locations
        docs: update docs with new tune locations
        arch-arm*: add better support for gcc march extensions
        tune-cortexr*: add support for all Arm Cortex-R processors
        arch-arm*: Fix bugs with dsp and simd feature include files
        tune-*: Use more specific DEFAULTTUNE

  Jose Quaresma (1):
        sstate.bbclass: get the number of threads from BB_NUMBER_THREADS

  Joshua Watt (17):
        bitbake: contrib: vim: Add "remove" override highlighting
        bitbake.conf: Add lz4c, pzstd and zstd
        bitbake: bitbake: asyncrpc: Defer all asyncio to child process
        conf/licenses: Add FreeType SPDX mapping
        tzdata: Remove BSD License specifier
        glib-2.0: Use specific BSD license variant
        e2fsprogs: Use specific BSD license variant
        shadow: Use specific BSD license variant
        libcap: Use specific BSD license variant
        sudo: Use specific BSD license variant
        libpam: Use specific BSD license variant
        libxfont2: Use specific BSD license variant
        libjitterentropy: Use specific BSD license variant
        libx11: Use specific BSD license variant
        font-util: Use specific BSD license variant
        flac: Use specific BSD license variant
        swig: Use specific BSD license variant

  Kai Kang (2):
        libcgroup: fix installed-vs-shipped qa issue
        rustfmt: fix SRC_URI

  Kevin Hao (2):
        meta-yocto-bsp: Set the default kernel to v5.13
        meta-yocto-bsp: Bump the kernel to v5.13.11

  Khem Raj (2):
        weston: Re-order gbm destruction at DRM-backend tear down
        musl: Update to latest tip of trunk

  Kristian Klausen (1):
        systemd: Add repart PACKAGECONFIG

  Marco Felsch (1):
        bitbake: bitbake: bitbake-layers: add skip reason to output

  Marek Vasut (1):
        weston: Add rdp PACKAGECONFIG

  Marta Rybczynska (1):
        lzo: add CVE_PRODUCT

  Martin Jansa (3):
        bitbake: prserv: handle PRSERV_HOST = "127.0.0.1:0" the same as "localhost:0"
        bitbake: cooker/process: Fix typos in exiting message
        rust: remove unused patches

  Michael Halstead (2):
        uninative: Upgrade to 3.3, support glibc 2.34
        uninative: Upgrade to 3.4

  Michael Opdenacker (2):
        maintainers.inc: maintainer for alsa-*, flac, lame and speex
        meta: stop using "virtual/" in RPROVIDES and RDEPENDS

  Mingli Yu (2):
        shadow: fix default value in SHA_get_salt_rounds()
        bitbake: prserv: make localhost work

  Oleksandr Popovych (1):
        utils: Reduce the number of calls to the "dirname" command

  Oliver Kranz (1):
        Allow global override of golang GO_DYNLINK

  Paul Barker (2):
        bitbake: prserv: Replace XML RPC with modern asyncrpc implementation
        bitbake: prserv: Add read-only mode

  Paul Gortmaker (1):
        ltp: backport ioctl_ns05 fix from upstream

  Peter Kjellerstedt (7):
        lttng-modules: Make it build when CONFIG_TRACEPOINTS is not enabled again
        poky-floating-revisions.inc: Use new override syntax for commented vars
        local.conf.sample: Use the new override syntax for a commented variable
        bitbake.conf: Use the new variable override syntax in a comment
        buildhistory-collect-srcrevs: Adapt to the new variable override syntax
        meson.bbclass: Make the default buildtype "debug" if DEBUG_BUILD is 1
        bitbake: providers: Use new override syntax when handling pn- "override"

  Purushottam Choudhary (1):
        assimp: added patch to fix hardcoded non-existing paths in CMake modules

  Randy MacLeod (8):
        openssl: upgrade from 1.1.1k to 1.1.1l
        rust: initial merge of most of meta-rust
        rust: mv README.md to recipes-devtools/rust/README-rust.md
        rust: update the README to conform to being in oe-core
        cargo/rust/rustfmt: exclude from world
        maintainers: Add myself as maintainer for rust pkgs
        cargo_common: remove http_proxy
        rust: remove Rust version 1.51.0 toolchain

  Richard Purdie (27):
        elfutils: Add zstd PACKAGECONFIG for determinism
        man-db: Add compression PACKAGECONFIG entries
        oeqa/selftest/glibc: Handle incorrect encoding issuesin glibc test results
        package/scripts: Fix FILES_INFO handling
        package: Fix overrides converion issue with PKGSIZE
        bitbake: bitbake: Make 3.6.0 the minimum python version
        elfutils: Fix ptest dependencies
        bsp-guide: Fix reference to bbappend section of dev-manual
        ref-manual: Fix reference to bbappend section of dev-manual
        gcc: Fix nativesdk builds and multilib fixes with gcc 11
        bitbake: README: Add note about test suite and new tests
        pseudo: Fix to work with glibc 2.34 systems
        bitbake: README: Fix typo
        rust-cross*: Fix OVERRRIDE references in task signature computation
        rust-cross-canadian-common: Use rust.inc directly, not rust-target
        cargo: Ensure cargo-cross-canadian doesn't have native/nativesdk versions
        rust-native: Avoid stripped warning
        rust-llvm: Add missing HOMEPAGE
        rust: Skip target recipe since it doesn't work
        oeqa/selftest/distrodata: Fix up rust maintainer testing
        rust: Avoid buildtools+uninative issues with glibc symbols mismatches
        rust-common: Add LDFLAGS to cc wrapper
        oeqa/selftest/reproducibile: Exclude rust packages
        kernel: Use unexpanded EXTENDPKGV
        oeqa/buildtools-cases: Allow bitbake time to shutdown
        cargo: Apply uninative fix to snapshot as with rust
        rust-common: Hack around LD_LIBRARY_PATH issues on centos7

  Robert P. J. Day (1):
        scripts/lib/wic/help/py: "Redhat" -> "Red Hat"

  Ross Burton (11):
        oeqa/selftest/buildoptions: test buildhistory PKGSIZE and FILELIST fields
        uninative: Improve glob to handle glibc 2.34
        oeqa/sdk: add relocation test for buildtools
        glibc: package the stub .a libaries into glibc-dev
        oeqa/sdk: add HTTPS test for buildtools
        libcgroup: upgrade to 2.0
        gcc: also relocate the musl loader
        local.conf.sample.extended: fix commented-out override syntax
        cpio: backport fix for CVE-2021-38185
        mesa: fix build on Arm V5 with soft float
        ptest: allow the ptest-packagelists.inc warning to be disabled

  Sakib Sajal (1):
        qemu: fix CVE-2021-3682

  Scott Murray (2):
        bitbake: bitbake: asyncrpc: always create new asyncio loops
        prservice: remove connection caching

  Stefan Herbrechtsmeier (4):
        u-boot: Remove redundancy from installed and deployed SPL artifact names
        u-boot: Remove misplaced configuration type variable
        u-boot: Make SPL suffix configurable
        u-boot: Make UBOOT_BINARYNAME configurable

  Tim Orling (7):
        python3-importlib-metadata: upgrade 4.6.3 -> 4.6.4
        python3-hypothesis: upgrade 6.14.5 -> 6.14.8
        python3-hypothesis: upgrade 6.14.8 -> 6.15.0
        python3-hypothesis: enable ptest
        python3-pluggy: upgrade 0.13.1 -> 1.0.0
        python3-pytest: allow python3-pluggy >=1.0.0
        rust-common.bbclass: export RUST_TARGET_PATH

  Trevor Gamblin (1):
        bluez: upgrade 5.60 -> 5.61

  Trevor Woerner (1):
        distro_features_check: expand with IMAGE_FEATURES

  Vinay Kumar (2):
        glibc: Fix CVE-2021-38604
        rust-common.inc: Fix build failure with qemuppc64.

  Yi Zhao (2):
        prelink: add PACKAGECONFIG for selinux
        shadow: add /etc/default/useradd

  Zoltán Böszörményi (4):
        kernel-module-split.bbclass: Support zstd-compressed modules
        Allow opt-out of split kernel modules
        kernel.bbclass: Use full versions for inter-package dependencies
        base/kernel: Support zstd-compressed squashfs and cpio initramfs

  leimaohui (2):
        Fix conflict error when enable multilib.
        wordsize.h: Fix a miss, this file in arm and aarch64 should be the same.

meta-raspberrypi: 32921fc9bd..a6fa6b3aec:
  Khem Raj (4):
        machines: Use tune files from new location in oe-core
        linux-raspberrypi: Update to 5.10.59
        raspberrypi-firmware: Update to latest
        raspberrypi4: Use full kms (vc4-kms-v3d) DT overlay

  Marcus Comstedt (1):
        pi-bluetooth: Add compatibility with non-systemd builds

  Tom Rini (1):
        xserver-xf86-config: Correctly append to FILES:${PN}

meta-security: c885d399cd..1f18c623e9:
  Armin Kuster (10):
        cryfs: add new package
        kas-security-bas: bump conf value
        kas: fix DISTRO appends
        dm-verity-img.bbclass: more overided fixups
        krill: Rust is in core now
        suricata: rust is in core
        layer.conf: drop dynamic-layer
        layer.conf: drop meta-rust
        harden-image-minimal: fix useradd inherit
        kas: remove rust layers

  Daiane Angolini (1):
        meta-integrity: kernel-modsign: Change weak default value

  George Liu (1):
        meta: Fix typos

  Marta Rybczynska (2):
        README: fix mailing lists
        README: fix mailing lists and a typo

meta-openembedded: a13db91f19..9fdc7960ba:
  Andreas Müller (6):
        catch2: upgrade 2.13.6 -> 2.13.7
        fltk/CMake: Do not export executable 'fluid'
        fltk: upgrade 1.3.6 -> 1.3.7
        network-manager-applet: upgrade 1.22.0 -> 1.24.0
        networkmanager: upgrade 1.32.4 -> 1.32.8
        udisks2: upgrade 2.9.2 -> 2.9.3

  Anton Blanchard (2):
        boost-url: Use GNUInstallDirs instead of hard wiring install directories
        cereal: Use GNUInstallDirs instead of hard wiring install directories

  Changqing Li (1):
        linuxptp: upgrade 3.1 -> 3.1.1

  Devendra Tewari (1):
        android-tools: Add flag to enable adbd service (#147)

  Dmitry Baryshkov (1):
        image_types_sparse: stop using ext2simg

  Easwar Hariharan (1):
        chrony: Fix privdrop packageconfig

  Joe Slater (1):
        nginx: fix CVE-2021-3618

  Justin Bronder (1):
        hidapi: add rdep on glibc-gconv-utf-16

  Khem Raj (7):
        layer.conf: Add ttf-ipa to SIGGEN_EXCLUDE_SAFE_RECIPE_DEPS on fontconfig
        mpich: link explictly with libgcc
        packagegroup-meta-networking: Add bmon
        libnss-nisplus: Remove
        pipewire: Upgrade to 0.3.34
        bluealsa: Add recipe
        apitrace: Enable on glibc >= 2.34

  Leon Anavi (21):
        python3-astroid: Upgrade 2.6.6 -> 2.7.0
        python3-ujson: Upgrade 4.0.2 -> 4.1.0
        python3-pycurl: Upgrade 7.44.0 -> 7.44.1
        python3-websocket-client: Upgrade 1.1.0 -> 1.2.1
        python3-bitarray: Upgrade 2.2.5 -> 2.3.0
        python3-langtable: Upgrade 0.0.54 -> 0.0.56
        python3-pandas: Upgrade 1.3.1 -> 1.3.2
        python3-tzlocal: Upgrade 2.1 -> 3.0
        python3-zeroconf: Upgrade 0.34.3 -> 0.36.0
        python3-dbus-next: Upgrade 0.2.2 -> 0.2.3
        python3-astroid: Upgrade 2.7.0 -> 2.7.1
        python3-ruamel-yaml: Upgrade 0.17.10 -> 0.17.11
        python3-unidiff: Upgrade 0.6.0 -> 0.7.0
        python3-qrcode: Upgrade 7.2 -> 7.3
        python3-simplejson: Upgrade 3.17.3 -> 3.17.4
        python3-regex: Upgrade 2021.7.6 -> 2021.8.3
        python3-colorlog: Upgrade 5.0.1 -> 6.4.1
        python3-ruamel-yaml: Upgrade 0.17.11 -> 0.17.13
        python3-simplejson: Upgrade 3.17.4 -> 3.17.5
        python3-bitarray: Upgrade 2.3.0 -> 2.3.2
        python3-watchdog: Upgrade 2.1.3 -> 2.1.5

  Martin Jansa (1):
        layer.conf: Add ttf-takao to SIGGEN_EXCLUDE_SAFE_RECIPE_DEPS on fontconfig

  Matija Tudan (1):
        gpsd: upgrade 3.20 -> 3.23

  Matteo Croce (1):
        libbpf: bump to 0.4.0

  Michael Opdenacker (2):
        meta-multimedia: stop using "virtual/" in RPROVIDES and RDEPENDS
        meta-oe: stop using "virtual/" in RPROVIDES and RDEPENDS

  Mingli Yu (4):
        polkit: fix CVE-2021-3560
        vsftpd: Upgrade to 3.0.5
        mariadb: Upgrade to 10.6.4
        jemalloc: improve reproducibility

  Nathan Rossi (1):
        nginx: Fix off_t size passed in configure

  Oleksandr Kravchuk (6):
        font-adobe-100dpi: fix UPSTREAM_CHECK_REGEX
        font-adobe-utopia-100dpi: fix UPSTREAM_CHECK_REGEX
        font-bh-100dpi: fix UPSTREAM_CHECK_REGEX
        font-bh-lucidatypewriter-100dpi: fix UPSTREAM_CHECK_REGEX
        font-bitstream-100dpi: fix UPSTREAM_CHECK_REGEX
        xf86-input-tslib: update to 1.1.1

  Patrick Areny (2):
        libConfuse: Add recipe
        bmon: Add recipe

  Peter Kjellerstedt (6):
        gpsd: Let scons install the udev and systemd files
        gpsd: Move /usr/share/gpsd/doc to the gpsd-doc package
        poppler: Explicitly enable/disable boost together with splash
        chrony: Use new override syntax for USERADD_PARAM
        gpsd: Correct the installation of gpsd.hotplug if systemd is not enabled
        gpsd: Do not install gpsd.hotplug unconditionally

  Peter Morrow (1):
        libbpf: remove stale comment

  Sakib Sajal (2):
        lmdb: use libprefix in Makefile to install libraries
        gd: fix CVE-2021-38115

  Sinan Kaya (4):
        c-ares: remove custom patches
        grpc: make SHARED library build optional
        libkcapi: add a hash only packageconfig
        libkcapi: allow an option to build natively

  Tim Orling (2):
        bootchart: drop; unfetchable
        python3-django_2.2.x: only check upstream 2.2.x

  Trevor Gamblin (5):
        python3-click: Add missing ptest artifacts
        python3-eventlet: add 0.30.2 to meta-python
        python3-gunicorn: tweak run-ptest, add RDEPENDS
        python3-license-expression: add ptest artifacts
        nftables: upgrade 0.9.9 -> 1.0.0

  Vesa Jääskeläinen (2):
        python3-cached-property: Add recipe for version 1.5.2
        python3-pkcs11: Add recipe for version 0.7.0

  Yi Zhao (2):
        audit: upgrade 3.0.4 -> 3.0.5
        krb5: filtering out -f*-prefix-map from krb5-config

  Zoltán Böszörményi (1):
        metacity: Add a patch to create build/src/core before moving generated sources to it

  leimaohui (3):
        packagegroup-meta-oe: Update ttf-ipa package name.
        uim: Dleted takao fonts from DEPENDS.
        takao-fonts: It should be in ttf-fonts directory as the other ttf fonts.

  wangmy (14):
        fetchmail: upgrade 6.4.20 -> 6.4.21
        c-ares: upgrade 1.17.1 -> 1.17.2
        icewm: upgrade 2.6.0 -> 2.7.0
        netplan: upgrade 0.102 -> 0.103
        ctags: upgrade 5.9.20210801.0 -> 5.9.20210815.0
        live555: upgrade 20210720 -> 20210809
        opensc: upgrade 0.21.0 -> 0.22.0
        xfsprogs: upgrade 5.12.0 -> 5.13.0
        networkmanager: upgrade 1.32.8 -> 1.32.10
        can-utils: upgrade 2021.06.0 -> 2021.08.0
        doxygen: upgrade 1.9.1 -> 1.9.2
        gensio: upgrade 2.2.8 -> 2.2.9
        live555: upgrade 20210809 -> 20210824
        sedutil: upgrade 1.15.1.01 -> 1.20.0

  zangrc (14):
        python3-flask-migrate: upgrade 3.0.1 -> 3.1.0
        python3-flask-socketio: upgrade 5.1.0 -> 5.1.1
        python3-google-api-python-client: upgrade 2.15.0 -> 2.17.0
        python3-grpcio-tools: upgrade 1.38.1 -> 1.39.0
        python3-grpcio: upgrade 1.38.1 -> 1.39.0
        python3-wheel: upgrade 0.36.2 -> 0.37.0
        libio-socket-ssl-perl: upgrade 2.071 -> 2.072
        python3-aiohttp-jinja2: upgrade 1.4.2 -> 1.5
        python3-gevent: upgrade 21.1.2 -> 21.8.0
        python3-google-api-python-client: upgrade 2.17.0 -> 2.18.0
        python3-h5py: upgrade 3.3.0 -> 3.4.0
        python3-haversine: upgrade 2.3.1 -> 2.4.0
        python3-pyephem: upgrade 3.7.7.1 -> 4.0.0.2
        rdma-core: upgrade 35.0 -> 36.0

  zhengruoqin (12):
        libqmi: upgrade 1.28.8 -> 1.30.0
        sedutil: upgrade 1.15.1 -> 1.15.1.01
        libencode-perl: upgrade 3.11 -> 3.12
        python3-pymisp: upgrade 2.4.144 -> 2.4.148
        python3-pyzmq: upgrade 22.1.0 -> 22.2.1
        python3-tqdm: upgrade 4.62.0 -> 4.62.2
        iwd: upgrade 1.16 -> 1.17
        xmlsec1: upgrade 1.2.31 -> 1.2.32
        xrdb: upgrade 1.2.0 -> 1.2.1
        python3-regex: upgrade 2021.8.3 -> 2021.8.27
        python3-sqlalchemy: upgrade 1.4.22 -> 1.4.23
        python3-stevedore: upgrade 3.3.0 -> 3.4.0

Signed-off-by: Andrew Geissler <geissonator@yahoo.com>
Change-Id: I2960f1ce53a1e2cde8b03b929829db9a2f105541

12 files changed, 371 insertions, 246 deletions

diff --git a/poky/bitbake/lib/bb/__init__.py b/poky/bitbake/lib/bb/__init__.py
index c1e30697b..5c248d365 100644
--- a/poky/bitbake/lib/bb/__init__.py
+++ b/poky/bitbake/lib/bb/__init__.py
@@ -12,8 +12,8 @@
 __version__ = "1.51.1"
 
 import sys
-if sys.version_info < (3, 5, 0):
-    raise RuntimeError("Sorry, python 3.5.0 or later is required for this version of bitbake")
+if sys.version_info < (3, 6, 0):
+    raise RuntimeError("Sorry, python 3.6.0 or later is required for this version of bitbake")
 
 
 class BBHandledException(Exception):
diff --git a/poky/bitbake/lib/bb/asyncrpc/client.py b/poky/bitbake/lib/bb/asyncrpc/client.py
index 3eb4fdde8..50e60d5c3 100644
--- a/poky/bitbake/lib/bb/asyncrpc/client.py
+++ b/poky/bitbake/lib/bb/asyncrpc/client.py
@@ -119,6 +119,16 @@ class Client(object):
         self.client = self._get_async_client()
         self.loop = asyncio.new_event_loop()
 
+        # Override any pre-existing loop.
+        # Without this, the PR server export selftest triggers a hang
+        # when running with Python 3.7.  The drawback is that there is
+        # potential for issues if the PR and hash equiv (or some new)
+        # clients need to both be instantiated in the same process.
+        # This should be revisited if/when Python 3.9 becomes the
+        # minimum required version for BitBake, as it seems not
+        # required (but harmless) with it.
+        asyncio.set_event_loop(self.loop)
+
         self._add_methods('connect_tcp', 'close', 'ping')
 
     @abc.abstractmethod
diff --git a/poky/bitbake/lib/bb/asyncrpc/serv.py b/poky/bitbake/lib/bb/asyncrpc/serv.py
index 4084f300d..b4cffff21 100644
--- a/poky/bitbake/lib/bb/asyncrpc/serv.py
+++ b/poky/bitbake/lib/bb/asyncrpc/serv.py
@@ -131,53 +131,55 @@ class AsyncServerConnection(object):
 
 
 class AsyncServer(object):
-    def __init__(self, logger, loop=None):
-        if loop is None:
-            self.loop = asyncio.new_event_loop()
-            self.close_loop = True
-        else:
-            self.loop = loop
-            self.close_loop = False
-
+    def __init__(self, logger):
         self._cleanup_socket = None
         self.logger = logger
+        self.start = None
+        self.address = None
+        self.loop = None
 
     def start_tcp_server(self, host, port):
-        self.server = self.loop.run_until_complete(
-            asyncio.start_server(self.handle_client, host, port, loop=self.loop)
-        )
-
-        for s in self.server.sockets:
-            self.logger.debug('Listening on %r' % (s.getsockname(),))
-            # Newer python does this automatically. Do it manually here for
-            # maximum compatibility
-            s.setsockopt(socket.SOL_TCP, socket.TCP_NODELAY, 1)
-            s.setsockopt(socket.SOL_TCP, socket.TCP_QUICKACK, 1)
-
-        name = self.server.sockets[0].getsockname()
-        if self.server.sockets[0].family == socket.AF_INET6:
-            self.address = "[%s]:%d" % (name[0], name[1])
-        else:
-            self.address = "%s:%d" % (name[0], name[1])
+        def start_tcp():
+            self.server = self.loop.run_until_complete(
+                asyncio.start_server(self.handle_client, host, port)
+            )
+
+            for s in self.server.sockets:
+                self.logger.debug('Listening on %r' % (s.getsockname(),))
+                # Newer python does this automatically. Do it manually here for
+                # maximum compatibility
+                s.setsockopt(socket.SOL_TCP, socket.TCP_NODELAY, 1)
+                s.setsockopt(socket.SOL_TCP, socket.TCP_QUICKACK, 1)
+
+            name = self.server.sockets[0].getsockname()
+            if self.server.sockets[0].family == socket.AF_INET6:
+                self.address = "[%s]:%d" % (name[0], name[1])
+            else:
+                self.address = "%s:%d" % (name[0], name[1])
+
+        self.start = start_tcp
 
     def start_unix_server(self, path):
         def cleanup():
             os.unlink(path)
 
-        cwd = os.getcwd()
-        try:
-            # Work around path length limits in AF_UNIX
-            os.chdir(os.path.dirname(path))
-            self.server = self.loop.run_until_complete(
-                asyncio.start_unix_server(self.handle_client, os.path.basename(path), loop=self.loop)
-            )
-        finally:
-            os.chdir(cwd)
+        def start_unix():
+            cwd = os.getcwd()
+            try:
+                # Work around path length limits in AF_UNIX
+                os.chdir(os.path.dirname(path))
+                self.server = self.loop.run_until_complete(
+                    asyncio.start_unix_server(self.handle_client, os.path.basename(path))
+                )
+            finally:
+                os.chdir(cwd)
+
+            self.logger.debug('Listening on %r' % path)
 
-        self.logger.debug('Listening on %r' % path)
+            self._cleanup_socket = cleanup
+            self.address = "unix://%s" % os.path.abspath(path)
 
-        self._cleanup_socket = cleanup
-        self.address = "unix://%s" % os.path.abspath(path)
+        self.start = start_unix
 
     @abc.abstractmethod
     def accept_client(self, reader, writer):
@@ -205,8 +207,7 @@ class AsyncServer(object):
         self.logger.debug("Got exit signal")
         self.loop.stop()
 
-    def serve_forever(self):
-        asyncio.set_event_loop(self.loop)
+    def _serve_forever(self):
         try:
             self.loop.add_signal_handler(signal.SIGTERM, self.signal_handler)
             signal.pthread_sigmask(signal.SIG_UNBLOCK, [signal.SIGTERM])
@@ -217,28 +218,69 @@ class AsyncServer(object):
             self.loop.run_until_complete(self.server.wait_closed())
             self.logger.debug('Server shutting down')
         finally:
-            if self.close_loop:
-                if sys.version_info >= (3, 6):
-                    self.loop.run_until_complete(self.loop.shutdown_asyncgens())
-                self.loop.close()
-
             if self._cleanup_socket is not None:
                 self._cleanup_socket()
 
+    def serve_forever(self):
+        """
+        Serve requests in the current process
+        """
+        # Create loop and override any loop that may have existed in
+        # a parent process.  It is possible that the usecases of
+        # serve_forever might be constrained enough to allow using
+        # get_event_loop here, but better safe than sorry for now.
+        self.loop = asyncio.new_event_loop()
+        asyncio.set_event_loop(self.loop)
+        self.start()
+        self._serve_forever()
+
     def serve_as_process(self, *, prefunc=None, args=()):
-        def run():
+        """
+        Serve requests in a child process
+        """
+        def run(queue):
+            # Create loop and override any loop that may have existed
+            # in a parent process.  Without doing this and instead
+            # using get_event_loop, at the very minimum the hashserv
+            # unit tests will hang when running the second test.
+            # This happens since get_event_loop in the spawned server
+            # process for the second testcase ends up with the loop
+            # from the hashserv client created in the unit test process
+            # when running the first testcase.  The problem is somewhat
+            # more general, though, as any potential use of asyncio in
+            # Cooker could create a loop that needs to replaced in this
+            # new process.
+            self.loop = asyncio.new_event_loop()
+            asyncio.set_event_loop(self.loop)
+            try:
+                self.start()
+            finally:
+                queue.put(self.address)
+                queue.close()
+
             if prefunc is not None:
                 prefunc(self, *args)
-            self.serve_forever()
+
+            self._serve_forever()
+
+            if sys.version_info >= (3, 6):
+                self.loop.run_until_complete(self.loop.shutdown_asyncgens())
+            self.loop.close()
+
+        queue = multiprocessing.Queue()
 
         # Temporarily block SIGTERM. The server process will inherit this
         # block which will ensure it doesn't receive the SIGTERM until the
         # handler is ready for it
         mask = signal.pthread_sigmask(signal.SIG_BLOCK, [signal.SIGTERM])
         try:
-            self.process = multiprocessing.Process(target=run)
+            self.process = multiprocessing.Process(target=run, args=(queue,))
             self.process.start()
 
+            self.address = queue.get()
+            queue.close()
+            queue.join_thread()
+
             return self.process
         finally:
             signal.pthread_sigmask(signal.SIG_SETMASK, mask)
diff --git a/poky/bitbake/lib/bb/cooker.py b/poky/bitbake/lib/bb/cooker.py
index b2d69c28c..db991702e 100644
--- a/poky/bitbake/lib/bb/cooker.py
+++ b/poky/bitbake/lib/bb/cooker.py
@@ -382,7 +382,7 @@ class BBCooker:
         try:
             self.prhost = prserv.serv.auto_start(self.data)
         except prserv.serv.PRServiceConfigError as e:
-            bb.fatal("Unable to start PR Server, exitting")
+            bb.fatal("Unable to start PR Server, exiting, check the bitbake-cookerdaemon.log")
 
         if self.data.getVar("BB_HASHSERVE") == "auto":
             # Create a new hash server bound to a unix domain socket
diff --git a/poky/bitbake/lib/bb/fetch2/wget.py b/poky/bitbake/lib/bb/fetch2/wget.py
index 29fcfbb3d..9a49e64a0 100644
--- a/poky/bitbake/lib/bb/fetch2/wget.py
+++ b/poky/bitbake/lib/bb/fetch2/wget.py
@@ -330,50 +330,51 @@ class Wget(FetchMethod):
                         urllib.request.HTTPSHandler(context=context)]
             opener = urllib.request.build_opener(*handlers)
 
-        try:
-            uri = ud.url.split(";")[0]
-            r = urllib.request.Request(uri)
-            r.get_method = lambda: "HEAD"
-            # Some servers (FusionForge, as used on Alioth) require that the
-            # optional Accept header is set.
-            r.add_header("Accept", "*/*")
-            r.add_header("User-Agent", self.user_agent)
-            def add_basic_auth(login_str, request):
-                '''Adds Basic auth to http request, pass in login:password as string'''
-                import base64
-                encodeuser = base64.b64encode(login_str.encode('utf-8')).decode("utf-8")
-                authheader = "Basic %s" % encodeuser
-                r.add_header("Authorization", authheader)
-
-            if ud.user and ud.pswd:
-                add_basic_auth(ud.user + ':' + ud.pswd, r)
-
             try:
-                import netrc
-                n = netrc.netrc()
-                login, unused, password = n.authenticators(urllib.parse.urlparse(uri).hostname)
-                add_basic_auth("%s:%s" % (login, password), r)
-            except (TypeError, ImportError, IOError, netrc.NetrcParseError):
-                pass
-
-            with opener.open(r) as response:
-                pass
-        except urllib.error.URLError as e:
-            if try_again:
-                logger.debug2("checkstatus: trying again")
-                return self.checkstatus(fetch, ud, d, False)
-            else:
-                # debug for now to avoid spamming the logs in e.g. remote sstate searches
-                logger.debug2("checkstatus() urlopen failed: %s" % e)
-                return False
-        except ConnectionResetError as e:
-            if try_again:
-                logger.debug2("checkstatus: trying again")
-                return self.checkstatus(fetch, ud, d, False)
-            else:
-                # debug for now to avoid spamming the logs in e.g. remote sstate searches
-                logger.debug2("checkstatus() urlopen failed: %s" % e)
-                return False
+                uri = ud.url.split(";")[0]
+                r = urllib.request.Request(uri)
+                r.get_method = lambda: "HEAD"
+                # Some servers (FusionForge, as used on Alioth) require that the
+                # optional Accept header is set.
+                r.add_header("Accept", "*/*")
+                r.add_header("User-Agent", self.user_agent)
+                def add_basic_auth(login_str, request):
+                    '''Adds Basic auth to http request, pass in login:password as string'''
+                    import base64
+                    encodeuser = base64.b64encode(login_str.encode('utf-8')).decode("utf-8")
+                    authheader = "Basic %s" % encodeuser
+                    r.add_header("Authorization", authheader)
+
+                if ud.user and ud.pswd:
+                    add_basic_auth(ud.user + ':' + ud.pswd, r)
+
+                try:
+                    import netrc
+                    n = netrc.netrc()
+                    login, unused, password = n.authenticators(urllib.parse.urlparse(uri).hostname)
+                    add_basic_auth("%s:%s" % (login, password), r)
+                except (TypeError, ImportError, IOError, netrc.NetrcParseError):
+                    pass
+
+                with opener.open(r) as response:
+                    pass
+            except urllib.error.URLError as e:
+                if try_again:
+                    logger.debug2("checkstatus: trying again")
+                    return self.checkstatus(fetch, ud, d, False)
+                else:
+                    # debug for now to avoid spamming the logs in e.g. remote sstate searches
+                    logger.debug2("checkstatus() urlopen failed: %s" % e)
+                    return False
+            except ConnectionResetError as e:
+                if try_again:
+                    logger.debug2("checkstatus: trying again")
+                    return self.checkstatus(fetch, ud, d, False)
+                else:
+                    # debug for now to avoid spamming the logs in e.g. remote sstate searches
+                    logger.debug2("checkstatus() urlopen failed: %s" % e)
+                    return False
+
         return True
 
     def _parse_path(self, regex, s):
diff --git a/poky/bitbake/lib/bb/providers.py b/poky/bitbake/lib/bb/providers.py
index 516d45e4a..8c1c31a5c 100644
--- a/poky/bitbake/lib/bb/providers.py
+++ b/poky/bitbake/lib/bb/providers.py
@@ -94,7 +94,7 @@ def versionVariableMatch(cfgData, keyword, pn):
 
     # pn can contain '_', e.g. gcc-cross-x86_64 and an override cannot
     # hence we do this manually rather than use OVERRIDES
-    ver = cfgData.getVar("%s_VERSION_pn-%s" % (keyword, pn))
+    ver = cfgData.getVar("%s_VERSION:pn-%s" % (keyword, pn))
     if not ver:
         ver = cfgData.getVar("%s_VERSION_%s" % (keyword, pn))
     if not ver:
diff --git a/poky/bitbake/lib/bb/server/process.py b/poky/bitbake/lib/bb/server/process.py
index 6127fd40e..b593830cc 100644
--- a/poky/bitbake/lib/bb/server/process.py
+++ b/poky/bitbake/lib/bb/server/process.py
@@ -473,7 +473,7 @@ class BitBakeServer(object):
             try:
                 r = ready.get()
             except EOFError:
-                # Trap the child exitting/closing the pipe and error out
+                # Trap the child exiting/closing the pipe and error out
                 r = None
         if not r or r[0] != "r":
             ready.close()
diff --git a/poky/bitbake/lib/bblayers/query.py b/poky/bitbake/lib/bblayers/query.py
index 947422a72..6e94c8307 100644
--- a/poky/bitbake/lib/bblayers/query.py
+++ b/poky/bitbake/lib/bblayers/query.py
@@ -154,7 +154,7 @@ skipped recipes will also be listed, with a " (skipped)" suffix.
         def print_item(f, pn, ver, layer, ispref):
             if not selected_layer or layer == selected_layer:
                 if not bare and f in skiplist:
-                    skipped = ' (skipped)'
+                    skipped = ' (skipped: %s)' % self.tinfoil.cooker.skiplist[f].skipreason
                 else:
                     skipped = ''
                 if show_filenames:
diff --git a/poky/bitbake/lib/hashserv/server.py b/poky/bitbake/lib/hashserv/server.py
index 8e8498973..a059e5211 100644
--- a/poky/bitbake/lib/hashserv/server.py
+++ b/poky/bitbake/lib/hashserv/server.py
@@ -410,11 +410,11 @@ class ServerClient(bb.asyncrpc.AsyncServerConnection):
 
 
 class Server(bb.asyncrpc.AsyncServer):
-    def __init__(self, db, loop=None, upstream=None, read_only=False):
+    def __init__(self, db, upstream=None, read_only=False):
         if upstream and read_only:
             raise bb.asyncrpc.ServerError("Read-only hashserv cannot pull from an upstream server")
 
-        super().__init__(logger, loop)
+        super().__init__(logger)
 
         self.request_stats = Stats()
         self.db = db
diff --git a/poky/bitbake/lib/prserv/client.py b/poky/bitbake/lib/prserv/client.py
new file mode 100644
index 000000000..a3f19ddaf
--- /dev/null
+++ b/poky/bitbake/lib/prserv/client.py
@@ -0,0 +1,48 @@
+#
+# SPDX-License-Identifier: GPL-2.0-only
+#
+
+import logging
+import bb.asyncrpc
+
+logger = logging.getLogger("BitBake.PRserv")
+
+class PRAsyncClient(bb.asyncrpc.AsyncClient):
+    def __init__(self):
+        super().__init__('PRSERVICE', '1.0', logger)
+
+    async def getPR(self, version, pkgarch, checksum):
+        response = await self.send_message(
+            {'get-pr': {'version': version, 'pkgarch': pkgarch, 'checksum': checksum}}
+        )
+        if response:
+            return response['value']
+
+    async def importone(self, version, pkgarch, checksum, value):
+        response = await self.send_message(
+            {'import-one': {'version': version, 'pkgarch': pkgarch, 'checksum': checksum, 'value': value}}
+        )
+        if response:
+            return response['value']
+
+    async def export(self, version, pkgarch, checksum, colinfo):
+        response = await self.send_message(
+            {'export': {'version': version, 'pkgarch': pkgarch, 'checksum': checksum, 'colinfo': colinfo}}
+        )
+        if response:
+            return (response['metainfo'], response['datainfo'])
+
+    async def is_readonly(self):
+        response = await self.send_message(
+            {'is-readonly': {}}
+        )
+        if response:
+            return response['readonly']
+
+class PRClient(bb.asyncrpc.Client):
+    def __init__(self):
+        super().__init__()
+        self._add_methods('getPR', 'importone', 'export', 'is_readonly')
+
+    def _get_async_client(self):
+        return PRAsyncClient()
diff --git a/poky/bitbake/lib/prserv/db.py b/poky/bitbake/lib/prserv/db.py
index cb2a2461e..2710d4a22 100644
--- a/poky/bitbake/lib/prserv/db.py
+++ b/poky/bitbake/lib/prserv/db.py
@@ -30,21 +30,29 @@ if sqlversion[0] < 3 or (sqlversion[0] == 3 and sqlversion[1] < 3):
 #
 
 class PRTable(object):
-    def __init__(self, conn, table, nohist):
+    def __init__(self, conn, table, nohist, read_only):
         self.conn = conn
         self.nohist = nohist
+        self.read_only = read_only
         self.dirty = False
         if nohist:
             self.table = "%s_nohist" % table 
         else:
             self.table = "%s_hist" % table 
 
-        self._execute("CREATE TABLE IF NOT EXISTS %s \
-                    (version TEXT NOT NULL, \
-                    pkgarch TEXT NOT NULL,  \
-                    checksum TEXT NOT NULL, \
-                    value INTEGER, \
-                    PRIMARY KEY (version, pkgarch, checksum));" % self.table)
+        if self.read_only:
+            table_exists = self._execute(
+                        "SELECT count(*) FROM sqlite_master \
+                        WHERE type='table' AND name='%s'" % (self.table))
+            if not table_exists:
+                raise prserv.NotFoundError
+        else:
+            self._execute("CREATE TABLE IF NOT EXISTS %s \
+                        (version TEXT NOT NULL, \
+                        pkgarch TEXT NOT NULL,  \
+                        checksum TEXT NOT NULL, \
+                        value INTEGER, \
+                        PRIMARY KEY (version, pkgarch, checksum));" % self.table)
 
     def _execute(self, *query):
         """Execute a query, waiting to acquire a lock if necessary"""
@@ -59,8 +67,9 @@ class PRTable(object):
                 raise exc
 
     def sync(self):
-        self.conn.commit()
-        self._execute("BEGIN EXCLUSIVE TRANSACTION")
+        if not self.read_only:
+            self.conn.commit()
+            self._execute("BEGIN EXCLUSIVE TRANSACTION")
 
     def sync_if_dirty(self):
         if self.dirty:
@@ -75,6 +84,15 @@ class PRTable(object):
             return row[0]
         else:
             #no value found, try to insert
+            if self.read_only:
+                data = self._execute("SELECT ifnull(max(value)+1,0) FROM %s where version=? AND pkgarch=?;" % (self.table),
+                                   (version, pkgarch))
+                row = data.fetchone()
+                if row is not None:
+                    return row[0]
+                else:
+                    return 0
+
             try:
                 self._execute("INSERT INTO %s VALUES (?, ?, ?, (select ifnull(max(value)+1,0) from %s where version=? AND pkgarch=?));"
                            % (self.table,self.table),
@@ -103,6 +121,15 @@ class PRTable(object):
             return row[0]
         else:
             #no value found, try to insert
+            if self.read_only:
+                data = self._execute("SELECT ifnull(max(value)+1,0) FROM %s where version=? AND pkgarch=?;" % (self.table),
+                                   (version, pkgarch))
+                row = data.fetchone()
+                if row is not None:
+                    return row[0]
+                else:
+                    return 0
+
             try:
                 self._execute("INSERT OR REPLACE INTO %s VALUES (?, ?, ?, (select ifnull(max(value)+1,0) from %s where version=? AND pkgarch=?));"
                                % (self.table,self.table),
@@ -128,6 +155,9 @@ class PRTable(object):
             return self._getValueHist(version, pkgarch, checksum)
 
     def _importHist(self, version, pkgarch, checksum, value):
+        if self.read_only:
+            return None
+
         val = None 
         data = self._execute("SELECT value FROM %s WHERE version=? AND pkgarch=? AND checksum=?;" % self.table,
                            (version, pkgarch, checksum))
@@ -152,6 +182,9 @@ class PRTable(object):
         return val
 
     def _importNohist(self, version, pkgarch, checksum, value):
+        if self.read_only:
+            return None
+
         try:
             #try to insert
             self._execute("INSERT INTO %s VALUES (?, ?, ?, ?);"  % (self.table),
@@ -245,19 +278,23 @@ class PRTable(object):
 
 class PRData(object):
     """Object representing the PR database"""
-    def __init__(self, filename, nohist=True):
+    def __init__(self, filename, nohist=True, read_only=False):
         self.filename=os.path.abspath(filename)
         self.nohist=nohist
+        self.read_only = read_only
         #build directory hierarchy
         try:
             os.makedirs(os.path.dirname(self.filename))
         except OSError as e:
             if e.errno != errno.EEXIST:
                 raise e
-        self.connection=sqlite3.connect(self.filename, isolation_level="EXCLUSIVE", check_same_thread = False)
+        uri = "file:%s%s" % (self.filename, "?mode=ro" if self.read_only else "")
+        logger.debug("Opening PRServ database '%s'" % (uri))
+        self.connection=sqlite3.connect(uri, uri=True, isolation_level="EXCLUSIVE", check_same_thread = False)
         self.connection.row_factory=sqlite3.Row
-        self.connection.execute("pragma synchronous = off;")
-        self.connection.execute("PRAGMA journal_mode = MEMORY;")
+        if not self.read_only:
+            self.connection.execute("pragma synchronous = off;")
+            self.connection.execute("PRAGMA journal_mode = MEMORY;")
         self._tables={}
 
     def disconnect(self):
@@ -270,7 +307,7 @@ class PRData(object):
         if tblname in self._tables:
             return self._tables[tblname]
         else:
-            tableobj = self._tables[tblname] = PRTable(self.connection, tblname, self.nohist)
+            tableobj = self._tables[tblname] = PRTable(self.connection, tblname, self.nohist, self.read_only)
             return tableobj
 
     def __delitem__(self, tblname):
diff --git a/poky/bitbake/lib/prserv/serv.py b/poky/bitbake/lib/prserv/serv.py
index 5e322bf83..0a20b927c 100644
--- a/poky/bitbake/lib/prserv/serv.py
+++ b/poky/bitbake/lib/prserv/serv.py
@@ -4,157 +4,135 @@
 
 import os,sys,logging
 import signal, time
-from xmlrpc.server import SimpleXMLRPCServer, SimpleXMLRPCRequestHandler
 import socket
 import io
 import sqlite3
-import bb.server.xmlrpcclient
 import prserv
 import prserv.db
 import errno
-import multiprocessing
+import bb.asyncrpc
 
 logger = logging.getLogger("BitBake.PRserv")
 
-class Handler(SimpleXMLRPCRequestHandler):
-    def _dispatch(self,method,params):
-        try:
-            value=self.server.funcs[method](*params)
-        except:
-            import traceback
-            traceback.print_exc()
-            raise
-        return value
-
 PIDPREFIX = "/tmp/PRServer_%s_%s.pid"
 singleton = None
 
-
-class PRServer(SimpleXMLRPCServer):
-    def __init__(self, dbfile, logfile, interface):
-        ''' constructor '''
+class PRServerClient(bb.asyncrpc.AsyncServerConnection):
+    def __init__(self, reader, writer, table, read_only):
+        super().__init__(reader, writer, 'PRSERVICE', logger)
+        self.handlers.update({
+            'get-pr': self.handle_get_pr,
+            'import-one': self.handle_import_one,
+            'export': self.handle_export,
+            'is-readonly': self.handle_is_readonly,
+        })
+        self.table = table
+        self.read_only = read_only
+
+    def validate_proto_version(self):
+        return (self.proto_version == (1, 0))
+
+    async def dispatch_message(self, msg):
         try:
-            SimpleXMLRPCServer.__init__(self, interface,
-                                        logRequests=False, allow_none=True)
-        except socket.error:
-            ip=socket.gethostbyname(interface[0])
-            port=interface[1]
-            msg="PR Server unable to bind to %s:%s\n" % (ip, port)
-            sys.stderr.write(msg)
-            raise PRServiceConfigError
-
-        self.dbfile=dbfile
-        self.logfile=logfile
-        self.host, self.port = self.socket.getsockname()
-
-        self.register_function(self.getPR, "getPR")
-        self.register_function(self.ping, "ping")
-        self.register_function(self.export, "export")
-        self.register_function(self.importone, "importone")
-        self.register_introspection_functions()
-
-        self.iter_count = 0
-        # 60 iterations between syncs or sync if dirty every ~30 seconds
-        self.iterations_between_sync = 60
-
-    def sigint_handler(self, signum, stack):
-        if self.table:
-            self.table.sync()
-
-    def sigterm_handler(self, signum, stack):
-        if self.table:
-            self.table.sync()
-        raise(SystemExit)
-
-    def process_request(self, request, client_address):
-        if request is None:
-            return
-        try:
-            self.finish_request(request, client_address)
-            self.shutdown_request(request)
-            self.iter_count = (self.iter_count + 1) % self.iterations_between_sync
-            if self.iter_count == 0:
-                self.table.sync_if_dirty()
+            await super().dispatch_message(msg)
         except:
-            self.handle_error(request, client_address)
-            self.shutdown_request(request)
             self.table.sync()
-        self.table.sync_if_dirty()
+            raise
 
-    def serve_forever(self, poll_interval=0.5):
-        signal.signal(signal.SIGINT, self.sigint_handler)
-        signal.signal(signal.SIGTERM, self.sigterm_handler)
+        self.table.sync_if_dirty()
 
-        self.db = prserv.db.PRData(self.dbfile)
-        self.table = self.db["PRMAIN"]
-        return super().serve_forever(poll_interval)
+    async def handle_get_pr(self, request):
+        version = request['version']
+        pkgarch = request['pkgarch']
+        checksum = request['checksum']
 
-    def export(self, version=None, pkgarch=None, checksum=None, colinfo=True):
+        response = None
         try:
-            return self.table.export(version, pkgarch, checksum, colinfo)
+            value = self.table.getValue(version, pkgarch, checksum)
+            response = {'value': value}
+        except prserv.NotFoundError:
+            logger.error("can not find value for (%s, %s)",version, checksum)
         except sqlite3.Error as exc:
             logger.error(str(exc))
-            return None
 
-    def importone(self, version, pkgarch, checksum, value):
-        return self.table.importone(version, pkgarch, checksum, value)
+        self.write_message(response)
 
-    def ping(self):
-        return True
+    async def handle_import_one(self, request):
+        response = None
+        if not self.read_only:
+            version = request['version']
+            pkgarch = request['pkgarch']
+            checksum = request['checksum']
+            value = request['value']
 
-    def getinfo(self):
-        return (self.host, self.port)
+            value = self.table.importone(version, pkgarch, checksum, value)
+            if value is not None:
+                response = {'value': value}
+
+        self.write_message(response)
+
+    async def handle_export(self, request):
+        version = request['version']
+        pkgarch = request['pkgarch']
+        checksum = request['checksum']
+        colinfo = request['colinfo']
 
-    def getPR(self, version, pkgarch, checksum):
         try:
-            return self.table.getValue(version, pkgarch, checksum)
-        except prserv.NotFoundError:
-            logger.error("can not find value for (%s, %s)",version, checksum)
-            return None
+            (metainfo, datainfo) = self.table.export(version, pkgarch, checksum, colinfo)
         except sqlite3.Error as exc:
             logger.error(str(exc))
-            return None
+            metainfo = datainfo = None
 
-class PRServSingleton(object):
-    def __init__(self, dbfile, logfile, interface):
+        response = {'metainfo': metainfo, 'datainfo': datainfo}
+        self.write_message(response)
+
+    async def handle_is_readonly(self, request):
+        response = {'readonly': self.read_only}
+        self.write_message(response)
+
+class PRServer(bb.asyncrpc.AsyncServer):
+    def __init__(self, dbfile, read_only=False):
+        super().__init__(logger)
         self.dbfile = dbfile
-        self.logfile = logfile
-        self.interface = interface
-        self.host = None
-        self.port = None
+        self.table = None
+        self.read_only = read_only
 
-    def start(self):
-        self.prserv = PRServer(self.dbfile, self.logfile, self.interface)
-        self.process = multiprocessing.Process(target=self.prserv.serve_forever)
-        self.process.start()
+    def accept_client(self, reader, writer):
+        return PRServerClient(reader, writer, self.table, self.read_only)
 
-        self.host, self.port = self.prserv.getinfo()
+    def _serve_forever(self):
+        self.db = prserv.db.PRData(self.dbfile, read_only=self.read_only)
+        self.table = self.db["PRMAIN"]
 
-    def getinfo(self):
-        return (self.host, self.port)
+        logger.info("Started PRServer with DBfile: %s, Address: %s, PID: %s" %
+                     (self.dbfile, self.address, str(os.getpid())))
 
-class PRServerConnection(object):
-    def __init__(self, host, port):
-        if is_local_special(host, port):
-            host, port = singleton.getinfo()
-        self.host = host
-        self.port = port
-        self.connection, self.transport = bb.server.xmlrpcclient._create_server(self.host, self.port)
+        super()._serve_forever()
 
-    def getPR(self, version, pkgarch, checksum):
-        return self.connection.getPR(version, pkgarch, checksum)
+        self.table.sync_if_dirty()
+        self.db.disconnect()
 
-    def ping(self):
-        return self.connection.ping()
+    def signal_handler(self):
+        super().signal_handler()
+        if self.table:
+            self.table.sync()
 
-    def export(self,version=None, pkgarch=None, checksum=None, colinfo=True):
-        return self.connection.export(version, pkgarch, checksum, colinfo)
+class PRServSingleton(object):
+    def __init__(self, dbfile, logfile, host, port):
+        self.dbfile = dbfile
+        self.logfile = logfile
+        self.host = host
+        self.port = port
 
-    def importone(self, version, pkgarch, checksum, value):
-        return self.connection.importone(version, pkgarch, checksum, value)
+    def start(self):
+        self.prserv = PRServer(self.dbfile)
+        self.prserv.start_tcp_server(socket.gethostbyname(self.host), self.port)
+        self.process = self.prserv.serve_as_process()
 
-    def getinfo(self):
-        return self.host, self.port
+        if not self.prserv.address:
+            raise PRServiceConfigError
+        if not self.port:
+            self.port = int(self.prserv.address.rsplit(':', 1)[1])
 
 def run_as_daemon(func, pidfile, logfile):
     """
@@ -226,7 +204,7 @@ def run_as_daemon(func, pidfile, logfile):
     os.remove(pidfile)
     os._exit(0)
 
-def start_daemon(dbfile, host, port, logfile):
+def start_daemon(dbfile, host, port, logfile, read_only=False):
     ip = socket.gethostbyname(host)
     pidfile = PIDPREFIX % (ip, port)
     try:
@@ -240,15 +218,13 @@ def start_daemon(dbfile, host, port, logfile):
                             % pidfile)
         return 1
 
-    server = PRServer(os.path.abspath(dbfile), os.path.abspath(logfile), (ip,port))
-    run_as_daemon(server.serve_forever, pidfile, os.path.abspath(logfile))
+    dbfile = os.path.abspath(dbfile)
+    def daemon_main():
+        server = PRServer(dbfile, read_only=read_only)
+        server.start_tcp_server(ip, port)
+        server.serve_forever()
 
-    # Sometimes, the port (i.e. localhost:0) indicated by the user does not match with
-    # the one the server actually is listening, so at least warn the user about it
-    _,rport = server.getinfo()
-    if port != rport:
-        sys.stdout.write("Server is listening at port %s instead of %s\n"
-                         % (rport,port))
+    run_as_daemon(daemon_main, pidfile, os.path.abspath(logfile))
     return 0
 
 def stop_daemon(host, port):
@@ -302,7 +278,7 @@ def is_running(pid):
     return True
 
 def is_local_special(host, port):
-    if host.strip().upper() == 'localhost'.upper() and (not port):
+    if (host == 'localhost' or host == '127.0.0.1') and not port:
         return True
     else:
         return False
@@ -326,7 +302,9 @@ def auto_start(d):
                 'Usage: PRSERV_HOST = "<hostname>:<port>"']))
         raise PRServiceConfigError
 
-    if is_local_special(host_params[0], int(host_params[1])):
+    host = host_params[0].strip().lower()
+    port = int(host_params[1])
+    if is_local_special(host, port):
         import bb.utils
         cachedir = (d.getVar("PERSISTENT_DIR") or d.getVar("CACHE"))
         if not cachedir:
@@ -340,20 +318,16 @@ def auto_start(d):
                auto_shutdown()
         if not singleton:
             bb.utils.mkdirhier(cachedir)
-            singleton = PRServSingleton(os.path.abspath(dbfile), os.path.abspath(logfile), ("localhost",0))
+            singleton = PRServSingleton(os.path.abspath(dbfile), os.path.abspath(logfile), host, port)
             singleton.start()
     if singleton:
-        host, port = singleton.getinfo()
-    else:
-        host = host_params[0]
-        port = int(host_params[1])
+        host = singleton.host
+        port = singleton.port
 
     try:
-        connection = PRServerConnection(host,port)
-        connection.ping()
-        realhost, realport = connection.getinfo()
-        return str(realhost) + ":" + str(realport)
-        
+        ping(host, port)
+        return str(host) + ":" + str(port)
+
     except Exception:
         logger.critical("PRservice %s:%d not available" % (host, port))
         raise PRServiceConfigError
@@ -366,8 +340,21 @@ def auto_shutdown():
         singleton = None
 
 def ping(host, port):
-    conn=PRServerConnection(host, port)
+    from . import client
+
+    conn = client.PRClient()
+    conn.connect_tcp(host, port)
     return conn.ping()
 
 def connect(host, port):
-    return PRServerConnection(host, port)
+    from . import client
+
+    global singleton
+
+    if host.strip().lower() == 'localhost' and not port:
+        host = 'localhost'
+        port = singleton.port
+
+    conn = client.PRClient()
+    conn.connect_tcp(host, port)
+    return conn