poky: subtree update:1203d1f24d..2dcd1f2a21

Alejandro Enedino Hernandez Samaniego (2):
      python3: Improve logging, syntax and update deprecated modules to create_manifest
      python3: Upgrade 3.9.2 -> 3.9.4

Alexander Kanavin (22):
      scripts/oe-debuginfod: correct several issues
      libmicrohttpd: add a recipe from meta-oe
      maintainers.inc: add libmicrohttpd entry
      xwayland: add a standalone recipe
      weston: use standalone xwayland instead of outdated xserver-xorg version
      elfutils: correct debuginfod builds on x32
      elfutils: adjust ptests for correct debuginfod testing
      default-distrovars.inc: add debuginfod to default DISTRO_FEATURES
      oeqa: tear down oeqa decorators if one of them raises an exception in setup
      meta/lib/oeqa/core/tests/cases/timeout.py: add a testcase for the previous fix
      core-image-weston: add sdk/ptest images
      oeqa/core/tests/test_data.py: use weston image instead of sato
      oeqa/selftest: transition to weston images
      core-image-multilib-example: base on weston, and not sato
      dev-manual/common-tasks.rst: correct the documentation for debuginfod
      diffoscope: add native libraries to LD_LIBRARY_PATH
      Revert "oeqa: Set LD_LIBRARY_PATH when executing native commands"
      boost: correct upstream version check
      vte: use tarballs again
      gdk-pixbuf: update 2.40.0 -> 2.42.6
      glib-2.0: update 2.68.0 -> 2.68.1
      gnu-config: update to latest revision

Anatol Belski (1):
      cross-canadian: Whitelist "mingw32" as TARGET_OS

Anders Wallin (3):
      lttng-tools: Fix missing legacy test files
      lttng-tools: Fix path for test_python_looging
      scripts/contrib/image-manifest: add new script

Andreas Müller (1):
      xwayland: remove protocol.txt - it clashes with xserver-xorg

Anthony Bagwell (1):
      systemd: upgrade 247.4 -> 247.6

Anuj Mittal (2):
      Revert "qemu: fix CVE-2021-3392"
      qemu: fix CVE-2021-3392

Armin Kuster (6):
      binutils: rename BRANCH var
      libseccomp: move recipe from meta-security to core
      gnutls: Enable seccomp if FEATURE is set
      systemd: Enable seccomp if FEATURE is set
      qemu: Enable seccomp if FEATURE is set
      default-distrovars.inc: Add seccomp to DISTRO_FEATURES_DEFAULT

Bastian Krause (1):
      ccache: add packageconfig docs option

Bruce Ashfield (20):
      kern-tools: add dropped options to audit output
      linux-yocto/5.4: update to v5.4.109
      linux-yocto/5.10: update to v5.10.27
      linux-yocto/5.10: BSP configuration fixes
      linux-yocto/5.10: update to v5.10.29
      linux-yocto/5.4: update to v5.4.111
      linux-yocto/5.10: update to v5.10.30
      linux-yocto-rt/5.10: update to -rt34
      linux-yocto/5.4: update to v5.4.112
      linux-yocto/5.4: fix arm defconfig warnings
      linux-yocto/5.10: fix arm defconfig warnings
      linux-yocto/5.10: aufs fixes
      linux-yocto/5.10: qemuriscv32.cfg: RV32 only supports 1G physical memory
      linux-yocto/5.10: update to v5.10.32
      perf: fix python-audit RDEPENDS
      linux-yocto/5.4: update to v5.4.114
      linux-yocto/5.10: update to v5.10.34
      linux-yocto/5.4: update to v5.4.116
      linux-yocto/5.10: qemuppc32: reduce serial shutdown issues
      yocto-check-layer: Only note a layer without a conf/layer.conf (versus error)

Changqing Li (2):
      libpam: make volatile files created successfully
      gcr: fix one parallel build failure

Chen Qi (3):
      busybox: fix CVE-2021-28831
      weston: fix build failure due to race condition
      rsync: fix CVE-2020-14387

Christophe Chapuis (1):
      rootfs.py: find .ko.gz and .ko.xz kernel modules as well

Daniel Ammann (1):
      archiver: Fix typos

Devendra Tewari (2):
      bitbake: lib/bb: Add bb.utils.rename() helper function and use for renaming
      classes/lib/scripts: Use bb.utils.rename() instead of os.rename()

Diego Sueiro (3):
      oeqa/selftest/bblayers: Add test case for bitbake-layers layerindex-show-depends
      bitbake: layerindex: Fix bitbake-layers layerindex-show-depends command
      bitbake: layerindex: Add --fetchdir parameter to layerindex-fetch

Douglas Royds (2):
      Revert "externalsrc: Detect code changes in submodules"
      externalsrc: Detect code changes in submodules

Gavin Li (1):
      kmod: do not symlink config.guess/config.sub during autoreconf

Harald Brinkmann (1):
      bitbake: fetch/svn: Fix parsing revision of SVN repos with redirects

He Zhe (1):
      linux-yocto-dev: add features/scsi/scsi-debug.scc features/gpio/mockup.scc to KERNEL_FEATURES

Henning Schild (3):
      bitbake: fetch/git: add support for disabling shared clones on unpack
      bitbake: tests/fetch: deduplicate local git testing code
      bitbake: tests/fetch: add tests for local and remote "noshared" git fetching

Jon Mason (1):
      oeqa/runtime: space needed

Jonas Höppner (1):
      ltp: fix empty ltp-dev package

Jose Quaresma (4):
      gstreamer1.0: update patch upstream status
      ptest-runner: libgcc must be installed for pthread_cancel to work
      gstreamer1.0: rename patches
      gstreamer1.0: update ptest patch

Joshua Watt (2):
      bitbake: knotty: Re-enable command line logging levels
      classes/image: Use xargs to set file timestamps

Kai Kang (2):
      cmake.bbclass: remove ${B} before cmake_do_configure
      kernel-yocto.bbclass: chdir to ${WORKDIR} for do_kernel_checkout

Kevin Hao (3):
      modutils-initscripts: Bail out when no module is installed
      sysvinit-inittab/start_getty: Check /sys for the tty device existence
      Revert "inittab: Add getty launch on hvc0 for qemuppc64"

Khairul Rohaizzat Jamaluddin (1):
      qemu: Fix CVE-2020-35517

Khem Raj (54):
      gcc: Upgrade to 10.3.0 bug-fix release
      glibc: Rename glibc src package
      gcc-runtime: Make DEBUG_PREFIX_MAP relative to S
      valgrind: Delete trailing whitespaces
      valgrind: Add glibc-src to ptest rdeps
      valgrind: Add libstdc++ debug symbols for ptest
      vte: Upgrade to 0.64.0 release
      systemd: Fix build on mips/musl
      epiphany: Add missing dependency on gnutls
      cups: Turn gnutls into a packageconfig knob
      wpa-supplicant: Enable openssl
      curl: Use openssl backend
      libpsl: Add config knobs for runtime/builtin conversion choices
      glib-networking: Prefer openssl backend instead of gnutls
      gstreamer1.0-plugins-bad: Add packageconfigs for hls crypto backends
      ca-certificates: Fix openssl runtime cert dependencies
      weston: Drop loading xwayland.so module
      elfutils: Make 64bit time_t fix generic
      binutils: Fix linking failures when using dwarf-5
      go: Use dl.google.com for SRC_URI
      musl: Update to latest master
      llvm: Upgrade to LLVM 12 release
      python3-docutils: Upgrade to 0.17.1
      python3-markupsafe: Enable ptests
      python3-jinja2: Enable ptests
      python3-pyyaml: Add recipe
      apt: Fix build on musl when seccomp is enabled
      default-distrovars.inc: Remove seccomp for riscv32
      gcc-target: Create a LTO plugin symlink in bfd-plugins directory
      bitbake.conf: Use gcc-nm as default NM
      gcc-cross: Install linker LTO plugin for binutils tools
      gcc-cross-canadian: Install LTO linker plugin to BFD searchable location
      gnutls: Point to staging area for finding seccomp libs and includes
      libjpeg-turbo: Use --reproducible option for nasm
      libid3tag: Filter -ffile-prefix-map too
      openssl: Filter out -ffile-prefix-map as well
      ltp: Filter out -ffile-prefix-map
      gcc-runtime: Fix __FILE__ related reproducablity issues
      reproducible_build.bbclass: Enable -Wdate-time
      pkgconfig: Fix nativesdk builds for mingw sdk hosts
      m4: Do not use SIGSTKSZ
      bluez: Fix shadowing of pause function from libc
      valgrind: Disable leak_cpp_interior test
      findutils: Do not use SIGSTKSZ
      bash: Include files needed for run-heredoc ptest
      libpam: Provide needed env for tst-pam_start_confdir ptest
      cml1.bbclass: Return sorted list of cfg files
      busybox: Enable long options for enabled applets
      webkitgtk: Fix reproducibility in minibrowser
      webkitgtk: Update patch status
      libgcc-initial: Do not build fp128 to decimal ppc functions
      gcc: Upgrade to GCC 11
      busybox: Fix reproducibility
      strace: Upgrade to 5.12

Konrad Weihmann (2):
      cpan-base: set default UPSTREAM_CHECK_REGEX
      cve-update-db-native: skip on empty cpe23Uri

Marek Vasut (1):
      linux-firmware: Package RSI 911x WiFi firmware

Martin Jansa (2):
      xwayland: add opengl to REQUIRED_DISTRO_FEATURES
      ofono: prevent using bundled ell headers and fix build with ell-0.39

Michael Halstead (1):
      releases: update to include 3.3

Michael Opdenacker (7):
      dev-manual: fix code insertion
      manuals: simplify code insertion
      manuals: code insertion simplification over two lines
      bitbake: doc: bitbake-user-manual: simplify colon usage
      bitbake: doc: bitbake-user-manual: code insertion simplification over two lines
      dev-manual: update references to Docker installation instructions
      sanity.bbclass: mention CONNECTIVITY_CHECK_URIS in network failure message

Mikko Rapeli (4):
      bitbake: bitbake: tests/fetch: fix test execution without .gitconfig
      bitbake: bitbake: tests/fetch: remove write protected files too
      lz4: use CFLAGS from bitbake
      unzip: use optimization from bitbake

Mingli Yu (6):
      libxshmfence: Build fixes for riscv32
      packagegroup-core-tools-profile: Remove valgrind for riscv32
      packagegroup-core-tools-testapps.bb: Remove kexec for riscv32
      libtool: make sure autoheader run before automake
      groff: not ship /usr/bin/grap2graph
      rpm: Upgrade to 4.16.1.3

Minjae Kim (1):
      qemu: fix CVE-2021-3392

Nicolas Dechesne (1):
      bitbake: doc: bitbake-user-manual: fix typo left over from Sphinx migration

Niels Avonds (1):
      bitbake: fetch/gitsm: Fix crash when using git LFS and submodules

Oleksandr Kravchuk (2):
      python3-setuptools: update to 56.0.0
      autoconf-archive: update to 2021.02.19

Otavio Salvador (2):
      gstreamer1.0-plugins-base: Add 'viv-fb' OpenGL Window System option
      gstreamer1.0-plugins-base: Use bb.utils.filter to reduce code

Paul Barker (10):
      bitbake: hashserv: Use generic ConnectionError
      bitbake: asyncrpc: Common implementation of RPC using json & asyncio
      bitbake: hashserv: Refactor to use asyncrpc
      bitbake: prserv: Drop obsolete python version check
      bitbake: prserv: Drop unused dump_db method
      bitbake: prserv: Add connect function
      prservice: Use new connect API
      bitbake: prserv: Use multiprocessing to auto start prserver
      bitbake: prserv: Extract daemonization from PRServer class
      bitbake: prserv: Handle requests in main thread

Paulo Cesar Zaneti (1):
      perl: fix startperl configuration option for perl-native

Peter Budny (1):
      lib/oe/terminal: Fix tmux new-session on older tmux versions (<1.9)

Petr Vorel (1):
      ltp: Replace musl patches with do_patch[postfuncs]

Przemyslaw Gorszkowski (2):
      bitbake: progress: LineFilterProgressHandler - Handle parsing line which ends with CR only
      bitbake: fetch/s3: Add progress handler for S3 cp command

Randy MacLeod (2):
      sqlite3: upgrade 3.35.0 -> 3.35.3
      oe-time-dd-test.sh: increase timeout to 15 sec

Reto Schneider (2):
      license_image.bbclass: Detect broken symlinks
      license_image.bbclass: Fix symlink to generic license files

Richard Purdie (32):
      oeqa/selftest: Hardcode test assumptions about heartbeat event timings
      pseudo: Upgrade to add trailing slashes ignore path fix
      oeqa/selftest: Ensure packages classes are set correctly for maintainers test
      layer.conf: Update to add post 3.3 release honister series
      sanity: Add error check for '%' in build path
      bitbake: runqueue: Fix deferred task issues
      bitbake: tinfoil/data_smart: Allow variable history emit() to function remotely
      sanity: Further improve directory sanity tests
      bitbake: bitbake-server: Remove now unneeded code
      bitbake: doc/user-manual-fetching: Remove basepath unpack parameter docs
      poky.conf: Post release version bump
      runqemu: Ensure we cleanup snapshot files after image run
      patchelf: Backport fix from upstream for note section overlap error
      pyyaml: Add missing HOMEPAGE
      yocto-check-layer: Avoid bug when iterating and autoadding dependencies
      libseccomp: Add MAINTAINERS entry and HOMEPAGE
      libseccomp: Fix reproducibility issue
      apt: Disable libseccomp
      libxcrypt: Update to 4.4.19 release and fix symbol version issues
      patchelf: Fix note section alignment issues
      bitbake: runqueue: Fix multiconfig deferred task sstate validity caching issue
      bitbake: runqueue: Handle deferred task rehashing in multiconfig builds
      patchelf: Fix alignment patch
      pybootchart/draw: Avoid divide by zero error
      yocto-uninative: Update to 3.1 which includes a patchelf fix
      Revert "perl: fix startperl configuration option for perl-native"
      bitbake: bin/bitbake-getvar: Add a new command to query a variable value (with history)
      bitbake: bitbake: Switch to post release version number 1.51.0
      sanity.conf: Require bitbake 1.51.0
      oeqa/qemurunner: Improve logging thread exit handling for qemu shutdown test
      oeqa/qemurunner: Handle path length issues for qmp socket
      lib/package_manager: Use shutil.copy instead of bb.utils.copyfile for intercepts

Robert Joslyn (3):
      btrfs-tools: Update to 5.11.1
      btrfs-tools: Add PACKAGECONFIG options
      btrfs-tools: Try to follow style guide

Robert P. J. Day (3):
      sdk-manual: "beablebone" -> "beaglebone"
      sdk-manual: fix broken formatting of sample command
      bitbake.conf: sort MIRROR list, add missing SAMBA_MIRROR

Ross Burton (4):
      glslang: strip whitespace in pkgconfig file
      insane: clean up some more warning messages
      bitbake: bitbake-server: ensure server timeout is a float
      oe-buildenv-internal: add BitBake's library to PYTHONPATH

Sakib Sajal (12):
      oe-time-dd-test.sh: make executable
      oe-time-dd-test.sh: provide more information from "top"
      qemu: fix CVE-2021-20181
      qemu: fix CVE-2020-29443
      qemu: fix CVE-2021-20221
      qemu: fix CVE-2021-3409
      qemu: fix CVE-2021-3416
      qemu: fix CVE-2021-20257
      oe-time-dd-test.sh: collect cooker log when timeout is exceeded
      buildstats.bbclass: collect data in the same file.
      qemu: fix CVE-2020-27821
      qemu: fix CVE-2021-20263

Samuli Piippo (1):
      assimp: BBCLASSEXTEND to native and nativesdk

Saul Wold (4):
      pango: re-enable ptest
      qemu-system-native: install qmp python module
      qemurunner: Add support for qmp commands
      qemurunner: change warning to info

Stefan Ghinea (3):
      wpa-supplicant: fix CVE-2021-30004
      libssh2: fix build failure with option no-ecdsa
      xserver-xorg: fix CVE-2021-3472

Stefano Babic (1):
      libubootenv: upgrade 0.3.1 -> 0.3.2

Teoh Jay Shen (6):
      oeqa/manual/bsp-hw.json : remove boot_from_runlevel_3 and boot_from_runlevel_5 manual test
      oeqa/manual/bsp-hw.json : remove ethernet_static_ip_set_in_connman and ethernet_get_IP_in_connman_via_DHCP manual test
      oeqa/manual/bsp-hw.json : remove standby and Test_if_LAN_device_works_well_after_resume_from_suspend_state manual test
      oeqa/manual/bsp-hw.json : remove click_terminal_icon_on_X_desktop manual test
      oeqa/manual/bsp-hw.json :remove Check_if_RTC_(Real_Time_Clock)_can_work_correctly manual test
      oeqa/manual/bsp-hw.json : remove Test_if_usb_hid_device_works_well_after_resume_from_suspend_state manual test

Trevor Gamblin (2):
      nettle: upgrade 3.7.1 -> 3.7.2
      ref-manual/variables.rst: Add incompatibility warning for SERIAL_CONSOLES_CHECK

Ulrich Ölmann (1):
      arch-armv6m.inc: fix access rights

Vinay Kumar (1):
      binutils: Fix CVE-2021-20197

Vineela Tummalapalli (1):
      Adding dunfell 3.1.7 to the switcher and release list.

Wang Mingyu (6):
      at-spi2-core: upgrade 2.38.0 -> 2.40.0
      babeltrace2: upgrade 2.0.3 -> 2.0.4
      boost-build-native: upgrade 4.3.0 -> 4.4.1
      libassuan: upgrade 2.5.4 -> 2.5.5
      webkitgtk: upgrade 2.30.5 -> 2.30.6
      vte: upgrade 0.62.2 -> 0.62.3

Wes Lindauer (1):
      oeqa/runtime/cases: Only disable/enable for current boot

Yanfei Xu (1):
      parselogs: ignore floppy error on qemu-system-x86 at boot stage

Yi Fan Yu (7):
      valgrind: update 3.16.1 -> 3.17.0
      valgrind: Disable ptest swapcontext.vgtest
      valgrind: Fix ptest swapcontext.vgtest
      Revert "glib-2.0: add workaround to fix codegen.py.test failing"
      re2c: Upgrade 2.0.3 -> 2.1.1
      valgrind: Enable drd/tests/bar_bad* ptest
      libevent: Increase ptest timing tolerance 50 ms -> 100 ms

Zqiang (1):
      rt-tests: Update rt-tests

hongxu (1):
      deb: apply postinstall on sdk

wangmy (34):
      ell: upgrade 0.38 -> 0.39
      dbus-glib: upgrade 0.110 -> 0.112
      ccache: upgrade 4.2 -> 4.2.1
      gcr: upgrade 3.38.1 -> 3.40.0
      ghostscript: upgrade 9.53.3 -> 9.54.0
      libsolv: upgrade 0.7.17 -> 0.7.18
      glib-2.0: upgrade 2.66.7 -> 2.68.0
      file: upgrade 5.39 -> 5.40
      curl: upgrade 7.75.0 -> 7.76.0
      acpica: upgrade 20210105 -> 20210331
      help2man: upgrade 1.48.2 -> 1.48.3
      libportal: upgrade 0.3 -> 0.4
      libksba: upgrade 1.5.0 -> 1.5.1
      go: upgrade 1.16.2 -> 1.16.3
      libcap: upgrade 2.48 -> 2.49
      libcomps: upgrade 0.1.15 -> 0.1.16
      icu: upgrade 68.2 -> 69.1
      mpg123: upgrade 1.26.4 -> 1.26.5
      man-pages: upgrade 5.10 -> 5.11
      go: update SRC_URI to use https protocol
      mesa: upgrade 21.0.1 -> 21.0.2
      openssh: upgrade 8.5p1 -> 8.6p1
      mtools: upgrade 4.0.26 -> 4.0.27
      python3-cython: upgrade 0.29.22 -> 0.29.23
      tiff: upgrade 4.2.0 -> 4.3.0
      boost: upgrade 1.75.0 -> 1.76.0
      wpebackend-fdo: upgrade 1.8.2 -> 1.8.3
      mesa: upgrade 21.0.2 -> 21.0.3
      gdb: upgrade 10.1 -> 10.2
      glib-networking: upgrade 2.66.0 -> 2.68.1
      glslang: upgrade 11.2.0 -> 11.4.0
      hdparm: upgrade 9.60 -> 9.61
      libhandy: upgrade 1.2.1 -> 1.2.2
      libjitterentropy: upgrade 3.0.1 -> 3.0.2

zangrc (1):
      maintainers.inc: Modify email address

zhengruoqin (19):
      epiphany: upgrade 3.38.2 -> 3.38.3
      wpebackend-fdo: upgrade 1.8.0 -> 1.8.2
      netbase: upgrade 6.2 -> 6.3
      python3-dbusmock: upgrade 0.22.0 -> 0.23.0
      python3-gitdb: upgrade 4.0.5 -> 4.0.7
      libva: upgrade 2.10.0 -> 2.11.0
      ruby: upgrade 3.0.0 -> 3.0.1
      libva-utils: upgrade 2.10.0 -> 2.11.1
      libdazzle: upgrade 3.38.0 -> 3.40.0
      librepo: upgrade 1.13.0 -> 1.14.0
      libdrm: upgrade 2.4.104 -> 2.4.105
      python3-pygobject: upgrade 3.38.0 -> 3.40.1
      libedit: upgrade 20210216-3.1 -> 20210419-3.1
      libhandy: upgrade 1.2.0 -> 1.2.1
      libical: upgrade 3.0.9 -> 3.0.10
      libsolv: upgrade 0.7.18 -> 0.7.19
      libmicrohttpd: upgrade 0.9.72 -> 0.9.73
      python3-numpy: upgrade 1.20.1 -> 1.20.2
      wireless-regdb: upgrade 2020.11.20 -> 2021.04.21

Signed-off-by: Andrew Geissler <geissonator@yahoo.com>
Change-Id: Ibdaea694cae40b0749d472bf08b53002a45b31d7

3 files changed, 82 insertions, 59 deletions

diff --git a/poky/meta/recipes-devtools/python/python3/0001-test_locale.py-correct-the-test-output-format.patch b/poky/meta/recipes-devtools/python/python3/0001-test_locale.py-correct-the-test-output-format.patch
index 35b7e0c480..c3d1e06d00 100644
--- a/poky/meta/recipes-devtools/python/python3/0001-test_locale.py-correct-the-test-output-format.patch
+++ b/poky/meta/recipes-devtools/python/python3/0001-test_locale.py-correct-the-test-output-format.patch
@@ -23,24 +23,24 @@ Before this patch:
 
 Upstream-Status: Submitted [https://github.com/python/cpython/pull/15132]
 
+
+Rebased for 3.9.4, still not accepted upstream Signed-off-by: Alejandro Hernandez <alejandro@enedino.org>
+
 Signed-off-by: Mingli Yu <mingli.yu@windriver.com>
 ---
  Lib/test/test_locale.py | 2 +-
  1 file changed, 1 insertion(+), 1 deletion(-)
 
-diff --git a/Lib/test/test_locale.py b/Lib/test/test_locale.py
-index e2c2178..558d63c 100644
---- a/Lib/test/test_locale.py
-+++ b/Lib/test/test_locale.py
-@@ -527,7 +527,7 @@ class TestMiscellaneous(unittest.TestCase):
+Index: Python-3.9.4/Lib/test/test_locale.py
+===================================================================
+--- Python-3.9.4.orig/Lib/test/test_locale.py
++++ Python-3.9.4/Lib/test/test_locale.py
+@@ -562,7 +562,7 @@ class TestMiscellaneous(unittest.TestCas
              self.skipTest('test needs Turkish locale')
          loc = locale.getlocale(locale.LC_CTYPE)
          if verbose:
 -            print('testing with %a' % (loc,), end=' ', flush=True)
 +            print('testing with %a...' % (loc,), end=' ', flush=True)
-         locale.setlocale(locale.LC_CTYPE, loc)
-         self.assertEqual(loc, locale.getlocale(locale.LC_CTYPE))
- 
--- 
-2.7.4
-
+         try:
+             locale.setlocale(locale.LC_CTYPE, loc)
+         except locale.Error as exc:
diff --git a/poky/meta/recipes-devtools/python/python3/create_manifest3.py b/poky/meta/recipes-devtools/python/python3/create_manifest3.py
index 4da02a2991..045240ea0b 100644
--- a/poky/meta/recipes-devtools/python/python3/create_manifest3.py
+++ b/poky/meta/recipes-devtools/python/python3/create_manifest3.py
@@ -36,7 +36,7 @@
 # Tha method to handle cached files does not work when a module includes a folder which
 # itself contains the pycache folder, gladly this is almost never the case.
 #
-# Author: Alejandro Enedino Hernandez Samaniego "aehs29" <aehs29 at gmail dot com>
+# Author: Alejandro Enedino Hernandez Samaniego <alejandro at enedino dot org>
 
 
 import sys
@@ -45,6 +45,11 @@ import json
 import os
 import collections
 
+if '-d' in sys.argv:
+    debugFlag = '-d'
+else:
+    debugFlag = ''
+
 # Get python version from ${PYTHON_MAJMIN}
 pyversion = str(sys.argv[1])
 
@@ -84,6 +89,12 @@ def prepend_comments(comments, json_manifest):
         manifest.seek(0, 0)
         manifest.write(comments + json_contents)
 
+def print_indent(msg, offset):
+    for l in msg.splitlines():
+        msg = ' ' * offset + l
+        print(msg)
+
+
 # Read existing JSON manifest
 with open('python3-manifest.json') as manifest:
     # The JSON format doesn't allow comments so we hack the call to keep the comments using a marker
@@ -99,7 +110,7 @@ with open('python3-manifest.json') as manifest:
 # Not exactly the same so it should not be a function
 #
 
-print ('Getting dependencies for package: core')
+print_indent('Getting dependencies for package: core', 0)
 
 
 # This special call gets the core dependencies and
@@ -109,7 +120,7 @@ print ('Getting dependencies for package: core')
 # on the new core package, they will still find them
 # even when checking the old_manifest
 
-output = subprocess.check_output([sys.executable, 'get_module_deps3.py', 'python-core-package']).decode('utf8')
+output = subprocess.check_output([sys.executable, 'get_module_deps3.py', 'python-core-package', '%s' % debugFlag]).decode('utf8')
 for coredep in output.split():
     coredep = coredep.replace(pyversion,'${PYTHON_MAJMIN}')
     if isCached(coredep):
@@ -149,17 +160,16 @@ for filedep in old_manifest['core']['files']:
     # Get actual module name , shouldnt be affected by libdir/bindir, etc.
     pymodule = os.path.splitext(os.path.basename(os.path.normpath(filedep)))[0]
 
-
     # We now know that were dealing with a python module, so we can import it
     # and check what its dependencies are.
     # We launch a separate task for each module for deterministic behavior.
     # Each module will only import what is necessary for it to work in specific.
     # The output of each task will contain each module's dependencies
 
-    print ('Getting dependencies for module: %s' % pymodule)
-    output = subprocess.check_output([sys.executable, 'get_module_deps3.py', '%s' % pymodule]).decode('utf8')
-    print ('The following dependencies were found for module %s:\n' % pymodule)
-    print (output)
+    print_indent('Getting dependencies for module: %s' % pymodule, 2)
+    output = subprocess.check_output([sys.executable, 'get_module_deps3.py', '%s' % pymodule, '%s' % debugFlag]).decode('utf8')
+    print_indent('The following dependencies were found for module %s:\n' % pymodule, 4)
+    print_indent(output, 6)
 
 
     for pymodule_dep in output.split():
@@ -178,12 +188,13 @@ for filedep in old_manifest['core']['files']:
 # all others will use this a base.
 
 
+print('\n\nChecking for directories...\n')
 # To improve the script speed, we check which packages contain directories
 # since we will be looping through (only) those later.
 for pypkg in old_manifest:
     for filedep in old_manifest[pypkg]['files']:
         if isFolder(filedep):
-            print ('%s is a folder' % filedep)
+            print_indent('%s is a directory' % filedep, 2)
             if pypkg not in hasfolders:
                 hasfolders.append(pypkg)
             if filedep not in allfolders:
@@ -221,14 +232,14 @@ for pypkg in old_manifest:
 
     print('\n')
     print('--------------------------')
-    print ('Handling package %s' % pypkg)
+    print('Handling package %s' % pypkg)
     print('--------------------------')
 
     # Handle special cases, we assume that when they were manually added 
     # to the manifest we knew what we were doing.
     special_packages = ['misc', 'modules', 'dev', 'tests']
     if pypkg in special_packages or 'staticdev' in pypkg:
-        print('Passing %s package directly' % pypkg)
+        print_indent('Passing %s package directly' % pypkg, 2)
         new_manifest[pypkg] = old_manifest[pypkg]
         continue
 
@@ -259,7 +270,7 @@ for pypkg in old_manifest:
 
         # Get actual module name , shouldnt be affected by libdir/bindir, etc.
         # We need to check if the imported module comes from another (e.g. sqlite3.dump)
-        path,pymodule = os.path.split(filedep)
+        path, pymodule = os.path.split(filedep)
         path = os.path.basename(path)
         pymodule = os.path.splitext(os.path.basename(pymodule))[0]
 
@@ -279,10 +290,10 @@ for pypkg in old_manifest:
         # Each module will only import what is necessary for it to work in specific.
         # The output of each task will contain each module's dependencies
 
-        print ('\nGetting dependencies for module: %s' % pymodule)
-        output = subprocess.check_output([sys.executable, 'get_module_deps3.py', '%s' % pymodule]).decode('utf8')
-        print ('The following dependencies were found for module %s:\n' % pymodule)
-        print (output)
+        print_indent('\nGetting dependencies for module: %s' % pymodule, 2)
+        output = subprocess.check_output([sys.executable, 'get_module_deps3.py', '%s' % pymodule, '%s' % debugFlag]).decode('utf8')
+        print_indent('The following dependencies were found for module %s:\n' % pymodule, 4)
+        print_indent(output, 6)
 
         reportFILES = []
         reportRDEPS = []
@@ -325,7 +336,7 @@ for pypkg in old_manifest:
                             # print('Checking folder %s on package %s' % (pymodule_dep,pypkg_with_folder))
                             for folder_dep in old_manifest[pypkg_with_folder]['files'] or folder_dep in old_manifest[pypkg_with_folder]['cached']:
                                 if folder_dep == folder:
-                                    print ('%s folder found in %s' % (folder, pypkg_with_folder))
+                                    print ('%s directory found in %s' % (folder, pypkg_with_folder))
                                     folderFound = True
                                     if pypkg_with_folder not in new_manifest[pypkg]['rdepends'] and pypkg_with_folder != pypkg:
                                         new_manifest[pypkg]['rdepends'].append(pypkg_with_folder)
@@ -424,7 +435,7 @@ prepend_comments(comments,'python3-manifest.json.new')
 
 if (repeated):
     error_msg = '\n\nERROR:\n'
-    error_msg += 'The following files are repeated (contained in more than one package),\n'
+    error_msg += 'The following files were found in more than one package),\n'
     error_msg += 'this is likely to happen when new files are introduced after an upgrade,\n'
     error_msg += 'please check which package should get it,\n modify the manifest accordingly and re-run the create_manifest task:\n'
     error_msg += '\n'.join(repeated)
diff --git a/poky/meta/recipes-devtools/python/python3/get_module_deps3.py b/poky/meta/recipes-devtools/python/python3/get_module_deps3.py
index 6806f23172..1f4c982aed 100644
--- a/poky/meta/recipes-devtools/python/python3/get_module_deps3.py
+++ b/poky/meta/recipes-devtools/python/python3/get_module_deps3.py
@@ -3,14 +3,18 @@
 # them out, the output of this execution will have all dependencies
 # for a specific module, which will be parsed an dealt on create_manifest.py
 #
-# Author: Alejandro Enedino Hernandez Samaniego "aehs29" <aehs29@gmail.com>
+# Author: Alejandro Enedino Hernandez Samaniego <alejandro at enedino dot org>
 
-# We can get a log per module, for all the dependencies that were found, but its messy.
-debug=False
 
 import sys
 import os
 
+# We can get a log per module, for all the dependencies that were found, but its messy.
+if '-d' in sys.argv:
+    debug = True
+else:
+    debug = False
+
 # We can get a list of the modules which are currently required to run python
 # so we run python-core and get its modules, we then import what we need
 # and check what modules are currently running, if we substract them from the
@@ -19,13 +23,13 @@ import os
 # We use importlib to achieve this, so we also need to know what modules importlib needs
 import importlib
 
-core_deps=set(sys.modules)
+core_deps = set(sys.modules)
 
 def fix_path(dep_path):
     import os
     # We DONT want the path on our HOST system
-    pivot='recipe-sysroot-native'
-    dep_path=dep_path[dep_path.find(pivot)+len(pivot):]
+    pivot = 'recipe-sysroot-native'
+    dep_path = dep_path[dep_path.find(pivot)+len(pivot):]
 
     if '/usr/bin' in dep_path:
         dep_path = dep_path.replace('/usr/bin''${bindir}')
@@ -46,8 +50,8 @@ def fix_path(dep_path):
 
 # Module to import was passed as an argument
 current_module =  str(sys.argv[1]).rstrip()
-if(debug==True):
-    log = open('log_%s' % current_module,'w')
+if debug == True:
+    log = open('temp/log_%s' % current_module.strip('.*'),'w')
     log.write('Module %s generated the following dependencies:\n' % current_module)
 try:
     m = importlib.import_module(current_module)
@@ -63,13 +67,13 @@ try:
             except:
                 pass # ignore all import or other exceptions raised during import
 except ImportError as e:
-    if (debug==True):
-        log.write('Module was not found')
+    if debug == True:
+        log.write('Module was not found\n')
     pass
 
 
 # Get current module dependencies, dif will contain a list of specific deps for this module
-module_deps=set(sys.modules)
+module_deps = set(sys.modules)
 
 # We handle the core package (1st pass on create_manifest.py) as a special case
 if current_module == 'python-core-package':
@@ -81,14 +85,18 @@ else:
 
 # Check where each dependency came from
 for item in dif:
-    dep_path=''
+    # Main module returns script filename, __main matches mp_main__ as well
+    if 'main__' in item:
+        continue
+
+    dep_path = ''
     try:
-        if (debug==True):
-            log.write('Calling: sys.modules[' + '%s' % item + '].__file__\n')
+        if debug == True:
+            log.write('\nCalling: sys.modules[' + '%s' % item + '].__file__\n')
         dep_path = sys.modules['%s' % item].__file__
     except AttributeError as e:
         # Deals with thread (builtin module) not having __file__ attribute
-        if debug==True:
+        if debug == True:
             log.write(item + ' ')
             log.write(str(e))
             log.write('\n')
@@ -96,11 +104,16 @@ for item in dif:
     except NameError as e:
         # Deals with NameError: name 'dep_path' is not defined
         # because module is not found (wasn't compiled?), e.g. bddsm
-        if (debug==True):
+        if debug == True:
             log.write(item+' ') 
             log.write(str(e))                                              
         pass
 
+    if dep_path == '':
+        continue
+    if debug == True:
+        log.write('Dependency path found:\n%s\n' % dep_path)
+
     # Site-customize is a special case since we (OpenEmbedded) put it there manually
     if 'sitecustomize' in dep_path:
         dep_path = '${libdir}/python${PYTHON_MAJMIN}/sitecustomize.py'
@@ -111,52 +124,51 @@ for item in dif:
     dep_path = fix_path(dep_path)
 
     import sysconfig
-    soabi=sysconfig.get_config_var('SOABI')
+    soabi = sysconfig.get_config_var('SOABI')
     # Check if its a shared library and deconstruct it
     if soabi in dep_path:
-        if (debug==True):
-            log.write('Shared library found in %s' % dep_path)
+        if debug == True:
+            log.write('Shared library found in %s\n' % dep_path)
         dep_path = dep_path.replace(soabi,'*')
         print (dep_path)
         continue
     if "_sysconfigdata" in dep_path:
         dep_path = dep_path.replace(sysconfig._get_sysconfigdata_name(), "_sysconfigdata*")
 
-    if (debug==True):
+    if debug == True:
         log.write(dep_path+'\n')
     # Prints out result, which is what will be used by create_manifest
     print (dep_path)
 
 
-    import imp
-    cpython_tag = imp.get_tag() 
-    cached=''
+    cpython_tag = sys.implementation.cache_tag
+    cached = ''
     # Theres no naive way to find *.pyc files on python3
     try:
-        if (debug==True):
-            log.write('Calling: sys.modules[' + '%s' % item + '].__cached__\n')
+        if debug == True:
+            log.write('\nCalling: sys.modules[' + '%s' % item + '].__cached__\n')
         cached = sys.modules['%s' % item].__cached__
     except AttributeError as e:
         # Deals with thread (builtin module) not having __cached__ attribute
-        if debug==True:
+        if debug == True:
             log.write(item + ' ')
             log.write(str(e))
             log.write('\n')
         pass
     except NameError as e:
         # Deals with NameError: name 'cached' is not defined
-        if (debug==True):
+        if debug == True:
             log.write(item+' ') 
             log.write(str(e))                                              
         pass
     if cached is not None:
-        if (debug==True):
-            log.write(cached)
+        if debug == True:
+            log.write(cached + '\n')
         cached = fix_path(cached)
         cached = cached.replace(cpython_tag,'*')
         if "_sysconfigdata" in cached:
             cached = cached.replace(sysconfig._get_sysconfigdata_name(), "_sysconfigdata*")
         print (cached)
 
-if debug==True:
+if debug == True:
     log.close()