From c926e17c956a1babdf42d31f644bf0eedfa7f5f6 Mon Sep 17 00:00:00 2001 From: Andrew Geissler Date: Fri, 7 May 2021 16:11:35 -0500 Subject: poky: subtree update:1203d1f24d..2dcd1f2a21 MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Alejandro Enedino Hernandez Samaniego (2): python3: Improve logging, syntax and update deprecated modules to create_manifest python3: Upgrade 3.9.2 -> 3.9.4 Alexander Kanavin (22): scripts/oe-debuginfod: correct several issues libmicrohttpd: add a recipe from meta-oe maintainers.inc: add libmicrohttpd entry xwayland: add a standalone recipe weston: use standalone xwayland instead of outdated xserver-xorg version elfutils: correct debuginfod builds on x32 elfutils: adjust ptests for correct debuginfod testing default-distrovars.inc: add debuginfod to default DISTRO_FEATURES oeqa: tear down oeqa decorators if one of them raises an exception in setup meta/lib/oeqa/core/tests/cases/timeout.py: add a testcase for the previous fix core-image-weston: add sdk/ptest images oeqa/core/tests/test_data.py: use weston image instead of sato oeqa/selftest: transition to weston images core-image-multilib-example: base on weston, and not sato dev-manual/common-tasks.rst: correct the documentation for debuginfod diffoscope: add native libraries to LD_LIBRARY_PATH Revert "oeqa: Set LD_LIBRARY_PATH when executing native commands" boost: correct upstream version check vte: use tarballs again gdk-pixbuf: update 2.40.0 -> 2.42.6 glib-2.0: update 2.68.0 -> 2.68.1 gnu-config: update to latest revision Anatol Belski (1): cross-canadian: Whitelist "mingw32" as TARGET_OS Anders Wallin (3): lttng-tools: Fix missing legacy test files lttng-tools: Fix path for test_python_looging scripts/contrib/image-manifest: add new script Andreas Müller (1): xwayland: remove protocol.txt - it clashes with xserver-xorg Anthony Bagwell (1): systemd: upgrade 247.4 -> 247.6 Anuj Mittal (2): Revert "qemu: fix CVE-2021-3392" qemu: fix CVE-2021-3392 Armin Kuster (6): binutils: rename BRANCH var libseccomp: move recipe from meta-security to core gnutls: Enable seccomp if FEATURE is set systemd: Enable seccomp if FEATURE is set qemu: Enable seccomp if FEATURE is set default-distrovars.inc: Add seccomp to DISTRO_FEATURES_DEFAULT Bastian Krause (1): ccache: add packageconfig docs option Bruce Ashfield (20): kern-tools: add dropped options to audit output linux-yocto/5.4: update to v5.4.109 linux-yocto/5.10: update to v5.10.27 linux-yocto/5.10: BSP configuration fixes linux-yocto/5.10: update to v5.10.29 linux-yocto/5.4: update to v5.4.111 linux-yocto/5.10: update to v5.10.30 linux-yocto-rt/5.10: update to -rt34 linux-yocto/5.4: update to v5.4.112 linux-yocto/5.4: fix arm defconfig warnings linux-yocto/5.10: fix arm defconfig warnings linux-yocto/5.10: aufs fixes linux-yocto/5.10: qemuriscv32.cfg: RV32 only supports 1G physical memory linux-yocto/5.10: update to v5.10.32 perf: fix python-audit RDEPENDS linux-yocto/5.4: update to v5.4.114 linux-yocto/5.10: update to v5.10.34 linux-yocto/5.4: update to v5.4.116 linux-yocto/5.10: qemuppc32: reduce serial shutdown issues yocto-check-layer: Only note a layer without a conf/layer.conf (versus error) Changqing Li (2): libpam: make volatile files created successfully gcr: fix one parallel build failure Chen Qi (3): busybox: fix CVE-2021-28831 weston: fix build failure due to race condition rsync: fix CVE-2020-14387 Christophe Chapuis (1): rootfs.py: find .ko.gz and .ko.xz kernel modules as well Daniel Ammann (1): archiver: Fix typos Devendra Tewari (2): bitbake: lib/bb: Add bb.utils.rename() helper function and use for renaming classes/lib/scripts: Use bb.utils.rename() instead of os.rename() Diego Sueiro (3): oeqa/selftest/bblayers: Add test case for bitbake-layers layerindex-show-depends bitbake: layerindex: Fix bitbake-layers layerindex-show-depends command bitbake: layerindex: Add --fetchdir parameter to layerindex-fetch Douglas Royds (2): Revert "externalsrc: Detect code changes in submodules" externalsrc: Detect code changes in submodules Gavin Li (1): kmod: do not symlink config.guess/config.sub during autoreconf Harald Brinkmann (1): bitbake: fetch/svn: Fix parsing revision of SVN repos with redirects He Zhe (1): linux-yocto-dev: add features/scsi/scsi-debug.scc features/gpio/mockup.scc to KERNEL_FEATURES Henning Schild (3): bitbake: fetch/git: add support for disabling shared clones on unpack bitbake: tests/fetch: deduplicate local git testing code bitbake: tests/fetch: add tests for local and remote "noshared" git fetching Jon Mason (1): oeqa/runtime: space needed Jonas Höppner (1): ltp: fix empty ltp-dev package Jose Quaresma (4): gstreamer1.0: update patch upstream status ptest-runner: libgcc must be installed for pthread_cancel to work gstreamer1.0: rename patches gstreamer1.0: update ptest patch Joshua Watt (2): bitbake: knotty: Re-enable command line logging levels classes/image: Use xargs to set file timestamps Kai Kang (2): cmake.bbclass: remove ${B} before cmake_do_configure kernel-yocto.bbclass: chdir to ${WORKDIR} for do_kernel_checkout Kevin Hao (3): modutils-initscripts: Bail out when no module is installed sysvinit-inittab/start_getty: Check /sys for the tty device existence Revert "inittab: Add getty launch on hvc0 for qemuppc64" Khairul Rohaizzat Jamaluddin (1): qemu: Fix CVE-2020-35517 Khem Raj (54): gcc: Upgrade to 10.3.0 bug-fix release glibc: Rename glibc src package gcc-runtime: Make DEBUG_PREFIX_MAP relative to S valgrind: Delete trailing whitespaces valgrind: Add glibc-src to ptest rdeps valgrind: Add libstdc++ debug symbols for ptest vte: Upgrade to 0.64.0 release systemd: Fix build on mips/musl epiphany: Add missing dependency on gnutls cups: Turn gnutls into a packageconfig knob wpa-supplicant: Enable openssl curl: Use openssl backend libpsl: Add config knobs for runtime/builtin conversion choices glib-networking: Prefer openssl backend instead of gnutls gstreamer1.0-plugins-bad: Add packageconfigs for hls crypto backends ca-certificates: Fix openssl runtime cert dependencies weston: Drop loading xwayland.so module elfutils: Make 64bit time_t fix generic binutils: Fix linking failures when using dwarf-5 go: Use dl.google.com for SRC_URI musl: Update to latest master llvm: Upgrade to LLVM 12 release python3-docutils: Upgrade to 0.17.1 python3-markupsafe: Enable ptests python3-jinja2: Enable ptests python3-pyyaml: Add recipe apt: Fix build on musl when seccomp is enabled default-distrovars.inc: Remove seccomp for riscv32 gcc-target: Create a LTO plugin symlink in bfd-plugins directory bitbake.conf: Use gcc-nm as default NM gcc-cross: Install linker LTO plugin for binutils tools gcc-cross-canadian: Install LTO linker plugin to BFD searchable location gnutls: Point to staging area for finding seccomp libs and includes libjpeg-turbo: Use --reproducible option for nasm libid3tag: Filter -ffile-prefix-map too openssl: Filter out -ffile-prefix-map as well ltp: Filter out -ffile-prefix-map gcc-runtime: Fix __FILE__ related reproducablity issues reproducible_build.bbclass: Enable -Wdate-time pkgconfig: Fix nativesdk builds for mingw sdk hosts m4: Do not use SIGSTKSZ bluez: Fix shadowing of pause function from libc valgrind: Disable leak_cpp_interior test findutils: Do not use SIGSTKSZ bash: Include files needed for run-heredoc ptest libpam: Provide needed env for tst-pam_start_confdir ptest cml1.bbclass: Return sorted list of cfg files busybox: Enable long options for enabled applets webkitgtk: Fix reproducibility in minibrowser webkitgtk: Update patch status libgcc-initial: Do not build fp128 to decimal ppc functions gcc: Upgrade to GCC 11 busybox: Fix reproducibility strace: Upgrade to 5.12 Konrad Weihmann (2): cpan-base: set default UPSTREAM_CHECK_REGEX cve-update-db-native: skip on empty cpe23Uri Marek Vasut (1): linux-firmware: Package RSI 911x WiFi firmware Martin Jansa (2): xwayland: add opengl to REQUIRED_DISTRO_FEATURES ofono: prevent using bundled ell headers and fix build with ell-0.39 Michael Halstead (1): releases: update to include 3.3 Michael Opdenacker (7): dev-manual: fix code insertion manuals: simplify code insertion manuals: code insertion simplification over two lines bitbake: doc: bitbake-user-manual: simplify colon usage bitbake: doc: bitbake-user-manual: code insertion simplification over two lines dev-manual: update references to Docker installation instructions sanity.bbclass: mention CONNECTIVITY_CHECK_URIS in network failure message Mikko Rapeli (4): bitbake: bitbake: tests/fetch: fix test execution without .gitconfig bitbake: bitbake: tests/fetch: remove write protected files too lz4: use CFLAGS from bitbake unzip: use optimization from bitbake Mingli Yu (6): libxshmfence: Build fixes for riscv32 packagegroup-core-tools-profile: Remove valgrind for riscv32 packagegroup-core-tools-testapps.bb: Remove kexec for riscv32 libtool: make sure autoheader run before automake groff: not ship /usr/bin/grap2graph rpm: Upgrade to 4.16.1.3 Minjae Kim (1): qemu: fix CVE-2021-3392 Nicolas Dechesne (1): bitbake: doc: bitbake-user-manual: fix typo left over from Sphinx migration Niels Avonds (1): bitbake: fetch/gitsm: Fix crash when using git LFS and submodules Oleksandr Kravchuk (2): python3-setuptools: update to 56.0.0 autoconf-archive: update to 2021.02.19 Otavio Salvador (2): gstreamer1.0-plugins-base: Add 'viv-fb' OpenGL Window System option gstreamer1.0-plugins-base: Use bb.utils.filter to reduce code Paul Barker (10): bitbake: hashserv: Use generic ConnectionError bitbake: asyncrpc: Common implementation of RPC using json & asyncio bitbake: hashserv: Refactor to use asyncrpc bitbake: prserv: Drop obsolete python version check bitbake: prserv: Drop unused dump_db method bitbake: prserv: Add connect function prservice: Use new connect API bitbake: prserv: Use multiprocessing to auto start prserver bitbake: prserv: Extract daemonization from PRServer class bitbake: prserv: Handle requests in main thread Paulo Cesar Zaneti (1): perl: fix startperl configuration option for perl-native Peter Budny (1): lib/oe/terminal: Fix tmux new-session on older tmux versions (<1.9) Petr Vorel (1): ltp: Replace musl patches with do_patch[postfuncs] Przemyslaw Gorszkowski (2): bitbake: progress: LineFilterProgressHandler - Handle parsing line which ends with CR only bitbake: fetch/s3: Add progress handler for S3 cp command Randy MacLeod (2): sqlite3: upgrade 3.35.0 -> 3.35.3 oe-time-dd-test.sh: increase timeout to 15 sec Reto Schneider (2): license_image.bbclass: Detect broken symlinks license_image.bbclass: Fix symlink to generic license files Richard Purdie (32): oeqa/selftest: Hardcode test assumptions about heartbeat event timings pseudo: Upgrade to add trailing slashes ignore path fix oeqa/selftest: Ensure packages classes are set correctly for maintainers test layer.conf: Update to add post 3.3 release honister series sanity: Add error check for '%' in build path bitbake: runqueue: Fix deferred task issues bitbake: tinfoil/data_smart: Allow variable history emit() to function remotely sanity: Further improve directory sanity tests bitbake: bitbake-server: Remove now unneeded code bitbake: doc/user-manual-fetching: Remove basepath unpack parameter docs poky.conf: Post release version bump runqemu: Ensure we cleanup snapshot files after image run patchelf: Backport fix from upstream for note section overlap error pyyaml: Add missing HOMEPAGE yocto-check-layer: Avoid bug when iterating and autoadding dependencies libseccomp: Add MAINTAINERS entry and HOMEPAGE libseccomp: Fix reproducibility issue apt: Disable libseccomp libxcrypt: Update to 4.4.19 release and fix symbol version issues patchelf: Fix note section alignment issues bitbake: runqueue: Fix multiconfig deferred task sstate validity caching issue bitbake: runqueue: Handle deferred task rehashing in multiconfig builds patchelf: Fix alignment patch pybootchart/draw: Avoid divide by zero error yocto-uninative: Update to 3.1 which includes a patchelf fix Revert "perl: fix startperl configuration option for perl-native" bitbake: bin/bitbake-getvar: Add a new command to query a variable value (with history) bitbake: bitbake: Switch to post release version number 1.51.0 sanity.conf: Require bitbake 1.51.0 oeqa/qemurunner: Improve logging thread exit handling for qemu shutdown test oeqa/qemurunner: Handle path length issues for qmp socket lib/package_manager: Use shutil.copy instead of bb.utils.copyfile for intercepts Robert Joslyn (3): btrfs-tools: Update to 5.11.1 btrfs-tools: Add PACKAGECONFIG options btrfs-tools: Try to follow style guide Robert P. J. Day (3): sdk-manual: "beablebone" -> "beaglebone" sdk-manual: fix broken formatting of sample command bitbake.conf: sort MIRROR list, add missing SAMBA_MIRROR Ross Burton (4): glslang: strip whitespace in pkgconfig file insane: clean up some more warning messages bitbake: bitbake-server: ensure server timeout is a float oe-buildenv-internal: add BitBake's library to PYTHONPATH Sakib Sajal (12): oe-time-dd-test.sh: make executable oe-time-dd-test.sh: provide more information from "top" qemu: fix CVE-2021-20181 qemu: fix CVE-2020-29443 qemu: fix CVE-2021-20221 qemu: fix CVE-2021-3409 qemu: fix CVE-2021-3416 qemu: fix CVE-2021-20257 oe-time-dd-test.sh: collect cooker log when timeout is exceeded buildstats.bbclass: collect data in the same file. qemu: fix CVE-2020-27821 qemu: fix CVE-2021-20263 Samuli Piippo (1): assimp: BBCLASSEXTEND to native and nativesdk Saul Wold (4): pango: re-enable ptest qemu-system-native: install qmp python module qemurunner: Add support for qmp commands qemurunner: change warning to info Stefan Ghinea (3): wpa-supplicant: fix CVE-2021-30004 libssh2: fix build failure with option no-ecdsa xserver-xorg: fix CVE-2021-3472 Stefano Babic (1): libubootenv: upgrade 0.3.1 -> 0.3.2 Teoh Jay Shen (6): oeqa/manual/bsp-hw.json : remove boot_from_runlevel_3 and boot_from_runlevel_5 manual test oeqa/manual/bsp-hw.json : remove ethernet_static_ip_set_in_connman and ethernet_get_IP_in_connman_via_DHCP manual test oeqa/manual/bsp-hw.json : remove standby and Test_if_LAN_device_works_well_after_resume_from_suspend_state manual test oeqa/manual/bsp-hw.json : remove click_terminal_icon_on_X_desktop manual test oeqa/manual/bsp-hw.json :remove Check_if_RTC_(Real_Time_Clock)_can_work_correctly manual test oeqa/manual/bsp-hw.json : remove Test_if_usb_hid_device_works_well_after_resume_from_suspend_state manual test Trevor Gamblin (2): nettle: upgrade 3.7.1 -> 3.7.2 ref-manual/variables.rst: Add incompatibility warning for SERIAL_CONSOLES_CHECK Ulrich Ölmann (1): arch-armv6m.inc: fix access rights Vinay Kumar (1): binutils: Fix CVE-2021-20197 Vineela Tummalapalli (1): Adding dunfell 3.1.7 to the switcher and release list. Wang Mingyu (6): at-spi2-core: upgrade 2.38.0 -> 2.40.0 babeltrace2: upgrade 2.0.3 -> 2.0.4 boost-build-native: upgrade 4.3.0 -> 4.4.1 libassuan: upgrade 2.5.4 -> 2.5.5 webkitgtk: upgrade 2.30.5 -> 2.30.6 vte: upgrade 0.62.2 -> 0.62.3 Wes Lindauer (1): oeqa/runtime/cases: Only disable/enable for current boot Yanfei Xu (1): parselogs: ignore floppy error on qemu-system-x86 at boot stage Yi Fan Yu (7): valgrind: update 3.16.1 -> 3.17.0 valgrind: Disable ptest swapcontext.vgtest valgrind: Fix ptest swapcontext.vgtest Revert "glib-2.0: add workaround to fix codegen.py.test failing" re2c: Upgrade 2.0.3 -> 2.1.1 valgrind: Enable drd/tests/bar_bad* ptest libevent: Increase ptest timing tolerance 50 ms -> 100 ms Zqiang (1): rt-tests: Update rt-tests hongxu (1): deb: apply postinstall on sdk wangmy (34): ell: upgrade 0.38 -> 0.39 dbus-glib: upgrade 0.110 -> 0.112 ccache: upgrade 4.2 -> 4.2.1 gcr: upgrade 3.38.1 -> 3.40.0 ghostscript: upgrade 9.53.3 -> 9.54.0 libsolv: upgrade 0.7.17 -> 0.7.18 glib-2.0: upgrade 2.66.7 -> 2.68.0 file: upgrade 5.39 -> 5.40 curl: upgrade 7.75.0 -> 7.76.0 acpica: upgrade 20210105 -> 20210331 help2man: upgrade 1.48.2 -> 1.48.3 libportal: upgrade 0.3 -> 0.4 libksba: upgrade 1.5.0 -> 1.5.1 go: upgrade 1.16.2 -> 1.16.3 libcap: upgrade 2.48 -> 2.49 libcomps: upgrade 0.1.15 -> 0.1.16 icu: upgrade 68.2 -> 69.1 mpg123: upgrade 1.26.4 -> 1.26.5 man-pages: upgrade 5.10 -> 5.11 go: update SRC_URI to use https protocol mesa: upgrade 21.0.1 -> 21.0.2 openssh: upgrade 8.5p1 -> 8.6p1 mtools: upgrade 4.0.26 -> 4.0.27 python3-cython: upgrade 0.29.22 -> 0.29.23 tiff: upgrade 4.2.0 -> 4.3.0 boost: upgrade 1.75.0 -> 1.76.0 wpebackend-fdo: upgrade 1.8.2 -> 1.8.3 mesa: upgrade 21.0.2 -> 21.0.3 gdb: upgrade 10.1 -> 10.2 glib-networking: upgrade 2.66.0 -> 2.68.1 glslang: upgrade 11.2.0 -> 11.4.0 hdparm: upgrade 9.60 -> 9.61 libhandy: upgrade 1.2.1 -> 1.2.2 libjitterentropy: upgrade 3.0.1 -> 3.0.2 zangrc (1): maintainers.inc: Modify email address zhengruoqin (19): epiphany: upgrade 3.38.2 -> 3.38.3 wpebackend-fdo: upgrade 1.8.0 -> 1.8.2 netbase: upgrade 6.2 -> 6.3 python3-dbusmock: upgrade 0.22.0 -> 0.23.0 python3-gitdb: upgrade 4.0.5 -> 4.0.7 libva: upgrade 2.10.0 -> 2.11.0 ruby: upgrade 3.0.0 -> 3.0.1 libva-utils: upgrade 2.10.0 -> 2.11.1 libdazzle: upgrade 3.38.0 -> 3.40.0 librepo: upgrade 1.13.0 -> 1.14.0 libdrm: upgrade 2.4.104 -> 2.4.105 python3-pygobject: upgrade 3.38.0 -> 3.40.1 libedit: upgrade 20210216-3.1 -> 20210419-3.1 libhandy: upgrade 1.2.0 -> 1.2.1 libical: upgrade 3.0.9 -> 3.0.10 libsolv: upgrade 0.7.18 -> 0.7.19 libmicrohttpd: upgrade 0.9.72 -> 0.9.73 python3-numpy: upgrade 1.20.1 -> 1.20.2 wireless-regdb: upgrade 2020.11.20 -> 2021.04.21 Signed-off-by: Andrew Geissler Change-Id: Ibdaea694cae40b0749d472bf08b53002a45b31d7 --- poky/scripts/contrib/image-manifest | 523 ++++++++++++++++++++++++++++++++++++ 1 file changed, 523 insertions(+) create mode 100755 poky/scripts/contrib/image-manifest (limited to 'poky/scripts/contrib') diff --git a/poky/scripts/contrib/image-manifest b/poky/scripts/contrib/image-manifest new file mode 100755 index 0000000000..3c07a73a4e --- /dev/null +++ b/poky/scripts/contrib/image-manifest @@ -0,0 +1,523 @@ +#!/usr/bin/env python3 + +# Script to extract information from image manifests +# +# Copyright (C) 2018 Intel Corporation +# Copyright (C) 2021 Wind River Systems, Inc. +# +# SPDX-License-Identifier: GPL-2.0-only +# + +import sys +import os +import argparse +import logging +import json +import shutil +import tempfile +import tarfile +from collections import OrderedDict + +scripts_path = os.path.dirname(__file__) +lib_path = scripts_path + '/../lib' +sys.path = sys.path + [lib_path] + +import scriptutils +logger = scriptutils.logger_create(os.path.basename(__file__)) + +import argparse_oe +import scriptpath +bitbakepath = scriptpath.add_bitbake_lib_path() +if not bitbakepath: + logger.error("Unable to find bitbake by searching parent directory of this script or PATH") + sys.exit(1) +logger.debug('Using standard bitbake path %s' % bitbakepath) +scriptpath.add_oe_lib_path() + +import bb.tinfoil +import bb.utils +import oe.utils +import oe.recipeutils + +def get_pkg_list(manifest): + pkglist = [] + with open(manifest, 'r') as f: + for line in f: + linesplit = line.split() + if len(linesplit) == 3: + # manifest file + pkglist.append(linesplit[0]) + elif len(linesplit) == 1: + # build dependency file + pkglist.append(linesplit[0]) + return sorted(pkglist) + +def list_packages(args): + pkglist = get_pkg_list(args.manifest) + for pkg in pkglist: + print('%s' % pkg) + +def pkg2recipe(tinfoil, pkg): + if "-native" in pkg: + logger.info('skipping %s' % pkg) + return None + + pkgdata_dir = tinfoil.config_data.getVar('PKGDATA_DIR') + pkgdatafile = os.path.join(pkgdata_dir, 'runtime-reverse', pkg) + logger.debug('pkgdatafile %s' % pkgdatafile) + try: + f = open(pkgdatafile, 'r') + for line in f: + if line.startswith('PN:'): + recipe = line.split(':', 1)[1].strip() + return recipe + except Exception: + logger.warning('%s is missing' % pkgdatafile) + return None + +def get_recipe_list(manifest, tinfoil): + pkglist = get_pkg_list(manifest) + recipelist = [] + for pkg in pkglist: + recipe = pkg2recipe(tinfoil,pkg) + if recipe: + if not recipe in recipelist: + recipelist.append(recipe) + + return sorted(recipelist) + +def list_recipes(args): + import bb.tinfoil + with bb.tinfoil.Tinfoil() as tinfoil: + tinfoil.logger.setLevel(logger.getEffectiveLevel()) + tinfoil.prepare(config_only=True) + recipelist = get_recipe_list(args.manifest, tinfoil) + for recipe in sorted(recipelist): + print('%s' % recipe) + +def list_layers(args): + + def find_git_repo(pth): + checkpth = pth + while checkpth != os.sep: + if os.path.exists(os.path.join(checkpth, '.git')): + return checkpth + checkpth = os.path.dirname(checkpth) + return None + + def get_git_remote_branch(repodir): + try: + stdout, _ = bb.process.run(['git', 'rev-parse', '--abbrev-ref', '--symbolic-full-name', '@{u}'], cwd=repodir) + except bb.process.ExecutionError as e: + stdout = None + if stdout: + return stdout.strip() + else: + return None + + def get_git_head_commit(repodir): + try: + stdout, _ = bb.process.run(['git', 'rev-parse', 'HEAD'], cwd=repodir) + except bb.process.ExecutionError as e: + stdout = None + if stdout: + return stdout.strip() + else: + return None + + def get_git_repo_url(repodir, remote='origin'): + import bb.process + # Try to get upstream repo location from origin remote + try: + stdout, _ = bb.process.run(['git', 'remote', '-v'], cwd=repodir) + except bb.process.ExecutionError as e: + stdout = None + if stdout: + for line in stdout.splitlines(): + splitline = line.split() + if len(splitline) > 1: + if splitline[0] == remote and scriptutils.is_src_url(splitline[1]): + return splitline[1] + return None + + with bb.tinfoil.Tinfoil() as tinfoil: + tinfoil.logger.setLevel(logger.getEffectiveLevel()) + tinfoil.prepare(config_only=False) + layers = OrderedDict() + for layerdir in tinfoil.config_data.getVar('BBLAYERS').split(): + layerdata = OrderedDict() + layername = os.path.basename(layerdir) + logger.debug('layername %s, layerdir %s' % (layername, layerdir)) + if layername in layers: + logger.warning('layername %s is not unique in configuration' % layername) + layername = os.path.basename(os.path.dirname(layerdir)) + '_' + os.path.basename(layerdir) + logger.debug('trying layername %s' % layername) + if layername in layers: + logger.error('Layer name %s is not unique in configuration' % layername) + sys.exit(2) + repodir = find_git_repo(layerdir) + if repodir: + remotebranch = get_git_remote_branch(repodir) + remote = 'origin' + if remotebranch and '/' in remotebranch: + rbsplit = remotebranch.split('/', 1) + layerdata['actual_branch'] = rbsplit[1] + remote = rbsplit[0] + layerdata['vcs_url'] = get_git_repo_url(repodir, remote) + if os.path.abspath(repodir) != os.path.abspath(layerdir): + layerdata['vcs_subdir'] = os.path.relpath(layerdir, repodir) + commit = get_git_head_commit(repodir) + if commit: + layerdata['vcs_commit'] = commit + layers[layername] = layerdata + + json.dump(layers, args.output, indent=2) + +def get_recipe(args): + with bb.tinfoil.Tinfoil() as tinfoil: + tinfoil.logger.setLevel(logger.getEffectiveLevel()) + tinfoil.prepare(config_only=True) + + recipe = pkg2recipe(tinfoil, args.package) + print(' %s package provided by %s' % (args.package, recipe)) + +def pkg_dependencies(args): + def get_recipe_info(tinfoil, recipe): + try: + info = tinfoil.get_recipe_info(recipe) + except Exception: + logger.error('Failed to get recipe info for: %s' % recipe) + sys.exit(1) + if not info: + logger.warning('No recipe info found for: %s' % recipe) + sys.exit(1) + append_files = tinfoil.get_file_appends(info.fn) + appends = True + data = tinfoil.parse_recipe_file(info.fn, appends, append_files) + data.pn = info.pn + data.pv = info.pv + return data + + def find_dependencies(tinfoil, assume_provided, recipe_info, packages, rn, order): + spaces = ' ' * order + data = recipe_info[rn] + if args.native: + logger.debug('%s- %s' % (spaces, data.pn)) + elif "-native" not in data.pn: + if "cross" not in data.pn: + logger.debug('%s- %s' % (spaces, data.pn)) + + depends = [] + for dep in data.depends: + if dep not in assume_provided: + depends.append(dep) + + # First find all dependencies not in package list. + for dep in depends: + if dep not in packages: + packages.append(dep) + dep_data = get_recipe_info(tinfoil, dep) + # Do this once now to reduce the number of bitbake calls. + dep_data.depends = dep_data.getVar('DEPENDS').split() + recipe_info[dep] = dep_data + + # Then recursively analyze all of the dependencies for the current recipe. + for dep in depends: + find_dependencies(tinfoil, assume_provided, recipe_info, packages, dep, order + 1) + + with bb.tinfoil.Tinfoil() as tinfoil: + tinfoil.logger.setLevel(logger.getEffectiveLevel()) + tinfoil.prepare() + + assume_provided = tinfoil.config_data.getVar('ASSUME_PROVIDED').split() + logger.debug('assumed provided:') + for ap in sorted(assume_provided): + logger.debug(' - %s' % ap) + + recipe = pkg2recipe(tinfoil, args.package) + data = get_recipe_info(tinfoil, recipe) + data.depends = [] + depends = data.getVar('DEPENDS').split() + for dep in depends: + if dep not in assume_provided: + data.depends.append(dep) + + recipe_info = dict([(recipe, data)]) + packages = [] + find_dependencies(tinfoil, assume_provided, recipe_info, packages, recipe, order=1) + + print('\nThe following packages are required to build %s' % recipe) + for p in sorted(packages): + data = recipe_info[p] + if "-native" not in data.pn: + if "cross" not in data.pn: + print(" %s (%s)" % (data.pn,p)) + + if args.native: + print('\nThe following native packages are required to build %s' % recipe) + for p in sorted(packages): + data = recipe_info[p] + if "-native" in data.pn: + print(" %s(%s)" % (data.pn,p)) + if "cross" in data.pn: + print(" %s(%s)" % (data.pn,p)) + +def default_config(): + vlist = OrderedDict() + vlist['PV'] = 'yes' + vlist['SUMMARY'] = 'no' + vlist['DESCRIPTION'] = 'no' + vlist['SECTION'] = 'no' + vlist['LICENSE'] = 'yes' + vlist['HOMEPAGE'] = 'no' + vlist['BUGTRACKER'] = 'no' + vlist['PROVIDES'] = 'no' + vlist['BBCLASSEXTEND'] = 'no' + vlist['DEPENDS'] = 'no' + vlist['PACKAGECONFIG'] = 'no' + vlist['SRC_URI'] = 'yes' + vlist['SRCREV'] = 'yes' + vlist['EXTRA_OECONF'] = 'no' + vlist['EXTRA_OESCONS'] = 'no' + vlist['EXTRA_OECMAKE'] = 'no' + vlist['EXTRA_OEMESON'] = 'no' + + clist = OrderedDict() + clist['variables'] = vlist + clist['filepath'] = 'no' + clist['sha256sum'] = 'no' + clist['layerdir'] = 'no' + clist['layer'] = 'no' + clist['inherits'] = 'no' + clist['source_urls'] = 'no' + clist['packageconfig_opts'] = 'no' + clist['patches'] = 'no' + clist['packagedir'] = 'no' + return clist + +def dump_config(args): + config = default_config() + f = open('default_config.json', 'w') + json.dump(config, f, indent=2) + logger.info('Default config list dumped to default_config.json') + +def export_manifest_info(args): + + def handle_value(value): + if value: + return oe.utils.squashspaces(value) + else: + return value + + if args.config: + logger.debug('config: %s' % args.config) + f = open(args.config, 'r') + config = json.load(f, object_pairs_hook=OrderedDict) + else: + config = default_config() + if logger.isEnabledFor(logging.DEBUG): + print('Configuration:') + json.dump(config, sys.stdout, indent=2) + print('') + + tmpoutdir = tempfile.mkdtemp(prefix=os.path.basename(__file__)+'-') + logger.debug('tmp dir: %s' % tmpoutdir) + + # export manifest + shutil.copy2(args.manifest,os.path.join(tmpoutdir, "manifest")) + + with bb.tinfoil.Tinfoil(tracking=True) as tinfoil: + tinfoil.logger.setLevel(logger.getEffectiveLevel()) + tinfoil.prepare(config_only=False) + + pkglist = get_pkg_list(args.manifest) + # export pkg list + f = open(os.path.join(tmpoutdir, "pkgs"), 'w') + for pkg in pkglist: + f.write('%s\n' % pkg) + f.close() + + recipelist = [] + for pkg in pkglist: + recipe = pkg2recipe(tinfoil,pkg) + if recipe: + if not recipe in recipelist: + recipelist.append(recipe) + recipelist.sort() + # export recipe list + f = open(os.path.join(tmpoutdir, "recipes"), 'w') + for recipe in recipelist: + f.write('%s\n' % recipe) + f.close() + + try: + rvalues = OrderedDict() + for pn in sorted(recipelist): + logger.debug('Package: %s' % pn) + rd = tinfoil.parse_recipe(pn) + + rvalues[pn] = OrderedDict() + + for varname in config['variables']: + if config['variables'][varname] == 'yes': + rvalues[pn][varname] = handle_value(rd.getVar(varname)) + + fpth = rd.getVar('FILE') + layerdir = oe.recipeutils.find_layerdir(fpth) + if config['filepath'] == 'yes': + rvalues[pn]['filepath'] = os.path.relpath(fpth, layerdir) + if config['sha256sum'] == 'yes': + rvalues[pn]['sha256sum'] = bb.utils.sha256_file(fpth) + + if config['layerdir'] == 'yes': + rvalues[pn]['layerdir'] = layerdir + + if config['layer'] == 'yes': + rvalues[pn]['layer'] = os.path.basename(layerdir) + + if config['inherits'] == 'yes': + gr = set(tinfoil.config_data.getVar("__inherit_cache") or []) + lr = set(rd.getVar("__inherit_cache") or []) + rvalues[pn]['inherits'] = sorted({os.path.splitext(os.path.basename(r))[0] for r in lr if r not in gr}) + + if config['source_urls'] == 'yes': + rvalues[pn]['source_urls'] = [] + for url in (rd.getVar('SRC_URI') or '').split(): + if not url.startswith('file://'): + url = url.split(';')[0] + rvalues[pn]['source_urls'].append(url) + + if config['packageconfig_opts'] == 'yes': + rvalues[pn]['packageconfig_opts'] = OrderedDict() + for key in rd.getVarFlags('PACKAGECONFIG').keys(): + if key == 'doc': + continue + rvalues[pn]['packageconfig_opts'][key] = rd.getVarFlag('PACKAGECONFIG', key, True) + + if config['patches'] == 'yes': + patches = oe.recipeutils.get_recipe_patches(rd) + rvalues[pn]['patches'] = [] + if patches: + recipeoutdir = os.path.join(tmpoutdir, pn, 'patches') + bb.utils.mkdirhier(recipeoutdir) + for patch in patches: + # Patches may be in other layers too + patchlayerdir = oe.recipeutils.find_layerdir(patch) + # patchlayerdir will be None for remote patches, which we ignore + # (since currently they are considered as part of sources) + if patchlayerdir: + rvalues[pn]['patches'].append((os.path.basename(patchlayerdir), os.path.relpath(patch, patchlayerdir))) + shutil.copy(patch, recipeoutdir) + + if config['packagedir'] == 'yes': + pn_dir = os.path.join(tmpoutdir, pn) + bb.utils.mkdirhier(pn_dir) + f = open(os.path.join(pn_dir, 'recipe.json'), 'w') + json.dump(rvalues[pn], f, indent=2) + f.close() + + with open(os.path.join(tmpoutdir, 'recipes.json'), 'w') as f: + json.dump(rvalues, f, indent=2) + + if args.output: + outname = os.path.basename(args.output) + else: + outname = os.path.splitext(os.path.basename(args.manifest))[0] + if outname.endswith('.tar.gz'): + outname = outname[:-7] + elif outname.endswith('.tgz'): + outname = outname[:-4] + + tarfn = outname + if tarfn.endswith(os.sep): + tarfn = tarfn[:-1] + if not tarfn.endswith(('.tar.gz', '.tgz')): + tarfn += '.tar.gz' + with open(tarfn, 'wb') as f: + with tarfile.open(None, "w:gz", f) as tar: + tar.add(tmpoutdir, outname) + finally: + shutil.rmtree(tmpoutdir) + + +def main(): + parser = argparse_oe.ArgumentParser(description="Image manifest utility", + epilog="Use %(prog)s --help to get help on a specific command") + parser.add_argument('-d', '--debug', help='Enable debug output', action='store_true') + parser.add_argument('-q', '--quiet', help='Print only errors', action='store_true') + subparsers = parser.add_subparsers(dest="subparser_name", title='subcommands', metavar='') + subparsers.required = True + + # get recipe info + parser_get_recipes = subparsers.add_parser('recipe-info', + help='Get recipe info', + description='Get recipe information for a package') + parser_get_recipes.add_argument('package', help='Package name') + parser_get_recipes.set_defaults(func=get_recipe) + + # list runtime dependencies + parser_pkg_dep = subparsers.add_parser('list-depends', + help='List dependencies', + description='List dependencies required to build the package') + parser_pkg_dep.add_argument('--native', help='also print native and cross packages', action='store_true') + parser_pkg_dep.add_argument('package', help='Package name') + parser_pkg_dep.set_defaults(func=pkg_dependencies) + + # list recipes + parser_recipes = subparsers.add_parser('list-recipes', + help='List recipes producing packages within an image', + description='Lists recipes producing the packages that went into an image, using the manifest and pkgdata') + parser_recipes.add_argument('manifest', help='Manifest file') + parser_recipes.set_defaults(func=list_recipes) + + # list packages + parser_packages = subparsers.add_parser('list-packages', + help='List packages within an image', + description='Lists packages that went into an image, using the manifest') + parser_packages.add_argument('manifest', help='Manifest file') + parser_packages.set_defaults(func=list_packages) + + # list layers + parser_layers = subparsers.add_parser('list-layers', + help='List included layers', + description='Lists included layers') + parser_layers.add_argument('-o', '--output', help='Output file - defaults to stdout if not specified', + default=sys.stdout, type=argparse.FileType('w')) + parser_layers.set_defaults(func=list_layers) + + # dump default configuration file + parser_dconfig = subparsers.add_parser('dump-config', + help='Dump default config', + description='Dump default config to default_config.json') + parser_dconfig.set_defaults(func=dump_config) + + # export recipe info for packages in manifest + parser_export = subparsers.add_parser('manifest-info', + help='Export recipe info for a manifest', + description='Export recipe information using the manifest') + parser_export.add_argument('-c', '--config', help='load config from json file') + parser_export.add_argument('-o', '--output', help='Output file (tarball) - defaults to manifest name if not specified') + parser_export.add_argument('manifest', help='Manifest file') + parser_export.set_defaults(func=export_manifest_info) + + args = parser.parse_args() + + if args.debug: + logger.setLevel(logging.DEBUG) + logger.debug("Debug Enabled") + elif args.quiet: + logger.setLevel(logging.ERROR) + + ret = args.func(args) + + return ret + + +if __name__ == "__main__": + try: + ret = main() + except Exception: + ret = 1 + import traceback + traceback.print_exc() + sys.exit(ret) -- cgit v1.2.3