diff options
author | Brad Bishop <bradleyb@fuzziesquirrel.com> | 2019-09-24 05:34:48 +0300 |
---|---|---|
committer | Brad Bishop <bradleyb@fuzziesquirrel.com> | 2019-09-24 05:35:28 +0300 |
commit | a34c030e5ec7021e7fb452410d38abfb3993ec68 (patch) | |
tree | b8f41acc23015f7f1de01de3c87c434195c681d4 /poky/scripts | |
parent | dec060e0fadb20cc708370ae192db4462283ee58 (diff) | |
download | openbmc-a34c030e5ec7021e7fb452410d38abfb3993ec68.tar.xz |
poky: subtree update:745e38ff0f..81f9e815d3
Adrian Bunk (6):
openssl: Upgrade 1.1.1c -> 1.1.1d
glib-2.0: Upgrade 2.60.6 -> 2.60.7
lttng-modules: Upgrade 2.10.10 -> 2.10.11
lttng-ust: Upgrade 2.10.4 -> 2.10.5
squashfs-tools: Remove UPSTREAM_CHECK_COMMITS
libmpc: Remove dead UPSTREAM_CHECK_URI
Alexander Kanavin (2):
runqemu: decouple gtk and gl options
strace: add a timeout for running ptests
Alistair Francis (1):
gdb: Mark gdbserver as ALLOW_EMPTY for riscv32
Andre McCurdy (9):
busybox: drop unused mount.busybox and umount.busybox wrappers
busybox: drop inittab from SRC_URI ( now moved to busybox-inittab )
busybox-inittab: minor formatting tweaks
base-files: drop legacy empty file /etc/default/usbd
busybox: rcS and rcK should not be writeable by everyone
ffmpeg: add PACKAGECONFIG controls for alsa and zlib (enable by default)
libwebp: apply ARM specific config options to big endian ARM
initscripts: enable alignment.sh init script for big endian ARM
libunwind: apply configure over-ride to both big and little endian ARM
Andrew F. Davis (4):
libepoxy: Disable x11 when not building for x11
cogl: Set depends to the virtual needed not explicitly on Mesa
gtk+3: Set depends to the virtual needed not explicitly on Mesa
weston: Set depends to the virtual needed not explicitly on Mesa
Armin Kuster (1):
gcc: Security fix for CVE-2019-15847
Changhyeok Bae (1):
iw: upgrade to 5.3
Changqing Li (2):
classextend.py: don't extend file for file dependency
report-error.bbclass: add local.conf/auto.conf into error report
Chen Qi (1):
python-numpy: fix build for libn32
Daniel Gomez (1):
lttng-modules: Add missing SRCREV_FORMAT
Diego Rondini (1):
initramfs-framework: support PARTLABEL option
Dmitry Eremin-Solenikov (7):
image-uefi.conf: add config file holding configuration for UEFI images
grub-bootconf: switch to image-uefi.conf
grub-efi: switch to image-uefi.conf
grub-efi.bbclass: switch to image-uefi.conf
systemd-boot: switch to image-uefi.conf
systemd-boot.bbclass: switch to image-uefi.conf
live-vm-common.bbclass: provide efi population functions for live images
Hector Palacios (1):
udev-extraconf: skip mounting partitions already mounted by systemd
Henning Schild (6):
oe-git-proxy: allow setting SOCAT from outside
oeqa: add case for oe-git-proxy
Revert "oe-git-proxy: Avoid resolving NO_PROXY against local files"
oe-git-proxy: disable shell pathname expansion for the whole script
oe-git-proxy: NO_PROXY suffix matching without wildcard for match_host
oe-git-proxy: fix dash "Bad substitution"
Hongxu Jia (1):
elfutils: 0.176 -> 0.177
Jack Mitchell (1):
iptables: add systemd helper unit to load/restore rules
Jaewon Lee (1):
populate_sdk_ext: Introduce mechanism to keep nativesdk* sstate in esdk
Jason Wessel (1):
gnupg: Extend -native wrapper to fix gpgme-native's gpgconf problems
Jiang Lu (2):
glib-networking:enable glib-networking build as native package
libsoup:enable libsoup build as native package
Joshua Watt (4):
sstatesig: Update server URI
Remove SSTATE_HASHEQUIV_SERVER
bitbake: bitbake: Rework hash equivalence
classes/archiver: Fix WORKDIR for shared source
Kai Kang (1):
systemd: provides ${base_sbindir}/udevadm
Khem Raj (10):
ptrace: Drop ptrace aid for musl/ppc
elfutils: Fix build on ppc/musl
cogl: Do not depend PN-dev on empty PN
musl: Update to latest master
glibc: Move DISTRO_FEATURE specific do_install code for target recipe only
populate_sdk_base.bbclass: nativesdk-glibc-locale is required on musl too
nativesdk.bbclass: Clear out LIBCEXTENSION and ABIEXTENSION
openssl: Enable os option for with-rand-seed as well
weston-init: Add possibility to run weston as non-root user
layer.conf: Remove weston-conf from SIGGEN_EXCLUDE_SAFE_RECIPE_DEPS
Li Zhou (1):
qemu: Security Advisory - qemu - CVE-2019-15890
Limeng (1):
tune-cortexa57-cortexa53: add tunes for ARM Cortex-A53-Cortex-A57
Martin Jansa (2):
perf: fix build on kernels which don't have ${S}/tools/include/linux/bits.h
bitbake: Revert "bitbake: cooker: Ensure bbappends are found in stable order"
Maxime Roussin-BĂ©langer (1):
meta: add missing descriptions and homepage in bsp
Mikko Rapeli (2):
busybox.inc: handle empty DEBUG_PREFIX_MAP
bitbake: svn fetcher: allow "svn propget svn:externals" to fail
Nathan Rossi (7):
resulttool: Handle multiple series containing ptestresults
gcc-cross.inc: Process binaries in build dir to be relocatable
oeqa/core/case.py: Add OEPTestResultTestCase for ptestresult helpers
oeqa/selftest: Rework toolchain tests to use OEPTestResultTestCase
glibc-testsuite: SkipRecipe if libc is not glibc
cmake: 3.15.2 -> 3.15.3
meson.bbclass: Handle microblaze* mapping to cpu family
Oleksandr Kravchuk (5):
python3-pygobject: update to 3.34.0
font-util: update to 1.3.2
expat: update to 2.2.8
curl: update to 7.66.0
python3-dbus: update to 1.2.12
Otavio Salvador (1):
mesa: Upgrade 19.1.1 -> 19.1.6
Peter Kjellerstedt (3):
glibc: Make it build without ldconfig in DISTRO_FEATURES
package_rpm.bbclass: Remove a misleading bb.note()
tzdata: Correct the packaging of /etc/localtime and /etc/timezone
Quentin Schulz (1):
externalsrc: stop rebuilds of 2+ externalsrc recipes sharing the same git repo
Randy MacLeod (4):
valgrind: enable ~500 more ptests
valgrind: make a few more ptests pass
valgrind: ptest improvements to run-ptest and more
valgrind: disable 256 ptests for aarch64
Richard Purdie (8):
bitbake: runqueue/siggen: Optimise hash equiv queries
runqemu: Mention snapshot in the help output
initramfs-framework: support PARTLABEL option
systemd: Handle slow to boot mips hwdb update timeouts
meta-extsdk: Either an sstate task is a proper task or it isn't
oeqa/concurrenttest: Use ionice to delete build directories
bitbake: utils: Add ionice option to prunedir
build-appliance-image: Update to master head revision
Robert Yang (2):
conf/multilib.conf: Add ovmf to NON_MULTILIB_RECIPES
bitbake: runqueue: validate_hashes(): currentcount should be a number
Ross Burton (16):
libtasn1: fix build with api-documentation enabled
gstreamer1.0-libav: enable gtk-doc again
python3: handle STAGING_LIBDIR/INCDIR being unset
mesa: no need to depend on target python3
adwaita-icon-theme: fix rare install race
oeqa/selftest/wic: improve assert messages in test_fixed_size
oeqa/selftest/imagefeatures: dump the JSON if it can't be parsed
libical: upgrade to 3.0.6
acpica: upgrade 20190509 -> 20190816
gdk-pixbuf: upgrade 2.38.1 -> 2.38.2
piglit: upgrade to latest revision
libinput: upgrade 1.14.0 -> 1.14.1
rootfs-postcommands: check /etc/gconf exists before working on it
systemd-systemctl-native: don't care about line endings
opkg-utils: respect SOURCE_DATE_EPOCH when building ipkgs
bitbake: fetch2/git: add git-lfs toggle option
Scott Murray (1):
systemd: upgrade to 243
Stefan Ghinea (1):
ghostscript: CVE-2019-14811, CVE-2019-14817
Tim Blechmann (1):
icecc: blacklist pixman
Yeoh Ee Peng (3):
bitbake: bitbake-layers: show-recipes: Show recipes only
bitbake: bitbake-layers: show-recipes: Select recipes from selected layer
bitbake: bitbake-layers: show-recipes: Enable bare output
Yi Zhao (3):
screen: add /etc/screenrc as global config file
nfs-utils: fix nfs mount error on 32bit nfs server
grub: remove diffutils and freetype runtime dependencies
Zang Ruochen (2):
btrfs-tools:upgrade 5.2.1 -> 5.2.2
timezone:upgrade 2019b -> 2019c
Change-Id: I1ec24480a8964e474cd99d60a0cb0975e49b46b8
Signed-off-by: Brad Bishop <bradleyb@fuzziesquirrel.com>
Diffstat (limited to 'poky/scripts')
-rw-r--r-- | poky/scripts/lib/resulttool/log.py | 43 | ||||
-rw-r--r-- | poky/scripts/lib/resulttool/report.py | 44 | ||||
-rw-r--r-- | poky/scripts/lib/resulttool/resultutils.py | 45 | ||||
-rwxr-xr-x | poky/scripts/oe-git-proxy | 21 | ||||
-rwxr-xr-x | poky/scripts/runqemu | 22 |
5 files changed, 120 insertions, 55 deletions
diff --git a/poky/scripts/lib/resulttool/log.py b/poky/scripts/lib/resulttool/log.py index 2352c767d..f1bfd9950 100644 --- a/poky/scripts/lib/resulttool/log.py +++ b/poky/scripts/lib/resulttool/log.py @@ -8,12 +8,12 @@ import os import resulttool.resultutils as resultutils def show_ptest(result, ptest, logger): - if 'ptestresult.sections' in result: - if ptest in result['ptestresult.sections'] and 'log' in result['ptestresult.sections'][ptest]: - print(result['ptestresult.sections'][ptest]['log']) - return 0 + logdata = resultutils.ptestresult_get_log(result, ptest) + if logdata is not None: + print(logdata) + return 0 - print("ptest '%s' not found" % ptest) + print("ptest '%s' log not found" % ptest) return 1 def show_reproducible(result, reproducible, logger): @@ -25,7 +25,6 @@ def show_reproducible(result, reproducible, logger): print("reproducible '%s' not found" % reproducible) return 1 - def log(args, logger): results = resultutils.load_resultsdata(args.source) @@ -35,24 +34,24 @@ def log(args, logger): return 1 for _, run_name, _, r in resultutils.test_run_results(results): - if args.dump_ptest: - if 'ptestresult.sections' in r: - for name, ptest in r['ptestresult.sections'].items(): - if 'log' in ptest: - dest_dir = args.dump_ptest - if args.prepend_run: - dest_dir = os.path.join(dest_dir, run_name) - - os.makedirs(dest_dir, exist_ok=True) - - dest = os.path.join(dest_dir, '%s.log' % name) - print(dest) - with open(dest, 'w') as f: - f.write(ptest['log']) + if args.dump_ptest and 'ptestresult.sections' in r: + for name, ptest in r['ptestresult.sections'].items(): + logdata = resultutils.ptestresult_get_log(r, name) + if logdata is not None: + dest_dir = args.dump_ptest + if args.prepend_run: + dest_dir = os.path.join(dest_dir, run_name) + + os.makedirs(dest_dir, exist_ok=True) + dest = os.path.join(dest_dir, '%s.log' % name) + print(dest) + with open(dest, 'w') as f: + f.write(logdata) if args.raw_ptest: - if 'ptestresult.rawlogs' in r: - print(r['ptestresult.rawlogs']['log']) + rawlog = resultutils.ptestresult_get_rawlogs(r) + if rawlog is not None: + print(rawlog) else: print('Raw ptest logs not found') return 1 diff --git a/poky/scripts/lib/resulttool/report.py b/poky/scripts/lib/resulttool/report.py index 8b03717d2..883b52517 100644 --- a/poky/scripts/lib/resulttool/report.py +++ b/poky/scripts/lib/resulttool/report.py @@ -32,16 +32,22 @@ class ResultsTextReport(object): # Ensure tests without any test results still show up on the report for suite in result['ptestresult.sections']: if suite not in self.ptests[machine]: - self.ptests[machine][suite] = {'passed': 0, 'failed': 0, 'skipped': 0, 'duration' : '-', 'failed_testcases': []} + self.ptests[machine][suite] = { + 'passed': 0, 'failed': 0, 'skipped': 0, 'duration' : '-', + 'failed_testcases': [], "testcases": set(), + } if 'duration' in result['ptestresult.sections'][suite]: self.ptests[machine][suite]['duration'] = result['ptestresult.sections'][suite]['duration'] if 'timeout' in result['ptestresult.sections'][suite]: self.ptests[machine][suite]['duration'] += " T" - return + return True + + # process test result try: _, suite, test = k.split(".", 2) except ValueError: - return + return True + # Handle 'glib-2.0' if 'ptestresult.sections' in result and suite not in result['ptestresult.sections']: try: @@ -50,11 +56,23 @@ class ResultsTextReport(object): suite = suite + "." + suite1 except ValueError: pass + if suite not in self.ptests[machine]: - self.ptests[machine][suite] = {'passed': 0, 'failed': 0, 'skipped': 0, 'duration' : '-', 'failed_testcases': []} + self.ptests[machine][suite] = { + 'passed': 0, 'failed': 0, 'skipped': 0, 'duration' : '-', + 'failed_testcases': [], "testcases": set(), + } + + # do not process duplicate results + if test in self.ptests[machine][suite]["testcases"]: + print("Warning duplicate ptest result '{}.{}' for {}".format(suite, test, machine)) + return False + for tk in self.result_types: if status in self.result_types[tk]: self.ptests[machine][suite][tk] += 1 + self.ptests[machine][suite]["testcases"].add(test) + return True def handle_ltptest_result(self, k, status, result, machine): if machine not in self.ltptests: @@ -124,17 +142,20 @@ class ResultsTextReport(object): result = testresult.get('result', []) for k in result: test_status = result[k].get('status', []) + if k.startswith("ptestresult."): + if not self.handle_ptest_result(k, test_status, result, machine): + continue + elif k.startswith("ltpresult."): + self.handle_ltptest_result(k, test_status, result, machine) + elif k.startswith("ltpposixresult."): + self.handle_ltpposixtest_result(k, test_status, result, machine) + + # process result if it was not skipped by a handler for tk in self.result_types: if test_status in self.result_types[tk]: test_count_report[tk] += 1 if test_status in self.result_types['failed']: test_count_report['failed_testcases'].append(k) - if k.startswith("ptestresult."): - self.handle_ptest_result(k, test_status, result, machine) - if k.startswith("ltpresult."): - self.handle_ltptest_result(k, test_status, result, machine) - if k.startswith("ltpposixresult."): - self.handle_ltpposixtest_result(k, test_status, result, machine) return test_count_report def print_test_report(self, template_file_name, test_count_reports): @@ -210,8 +231,7 @@ class ResultsTextReport(object): # Check to see if there is already results for these kinds of tests for the machine for key in result['result'].keys(): testtype = str(key).split('.')[0] - if ((machine in self.ptests and testtype == "ptestresult" and self.ptests[machine]) or - (machine in self.ltptests and testtype == "ltpiresult" and self.ltptests[machine]) or + if ((machine in self.ltptests and testtype == "ltpiresult" and self.ltptests[machine]) or (machine in self.ltpposixtests and testtype == "ltpposixresult" and self.ltpposixtests[machine])): print("Already have test results for %s on %s, skipping %s" %(str(key).split('.')[0], machine, resultid)) skip = True diff --git a/poky/scripts/lib/resulttool/resultutils.py b/poky/scripts/lib/resulttool/resultutils.py index e595c185d..177fb25f9 100644 --- a/poky/scripts/lib/resulttool/resultutils.py +++ b/poky/scripts/lib/resulttool/resultutils.py @@ -7,6 +7,8 @@ # import os +import base64 +import zlib import json import scriptpath import copy @@ -117,6 +119,34 @@ def strip_ptestresults(results): del newresults[res]['result']['ptestresult.sections'][i]['log'] return newresults +def decode_log(logdata): + if isinstance(logdata, str): + return logdata + elif isinstance(logdata, dict): + if "compressed" in logdata: + data = logdata.get("compressed") + data = base64.b64decode(data.encode("utf-8")) + return zlib.decompress(data).decode("utf-8") + return None + +def ptestresult_get_log(results, section): + if 'ptestresult.sections' not in results: + return None + if section not in results['ptestresult.sections']: + return None + + ptest = results['ptestresult.sections'][section] + if 'log' not in ptest: + return None + return decode_log(ptest['log']) + +def ptestresult_get_rawlogs(results): + if 'ptestresult.rawlogs' not in results: + return None + if 'log' not in results['ptestresult.rawlogs']: + return None + return decode_log(results['ptestresult.rawlogs']['log']) + def save_resultsdata(results, destdir, fn="testresults.json", ptestjson=False, ptestlogs=False): for res in results: if res: @@ -131,14 +161,17 @@ def save_resultsdata(results, destdir, fn="testresults.json", ptestjson=False, p f.write(json.dumps(resultsout, sort_keys=True, indent=4)) for res2 in results[res]: if ptestlogs and 'result' in results[res][res2]: - if 'ptestresult.rawlogs' in results[res][res2]['result']: + seriesresults = results[res][res2]['result'] + rawlogs = ptestresult_get_rawlogs(seriesresults) + if rawlogs is not None: with open(dst.replace(fn, "ptest-raw.log"), "w+") as f: - f.write(results[res][res2]['result']['ptestresult.rawlogs']['log']) - if 'ptestresult.sections' in results[res][res2]['result']: - for i in results[res][res2]['result']['ptestresult.sections']: - if 'log' in results[res][res2]['result']['ptestresult.sections'][i]: + f.write(rawlogs) + if 'ptestresult.sections' in seriesresults: + for i in seriesresults['ptestresult.sections']: + sectionlog = ptestresult_get_log(seriesresults, i) + if sectionlog is not None: with open(dst.replace(fn, "ptest-%s.log" % i), "w+") as f: - f.write(results[res][res2]['result']['ptestresult.sections'][i]['log']) + f.write(sectionlog) def git_get_result(repo, tags): git_objs = [] diff --git a/poky/scripts/oe-git-proxy b/poky/scripts/oe-git-proxy index bb2ed2a46..aa9b9dc9a 100755 --- a/poky/scripts/oe-git-proxy +++ b/poky/scripts/oe-git-proxy @@ -19,6 +19,9 @@ # AUTHORS # Darren Hart <dvhart@linux.intel.com> +# disable pathname expansion, NO_PROXY fields could start with "*" or be it +set -f + if [ $# -lt 2 -o "$1" = '--help' -o "$1" = '-h' ] ; then echo 'oe-git-proxy: error: the following arguments are required: host port' echo 'Usage: oe-git-proxy host port' @@ -41,10 +44,12 @@ if [ $# -lt 2 -o "$1" = '--help' -o "$1" = '-h' ] ; then fi # Locate the netcat binary -SOCAT=$(which socat 2>/dev/null) -if [ $? -ne 0 ]; then - echo "ERROR: socat binary not in PATH" 1>&2 - exit 1 +if [ -z "$SOCAT" ]; then + SOCAT=$(which socat 2>/dev/null) + if [ $? -ne 0 ]; then + echo "ERROR: socat binary not in PATH" 1>&2 + exit 1 + fi fi METHOD="" @@ -59,7 +64,7 @@ ipv4_val() { IP="$1" SHIFT=24 VAL=0 - for B in ${IP//./ }; do + for B in $( echo "$IP" | tr '.' ' ' ); do VAL=$(($VAL+$(($B<<$SHIFT)))) SHIFT=$(($SHIFT-8)) done @@ -102,7 +107,7 @@ match_host() { HOST=$1 GLOB=$2 - if [ -z "${HOST%%$GLOB}" ]; then + if [ -z "${HOST%%*$GLOB}" ]; then return 0 fi @@ -132,8 +137,8 @@ if [ -z "$ALL_PROXY" ]; then fi # Connect directly to hosts in NO_PROXY -for H in "${NO_PROXY//,/ }"; do - if match_host $1 "$H"; then +for H in $( echo "$NO_PROXY" | tr ',' ' ' ); do + if match_host $1 $H; then exec $SOCAT STDIO $METHOD fi done diff --git a/poky/scripts/runqemu b/poky/scripts/runqemu index 4415e9b69..1a5aca98a 100755 --- a/poky/scripts/runqemu +++ b/poky/scripts/runqemu @@ -65,13 +65,15 @@ of the following environment variables (in any order): MACHINE - the machine name (optional, autodetected from KERNEL filename if unspecified) Simplified QEMU command-line options can be passed with: nographic - disable video console - sdl - choose the SDL frontend instead of the Gtk+ default - gtk-gl - enable virgl-based GL acceleration using Gtk+ frontend - gtk-gl-es - enable virgl-based GL acceleration, using OpenGL ES and Gtk+ frontend + sdl - choose the SDL UI frontend + gtk - choose the Gtk UI frontend + gl - enable virgl-based GL acceleration (also needs gtk option) + gl-es - enable virgl-based GL acceleration, using OpenGL ES (also needs gtk option) egl-headless - enable headless EGL output; use vnc or spice to see it serial - enable a serial console on /dev/ttyS0 serialstdio - enable a serial console on the console (regardless of graphics mode) slirp - enable user networking, no root privileges is required + snapshot - don't write changes to back to images kvm - enable KVM when running x86/x86_64 (VT-capable CPU required) kvm-vhost - enable KVM with vhost when running x86/x86_64 (VT-capable CPU required) publicvnc - enable a VNC server open to all hosts @@ -436,10 +438,16 @@ class BaseConfig(object): self.kernel_cmdline_script += ' console=ttyS0' elif arg == 'sdl': self.qemu_opt_script += ' -display sdl' - elif arg == 'gtk-gl': - self.qemu_opt_script += ' -vga virtio -display gtk,gl=on' - elif arg == 'gtk-gl-es': - self.qemu_opt_script += ' -vga virtio -display gtk,gl=es' + elif arg == 'gtk': + if 'gl' in sys.argv[1:]: + self.qemu_opt_script += ' -vga virtio -display gtk,gl=on' + elif 'gl-es' in sys.argv[1:]: + self.qemu_opt_script += ' -vga virtio -display gtk,gl=es' + else: + self.qemu_opt_script += ' -display gtk' + elif arg == 'gl' or arg == 'gl-es': + # These args are handled inside sdl or gtk blocks above + pass elif arg == 'egl-headless': self.qemu_opt_script += ' -vga virtio -display egl-headless' # As runqemu can be run within bitbake (when using testimage, for example), |