diff options
author | Brad Bishop <bradleyb@fuzziesquirrel.com> | 2019-09-24 05:34:48 +0300 |
---|---|---|
committer | Brad Bishop <bradleyb@fuzziesquirrel.com> | 2019-09-24 05:35:28 +0300 |
commit | a34c030e5ec7021e7fb452410d38abfb3993ec68 (patch) | |
tree | b8f41acc23015f7f1de01de3c87c434195c681d4 /poky/bitbake/bin | |
parent | dec060e0fadb20cc708370ae192db4462283ee58 (diff) | |
download | openbmc-a34c030e5ec7021e7fb452410d38abfb3993ec68.tar.xz |
poky: subtree update:745e38ff0f..81f9e815d3
Adrian Bunk (6):
openssl: Upgrade 1.1.1c -> 1.1.1d
glib-2.0: Upgrade 2.60.6 -> 2.60.7
lttng-modules: Upgrade 2.10.10 -> 2.10.11
lttng-ust: Upgrade 2.10.4 -> 2.10.5
squashfs-tools: Remove UPSTREAM_CHECK_COMMITS
libmpc: Remove dead UPSTREAM_CHECK_URI
Alexander Kanavin (2):
runqemu: decouple gtk and gl options
strace: add a timeout for running ptests
Alistair Francis (1):
gdb: Mark gdbserver as ALLOW_EMPTY for riscv32
Andre McCurdy (9):
busybox: drop unused mount.busybox and umount.busybox wrappers
busybox: drop inittab from SRC_URI ( now moved to busybox-inittab )
busybox-inittab: minor formatting tweaks
base-files: drop legacy empty file /etc/default/usbd
busybox: rcS and rcK should not be writeable by everyone
ffmpeg: add PACKAGECONFIG controls for alsa and zlib (enable by default)
libwebp: apply ARM specific config options to big endian ARM
initscripts: enable alignment.sh init script for big endian ARM
libunwind: apply configure over-ride to both big and little endian ARM
Andrew F. Davis (4):
libepoxy: Disable x11 when not building for x11
cogl: Set depends to the virtual needed not explicitly on Mesa
gtk+3: Set depends to the virtual needed not explicitly on Mesa
weston: Set depends to the virtual needed not explicitly on Mesa
Armin Kuster (1):
gcc: Security fix for CVE-2019-15847
Changhyeok Bae (1):
iw: upgrade to 5.3
Changqing Li (2):
classextend.py: don't extend file for file dependency
report-error.bbclass: add local.conf/auto.conf into error report
Chen Qi (1):
python-numpy: fix build for libn32
Daniel Gomez (1):
lttng-modules: Add missing SRCREV_FORMAT
Diego Rondini (1):
initramfs-framework: support PARTLABEL option
Dmitry Eremin-Solenikov (7):
image-uefi.conf: add config file holding configuration for UEFI images
grub-bootconf: switch to image-uefi.conf
grub-efi: switch to image-uefi.conf
grub-efi.bbclass: switch to image-uefi.conf
systemd-boot: switch to image-uefi.conf
systemd-boot.bbclass: switch to image-uefi.conf
live-vm-common.bbclass: provide efi population functions for live images
Hector Palacios (1):
udev-extraconf: skip mounting partitions already mounted by systemd
Henning Schild (6):
oe-git-proxy: allow setting SOCAT from outside
oeqa: add case for oe-git-proxy
Revert "oe-git-proxy: Avoid resolving NO_PROXY against local files"
oe-git-proxy: disable shell pathname expansion for the whole script
oe-git-proxy: NO_PROXY suffix matching without wildcard for match_host
oe-git-proxy: fix dash "Bad substitution"
Hongxu Jia (1):
elfutils: 0.176 -> 0.177
Jack Mitchell (1):
iptables: add systemd helper unit to load/restore rules
Jaewon Lee (1):
populate_sdk_ext: Introduce mechanism to keep nativesdk* sstate in esdk
Jason Wessel (1):
gnupg: Extend -native wrapper to fix gpgme-native's gpgconf problems
Jiang Lu (2):
glib-networking:enable glib-networking build as native package
libsoup:enable libsoup build as native package
Joshua Watt (4):
sstatesig: Update server URI
Remove SSTATE_HASHEQUIV_SERVER
bitbake: bitbake: Rework hash equivalence
classes/archiver: Fix WORKDIR for shared source
Kai Kang (1):
systemd: provides ${base_sbindir}/udevadm
Khem Raj (10):
ptrace: Drop ptrace aid for musl/ppc
elfutils: Fix build on ppc/musl
cogl: Do not depend PN-dev on empty PN
musl: Update to latest master
glibc: Move DISTRO_FEATURE specific do_install code for target recipe only
populate_sdk_base.bbclass: nativesdk-glibc-locale is required on musl too
nativesdk.bbclass: Clear out LIBCEXTENSION and ABIEXTENSION
openssl: Enable os option for with-rand-seed as well
weston-init: Add possibility to run weston as non-root user
layer.conf: Remove weston-conf from SIGGEN_EXCLUDE_SAFE_RECIPE_DEPS
Li Zhou (1):
qemu: Security Advisory - qemu - CVE-2019-15890
Limeng (1):
tune-cortexa57-cortexa53: add tunes for ARM Cortex-A53-Cortex-A57
Martin Jansa (2):
perf: fix build on kernels which don't have ${S}/tools/include/linux/bits.h
bitbake: Revert "bitbake: cooker: Ensure bbappends are found in stable order"
Maxime Roussin-BĂ©langer (1):
meta: add missing descriptions and homepage in bsp
Mikko Rapeli (2):
busybox.inc: handle empty DEBUG_PREFIX_MAP
bitbake: svn fetcher: allow "svn propget svn:externals" to fail
Nathan Rossi (7):
resulttool: Handle multiple series containing ptestresults
gcc-cross.inc: Process binaries in build dir to be relocatable
oeqa/core/case.py: Add OEPTestResultTestCase for ptestresult helpers
oeqa/selftest: Rework toolchain tests to use OEPTestResultTestCase
glibc-testsuite: SkipRecipe if libc is not glibc
cmake: 3.15.2 -> 3.15.3
meson.bbclass: Handle microblaze* mapping to cpu family
Oleksandr Kravchuk (5):
python3-pygobject: update to 3.34.0
font-util: update to 1.3.2
expat: update to 2.2.8
curl: update to 7.66.0
python3-dbus: update to 1.2.12
Otavio Salvador (1):
mesa: Upgrade 19.1.1 -> 19.1.6
Peter Kjellerstedt (3):
glibc: Make it build without ldconfig in DISTRO_FEATURES
package_rpm.bbclass: Remove a misleading bb.note()
tzdata: Correct the packaging of /etc/localtime and /etc/timezone
Quentin Schulz (1):
externalsrc: stop rebuilds of 2+ externalsrc recipes sharing the same git repo
Randy MacLeod (4):
valgrind: enable ~500 more ptests
valgrind: make a few more ptests pass
valgrind: ptest improvements to run-ptest and more
valgrind: disable 256 ptests for aarch64
Richard Purdie (8):
bitbake: runqueue/siggen: Optimise hash equiv queries
runqemu: Mention snapshot in the help output
initramfs-framework: support PARTLABEL option
systemd: Handle slow to boot mips hwdb update timeouts
meta-extsdk: Either an sstate task is a proper task or it isn't
oeqa/concurrenttest: Use ionice to delete build directories
bitbake: utils: Add ionice option to prunedir
build-appliance-image: Update to master head revision
Robert Yang (2):
conf/multilib.conf: Add ovmf to NON_MULTILIB_RECIPES
bitbake: runqueue: validate_hashes(): currentcount should be a number
Ross Burton (16):
libtasn1: fix build with api-documentation enabled
gstreamer1.0-libav: enable gtk-doc again
python3: handle STAGING_LIBDIR/INCDIR being unset
mesa: no need to depend on target python3
adwaita-icon-theme: fix rare install race
oeqa/selftest/wic: improve assert messages in test_fixed_size
oeqa/selftest/imagefeatures: dump the JSON if it can't be parsed
libical: upgrade to 3.0.6
acpica: upgrade 20190509 -> 20190816
gdk-pixbuf: upgrade 2.38.1 -> 2.38.2
piglit: upgrade to latest revision
libinput: upgrade 1.14.0 -> 1.14.1
rootfs-postcommands: check /etc/gconf exists before working on it
systemd-systemctl-native: don't care about line endings
opkg-utils: respect SOURCE_DATE_EPOCH when building ipkgs
bitbake: fetch2/git: add git-lfs toggle option
Scott Murray (1):
systemd: upgrade to 243
Stefan Ghinea (1):
ghostscript: CVE-2019-14811, CVE-2019-14817
Tim Blechmann (1):
icecc: blacklist pixman
Yeoh Ee Peng (3):
bitbake: bitbake-layers: show-recipes: Show recipes only
bitbake: bitbake-layers: show-recipes: Select recipes from selected layer
bitbake: bitbake-layers: show-recipes: Enable bare output
Yi Zhao (3):
screen: add /etc/screenrc as global config file
nfs-utils: fix nfs mount error on 32bit nfs server
grub: remove diffutils and freetype runtime dependencies
Zang Ruochen (2):
btrfs-tools:upgrade 5.2.1 -> 5.2.2
timezone:upgrade 2019b -> 2019c
Change-Id: I1ec24480a8964e474cd99d60a0cb0975e49b46b8
Signed-off-by: Brad Bishop <bradleyb@fuzziesquirrel.com>
Diffstat (limited to 'poky/bitbake/bin')
-rwxr-xr-x | poky/bitbake/bin/bitbake-hashclient | 170 | ||||
-rwxr-xr-x | poky/bitbake/bin/bitbake-hashserv | 24 | ||||
-rwxr-xr-x | poky/bitbake/bin/bitbake-worker | 2 |
3 files changed, 186 insertions, 10 deletions
diff --git a/poky/bitbake/bin/bitbake-hashclient b/poky/bitbake/bin/bitbake-hashclient new file mode 100755 index 0000000000..29ab65f177 --- /dev/null +++ b/poky/bitbake/bin/bitbake-hashclient @@ -0,0 +1,170 @@ +#! /usr/bin/env python3 +# +# Copyright (C) 2019 Garmin Ltd. +# +# SPDX-License-Identifier: GPL-2.0-only +# + +import argparse +import hashlib +import logging +import os +import pprint +import sys +import threading +import time + +try: + import tqdm + ProgressBar = tqdm.tqdm +except ImportError: + class ProgressBar(object): + def __init__(self, *args, **kwargs): + pass + + def __enter__(self): + return self + + def __exit__(self, *args, **kwargs): + pass + + def update(self): + pass + +sys.path.insert(0, os.path.join(os.path.dirname(os.path.dirname(__file__)), 'lib')) + +import hashserv + +DEFAULT_ADDRESS = 'unix://./hashserve.sock' +METHOD = 'stress.test.method' + + +def main(): + def handle_stats(args, client): + if args.reset: + s = client.reset_stats() + else: + s = client.get_stats() + pprint.pprint(s) + return 0 + + def handle_stress(args, client): + def thread_main(pbar, lock): + nonlocal found_hashes + nonlocal missed_hashes + nonlocal max_time + + client = hashserv.create_client(args.address) + + for i in range(args.requests): + taskhash = hashlib.sha256() + taskhash.update(args.taskhash_seed.encode('utf-8')) + taskhash.update(str(i).encode('utf-8')) + + start_time = time.perf_counter() + l = client.get_unihash(METHOD, taskhash.hexdigest()) + elapsed = time.perf_counter() - start_time + + with lock: + if l: + found_hashes += 1 + else: + missed_hashes += 1 + + max_time = max(elapsed, max_time) + pbar.update() + + max_time = 0 + found_hashes = 0 + missed_hashes = 0 + lock = threading.Lock() + total_requests = args.clients * args.requests + start_time = time.perf_counter() + with ProgressBar(total=total_requests) as pbar: + threads = [threading.Thread(target=thread_main, args=(pbar, lock), daemon=False) for _ in range(args.clients)] + for t in threads: + t.start() + + for t in threads: + t.join() + + elapsed = time.perf_counter() - start_time + with lock: + print("%d requests in %.1fs. %.1f requests per second" % (total_requests, elapsed, total_requests / elapsed)) + print("Average request time %.8fs" % (elapsed / total_requests)) + print("Max request time was %.8fs" % max_time) + print("Found %d hashes, missed %d" % (found_hashes, missed_hashes)) + + if args.report: + with ProgressBar(total=args.requests) as pbar: + for i in range(args.requests): + taskhash = hashlib.sha256() + taskhash.update(args.taskhash_seed.encode('utf-8')) + taskhash.update(str(i).encode('utf-8')) + + outhash = hashlib.sha256() + outhash.update(args.outhash_seed.encode('utf-8')) + outhash.update(str(i).encode('utf-8')) + + client.report_unihash(taskhash.hexdigest(), METHOD, outhash.hexdigest(), taskhash.hexdigest()) + + with lock: + pbar.update() + + parser = argparse.ArgumentParser(description='Hash Equivalence Client') + parser.add_argument('--address', default=DEFAULT_ADDRESS, help='Server address (default "%(default)s")') + parser.add_argument('--log', default='WARNING', help='Set logging level') + + subparsers = parser.add_subparsers() + + stats_parser = subparsers.add_parser('stats', help='Show server stats') + stats_parser.add_argument('--reset', action='store_true', + help='Reset server stats') + stats_parser.set_defaults(func=handle_stats) + + stress_parser = subparsers.add_parser('stress', help='Run stress test') + stress_parser.add_argument('--clients', type=int, default=10, + help='Number of simultaneous clients') + stress_parser.add_argument('--requests', type=int, default=1000, + help='Number of requests each client will perform') + stress_parser.add_argument('--report', action='store_true', + help='Report new hashes') + stress_parser.add_argument('--taskhash-seed', default='', + help='Include string in taskhash') + stress_parser.add_argument('--outhash-seed', default='', + help='Include string in outhash') + stress_parser.set_defaults(func=handle_stress) + + args = parser.parse_args() + + logger = logging.getLogger('hashserv') + + level = getattr(logging, args.log.upper(), None) + if not isinstance(level, int): + raise ValueError('Invalid log level: %s' % args.log) + + logger.setLevel(level) + console = logging.StreamHandler() + console.setLevel(level) + logger.addHandler(console) + + func = getattr(args, 'func', None) + if func: + client = hashserv.create_client(args.address) + # Try to establish a connection to the server now to detect failures + # early + client.connect() + + return func(args, client) + + return 0 + + +if __name__ == '__main__': + try: + ret = main() + except Exception: + ret = 1 + import traceback + traceback.print_exc() + sys.exit(ret) diff --git a/poky/bitbake/bin/bitbake-hashserv b/poky/bitbake/bin/bitbake-hashserv index 6c911c098a..1bc1f91f38 100755 --- a/poky/bitbake/bin/bitbake-hashserv +++ b/poky/bitbake/bin/bitbake-hashserv @@ -11,20 +11,26 @@ import logging import argparse import sqlite3 -sys.path.insert(0, os.path.join(os.path.dirname(os.path.dirname(__file__)),'lib')) +sys.path.insert(0, os.path.join(os.path.dirname(os.path.dirname(__file__)), 'lib')) import hashserv VERSION = "1.0.0" -DEFAULT_HOST = '' -DEFAULT_PORT = 8686 +DEFAULT_BIND = 'unix://./hashserve.sock' + def main(): - parser = argparse.ArgumentParser(description='HTTP Equivalence Reference Server. Version=%s' % VERSION) - parser.add_argument('--address', default=DEFAULT_HOST, help='Bind address (default "%(default)s")') - parser.add_argument('--port', type=int, default=DEFAULT_PORT, help='Bind port (default %(default)d)') - parser.add_argument('--prefix', default='', help='HTTP path prefix (default "%(default)s")') + parser = argparse.ArgumentParser(description='Hash Equivalence Reference Server. Version=%s' % VERSION, + epilog='''The bind address is the path to a unix domain socket if it is + prefixed with "unix://". Otherwise, it is an IP address + and port in form ADDRESS:PORT. To bind to all addresses, leave + the ADDRESS empty, e.g. "--bind :8686". To bind to a specific + IPv6 address, enclose the address in "[]", e.g. + "--bind [::1]:8686"''' + ) + + parser.add_argument('--bind', default=DEFAULT_BIND, help='Bind address (default "%(default)s")') parser.add_argument('--database', default='./hashserv.db', help='Database file (default "%(default)s")') parser.add_argument('--log', default='WARNING', help='Set logging level') @@ -41,10 +47,11 @@ def main(): console.setLevel(level) logger.addHandler(console) - server = hashserv.create_server((args.address, args.port), args.database, args.prefix) + server = hashserv.create_server(args.bind, args.database) server.serve_forever() return 0 + if __name__ == '__main__': try: ret = main() @@ -53,4 +60,3 @@ if __name__ == '__main__': import traceback traceback.print_exc() sys.exit(ret) - diff --git a/poky/bitbake/bin/bitbake-worker b/poky/bitbake/bin/bitbake-worker index 96369199f2..6776cadda3 100755 --- a/poky/bitbake/bin/bitbake-worker +++ b/poky/bitbake/bin/bitbake-worker @@ -418,7 +418,7 @@ class BitbakeWorker(object): bb.msg.loggerDefaultDomains = self.workerdata["logdefaultdomain"] for mc in self.databuilder.mcdata: self.databuilder.mcdata[mc].setVar("PRSERV_HOST", self.workerdata["prhost"]) - self.databuilder.mcdata[mc].setVar("BB_HASHSERVE", self.workerdata["hashservport"]) + self.databuilder.mcdata[mc].setVar("BB_HASHSERVE", self.workerdata["hashservaddr"]) def handle_newtaskhashes(self, data): self.workerdata["newhashes"] = pickle.loads(data) |