summaryrefslogtreecommitdiff
path: root/poky/bitbake/bin/bitbake-hashclient
diff options
context:
space:
mode:
authorBrad Bishop <bradleyb@fuzziesquirrel.com>2019-09-24 05:34:48 +0300
committerBrad Bishop <bradleyb@fuzziesquirrel.com>2019-09-24 05:35:28 +0300
commita34c030e5ec7021e7fb452410d38abfb3993ec68 (patch)
treeb8f41acc23015f7f1de01de3c87c434195c681d4 /poky/bitbake/bin/bitbake-hashclient
parentdec060e0fadb20cc708370ae192db4462283ee58 (diff)
downloadopenbmc-a34c030e5ec7021e7fb452410d38abfb3993ec68.tar.xz
poky: subtree update:745e38ff0f..81f9e815d3
Adrian Bunk (6): openssl: Upgrade 1.1.1c -> 1.1.1d glib-2.0: Upgrade 2.60.6 -> 2.60.7 lttng-modules: Upgrade 2.10.10 -> 2.10.11 lttng-ust: Upgrade 2.10.4 -> 2.10.5 squashfs-tools: Remove UPSTREAM_CHECK_COMMITS libmpc: Remove dead UPSTREAM_CHECK_URI Alexander Kanavin (2): runqemu: decouple gtk and gl options strace: add a timeout for running ptests Alistair Francis (1): gdb: Mark gdbserver as ALLOW_EMPTY for riscv32 Andre McCurdy (9): busybox: drop unused mount.busybox and umount.busybox wrappers busybox: drop inittab from SRC_URI ( now moved to busybox-inittab ) busybox-inittab: minor formatting tweaks base-files: drop legacy empty file /etc/default/usbd busybox: rcS and rcK should not be writeable by everyone ffmpeg: add PACKAGECONFIG controls for alsa and zlib (enable by default) libwebp: apply ARM specific config options to big endian ARM initscripts: enable alignment.sh init script for big endian ARM libunwind: apply configure over-ride to both big and little endian ARM Andrew F. Davis (4): libepoxy: Disable x11 when not building for x11 cogl: Set depends to the virtual needed not explicitly on Mesa gtk+3: Set depends to the virtual needed not explicitly on Mesa weston: Set depends to the virtual needed not explicitly on Mesa Armin Kuster (1): gcc: Security fix for CVE-2019-15847 Changhyeok Bae (1): iw: upgrade to 5.3 Changqing Li (2): classextend.py: don't extend file for file dependency report-error.bbclass: add local.conf/auto.conf into error report Chen Qi (1): python-numpy: fix build for libn32 Daniel Gomez (1): lttng-modules: Add missing SRCREV_FORMAT Diego Rondini (1): initramfs-framework: support PARTLABEL option Dmitry Eremin-Solenikov (7): image-uefi.conf: add config file holding configuration for UEFI images grub-bootconf: switch to image-uefi.conf grub-efi: switch to image-uefi.conf grub-efi.bbclass: switch to image-uefi.conf systemd-boot: switch to image-uefi.conf systemd-boot.bbclass: switch to image-uefi.conf live-vm-common.bbclass: provide efi population functions for live images Hector Palacios (1): udev-extraconf: skip mounting partitions already mounted by systemd Henning Schild (6): oe-git-proxy: allow setting SOCAT from outside oeqa: add case for oe-git-proxy Revert "oe-git-proxy: Avoid resolving NO_PROXY against local files" oe-git-proxy: disable shell pathname expansion for the whole script oe-git-proxy: NO_PROXY suffix matching without wildcard for match_host oe-git-proxy: fix dash "Bad substitution" Hongxu Jia (1): elfutils: 0.176 -> 0.177 Jack Mitchell (1): iptables: add systemd helper unit to load/restore rules Jaewon Lee (1): populate_sdk_ext: Introduce mechanism to keep nativesdk* sstate in esdk Jason Wessel (1): gnupg: Extend -native wrapper to fix gpgme-native's gpgconf problems Jiang Lu (2): glib-networking:enable glib-networking build as native package libsoup:enable libsoup build as native package Joshua Watt (4): sstatesig: Update server URI Remove SSTATE_HASHEQUIV_SERVER bitbake: bitbake: Rework hash equivalence classes/archiver: Fix WORKDIR for shared source Kai Kang (1): systemd: provides ${base_sbindir}/udevadm Khem Raj (10): ptrace: Drop ptrace aid for musl/ppc elfutils: Fix build on ppc/musl cogl: Do not depend PN-dev on empty PN musl: Update to latest master glibc: Move DISTRO_FEATURE specific do_install code for target recipe only populate_sdk_base.bbclass: nativesdk-glibc-locale is required on musl too nativesdk.bbclass: Clear out LIBCEXTENSION and ABIEXTENSION openssl: Enable os option for with-rand-seed as well weston-init: Add possibility to run weston as non-root user layer.conf: Remove weston-conf from SIGGEN_EXCLUDE_SAFE_RECIPE_DEPS Li Zhou (1): qemu: Security Advisory - qemu - CVE-2019-15890 Limeng (1): tune-cortexa57-cortexa53: add tunes for ARM Cortex-A53-Cortex-A57 Martin Jansa (2): perf: fix build on kernels which don't have ${S}/tools/include/linux/bits.h bitbake: Revert "bitbake: cooker: Ensure bbappends are found in stable order" Maxime Roussin-BĂ©langer (1): meta: add missing descriptions and homepage in bsp Mikko Rapeli (2): busybox.inc: handle empty DEBUG_PREFIX_MAP bitbake: svn fetcher: allow "svn propget svn:externals" to fail Nathan Rossi (7): resulttool: Handle multiple series containing ptestresults gcc-cross.inc: Process binaries in build dir to be relocatable oeqa/core/case.py: Add OEPTestResultTestCase for ptestresult helpers oeqa/selftest: Rework toolchain tests to use OEPTestResultTestCase glibc-testsuite: SkipRecipe if libc is not glibc cmake: 3.15.2 -> 3.15.3 meson.bbclass: Handle microblaze* mapping to cpu family Oleksandr Kravchuk (5): python3-pygobject: update to 3.34.0 font-util: update to 1.3.2 expat: update to 2.2.8 curl: update to 7.66.0 python3-dbus: update to 1.2.12 Otavio Salvador (1): mesa: Upgrade 19.1.1 -> 19.1.6 Peter Kjellerstedt (3): glibc: Make it build without ldconfig in DISTRO_FEATURES package_rpm.bbclass: Remove a misleading bb.note() tzdata: Correct the packaging of /etc/localtime and /etc/timezone Quentin Schulz (1): externalsrc: stop rebuilds of 2+ externalsrc recipes sharing the same git repo Randy MacLeod (4): valgrind: enable ~500 more ptests valgrind: make a few more ptests pass valgrind: ptest improvements to run-ptest and more valgrind: disable 256 ptests for aarch64 Richard Purdie (8): bitbake: runqueue/siggen: Optimise hash equiv queries runqemu: Mention snapshot in the help output initramfs-framework: support PARTLABEL option systemd: Handle slow to boot mips hwdb update timeouts meta-extsdk: Either an sstate task is a proper task or it isn't oeqa/concurrenttest: Use ionice to delete build directories bitbake: utils: Add ionice option to prunedir build-appliance-image: Update to master head revision Robert Yang (2): conf/multilib.conf: Add ovmf to NON_MULTILIB_RECIPES bitbake: runqueue: validate_hashes(): currentcount should be a number Ross Burton (16): libtasn1: fix build with api-documentation enabled gstreamer1.0-libav: enable gtk-doc again python3: handle STAGING_LIBDIR/INCDIR being unset mesa: no need to depend on target python3 adwaita-icon-theme: fix rare install race oeqa/selftest/wic: improve assert messages in test_fixed_size oeqa/selftest/imagefeatures: dump the JSON if it can't be parsed libical: upgrade to 3.0.6 acpica: upgrade 20190509 -> 20190816 gdk-pixbuf: upgrade 2.38.1 -> 2.38.2 piglit: upgrade to latest revision libinput: upgrade 1.14.0 -> 1.14.1 rootfs-postcommands: check /etc/gconf exists before working on it systemd-systemctl-native: don't care about line endings opkg-utils: respect SOURCE_DATE_EPOCH when building ipkgs bitbake: fetch2/git: add git-lfs toggle option Scott Murray (1): systemd: upgrade to 243 Stefan Ghinea (1): ghostscript: CVE-2019-14811, CVE-2019-14817 Tim Blechmann (1): icecc: blacklist pixman Yeoh Ee Peng (3): bitbake: bitbake-layers: show-recipes: Show recipes only bitbake: bitbake-layers: show-recipes: Select recipes from selected layer bitbake: bitbake-layers: show-recipes: Enable bare output Yi Zhao (3): screen: add /etc/screenrc as global config file nfs-utils: fix nfs mount error on 32bit nfs server grub: remove diffutils and freetype runtime dependencies Zang Ruochen (2): btrfs-tools:upgrade 5.2.1 -> 5.2.2 timezone:upgrade 2019b -> 2019c Change-Id: I1ec24480a8964e474cd99d60a0cb0975e49b46b8 Signed-off-by: Brad Bishop <bradleyb@fuzziesquirrel.com>
Diffstat (limited to 'poky/bitbake/bin/bitbake-hashclient')
-rwxr-xr-xpoky/bitbake/bin/bitbake-hashclient170
1 files changed, 170 insertions, 0 deletions
diff --git a/poky/bitbake/bin/bitbake-hashclient b/poky/bitbake/bin/bitbake-hashclient
new file mode 100755
index 000000000..29ab65f17
--- /dev/null
+++ b/poky/bitbake/bin/bitbake-hashclient
@@ -0,0 +1,170 @@
+#! /usr/bin/env python3
+#
+# Copyright (C) 2019 Garmin Ltd.
+#
+# SPDX-License-Identifier: GPL-2.0-only
+#
+
+import argparse
+import hashlib
+import logging
+import os
+import pprint
+import sys
+import threading
+import time
+
+try:
+ import tqdm
+ ProgressBar = tqdm.tqdm
+except ImportError:
+ class ProgressBar(object):
+ def __init__(self, *args, **kwargs):
+ pass
+
+ def __enter__(self):
+ return self
+
+ def __exit__(self, *args, **kwargs):
+ pass
+
+ def update(self):
+ pass
+
+sys.path.insert(0, os.path.join(os.path.dirname(os.path.dirname(__file__)), 'lib'))
+
+import hashserv
+
+DEFAULT_ADDRESS = 'unix://./hashserve.sock'
+METHOD = 'stress.test.method'
+
+
+def main():
+ def handle_stats(args, client):
+ if args.reset:
+ s = client.reset_stats()
+ else:
+ s = client.get_stats()
+ pprint.pprint(s)
+ return 0
+
+ def handle_stress(args, client):
+ def thread_main(pbar, lock):
+ nonlocal found_hashes
+ nonlocal missed_hashes
+ nonlocal max_time
+
+ client = hashserv.create_client(args.address)
+
+ for i in range(args.requests):
+ taskhash = hashlib.sha256()
+ taskhash.update(args.taskhash_seed.encode('utf-8'))
+ taskhash.update(str(i).encode('utf-8'))
+
+ start_time = time.perf_counter()
+ l = client.get_unihash(METHOD, taskhash.hexdigest())
+ elapsed = time.perf_counter() - start_time
+
+ with lock:
+ if l:
+ found_hashes += 1
+ else:
+ missed_hashes += 1
+
+ max_time = max(elapsed, max_time)
+ pbar.update()
+
+ max_time = 0
+ found_hashes = 0
+ missed_hashes = 0
+ lock = threading.Lock()
+ total_requests = args.clients * args.requests
+ start_time = time.perf_counter()
+ with ProgressBar(total=total_requests) as pbar:
+ threads = [threading.Thread(target=thread_main, args=(pbar, lock), daemon=False) for _ in range(args.clients)]
+ for t in threads:
+ t.start()
+
+ for t in threads:
+ t.join()
+
+ elapsed = time.perf_counter() - start_time
+ with lock:
+ print("%d requests in %.1fs. %.1f requests per second" % (total_requests, elapsed, total_requests / elapsed))
+ print("Average request time %.8fs" % (elapsed / total_requests))
+ print("Max request time was %.8fs" % max_time)
+ print("Found %d hashes, missed %d" % (found_hashes, missed_hashes))
+
+ if args.report:
+ with ProgressBar(total=args.requests) as pbar:
+ for i in range(args.requests):
+ taskhash = hashlib.sha256()
+ taskhash.update(args.taskhash_seed.encode('utf-8'))
+ taskhash.update(str(i).encode('utf-8'))
+
+ outhash = hashlib.sha256()
+ outhash.update(args.outhash_seed.encode('utf-8'))
+ outhash.update(str(i).encode('utf-8'))
+
+ client.report_unihash(taskhash.hexdigest(), METHOD, outhash.hexdigest(), taskhash.hexdigest())
+
+ with lock:
+ pbar.update()
+
+ parser = argparse.ArgumentParser(description='Hash Equivalence Client')
+ parser.add_argument('--address', default=DEFAULT_ADDRESS, help='Server address (default "%(default)s")')
+ parser.add_argument('--log', default='WARNING', help='Set logging level')
+
+ subparsers = parser.add_subparsers()
+
+ stats_parser = subparsers.add_parser('stats', help='Show server stats')
+ stats_parser.add_argument('--reset', action='store_true',
+ help='Reset server stats')
+ stats_parser.set_defaults(func=handle_stats)
+
+ stress_parser = subparsers.add_parser('stress', help='Run stress test')
+ stress_parser.add_argument('--clients', type=int, default=10,
+ help='Number of simultaneous clients')
+ stress_parser.add_argument('--requests', type=int, default=1000,
+ help='Number of requests each client will perform')
+ stress_parser.add_argument('--report', action='store_true',
+ help='Report new hashes')
+ stress_parser.add_argument('--taskhash-seed', default='',
+ help='Include string in taskhash')
+ stress_parser.add_argument('--outhash-seed', default='',
+ help='Include string in outhash')
+ stress_parser.set_defaults(func=handle_stress)
+
+ args = parser.parse_args()
+
+ logger = logging.getLogger('hashserv')
+
+ level = getattr(logging, args.log.upper(), None)
+ if not isinstance(level, int):
+ raise ValueError('Invalid log level: %s' % args.log)
+
+ logger.setLevel(level)
+ console = logging.StreamHandler()
+ console.setLevel(level)
+ logger.addHandler(console)
+
+ func = getattr(args, 'func', None)
+ if func:
+ client = hashserv.create_client(args.address)
+ # Try to establish a connection to the server now to detect failures
+ # early
+ client.connect()
+
+ return func(args, client)
+
+ return 0
+
+
+if __name__ == '__main__':
+ try:
+ ret = main()
+ except Exception:
+ ret = 1
+ import traceback
+ traceback.print_exc()
+ sys.exit(ret)