diff options
| author | Brad Bishop <bradleyb@fuzziesquirrel.com> | 2019-09-10 14:20:22 +0300 |
|---|---|---|
| committer | Brad Bishop <bradleyb@fuzziesquirrel.com> | 2019-09-10 14:21:34 +0300 |
| commit | 79641f25e882b55af6a647cfc0bf6bcc025661a5 (patch) | |
| tree | 2b5fb3c59e5b6e85fd6fb8a94b6e15989d974336 /poky/meta/lib | |
| parent | 39ae1a1dce50fde698688fa5ac8eb737b302f0fd (diff) | |
| download | openbmc-79641f25e882b55af6a647cfc0bf6bcc025661a5.tar.xz | |
poky: subtree update:c17113f1e2..3b87508a9a
Adrian Bunk (18):
help2man: Upgrade 1.47.10 -> 1.47.11
glib-networking: Upgrade 2.60.2 -> 2.60.3
flac: Upgrade 1.3.2 -> 1.3.3
subversion: Upgrade 1.12.0 -> 1.12.2
world-broken.inc: Remove
meta/recipes.txt: Stop documenting the long removed recipes-lsb4
python: Remove PYTHONLSBOPTS that hasn't been set for some time
systemd: Print a buildtime warning about the status with musl
linux-dtb.inc: Remove
systemd: Update to the latest 242 branch
Remove LSB support
Remove LSB support bits
libxcrypt: Fix the build with -Os
systemd: Update to the latest 242 branch
Remove leftovers from the eglinfo removal
Update to KERNEL_DEVICETREE variable.
ref-manual: Remove documentation for the removed bluez5 distro feature
ref-manual: Remove documentation for the removed gnome class
Alex Kiernan (1):
iputils: Set prefix correctly for usrmerge
Alexander Kanavin (23):
python3-numpy: update to 1.17.0
images: use 512M of RAM in qemu if 'opengl" is in DISTRO_FEATURES
boost: update to 1.71.0
beaglebone: use the default of 256M rather than 512M
rpm: resolve a host contamination issue for mono packaging
qemu: set default RAM to 256M for all machines
mesa: disable gallium swrast driver on x86 x32
ltp: move from python 2.x to python 3.x
lsb-release: fix upstream version check
libffi: fix upstream version check
qemu: switch to '-vga std' emulated hardware from vmware/cirrus for x86/mips
ca-certificates: use python 3 for builds
libpsl: use python 3 during builds
pseudo: use python 3 during builds
llvm: use python 3 during builds
virglrenderer: update 0.7.0 -> 0.8.0
webkitgtk: use Python 3 for builds
webkitgtk: update 2.24.3 -> 2.24.4
default-providers.inc: set mesa as the provider for native(sdk) virtual/libgl
libepoxy: enable x11 PACKAGECONFIG for native/nativesdk
eglinfo: remove the recipe
local.conf.sample: add a commented-out option to enable the gtk UI qemu frontend
package.bbclass: allow shell-style wildcards in PRIVATE_LIBS
Alistair Francis (2):
linux-yocto-dev: Mark qemuriscv64 as compatible
qemuriscv: Generate a wic rootFS with a larger filesystem
Andre McCurdy (11):
bitbake.conf: drop FETCHCMD, GITDIR, etc fetcher over-rides
tune-arm1136jf-s.inc: add hardfloat support
tune-arm1176jz-s.inc: add support for arm1176jz-s CPU specific tuning
tune-ep9312.inc: fix TUNEVALID description
packagegroup-core-boot: drop duplicate MACHINE_ESSENTIAL_EXTRA_RDEPENDS, etc
rootfs-postcommands.bbclass: add inittab tweak to read_only_rootfs_hook()
base-files: nsswitch.conf is glibc specific
sqlite3: make CFLAGS consistent across native, nativesdk and target cases
sqlite3: explicitly set target endian-ness
feature-arm-thumb.inc: fix ARM_THUMB_SUFFIX
dropbear: don't create invalid symlinks if ssh, scp, etc are disabled
Anuj Mittal (1):
libjpeg-turbo: upgrade 2.0.2 -> 2.0.3
Armin Kuster (4):
bitbake: tests/fetch: Resolve fetch error in bitbake-selftest
gcc-9.2: Security fix for CVE-2019-14250
gcc-8.3: Security fix for CVE-2019-14250
gcc-8.3: remove 8.3 from zeus
Bruce Ashfield (11):
bluez5: fix build against libc-headers 5.2+
kernel-devsrc: remove python2 dependency
perf: change dependencies on python to python3
perf: v5.3+ build fixes
devsrc: update arm64 scripts/prepare for kernels 5.3+
linux-libc-headers: update to v5.2 headers
linux-yocto: introduce 5.2 recipes
qemu: bump linux-yocto preferred version to 5.2
linux-yocto/5.2: update qemuriscv64 SRCREV
linux-yocto/5.2: config tweaks and kallsyms fix
linux-yocto: drop 5.0 recipes
Changhyeok Bae (1):
iputils: upgrade to s20190709
Changqing Li (8):
libedit: fix failed to get history
avahi-ui: support usrmerge
bootchart2: support usrmerge
systemd-bootchart: support usrmerge
bootchart2: switch to add patch from change source in do_install
mdadm: skip timeout testcase 11spare-migration
dnf.py: installroot support usrmerge
man-db: support usrmerge
Chen Qi (7):
bbtests.py: remove recipe-depends.dot
commands.py: fix typo
bitbake: cooker.py: remove generation of recipe-depends.dot
bind: fix build with python3 PACKAGECONFIG enabled
python-testtools: rdepend on doctest module
python-numpy: rdepend on multiprocessing module
python3: make misc package rdepend on pydoc and pickle modules
Christophe PRIOUZEAU (1):
common-licenses: update BSD-2-CLAUSE license text
Dan Callaghan (1):
autotools: always include config_site.d files in CONFIG_SITE
Daniel Klauer (1):
image.bbclass: Fix debug output for rootfs size
Denys Zagorui (1):
gzip: add nativesdk support
Drew Moseley (1):
runqemu: Fix typo in help text.
He Zhe (4):
ltp: cve/meltdown.c: Fix kernel symbol finding
ltp: syscalls: rt_sigwaitinfo01: Fix failure for MIPS arches
qemu: Add pkg-config handling for libgcrypt
qemu: Fix dependency of PACKAGECONFIG libcurl
Jacob Kroon (2):
btrfs-tools: Add PACKAGECONFIG knob for controlling python support
rm_work: Remove redundant 'after' in addtask statement
Jason Wessel (2):
serial-getty@.service: Allow device to fast fail if it does not exist
Revert "kernel.bbclass: adjust a condition checking"
Joshua Watt (9):
bitbake: bitbake: respect force flag in runall and runonly
oeqa: reproducible: Cleanup reproducible build
oeqa: reproducible: Use subTest for packages
oeqa: Set LD_LIBRARY_PATH when executing native commands
oeqa: reproducible: Record packages in test results
oeqa: Enable reproducible build test
oeqa: reproducible: Do two clean builds
oeqa: reproducible: Preserve sstate mirror for first build
resulttool: Add reproducible log extraction
Kai Kang (4):
avahi: launch avahi-daemon after connman
cups: redefine CUPS_SERVERBIN
texinfo: fix multilib file confliction
sysstat: 12.1.3 -> 12.1.6
Kevin Hao (1):
meta-yocto-bsp: Bump to the v5.2 kernel
Khem Raj (12):
go-1.12: update to 1.12.9 minor release
powerpc-tunes: Introduce bigendian tune
tune-riscv: Drop littleendian and introduce bigendian tune
gcc: Move STANDARD_STARTFILE_PREFIX_1 and STANDARD_STARTFILE_PREFIX_2 back to gcc.c
gcc: Fix ldbl-128 support for musl
libgpg-error: Fix build with gawk 5.x
libgpg-error: Fix build with gawk 5.x
musl: Update to latest master
gcc-9.2: Fix risc-v dynamic linker relocation
python3native, pythonnative: export PYTHON_LIBRARY and PYTHON_INCLUDE_DIR
oeqa: Recognise svlogd as another logger
python3-manifest.json: Fix typo in summary
Mark Asselstine (1):
sdkext: use simpler kernel module for devtool test
Mark Hatle (1):
yocto-check-layer: Allow any case for README file detection
Martin Hundebøll (1):
initscripts: remove warnings on read-only-rootfs (again)
Martin Jansa (3):
bitbake: utils: Fix movefile() exception handling with python3
bitbake: fetch2: show warning when renaming the archive with bad checksum failed
bitbake: bitbake-user-manual: Improve the example for Removal (Override Style Syntax)
Mazliana (1):
oeqa/kerneldevelopment: Able to apply a single patch to the Linux kernel source
Ming Liu (1):
weston: uprev to 7.0.0
Nathan Rossi (30):
binutils: Fix mips patch which changes default emulation
oeqa/utils/nfs: Add unfs_server function to setup a userspace NFS server
scripts/lib/resulttool/report.py: Add more result types
gcc-common.inc: Process staging fixme with correct target/native sysroot
oeqa/core/tests: Skip test_fail_duplicated_module
oeqa/core/tests: Fix test_data module tests
Clean up remnants of glibc-initial
oe-selftest: Implement console 'keepalive' output
dejagnu: Add dejagnu for binutils/gcc test suites
binutils-cross-testsuite: Create recipe for test suite execution
gcc-runtime: Add do_check task for executing gcc test suites
glibc-testsuite: Create a recipe to implement glibc test suite
gcc-testsuite.inc: Fix ssh.exp, ensure multilib_flags are populated
security_flags.inc: Add overrides for glibc-testsuite to match glibc
archiver.bbclass: Do not archive srpm's if there are no package tasks
oeqa/core: Implement proper extra result collection and serialization
oeqa/core: Rework OETestTag and remove unused OETestFilter
oeqa/selftest: Add test run filtering based on test tags
oeqa/core/runner.py: Fix OETestTag listing
oeqa/selftest/binutils: Create selftest case for binutils test suite
oeqa/selftest/gcc: Create selftest case for gcc test suite
oeqa/selftest/glibc: Create selftest case for glibc test suite
oeqa/selftest/cases/glibc.py: Don't install python3 and pexpect
oeqa/selftest: For toolchain tests do not fail if a test failed
oeqa/selftest: Toolchain tests suffix "-user" for qemu usermode results
oeqa/selftest: Use extraresults on self instead of self.tc
oeqa/selftest/context.py: Change -t/-T args to be optional
oeqa/core/decorator: Fix super class modifying subclass tags
oeqa/selftest/cases/gcc.py: Split into classes for parallelism
oeqa/selftest/cases/glibc.py: Rework and tag with toolchain-user/system
Neil Armstrong (1):
mesa: Add support for the panfrost PACKAGECONFIG
Oleksandr Kravchuk (8):
libsolv: update to 0.7.6
libevdev: update to 1.8.0
ell: update to 0.22
go-dep: update to 0.5.4
libsecret: 0.19.0
python3-pbr: update to 5.4.3
python3-git: update to 3.0.2
libxcrypt: update to 4.4.8
Peiran Hong (1):
diffutils: Added perl to support ptest & Skipped one test case
Peter Kjellerstedt (2):
libffi: Make it build for MIPS o32
devtool: Avoid failure for recipes with S == WORKDIR and no local files
Richard Purdie (11):
poky-altcfg: Replace poky-lsb with poky-altcfg
ca-certificates: Fix reproducibilty and multilib issue
gcc-cross: Clean up fixed-includes
systemtap: Use latest and greatest git version for 5.2 kernel support
gcc-cross: Fix header file corruption problems
poky: Use 5.2 kernel by default
binutils/glibc-testsuite: inherit nopackages to fix world builds
systemtap: Drop patches merged upstream
poky-tiny/poky-altcfg: Use busybox-mdev and systemd
oeqa/selftest: Markup 'machine' specific test cases
oeqa/selftest: Tweak binutils tests tags as toolchain/user and system
Robert Yang (3):
ccache: 3.7.1 -> 3.7.3
nfs-utils: 2.3.3 -> 2.4.1
build-compare: 2015.02.10 -> 2019.08.14
Ross Burton (11):
libedit: remove FETCHCMD append
libnl: upgrade to 3.5.0
libnl: don't patch libnl-3.0.pc
systemd: explicitly set the path to nologin
core-image-sato: don't use 512MB in qemumips
patch: add CVE tags to patches
pango: 1.44.6 upgrade
pango: add PACKAGECONFIG for libthai support
harfbuzz: upgrade to 2.6.1
systemd: ensure reproducible builds by clearly exposing the time epoch support
ref-system-requirements: add Debian 10 to supported distribution list
Scott Rifenbark (5):
ref-manual: Updated the "AS" variable
ref-manual: Updates to "Building Your Recipe"
dev-manual: Updated "Creating a Team Development Environment"
poky-end: Added "pylint3" package to Ubuntu essentials.
ref-manual: sync list of image types with source
Tim Blechmann (1):
icecc: blacklist systemtap
Trevor Gamblin (2):
lighttpd: drop hard-coded lua version in PACKAGECONFIG
libevent: add granularity to ptest log
Vijai Kumar K (1):
lib/oe/gpg_sign.py: Remove unwanted import
Wes Lindauer (1):
bitbake: bitbake: cooker: Ensure bbappends are found in stable order
Yi Zhao (1):
libbsd: upgrade 0.9.1 -> 0.10.0
Yuan Chao (1):
man-db: upgrade 2.8.6.1 -> 2.8.7
Zang Ruochen (7):
libarchive:upgrade 3.3.3 -> 3.4.0
python3-pip:upgrade 19.2.2 -> 19.2.3
logrotate:upgrade 3.15.0 -> 3.15.1
libogg:upgrade 1.3.3 -> 1.3.4
dtc:upgrade 1.5.0 -> 1.5.1
python3-dbus:upgrade 1.2.8 -> 1.2.10
meson:upgrade 0.51.1 -> 0.51.2
Change-Id: I341e8df65e8196114088a2d6d6eb3a33296188b4
Signed-off-by: Brad Bishop <bradleyb@fuzziesquirrel.com>
Diffstat (limited to 'poky/meta/lib')
33 files changed, 734 insertions, 243 deletions
diff --git a/poky/meta/lib/oe/gpg_sign.py b/poky/meta/lib/oe/gpg_sign.py index d7624804d..7634d7ef1 100644 --- a/poky/meta/lib/oe/gpg_sign.py +++ b/poky/meta/lib/oe/gpg_sign.py @@ -6,7 +6,6 @@ import os import bb -import oe.utils import subprocess import shlex diff --git a/poky/meta/lib/oeqa/core/context.py b/poky/meta/lib/oeqa/core/context.py index 68819cc33..14fc6a54f 100644 --- a/poky/meta/lib/oeqa/core/context.py +++ b/poky/meta/lib/oeqa/core/context.py @@ -64,12 +64,12 @@ class OETestContext(object): setattr(tclass, 'setUpHooker', skipfuncgen('Skip by the command line argument "%s"' % skip)) def loadTests(self, module_paths, modules=[], tests=[], - modules_manifest="", modules_required=[], filters={}): + modules_manifest="", modules_required=[], **kwargs): if modules_manifest: modules = self._read_modules_from_manifest(modules_manifest) self.loader = self.loaderClass(self, module_paths, modules, tests, - modules_required, filters) + modules_required, **kwargs) self.suites = self.loader.discover() def runTests(self, processes=None, skips=[]): diff --git a/poky/meta/lib/oeqa/core/decorator/__init__.py b/poky/meta/lib/oeqa/core/decorator/__init__.py index 923b21826..1a82518ab 100644 --- a/poky/meta/lib/oeqa/core/decorator/__init__.py +++ b/poky/meta/lib/oeqa/core/decorator/__init__.py @@ -6,6 +6,7 @@ from functools import wraps from abc import abstractmethod, ABCMeta +from oeqa.core.utils.misc import strToList decoratorClasses = set() @@ -63,12 +64,16 @@ class OETestDiscover(OETestDecorator): def discover(registry): return registry['cases'] -class OETestFilter(OETestDecorator): +def OETestTag(*tags): + expandedtags = [] + for tag in tags: + expandedtags += strToList(tag) + def decorator(item): + if hasattr(item, "__oeqa_testtags"): + # do not append, create a new list (to handle classes with inheritance) + item.__oeqa_testtags = list(item.__oeqa_testtags) + expandedtags + else: + item.__oeqa_testtags = expandedtags + return item + return decorator - # OETestLoader call it while loading the tests - # in loadTestsFromTestCase method, it needs to - # return a bool, True if needs to be filtered. - # This method must consume the filter used. - @abstractmethod - def filtrate(self, filters): - return False diff --git a/poky/meta/lib/oeqa/core/decorator/oetag.py b/poky/meta/lib/oeqa/core/decorator/oetag.py deleted file mode 100644 index 8c31138da..000000000 --- a/poky/meta/lib/oeqa/core/decorator/oetag.py +++ /dev/null @@ -1,27 +0,0 @@ -# -# Copyright (C) 2016 Intel Corporation -# -# SPDX-License-Identifier: MIT -# - -from . import OETestFilter, registerDecorator -from oeqa.core.utils.misc import strToList - -def _tagFilter(tags, filters): - return False if set(tags) & set(filters) else True - -@registerDecorator -class OETestTag(OETestFilter): - attrs = ('oetag',) - - def bind(self, registry, case): - super(OETestTag, self).bind(registry, case) - self.oetag = strToList(self.oetag, 'oetag') - - def filtrate(self, filters): - if filters.get('oetag'): - filterx = strToList(filters['oetag'], 'oetag') - del filters['oetag'] - if _tagFilter(self.oetag, filterx): - return True - return False diff --git a/poky/meta/lib/oeqa/core/loader.py b/poky/meta/lib/oeqa/core/loader.py index 7fea0585c..0d7970d49 100644 --- a/poky/meta/lib/oeqa/core/loader.py +++ b/poky/meta/lib/oeqa/core/loader.py @@ -16,7 +16,7 @@ from oeqa.core.utils.test import getSuiteModules, getCaseID from oeqa.core.exception import OEQATestNotFound from oeqa.core.case import OETestCase from oeqa.core.decorator import decoratorClasses, OETestDecorator, \ - OETestFilter, OETestDiscover + OETestDiscover # When loading tests, the unittest framework stores any exceptions and # displays them only when the run method is called. @@ -68,7 +68,7 @@ class OETestLoader(unittest.TestLoader): '_top_level_dir'] def __init__(self, tc, module_paths, modules, tests, modules_required, - filters, *args, **kwargs): + *args, **kwargs): self.tc = tc self.modules = _built_modules_dict(modules) @@ -76,13 +76,7 @@ class OETestLoader(unittest.TestLoader): self.tests = tests self.modules_required = modules_required - self.filters = filters - self.decorator_filters = [d for d in decoratorClasses if \ - issubclass(d, OETestFilter)] - self._validateFilters(self.filters, self.decorator_filters) - self.used_filters = [d for d in self.decorator_filters - for f in self.filters - if f in d.attrs] + self.tags_filter = kwargs.get("tags_filter", None) if isinstance(module_paths, str): module_paths = [module_paths] @@ -104,28 +98,6 @@ class OETestLoader(unittest.TestLoader): setattr(testCaseClass, 'td', self.tc.td) setattr(testCaseClass, 'logger', self.tc.logger) - def _validateFilters(self, filters, decorator_filters): - # Validate if filter isn't empty - for key,value in filters.items(): - if not value: - raise TypeError("Filter %s specified is empty" % key) - - # Validate unique attributes - attr_filters = [attr for clss in decorator_filters \ - for attr in clss.attrs] - dup_attr = [attr for attr in attr_filters - if attr_filters.count(attr) > 1] - if dup_attr: - raise TypeError('Detected duplicated attribute(s) %s in filter' - ' decorators' % ' ,'.join(dup_attr)) - - # Validate if filter is supported - for f in filters: - if f not in attr_filters: - classes = ', '.join([d.__name__ for d in decorator_filters]) - raise TypeError('Found "%s" filter but not declared in any of ' - '%s decorators' % (f, classes)) - def _registerTestCase(self, case): case_id = case.id() self.tc._registry['cases'][case_id] = case @@ -188,19 +160,20 @@ class OETestLoader(unittest.TestLoader): return True # Decorator filters - if self.filters and isinstance(case, OETestCase): - filters = self.filters.copy() - case_decorators = [cd for cd in case.decorators - if cd.__class__ in self.used_filters] - - # Iterate over case decorators to check if needs to be filtered. - for cd in case_decorators: - if cd.filtrate(filters): - return True - - # Case is missing one or more decorators for all the filters - # being used, so filter test case. - if filters: + if self.tags_filter is not None and callable(self.tags_filter): + alltags = set() + # pull tags from the case class + if hasattr(case, "__oeqa_testtags"): + for t in getattr(case, "__oeqa_testtags"): + alltags.add(t) + # pull tags from the method itself + if hasattr(case, test_name): + method = getattr(case, test_name) + if hasattr(method, "__oeqa_testtags"): + for t in getattr(method, "__oeqa_testtags"): + alltags.add(t) + + if self.tags_filter(alltags): return True return False diff --git a/poky/meta/lib/oeqa/core/runner.py b/poky/meta/lib/oeqa/core/runner.py index 930620ea1..f656e1a9c 100644 --- a/poky/meta/lib/oeqa/core/runner.py +++ b/poky/meta/lib/oeqa/core/runner.py @@ -43,6 +43,7 @@ class OETestResult(_TestResult): self.starttime = {} self.endtime = {} self.progressinfo = {} + self.extraresults = {} # Inject into tc so that TestDepends decorator can see results tc.results = self @@ -129,19 +130,51 @@ class OETestResult(_TestResult): return 'UNKNOWN', None - def addSuccess(self, test): + def extractExtraResults(self, test, details = None): + extraresults = None + if details is not None and "extraresults" in details: + extraresults = details.get("extraresults", {}) + elif hasattr(test, "extraresults"): + extraresults = test.extraresults + + if extraresults is not None: + for k, v in extraresults.items(): + # handle updating already existing entries (e.g. ptestresults.sections) + if k in self.extraresults: + self.extraresults[k].update(v) + else: + self.extraresults[k] = v + + def addError(self, test, *args, details = None): + self.extractExtraResults(test, details = details) + return super(OETestResult, self).addError(test, *args) + + def addFailure(self, test, *args, details = None): + self.extractExtraResults(test, details = details) + return super(OETestResult, self).addFailure(test, *args) + + def addSuccess(self, test, details = None): #Added so we can keep track of successes too self.successes.append((test, None)) - super(OETestResult, self).addSuccess(test) + self.extractExtraResults(test, details = details) + return super(OETestResult, self).addSuccess(test) + + def addExpectedFailure(self, test, *args, details = None): + self.extractExtraResults(test, details = details) + return super(OETestResult, self).addExpectedFailure(test, *args) + + def addUnexpectedSuccess(self, test, details = None): + self.extractExtraResults(test, details = details) + return super(OETestResult, self).addUnexpectedSuccess(test) def logDetails(self, json_file_dir=None, configuration=None, result_id=None, dump_streams=False): self.tc.logger.info("RESULTS:") - result = {} + result = self.extraresults logs = {} if hasattr(self.tc, "extraresults"): - result = self.tc.extraresults + result.update(self.tc.extraresults) for case_name in self.tc._registry['cases']: case = self.tc._registry['cases'][case_name] @@ -205,23 +238,20 @@ class OETestRunner(_TestRunner): self._walked_cases = self._walked_cases + 1 def _list_tests_name(self, suite): - from oeqa.core.decorator.oetag import OETestTag - self._walked_cases = 0 def _list_cases(logger, case): - oetag = None - - if hasattr(case, 'decorators'): - for d in case.decorators: - if isinstance(d, OETestTag): - oetag = d.oetag - - logger.info("%s\t\t%s" % (oetag, case.id())) + oetags = [] + if hasattr(case, '__oeqa_testtags'): + oetags = getattr(case, '__oeqa_testtags') + if oetags: + logger.info("%s (%s)" % (case.id(), ",".join(oetags))) + else: + logger.info("%s" % (case.id())) self.tc.logger.info("Listing all available tests:") self._walked_cases = 0 - self.tc.logger.info("id\ttag\t\ttest") + self.tc.logger.info("test (tags)") self.tc.logger.info("-" * 80) self._walk_suite(suite, _list_cases) self.tc.logger.info("-" * 80) diff --git a/poky/meta/lib/oeqa/core/tests/cases/data.py b/poky/meta/lib/oeqa/core/tests/cases/data.py index 0d8de87ae..61f88547f 100644 --- a/poky/meta/lib/oeqa/core/tests/cases/data.py +++ b/poky/meta/lib/oeqa/core/tests/cases/data.py @@ -5,7 +5,7 @@ # from oeqa.core.case import OETestCase -from oeqa.core.decorator.oetag import OETestTag +from oeqa.core.decorator import OETestTag from oeqa.core.decorator.data import OETestDataDepends class DataTest(OETestCase): diff --git a/poky/meta/lib/oeqa/core/tests/cases/oetag.py b/poky/meta/lib/oeqa/core/tests/cases/oetag.py index 4e1d08098..52f97dfda 100644 --- a/poky/meta/lib/oeqa/core/tests/cases/oetag.py +++ b/poky/meta/lib/oeqa/core/tests/cases/oetag.py @@ -5,10 +5,9 @@ # from oeqa.core.case import OETestCase -from oeqa.core.decorator.oetag import OETestTag +from oeqa.core.decorator import OETestTag class TagTest(OETestCase): - @OETestTag('goodTag') def testTagGood(self): self.assertTrue(True, msg='How is this possible?') @@ -17,5 +16,23 @@ class TagTest(OETestCase): def testTagOther(self): self.assertTrue(True, msg='How is this possible?') + @OETestTag('otherTag', 'multiTag') + def testTagOtherMulti(self): + self.assertTrue(True, msg='How is this possible?') + def testTagNone(self): self.assertTrue(True, msg='How is this possible?') + +@OETestTag('classTag') +class TagClassTest(OETestCase): + @OETestTag('otherTag') + def testTagOther(self): + self.assertTrue(True, msg='How is this possible?') + + @OETestTag('otherTag', 'multiTag') + def testTagOtherMulti(self): + self.assertTrue(True, msg='How is this possible?') + + def testTagNone(self): + self.assertTrue(True, msg='How is this possible?') + diff --git a/poky/meta/lib/oeqa/core/tests/common.py b/poky/meta/lib/oeqa/core/tests/common.py index 39efd504c..88cc758ad 100644 --- a/poky/meta/lib/oeqa/core/tests/common.py +++ b/poky/meta/lib/oeqa/core/tests/common.py @@ -30,9 +30,9 @@ class TestBase(unittest.TestCase): directory = os.path.dirname(os.path.abspath(__file__)) self.cases_path = os.path.join(directory, 'cases') - def _testLoader(self, d={}, modules=[], tests=[], filters={}): + def _testLoader(self, d={}, modules=[], tests=[], **kwargs): from oeqa.core.context import OETestContext tc = OETestContext(d, self.logger) tc.loadTests(self.cases_path, modules=modules, tests=tests, - filters=filters) + **kwargs) return tc diff --git a/poky/meta/lib/oeqa/core/tests/test_data.py b/poky/meta/lib/oeqa/core/tests/test_data.py index 50811bb38..ac74098b7 100755 --- a/poky/meta/lib/oeqa/core/tests/test_data.py +++ b/poky/meta/lib/oeqa/core/tests/test_data.py @@ -22,8 +22,9 @@ class TestData(TestBase): expectedException = "oeqa.core.exception.OEQAMissingVariable" tc = self._testLoader(modules=self.modules) - self.assertEqual(False, tc.runTests().wasSuccessful()) - for test, data in tc.errors: + results = tc.runTests() + self.assertFalse(results.wasSuccessful()) + for test, data in results.errors: expect = False if expectedException in data: expect = True @@ -35,8 +36,9 @@ class TestData(TestBase): d = {'IMAGE' : 'core-image-sato', 'ARCH' : 'arm'} tc = self._testLoader(d=d, modules=self.modules) - self.assertEqual(False, tc.runTests().wasSuccessful()) - for test, data in tc.failures: + results = tc.runTests() + self.assertFalse(results.wasSuccessful()) + for test, data in results.failures: expect = False if expectedError in data: expect = True diff --git a/poky/meta/lib/oeqa/core/tests/test_decorators.py b/poky/meta/lib/oeqa/core/tests/test_decorators.py index 499cd66ff..b798bf7d3 100755 --- a/poky/meta/lib/oeqa/core/tests/test_decorators.py +++ b/poky/meta/lib/oeqa/core/tests/test_decorators.py @@ -14,35 +14,58 @@ setup_sys_path() from oeqa.core.exception import OEQADependency from oeqa.core.utils.test import getCaseMethod, getSuiteCasesNames, getSuiteCasesIDs -class TestFilterDecorator(TestBase): - - def _runFilterTest(self, modules, filters, expect, msg): - tc = self._testLoader(modules=modules, filters=filters) - test_loaded = set(getSuiteCasesNames(tc.suites)) - self.assertEqual(expect, test_loaded, msg=msg) +class TestTagDecorator(TestBase): + def _runTest(self, modules, filterfn, expect): + tc = self._testLoader(modules = modules, tags_filter = filterfn) + test_loaded = set(getSuiteCasesIDs(tc.suites)) + self.assertEqual(expect, test_loaded) def test_oetag(self): - # Get all cases without filtering. - filter_all = {} - test_all = {'testTagGood', 'testTagOther', 'testTagNone'} - msg_all = 'Failed to get all oetag cases without filtering.' - - # Get cases with 'goodTag'. - filter_good = {'oetag':'goodTag'} - test_good = {'testTagGood'} - msg_good = 'Failed to get just one test filtering with "goodTag" oetag.' - - # Get cases with an invalid tag. - filter_invalid = {'oetag':'invalidTag'} - test_invalid = set() - msg_invalid = 'Failed to filter all test using an invalid oetag.' - - tests = ((filter_all, test_all, msg_all), - (filter_good, test_good, msg_good), - (filter_invalid, test_invalid, msg_invalid)) - - for test in tests: - self._runFilterTest(['oetag'], test[0], test[1], test[2]) + # get all cases without any filtering + self._runTest(['oetag'], None, { + 'oetag.TagTest.testTagGood', + 'oetag.TagTest.testTagOther', + 'oetag.TagTest.testTagOtherMulti', + 'oetag.TagTest.testTagNone', + 'oetag.TagClassTest.testTagOther', + 'oetag.TagClassTest.testTagOtherMulti', + 'oetag.TagClassTest.testTagNone', + }) + + # exclude any case with tags + self._runTest(['oetag'], lambda tags: tags, { + 'oetag.TagTest.testTagNone', + }) + + # exclude any case with otherTag + self._runTest(['oetag'], lambda tags: "otherTag" in tags, { + 'oetag.TagTest.testTagGood', + 'oetag.TagTest.testTagNone', + 'oetag.TagClassTest.testTagNone', + }) + + # exclude any case with classTag + self._runTest(['oetag'], lambda tags: "classTag" in tags, { + 'oetag.TagTest.testTagGood', + 'oetag.TagTest.testTagOther', + 'oetag.TagTest.testTagOtherMulti', + 'oetag.TagTest.testTagNone', + }) + + # include any case with classTag + self._runTest(['oetag'], lambda tags: "classTag" not in tags, { + 'oetag.TagClassTest.testTagOther', + 'oetag.TagClassTest.testTagOtherMulti', + 'oetag.TagClassTest.testTagNone', + }) + + # include any case with classTag or no tags + self._runTest(['oetag'], lambda tags: tags and "classTag" not in tags, { + 'oetag.TagTest.testTagNone', + 'oetag.TagClassTest.testTagOther', + 'oetag.TagClassTest.testTagOtherMulti', + 'oetag.TagClassTest.testTagNone', + }) class TestDependsDecorator(TestBase): modules = ['depends'] diff --git a/poky/meta/lib/oeqa/core/tests/test_loader.py b/poky/meta/lib/oeqa/core/tests/test_loader.py index 519ba9609..cb38ac845 100755 --- a/poky/meta/lib/oeqa/core/tests/test_loader.py +++ b/poky/meta/lib/oeqa/core/tests/test_loader.py @@ -15,31 +15,7 @@ from oeqa.core.exception import OEQADependency from oeqa.core.utils.test import getSuiteModules, getSuiteCasesIDs class TestLoader(TestBase): - - def test_fail_empty_filter(self): - filters = {'oetag' : ''} - expect = 'Filter oetag specified is empty' - msg = 'Expected TypeError exception for having invalid filter' - try: - # Must throw TypeError because empty filter - tc = self._testLoader(filters=filters) - self.fail(msg) - except TypeError as e: - result = True if expect in str(e) else False - self.assertTrue(result, msg=msg) - - def test_fail_invalid_filter(self): - filters = {'invalid' : 'good'} - expect = 'filter but not declared in any of' - msg = 'Expected TypeError exception for having invalid filter' - try: - # Must throw TypeError because invalid filter - tc = self._testLoader(filters=filters) - self.fail(msg) - except TypeError as e: - result = True if expect in str(e) else False - self.assertTrue(result, msg=msg) - + @unittest.skip("invalid directory is missing oetag.py") def test_fail_duplicated_module(self): cases_path = self.cases_path invalid_path = os.path.join(cases_path, 'loader', 'invalid') diff --git a/poky/meta/lib/oeqa/core/utils/concurrencytest.py b/poky/meta/lib/oeqa/core/utils/concurrencytest.py index 6bf771886..fa6fa34b0 100644 --- a/poky/meta/lib/oeqa/core/utils/concurrencytest.py +++ b/poky/meta/lib/oeqa/core/utils/concurrencytest.py @@ -21,6 +21,7 @@ import testtools import threading import time import io +import json import subunit from queue import Queue @@ -28,6 +29,8 @@ from itertools import cycle from subunit import ProtocolTestCase, TestProtocolClient from subunit.test_results import AutoTimingTestResultDecorator from testtools import ThreadsafeForwardingResult, iterate_tests +from testtools.content import Content +from testtools.content_type import ContentType from oeqa.utils.commands import get_test_layer import bb.utils @@ -70,6 +73,58 @@ class BBThreadsafeForwardingResult(ThreadsafeForwardingResult): self.semaphore.release() super(BBThreadsafeForwardingResult, self)._add_result_with_semaphore(method, test, *args, **kwargs) +class ProxyTestResult: + # a very basic TestResult proxy, in order to modify add* calls + def __init__(self, target): + self.result = target + + def _addResult(self, method, test, *args, **kwargs): + return method(test, *args, **kwargs) + + def addError(self, test, *args, **kwargs): + self._addResult(self.result.addError, test, *args, **kwargs) + + def addFailure(self, test, *args, **kwargs): + self._addResult(self.result.addFailure, test, *args, **kwargs) + + def addSuccess(self, test, *args, **kwargs): + self._addResult(self.result.addSuccess, test, *args, **kwargs) + + def addExpectedFailure(self, test, *args, **kwargs): + self._addResult(self.result.addExpectedFailure, test, *args, **kwargs) + + def addUnexpectedSuccess(self, test, *args, **kwargs): + self._addResult(self.result.addUnexpectedSuccess, test, *args, **kwargs) + + def __getattr__(self, attr): + return getattr(self.result, attr) + +class ExtraResultsDecoderTestResult(ProxyTestResult): + def _addResult(self, method, test, *args, **kwargs): + if "details" in kwargs and "extraresults" in kwargs["details"]: + if isinstance(kwargs["details"]["extraresults"], Content): + kwargs = kwargs.copy() + kwargs["details"] = kwargs["details"].copy() + extraresults = kwargs["details"]["extraresults"] + data = bytearray() + for b in extraresults.iter_bytes(): + data += b + extraresults = json.loads(data.decode()) + kwargs["details"]["extraresults"] = extraresults + return method(test, *args, **kwargs) + +class ExtraResultsEncoderTestResult(ProxyTestResult): + def _addResult(self, method, test, *args, **kwargs): + if hasattr(test, "extraresults"): + extras = lambda : [json.dumps(test.extraresults).encode()] + kwargs = kwargs.copy() + if "details" not in kwargs: + kwargs["details"] = {} + else: + kwargs["details"] = kwargs["details"].copy() + kwargs["details"]["extraresults"] = Content(ContentType("application", "json", {'charset': 'utf8'}), extras) + return method(test, *args, **kwargs) + # # We have to patch subunit since it doesn't understand how to handle addError # outside of a running test case. This can happen if classSetUp() fails @@ -116,7 +171,9 @@ class ConcurrentTestSuite(unittest.TestSuite): result.threadprogress = {} for i, (test, testnum) in enumerate(tests): result.threadprogress[i] = [] - process_result = BBThreadsafeForwardingResult(result, semaphore, i, testnum, totaltests) + process_result = BBThreadsafeForwardingResult( + ExtraResultsDecoderTestResult(result), + semaphore, i, testnum, totaltests) # Force buffering of stdout/stderr so the console doesn't get corrupted by test output # as per default in parent code process_result.buffer = True @@ -231,7 +288,7 @@ def fork_for_tests(concurrency_num, suite): # as per default in parent code subunit_client.buffer = True subunit_result = AutoTimingTestResultDecorator(subunit_client) - process_suite.run(subunit_result) + process_suite.run(ExtraResultsEncoderTestResult(subunit_result)) if ourpid != os.getpid(): os._exit(0) if newbuilddir: diff --git a/poky/meta/lib/oeqa/manual/bsp-hw.json b/poky/meta/lib/oeqa/manual/bsp-hw.json index 4b7c76f70..18cec222f 100644 --- a/poky/meta/lib/oeqa/manual/bsp-hw.json +++ b/poky/meta/lib/oeqa/manual/bsp-hw.json @@ -526,7 +526,7 @@ "expected_results": "" }, "4": { - "action": "check ping status \n\nNote: This TC apply only for core-image-full-cmd and core-image-lsb .", + "action": "check ping status \n\nNote: This TC apply only for core-image-full-cmd.", "expected_results": "ping should always work before/after standby" } }, diff --git a/poky/meta/lib/oeqa/manual/compliance-test.json b/poky/meta/lib/oeqa/manual/compliance-test.json index 8c13b68ec..367a416f0 100644 --- a/poky/meta/lib/oeqa/manual/compliance-test.json +++ b/poky/meta/lib/oeqa/manual/compliance-test.json @@ -1,36 +1,6 @@ [ { "test": { - "@alias": "compliance-test.compliance-test.LSB_subset_test_suite", - "author": [ - { - "email": "corneliux.stoicescu@intel.com", - "name": "corneliux.stoicescu@intel.com" - } - ], - "execution": { - "1": { - "action": "Get lsd-sdk image and install it on target device or start the image(if it is QEMU) with option \"-m 512M\"", - "expected_results": "" - }, - "2": { - "action": "Comment in /opt/lsb-test/session any tests you don't want to run.", - "expected_results": "" - }, - "3": { - "action": "Run /usr/bin/LSB_Test.sh which should download the LSB suite and set it up. Some packages may fail to download because their location changed on ftp.linuxfoundation.org. You need to manually update /opt/lsb-test/packages_list", - "expected_results": "" - }, - "4": { - "action": "Tests should start automatically, you can use the web interface to reconfigure the setup. ", - "expected_results": "Check the result on wiki https://wiki.yoctoproject.org/wiki/LSB_Result No regression failures should be met." - } - }, - "summary": "LSB_subset_test_suite" - } - }, - { - "test": { "@alias": "compliance-test.compliance-test.stress_test_-_Genericx86-64", "author": [ { @@ -40,7 +10,7 @@ ], "execution": { "1": { - "action": "Bootup with core-image-lsb-sdk image", + "action": "Bootup with core-image-sato-sdk image", "expected_results": "" }, "2": { diff --git a/poky/meta/lib/oeqa/manual/crops.json b/poky/meta/lib/oeqa/manual/crops.json index 1cf3c8f30..5cfa65384 100644 --- a/poky/meta/lib/oeqa/manual/crops.json +++ b/poky/meta/lib/oeqa/manual/crops.json @@ -234,16 +234,16 @@ "expected_results": "this should output the directory of the devtool script and it should be within the sdk workdir you are working in. \n\n" }, "5": { - "action": "devtool add v4l2loopback-driver https://github.com/umlaeute/v4l2loopback.git \n\n", - "expected_results": "This should automatically create the recipe v4l2loopback-driver.bb under <crops-esdk-workdir-workspace>/recipes/v4l2loopback-driver/v4l2loopback-driver.bb " + "action": "devtool add kernel-module-hello-world https://git.yoctoproject.org/git/kernel-module-hello-world \n\n", + "expected_results": "This should automatically create the recipe kernel-module-hello-world.bb under <crops-esdk-workdir-workspace>/recipes/kernel-module-hello-world/kernel-module-hello-world.bb " }, "6": { - "action": "devtool build v4l2loopback-driver \n\n", + "action": "devtool build kernel-module-hello-world \n\n", "expected_results": "This should compile an image \n\n" }, "7": { - "action": "devtool reset v4l2loopback-driver ", - "expected_results": "This cleans sysroot of the v4l2loopback-driver recipe, but it leaves the source tree intact. meaning it does not erase." + "action": "devtool reset kernel-module-hello-world ", + "expected_results": "This cleans sysroot of the kernel-module-hello-world recipe, but it leaves the source tree intact. meaning it does not erase." } }, "summary": "sdkext_devtool_kernelmodule" @@ -291,4 +291,4 @@ "summary": "sdkext_recipes_for_nodejs" } } -]
\ No newline at end of file +] diff --git a/poky/meta/lib/oeqa/manual/toaster-managed-mode.json b/poky/meta/lib/oeqa/manual/toaster-managed-mode.json index 812f57da3..12374c7c6 100644 --- a/poky/meta/lib/oeqa/manual/toaster-managed-mode.json +++ b/poky/meta/lib/oeqa/manual/toaster-managed-mode.json @@ -1494,7 +1494,7 @@ "expected_results": "A type in form appears. \n\n\t" }, "6": { - "action": "Change distro (ex: poky-lsb). \n\n\t", + "action": "Change distro (ex: poky-tiny). \n\n\t", "expected_results": "Distro has changed. \n\n\t" }, "7": { @@ -1503,7 +1503,7 @@ }, "8": { "action": " Build a recipe (ex: core-image-minimal) and wait until build finish.", - "expected_results": "Build finishes successfully. \n\nThe 'success' criteria for this one should be that the build is reported as using the poky-lsb distro in the build summary page, and that the DISTRO variable value in the bitbake variables table is set to the value specified in toaster (poky-lsb again)." + "expected_results": "Build finishes successfully. \n\nThe 'success' criteria for this one should be that the build is reported as using the poky-tiny distro in the build summary page, and that the DISTRO variable value in the bitbake variables table is set to the value specified in toaster (poky-tiny again)." } }, "summary": "Build_a_recipe_with_different_distro" @@ -1612,7 +1612,7 @@ "expected_results": "" }, "3": { - "action": "Check that the table is populated with the list of image recipes (eg. core-image minimal, core-image-lsb) \n\n\n", + "action": "Check that the table is populated with the list of image recipes (eg. core-image minimal) \n\n\n", "expected_results": "" }, "4": { @@ -1670,7 +1670,7 @@ "expected_results": "" }, "7": { - "action": "Sort the table by \"Layer\" and then navigate away by selecting an image (such as core-image-lsb). When you click the \"back\" button in the web-browser to go back, the \"New custom image\" table should still be sorted by \"Layer\". \nThis should apply also by navigating back to the page by any other means. \n\n", + "action": "Sort the table by \"Layer\" and then navigate away by selecting an image. When you click the \"back\" button in the web-browser to go back, the \"New custom image\" table should still be sorted by \"Layer\". \nThis should apply also by navigating back to the page by any other means. \n\n", "expected_results": "" }, "8": { @@ -2348,7 +2348,7 @@ "expected_results": "" }, "3": { - "action": "Build 6 recipes example (core-image-sato, core-image-minimal, core-image-base, core-image-lsb, core-image-clutter) to name a few. ", + "action": "Build 6 recipes example (core-image-sato, core-image-minimal, core-image-base, core-image-clutter) to name a few. ", "expected_results": " All recipes are built correctly \n\n" }, "4": { @@ -2382,7 +2382,7 @@ "expected_results": "" }, "3": { - "action": "Build 6 recipes example (core-image-sato, core-image-minimal, core-image-base, core-image-lsb, core-image-clutter) to name a few. \n\n", + "action": "Build 6 recipes example (core-image-sato, core-image-minimal, core-image-base, core-image-clutter) to name a few. \n\n", "expected_results": "All recipes are built correctly \n\n" }, "4": { @@ -2420,7 +2420,7 @@ "expected_results": "" }, "3": { - "action": "Build 4 recipes example (core-image-sato, core-image-minimal, core-image-base, core-image-lsb, core-image-clutter) to name a few. \n\n", + "action": "Build 4 recipes example (core-image-sato, core-image-minimal, core-image-base, core-image-clutter) to name a few. \n\n", "expected_results": " All recipes are built correctly \n\n" }, "4": { diff --git a/poky/meta/lib/oeqa/runtime/cases/dnf.py b/poky/meta/lib/oeqa/runtime/cases/dnf.py index 629b9af3e..80cc86a4f 100644 --- a/poky/meta/lib/oeqa/runtime/cases/dnf.py +++ b/poky/meta/lib/oeqa/runtime/cases/dnf.py @@ -9,7 +9,7 @@ from oeqa.utils.httpserver import HTTPService from oeqa.runtime.case import OERuntimeTestCase from oeqa.core.decorator.depends import OETestDepends -from oeqa.core.decorator.data import skipIfNotDataVar, skipIfNotFeature +from oeqa.core.decorator.data import skipIfNotDataVar, skipIfNotFeature, skipIfInDataVar, skipIfNotInDataVar from oeqa.runtime.decorator.package import OEHasPackage class DnfTest(OERuntimeTestCase): @@ -116,6 +116,7 @@ class DnfRepoTest(DnfTest): self.dnf_with_repo('reinstall -y run-postinsts-dev') @OETestDepends(['dnf.DnfRepoTest.test_dnf_makecache']) + @skipIfInDataVar('DISTRO_FEATURES', 'usrmerge', 'Test run when not enable usrmerge') def test_dnf_installroot(self): rootpath = '/home/root/chroot/test' #Copy necessary files to avoid errors with not yet installed tools on @@ -141,6 +142,37 @@ class DnfRepoTest(DnfTest): self.assertEqual(0, status, output) @OETestDepends(['dnf.DnfRepoTest.test_dnf_makecache']) + @skipIfNotInDataVar('DISTRO_FEATURES', 'usrmerge', 'Test run when enable usrmege') + def test_dnf_installroot_usrmerge(self): + rootpath = '/home/root/chroot/test' + #Copy necessary files to avoid errors with not yet installed tools on + #installroot directory. + self.target.run('mkdir -p %s/etc' % rootpath, 1500) + self.target.run('mkdir -p %s/usr/bin %s/usr/sbin' % (rootpath, rootpath), 1500) + self.target.run('ln -sf -r %s/usr/bin %s/bin' % (rootpath, rootpath), 1500) + self.target.run('ln -sf -r %s/usr/sbin %s/sbin' % (rootpath, rootpath), 1500) + self.target.run('mkdir -p %s/dev' % rootpath, 1500) + #Handle different architectures lib dirs + self.target.run('mkdir -p %s/usr/lib' % rootpath, 1500) + self.target.run('mkdir -p %s/usr/libx32' % rootpath, 1500) + self.target.run('mkdir -p %s/usr/lib64' % rootpath, 1500) + self.target.run('cp /lib/libtinfo.so.5 %s/usr/lib' % rootpath, 1500) + self.target.run('cp /libx32/libtinfo.so.5 %s/usr/libx32' % rootpath, 1500) + self.target.run('cp /lib64/libtinfo.so.5 %s/usr/lib64' % rootpath, 1500) + self.target.run('ln -sf -r %s/lib %s/usr/lib' % (rootpath,rootpath), 1500) + self.target.run('ln -sf -r %s/libx32 %s/usr/libx32' % (rootpath,rootpath), 1500) + self.target.run('ln -sf -r %s/lib64 %s/usr/lib64' % (rootpath,rootpath), 1500) + self.target.run('cp -r /etc/rpm %s/etc' % rootpath, 1500) + self.target.run('cp -r /etc/dnf %s/etc' % rootpath, 1500) + self.target.run('cp /bin/sh %s/bin' % rootpath, 1500) + self.target.run('mount -o bind /dev %s/dev/' % rootpath, 1500) + self.dnf_with_repo('install --installroot=%s -v -y --rpmverbosity=debug busybox run-postinsts' % rootpath) + status, output = self.target.run('test -e %s/var/cache/dnf' % rootpath, 1500) + self.assertEqual(0, status, output) + status, output = self.target.run('test -e %s/bin/busybox' % rootpath, 1500) + self.assertEqual(0, status, output) + + @OETestDepends(['dnf.DnfRepoTest.test_dnf_makecache']) def test_dnf_exclude(self): excludepkg = 'curl-dev' self.dnf_with_repo('install -y curl*') diff --git a/poky/meta/lib/oeqa/runtime/cases/oe_syslog.py b/poky/meta/lib/oeqa/runtime/cases/oe_syslog.py index 3a8271a53..f3c2bedba 100644 --- a/poky/meta/lib/oeqa/runtime/cases/oe_syslog.py +++ b/poky/meta/lib/oeqa/runtime/cases/oe_syslog.py @@ -17,7 +17,7 @@ class SyslogTest(OERuntimeTestCase): msg = "Failed to execute %s" % self.tc.target_cmds['ps'] self.assertEqual(status, 0, msg=msg) msg = "No syslog daemon process; %s output:\n%s" % (self.tc.target_cmds['ps'], output) - hasdaemon = "syslogd" in output or "syslog-ng" in output + hasdaemon = "syslogd" in output or "syslog-ng" in output or "svlogd" in output self.assertTrue(hasdaemon, msg=msg) class SyslogTestConfig(OERuntimeTestCase): diff --git a/poky/meta/lib/oeqa/runtime/cases/parselogs.py b/poky/meta/lib/oeqa/runtime/cases/parselogs.py index 19c9c52a0..15343d7ab 100644 --- a/poky/meta/lib/oeqa/runtime/cases/parselogs.py +++ b/poky/meta/lib/oeqa/runtime/cases/parselogs.py @@ -83,6 +83,10 @@ qemux86_common = [ 'amd_nb: Cannot enumerate AMD northbridges', 'uvesafb: 5000 ms task timeout, infinitely waiting', 'tsc: HPET/PMTIMER calibration failed', + "modeset(0): Failed to initialize the DRI2 extension", + "uvesafb: cannot reserve video memory at", + "uvesafb: probe of uvesafb.0 failed with error", + "glamor initialization failed", ] + common_errors ignore_errors = { diff --git a/poky/meta/lib/oeqa/sdkext/cases/devtool.py b/poky/meta/lib/oeqa/sdkext/cases/devtool.py index 5a02add76..8e92bf806 100644 --- a/poky/meta/lib/oeqa/sdkext/cases/devtool.py +++ b/poky/meta/lib/oeqa/sdkext/cases/devtool.py @@ -73,8 +73,8 @@ class DevtoolTest(OESDKExtTestCase): self._run('devtool reset %s' % recipe) def test_devtool_kernelmodule(self): - docfile = 'https://github.com/umlaeute/v4l2loopback.git' - recipe = 'v4l2loopback-driver' + docfile = 'https://git.yoctoproject.org/git/kernel-module-hello-world' + recipe = 'kernel-module-hello-world' self._run('devtool add %s %s' % (recipe, docfile) ) try: self._run('devtool build %s' % recipe) diff --git a/poky/meta/lib/oeqa/selftest/cases/bbtests.py b/poky/meta/lib/oeqa/selftest/cases/bbtests.py index 0693ba8cf..8e59bafae 100644 --- a/poky/meta/lib/oeqa/selftest/cases/bbtests.py +++ b/poky/meta/lib/oeqa/selftest/cases/bbtests.py @@ -119,7 +119,7 @@ class BitbakeTests(OESelftestTestCase): def test_bitbake_g(self): result = bitbake('-g core-image-minimal') - for f in ['pn-buildlist', 'recipe-depends.dot', 'task-depends.dot']: + for f in ['pn-buildlist', 'task-depends.dot']: self.addCleanup(os.remove, f) self.assertTrue('Task dependencies saved to \'task-depends.dot\'' in result.output, msg = "No task dependency \"task-depends.dot\" file was generated for the given task target. bitbake output: %s" % result.output) self.assertTrue('busybox' in ftools.read_file(os.path.join(self.builddir, 'task-depends.dot')), msg = "No \"busybox\" dependency found in task-depends.dot file.") diff --git a/poky/meta/lib/oeqa/selftest/cases/binutils.py b/poky/meta/lib/oeqa/selftest/cases/binutils.py new file mode 100644 index 000000000..9bc752040 --- /dev/null +++ b/poky/meta/lib/oeqa/selftest/cases/binutils.py @@ -0,0 +1,49 @@ +# SPDX-License-Identifier: MIT +import os +import sys +import re +import logging +from oeqa.core.decorator import OETestTag +from oeqa.selftest.case import OESelftestTestCase +from oeqa.utils.commands import bitbake, get_bb_var, get_bb_vars + +def parse_values(content): + for i in content: + for v in ["PASS", "FAIL", "XPASS", "XFAIL", "UNRESOLVED", "UNSUPPORTED", "UNTESTED", "ERROR", "WARNING"]: + if i.startswith(v + ": "): + yield i[len(v) + 2:].strip(), v + break + +@OETestTag("toolchain-user", "toolchain-system") +class BinutilsCrossSelfTest(OESelftestTestCase): + def test_binutils(self): + self.run_binutils("binutils") + + def test_gas(self): + self.run_binutils("gas") + + def test_ld(self): + self.run_binutils("ld") + + def run_binutils(self, suite): + features = [] + features.append('CHECK_TARGETS = "{0}"'.format(suite)) + self.write_config("\n".join(features)) + + recipe = "binutils-cross-testsuite" + bb_vars = get_bb_vars(["B", "TARGET_SYS", "T"], recipe) + builddir, target_sys, tdir = bb_vars["B"], bb_vars["TARGET_SYS"], bb_vars["T"] + + bitbake("{0} -c check".format(recipe)) + + ptestsuite = "binutils-{}".format(suite) if suite != "binutils" else suite + self.extraresults = {"ptestresult.sections" : {ptestsuite : {}}} + + sumspath = os.path.join(builddir, suite, "{0}.sum".format(suite)) + if not os.path.exists(sumspath): + sumspath = os.path.join(builddir, suite, "testsuite", "{0}.sum".format(suite)) + + with open(sumspath, "r") as f: + for test, result in parse_values(f): + self.extraresults["ptestresult.{}.{}".format(ptestsuite, test)] = {"status" : result} + diff --git a/poky/meta/lib/oeqa/selftest/cases/gcc.py b/poky/meta/lib/oeqa/selftest/cases/gcc.py new file mode 100644 index 000000000..2c25b5904 --- /dev/null +++ b/poky/meta/lib/oeqa/selftest/cases/gcc.py @@ -0,0 +1,139 @@ +# SPDX-License-Identifier: MIT +import os +from oeqa.core.decorator import OETestTag +from oeqa.selftest.case import OESelftestTestCase +from oeqa.utils.commands import bitbake, get_bb_var, get_bb_vars, runqemu, Command + +def parse_values(content): + for i in content: + for v in ["PASS", "FAIL", "XPASS", "XFAIL", "UNRESOLVED", "UNSUPPORTED", "UNTESTED", "ERROR", "WARNING"]: + if i.startswith(v + ": "): + yield i[len(v) + 2:].strip(), v + break + +class GccSelfTestBase(OESelftestTestCase): + def check_skip(self, suite): + targets = get_bb_var("RUNTIMETARGET", "gcc-runtime").split() + if suite not in targets: + self.skipTest("Target does not use {0}".format(suite)) + + def run_check(self, *suites, ssh = None): + targets = set() + for s in suites: + if s in ["gcc", "g++"]: + targets.add("check-gcc") + else: + targets.add("check-target-{}".format(s)) + + # configure ssh target + features = [] + features.append('MAKE_CHECK_TARGETS = "{0}"'.format(" ".join(targets))) + if ssh is not None: + features.append('TOOLCHAIN_TEST_TARGET = "ssh"') + features.append('TOOLCHAIN_TEST_HOST = "{0}"'.format(ssh)) + features.append('TOOLCHAIN_TEST_HOST_USER = "root"') + features.append('TOOLCHAIN_TEST_HOST_PORT = "22"') + self.write_config("\n".join(features)) + + recipe = "gcc-runtime" + bitbake("{} -c check".format(recipe)) + + bb_vars = get_bb_vars(["B", "TARGET_SYS"], recipe) + builddir, target_sys = bb_vars["B"], bb_vars["TARGET_SYS"] + + self.extraresults = {"ptestresult.sections" : {}} + for suite in suites: + sumspath = os.path.join(builddir, "gcc", "testsuite", suite, "{0}.sum".format(suite)) + if not os.path.exists(sumspath): # check in target dirs + sumspath = os.path.join(builddir, target_sys, suite, "testsuite", "{0}.sum".format(suite)) + if not os.path.exists(sumspath): # handle libstdc++-v3 -> libstdc++ + sumspath = os.path.join(builddir, target_sys, suite, "testsuite", "{0}.sum".format(suite.split("-")[0])) + + ptestsuite = "gcc-{}".format(suite) if suite != "gcc" else suite + ptestsuite = ptestsuite + "-user" if ssh is None else ptestsuite + self.extraresults["ptestresult.sections"][ptestsuite] = {} + with open(sumspath, "r") as f: + for test, result in parse_values(f): + self.extraresults["ptestresult.{}.{}".format(ptestsuite, test)] = {"status" : result} + + def run_check_emulated(self, *args, **kwargs): + # build core-image-minimal with required packages + default_installed_packages = ["libgcc", "libstdc++", "libatomic", "libgomp"] + features = [] + features.append('IMAGE_FEATURES += "ssh-server-openssh"') + features.append('CORE_IMAGE_EXTRA_INSTALL += "{0}"'.format(" ".join(default_installed_packages))) + self.write_config("\n".join(features)) + bitbake("core-image-minimal") + + # wrap the execution with a qemu instance + with runqemu("core-image-minimal", runqemuparams = "nographic") as qemu: + # validate that SSH is working + status, _ = qemu.run("uname") + self.assertEqual(status, 0) + + return self.run_check(*args, ssh=qemu.ip, **kwargs) + +@OETestTag("toolchain-user") +class GccCrossSelfTest(GccSelfTestBase): + def test_cross_gcc(self): + self.run_check("gcc", "g++") + +@OETestTag("toolchain-user") +class GccLibAtomicSelfTest(GccSelfTestBase): + def test_libatomic(self): + self.run_check("libatomic") + +@OETestTag("toolchain-user") +class GccLibGompSelfTest(GccSelfTestBase): + def test_libgomp(self): + self.run_check("libgomp") + +@OETestTag("toolchain-user") +class GccLibStdCxxSelfTest(GccSelfTestBase): + def test_libstdcxx(self): + self.run_check("libstdc++-v3") + +@OETestTag("toolchain-user") +class GccLibSspSelfTest(GccSelfTestBase): + def test_libssp(self): + self.check_skip("libssp") + self.run_check("libssp") + +@OETestTag("toolchain-user") +class GccLibItmSelfTest(GccSelfTestBase): + def test_libitm(self): + self.check_skip("libitm") + self.run_check("libitm") + +@OETestTag("toolchain-system") +class GccCrossSelfTestSystemEmulated(GccSelfTestBase): + def test_cross_gcc(self): + self.run_check_emulated("gcc", "g++") + +@OETestTag("toolchain-system") +class GccLibAtomicSelfTestSystemEmulated(GccSelfTestBase): + def test_libatomic(self): + self.run_check_emulated("libatomic") + +@OETestTag("toolchain-system") +class GccLibGompSelfTestSystemEmulated(GccSelfTestBase): + def test_libgomp(self): + self.run_check_emulated("libgomp") + +@OETestTag("toolchain-system") +class GccLibStdCxxSelfTestSystemEmulated(GccSelfTestBase): + def test_libstdcxx(self): + self.run_check_emulated("libstdc++-v3") + +@OETestTag("toolchain-system") +class GccLibSspSelfTestSystemEmulated(GccSelfTestBase): + def test_libssp(self): + self.check_skip("libssp") + self.run_check_emulated("libssp") + +@OETestTag("toolchain-system") +class GccLibItmSelfTestSystemEmulated(GccSelfTestBase): + def test_libitm(self): + self.check_skip("libitm") + self.run_check_emulated("libitm") + diff --git a/poky/meta/lib/oeqa/selftest/cases/glibc.py b/poky/meta/lib/oeqa/selftest/cases/glibc.py new file mode 100644 index 000000000..2e42485dd --- /dev/null +++ b/poky/meta/lib/oeqa/selftest/cases/glibc.py @@ -0,0 +1,88 @@ +# SPDX-License-Identifier: MIT +import os +import contextlib +from oeqa.core.decorator import OETestTag +from oeqa.selftest.case import OESelftestTestCase +from oeqa.utils.commands import bitbake, get_bb_var, get_bb_vars, runqemu, Command +from oeqa.utils.nfs import unfs_server + +def parse_values(content): + for i in content: + for v in ["PASS", "FAIL", "XPASS", "XFAIL", "UNRESOLVED", "UNSUPPORTED", "UNTESTED", "ERROR", "WARNING"]: + if i.startswith(v + ": "): + yield i[len(v) + 2:].strip(), v + break + +class GlibcSelfTestBase(OESelftestTestCase): + def run_check(self, ssh = None): + # configure ssh target + features = [] + if ssh is not None: + features.append('TOOLCHAIN_TEST_TARGET = "ssh"') + features.append('TOOLCHAIN_TEST_HOST = "{0}"'.format(ssh)) + features.append('TOOLCHAIN_TEST_HOST_USER = "root"') + features.append('TOOLCHAIN_TEST_HOST_PORT = "22"') + # force single threaded test execution + features.append('EGLIBCPARALLELISM_task-check_pn-glibc-testsuite = "PARALLELMFLAGS="-j1""') + self.write_config("\n".join(features)) + + bitbake("glibc-testsuite -c check") + + builddir = get_bb_var("B", "glibc-testsuite") + + ptestsuite = "glibc-user" if ssh is None else "glibc" + self.extraresults = {"ptestresult.sections" : {ptestsuite : {}}} + with open(os.path.join(builddir, "tests.sum"), "r") as f: + for test, result in parse_values(f): + self.extraresults["ptestresult.{}.{}".format(ptestsuite, test)] = {"status" : result} + + def run_check_emulated(self): + with contextlib.ExitStack() as s: + # use the base work dir, as the nfs mount, since the recipe directory may not exist + tmpdir = get_bb_var("BASE_WORKDIR") + nfsport, mountport = s.enter_context(unfs_server(tmpdir)) + + # build core-image-minimal with required packages + default_installed_packages = [ + "glibc-charmaps", + "libgcc", + "libstdc++", + "libatomic", + "libgomp", + # "python3", + # "python3-pexpect", + "nfs-utils", + ] + features = [] + features.append('IMAGE_FEATURES += "ssh-server-openssh"') + features.append('CORE_IMAGE_EXTRA_INSTALL += "{0}"'.format(" ".join(default_installed_packages))) + self.write_config("\n".join(features)) + bitbake("core-image-minimal") + + # start runqemu + qemu = s.enter_context(runqemu("core-image-minimal", runqemuparams = "nographic")) + + # validate that SSH is working + status, _ = qemu.run("uname") + self.assertEqual(status, 0) + + # setup nfs mount + if qemu.run("mkdir -p \"{0}\"".format(tmpdir))[0] != 0: + raise Exception("Failed to setup NFS mount directory on target") + mountcmd = "mount -o noac,nfsvers=3,port={0},udp,mountport={1} \"{2}:{3}\" \"{3}\"".format(nfsport, mountport, qemu.server_ip, tmpdir) + status, output = qemu.run(mountcmd) + if status != 0: + raise Exception("Failed to setup NFS mount on target ({})".format(repr(output))) + + self.run_check(ssh = qemu.ip) + +@OETestTag("toolchain-user") +class GlibcSelfTest(GlibcSelfTestBase): + def test_glibc(self): + self.run_check() + +@OETestTag("toolchain-system") +class GlibcSelfTestSystemEmulated(GlibcSelfTestBase): + def test_glibc(self): + self.run_check_emulated() + diff --git a/poky/meta/lib/oeqa/selftest/cases/kerneldevelopment.py b/poky/meta/lib/oeqa/selftest/cases/kerneldevelopment.py new file mode 100644 index 000000000..a61876ee6 --- /dev/null +++ b/poky/meta/lib/oeqa/selftest/cases/kerneldevelopment.py @@ -0,0 +1,67 @@ +import os +from oeqa.selftest.case import OESelftestTestCase +from oeqa.utils.commands import runCmd, get_bb_var +from oeqa.utils.git import GitRepo + +class KernelDev(OESelftestTestCase): + + @classmethod + def setUpClass(cls): + super(KernelDev, cls).setUpClass() + # Create the recipe directory structure inside the created layer + cls.layername = 'meta-kerneltest' + runCmd('bitbake-layers create-layer %s' % cls.layername) + runCmd('mkdir -p %s/recipes-kernel/linux/linux-yocto' % cls.layername) + cls.recipes_linuxyocto_dir = os.path.join \ + (cls.builddir, cls.layername, 'recipes-kernel', 'linux', 'linux-yocto') + cls.recipeskernel_dir = os.path.dirname(cls.recipes_linuxyocto_dir) + runCmd('bitbake-layers add-layer %s' % cls.layername) + + @classmethod + def tearDownClass(cls): + runCmd('bitbake-layers remove-layer %s' % cls.layername, ignore_status=True) + runCmd('rm -rf %s' % cls.layername) + super(KernelDev, cls).tearDownClass() + + def setUp(self): + super(KernelDev, self).setUp() + self.set_machine_config('MACHINE = "qemux86-64"\n') + + def test_apply_patches(self): + """ + Summary: Able to apply a single patch to the Linux kernel source + Expected: The README file should exist and the patch changes should be + displayed at the end of the file. + Product: Kernel Development + Author: Yeoh Ee Peng <ee.peng.yeoh@intel.com> + AutomatedBy: Mazliana Mohamad <mazliana.mohamad@intel.com> + """ + runCmd('bitbake virtual/kernel -c patch') + kernel_source = get_bb_var('STAGING_KERNEL_DIR') + readme = os.path.join(kernel_source, 'README') + + # This test step adds modified file 'README' to git and creates a + # patch file '0001-KERNEL_DEV_TEST_CASE.patch' at the same location as file + patch_content = 'This is a test to apply a patch to the kernel' + with open(readme, 'a+') as f: + f.write(patch_content) + repo = GitRepo('%s' % kernel_source, is_topdir=True) + repo.run_cmd('add %s' % readme) + repo.run_cmd(['commit', '-m', 'KERNEL_DEV_TEST_CASE']) + repo.run_cmd(['format-patch', '-1']) + patch_name = '0001-KERNEL_DEV_TEST_CASE.patch' + patchpath = os.path.join(kernel_source, patch_name) + runCmd('mv %s %s' % (patchpath, self.recipes_linuxyocto_dir)) + runCmd('rm %s ' % readme) + self.assertFalse(os.path.exists(readme)) + + recipe_append = os.path.join(self.recipeskernel_dir, 'linux-yocto_%.bbappend') + with open(recipe_append, 'w+') as fh: + fh.write('SRC_URI += "file://%s"\n' % patch_name) + fh.write('FILESEXTRAPATHS_prepend := "${THISDIR}/${PN}:"') + + runCmd('bitbake virtual/kernel -c clean') + runCmd('bitbake virtual/kernel -c patch') + self.assertTrue(os.path.exists(readme)) + result = runCmd('tail -n 1 %s' % readme) + self.assertEqual(result.output, patch_content) diff --git a/poky/meta/lib/oeqa/selftest/cases/meta_ide.py b/poky/meta/lib/oeqa/selftest/cases/meta_ide.py index f47bc7091..03901a2f3 100644 --- a/poky/meta/lib/oeqa/selftest/cases/meta_ide.py +++ b/poky/meta/lib/oeqa/selftest/cases/meta_ide.py @@ -5,9 +5,11 @@ from oeqa.selftest.case import OESelftestTestCase from oeqa.sdk.utils.sdkbuildproject import SDKBuildProject from oeqa.utils.commands import bitbake, get_bb_vars, runCmd +from oeqa.core.decorator import OETestTag import tempfile import shutil +@OETestTag("machine") class MetaIDE(OESelftestTestCase): @classmethod diff --git a/poky/meta/lib/oeqa/selftest/cases/reproducible.py b/poky/meta/lib/oeqa/selftest/cases/reproducible.py index 6dc83d284..eee09d3fb 100644 --- a/poky/meta/lib/oeqa/selftest/cases/reproducible.py +++ b/poky/meta/lib/oeqa/selftest/cases/reproducible.py @@ -8,6 +8,7 @@ from oeqa.utils.commands import runCmd, bitbake, get_bb_var, get_bb_vars import functools import multiprocessing import textwrap +import json import unittest MISSING = 'MISSING' @@ -81,14 +82,12 @@ class ReproducibleTests(OESelftestTestCase): for v in needed_vars: setattr(self, v.lower(), bb_vars[v]) - if not hasattr(self.tc, "extraresults"): - self.tc.extraresults = {} - self.extras = self.tc.extraresults - - self.extras.setdefault('reproducible.rawlogs', {})['log'] = '' + self.extrasresults = {} + self.extrasresults.setdefault('reproducible.rawlogs', {})['log'] = '' + self.extrasresults.setdefault('reproducible', {}).setdefault('files', {}) def append_to_log(self, msg): - self.extras['reproducible.rawlogs']['log'] += msg + self.extrasresults['reproducible.rawlogs']['log'] += msg def compare_packages(self, reference_dir, test_dir, diffutils_sysroot): result = PackageCompareResults() @@ -114,47 +113,70 @@ class ReproducibleTests(OESelftestTestCase): result.sort() return result - @unittest.skip("Reproducible builds do not yet pass") + def write_package_list(self, package_class, name, packages): + self.extrasresults['reproducible']['files'].setdefault(package_class, {})[name] = [ + {'reference': p.reference, 'test': p.test} for p in packages] + def test_reproducible_builds(self): capture_vars = ['DEPLOY_DIR_' + c.upper() for c in self.package_classes] + # Build native utilities + self.write_config('') + bitbake("diffutils-native -c addto_recipe_sysroot") + diffutils_sysroot = get_bb_var("RECIPE_SYSROOT_NATIVE", "diffutils-native") + + # Reproducible builds should not pull from sstate or mirrors, but + # sharing DL_DIR is fine common_config = textwrap.dedent('''\ INHERIT += "reproducible_build" PACKAGE_CLASSES = "%s" + SSTATE_DIR = "${TMPDIR}/sstate" ''') % (' '.join('package_%s' % c for c in self.package_classes)) - # Do an initial build. It's acceptable for this build to use sstate - self.write_config(common_config) - vars_reference = get_bb_vars(capture_vars) + # Perform a build. + reproducibleA_tmp = os.path.join(self.topdir, 'reproducibleA', 'tmp') + if os.path.exists(reproducibleA_tmp): + bb.utils.remove(reproducibleA_tmp, recurse=True) + + self.write_config((textwrap.dedent('''\ + TMPDIR = "%s" + ''') % reproducibleA_tmp) + common_config) + vars_A = get_bb_vars(capture_vars) bitbake(' '.join(self.images)) - # Build native utilities - bitbake("diffutils-native -c addto_recipe_sysroot") - diffutils_sysroot = get_bb_var("RECIPE_SYSROOT_NATIVE", "diffutils-native") + # Perform another build. + reproducibleB_tmp = os.path.join(self.topdir, 'reproducibleB', 'tmp') + if os.path.exists(reproducibleB_tmp): + bb.utils.remove(reproducibleB_tmp, recurse=True) - # Perform another build. This build should *not* share sstate or pull - # from any mirrors, but sharing a DL_DIR is fine - self.write_config(textwrap.dedent('''\ - TMPDIR = "${TOPDIR}/reproducible/tmp" - SSTATE_DIR = "${TMPDIR}/sstate" + self.write_config((textwrap.dedent('''\ SSTATE_MIRROR = "" - ''') + common_config) - vars_test = get_bb_vars(capture_vars) + TMPDIR = "%s" + ''') % reproducibleB_tmp) + common_config) + vars_B = get_bb_vars(capture_vars) bitbake(' '.join(self.images)) + # NOTE: The temp directories from the reproducible build are purposely + # kept after the build so it can be diffed for debugging. + for c in self.package_classes: - package_class = 'package_' + c + with self.subTest(package_class=c): + package_class = 'package_' + c + + deploy_A = vars_A['DEPLOY_DIR_' + c.upper()] + deploy_B = vars_B['DEPLOY_DIR_' + c.upper()] - deploy_reference = vars_reference['DEPLOY_DIR_' + c.upper()] - deploy_test = vars_test['DEPLOY_DIR_' + c.upper()] + result = self.compare_packages(deploy_A, deploy_B, diffutils_sysroot) - result = self.compare_packages(deploy_reference, deploy_test, diffutils_sysroot) + self.logger.info('Reproducibility summary for %s: %s' % (c, result)) - self.logger.info('Reproducibility summary for %s: %s' % (c, result)) + self.append_to_log('\n'.join("%s: %s" % (r.status, r.test) for r in result.total)) - self.append_to_log('\n'.join("%s: %s" % (r.status, r.test) for r in result.total)) + self.write_package_list(package_class, 'missing', result.missing) + self.write_package_list(package_class, 'different', result.different) + self.write_package_list(package_class, 'same', result.same) - if result.missing or result.different: - self.fail("The following %s packages are missing or different: %s" % - (c, ' '.join(r.test for r in (result.missing + result.different)))) + if result.missing or result.different: + self.fail("The following %s packages are missing or different: %s" % + (c, ' '.join(r.test for r in (result.missing + result.different)))) diff --git a/poky/meta/lib/oeqa/selftest/cases/runqemu.py b/poky/meta/lib/oeqa/selftest/cases/runqemu.py index b88ae306c..7e676bcb4 100644 --- a/poky/meta/lib/oeqa/selftest/cases/runqemu.py +++ b/poky/meta/lib/oeqa/selftest/cases/runqemu.py @@ -8,6 +8,7 @@ import re import tempfile import time import oe.types +from oeqa.core.decorator import OETestTag from oeqa.selftest.case import OESelftestTestCase from oeqa.utils.commands import bitbake, runqemu, get_bb_var, runCmd @@ -147,6 +148,7 @@ SYSLINUX_TIMEOUT = "10" # dedicated for MACHINE=qemux86-64 where it test that qemux86-64 will # bootup various filesystem types, including live image(iso and hddimg) # where live image was not supported on all qemu architecture. +@OETestTag("machine") class QemuTest(OESelftestTestCase): @classmethod diff --git a/poky/meta/lib/oeqa/selftest/context.py b/poky/meta/lib/oeqa/selftest/context.py index d279994dd..3126ada71 100644 --- a/poky/meta/lib/oeqa/selftest/context.py +++ b/poky/meta/lib/oeqa/selftest/context.py @@ -77,7 +77,14 @@ class OESelftestTestContextExecutor(OETestContextExecutor): parser.add_argument('--machine', required=False, choices=['random', 'all'], help='Run tests on different machines (random/all).') - + + parser.add_argument('-t', '--select-tags', dest="select_tags", + nargs='*', default=None, + help='Filter all (unhidden) tests to any that match any of the specified tags.') + parser.add_argument('-T', '--exclude-tags', dest="exclude_tags", + nargs='*', default=None, + help='Exclude all (unhidden) tests that match any of the specified tags. (exclude applies before select)') + parser.set_defaults(func=self.run) def _get_available_machines(self): @@ -149,6 +156,18 @@ class OESelftestTestContextExecutor(OETestContextExecutor): copyfile(self.tc_kwargs['init']['config_paths']['bblayers'], self.tc_kwargs['init']['config_paths']['bblayers_backup']) + def tag_filter(tags): + if args.exclude_tags: + if any(tag in args.exclude_tags for tag in tags): + return True + if args.select_tags: + if not tags or not any(tag in args.select_tags for tag in tags): + return True + return False + + if args.select_tags or args.exclude_tags: + self.tc_kwargs['load']['tags_filter'] = tag_filter + self.tc_kwargs['run']['skips'] = args.skips self.tc_kwargs['run']['processes'] = args.processes diff --git a/poky/meta/lib/oeqa/utils/commands.py b/poky/meta/lib/oeqa/utils/commands.py index 7140bc73d..dc1e286da 100644 --- a/poky/meta/lib/oeqa/utils/commands.py +++ b/poky/meta/lib/oeqa/utils/commands.py @@ -172,8 +172,11 @@ def runCmd(command, ignore_status=False, timeout=None, assert_error=True, if native_sysroot: extra_paths = "%s/sbin:%s/usr/sbin:%s/usr/bin" % \ (native_sysroot, native_sysroot, native_sysroot) + extra_libpaths = "%s/lib:%s/usr/lib" % \ + (native_sysroot, native_sysroot) nenv = dict(options.get('env', os.environ)) nenv['PATH'] = extra_paths + ':' + nenv.get('PATH', '') + nenv['LD_LIBRARY_PATH'] = extra_libpaths + ':' + nenv.get('LD_LIBRARY_PATH', '') options['env'] = nenv cmd = Command(command, timeout=timeout, output_log=output_log, **options) @@ -337,7 +340,7 @@ def runqemu(pn, ssh=True, runqemuparams='', image_fstype=None, launch_cmd=None, qemu.deploy() try: qemu.start(params=qemuparams, ssh=ssh, runqemuparams=runqemuparams, launch_cmd=launch_cmd, discard_writes=discard_writes) - except EXception as e: + except Exception as e: msg = str(e) + '\nFailed to start QEMU - see the logs in %s' % logdir if os.path.exists(qemu.qemurunnerlog): with open(qemu.qemurunnerlog, 'r') as f: diff --git a/poky/meta/lib/oeqa/utils/network.py b/poky/meta/lib/oeqa/utils/network.py index 59cbbc4f1..59d01723a 100644 --- a/poky/meta/lib/oeqa/utils/network.py +++ b/poky/meta/lib/oeqa/utils/network.py @@ -4,8 +4,8 @@ import socket -def get_free_port(): - s = socket.socket(socket.AF_INET, socket.SOCK_STREAM) +def get_free_port(udp = False): + s = socket.socket(socket.AF_INET, socket.SOCK_STREAM if not udp else socket.SOCK_DGRAM) s.bind(('', 0)) addr = s.getsockname() s.close() diff --git a/poky/meta/lib/oeqa/utils/nfs.py b/poky/meta/lib/oeqa/utils/nfs.py new file mode 100644 index 000000000..a37686c91 --- /dev/null +++ b/poky/meta/lib/oeqa/utils/nfs.py @@ -0,0 +1,39 @@ +# SPDX-License-Identifier: MIT +import os +import sys +import tempfile +import contextlib +import socket +from oeqa.utils.commands import bitbake, get_bb_var, Command +from oeqa.utils.network import get_free_port + +@contextlib.contextmanager +def unfs_server(directory, logger = None): + unfs_sysroot = get_bb_var("RECIPE_SYSROOT_NATIVE", "unfs3-native") + if not os.path.exists(os.path.join(unfs_sysroot, "usr", "bin", "unfsd")): + # build native tool + bitbake("unfs3-native -c addto_recipe_sysroot") + + exports = None + cmd = None + try: + # create the exports file + with tempfile.NamedTemporaryFile(delete = False) as exports: + exports.write("{0} (rw,no_root_squash,no_all_squash,insecure)\n".format(directory).encode()) + + # find some ports for the server + nfsport, mountport = get_free_port(udp = True), get_free_port(udp = True) + + nenv = dict(os.environ) + nenv['PATH'] = "{0}/sbin:{0}/usr/sbin:{0}/usr/bin:".format(unfs_sysroot) + nenv.get('PATH', '') + cmd = Command(["unfsd", "-d", "-p", "-N", "-e", exports.name, "-n", str(nfsport), "-m", str(mountport)], + bg = True, env = nenv, output_log = logger) + cmd.run() + yield nfsport, mountport + finally: + if cmd is not None: + cmd.stop() + if exports is not None: + # clean up exports file + os.unlink(exports.name) + |