summaryrefslogtreecommitdiff
path: root/poky/meta/lib/oeqa/selftest
diff options
context:
space:
mode:
authorBrad Bishop <bradleyb@fuzziesquirrel.com>2019-09-10 14:20:22 +0300
committerBrad Bishop <bradleyb@fuzziesquirrel.com>2019-09-10 14:21:34 +0300
commit79641f25e882b55af6a647cfc0bf6bcc025661a5 (patch)
tree2b5fb3c59e5b6e85fd6fb8a94b6e15989d974336 /poky/meta/lib/oeqa/selftest
parent39ae1a1dce50fde698688fa5ac8eb737b302f0fd (diff)
downloadopenbmc-79641f25e882b55af6a647cfc0bf6bcc025661a5.tar.xz
poky: subtree update:c17113f1e2..3b87508a9a
Adrian Bunk (18): help2man: Upgrade 1.47.10 -> 1.47.11 glib-networking: Upgrade 2.60.2 -> 2.60.3 flac: Upgrade 1.3.2 -> 1.3.3 subversion: Upgrade 1.12.0 -> 1.12.2 world-broken.inc: Remove meta/recipes.txt: Stop documenting the long removed recipes-lsb4 python: Remove PYTHONLSBOPTS that hasn't been set for some time systemd: Print a buildtime warning about the status with musl linux-dtb.inc: Remove systemd: Update to the latest 242 branch Remove LSB support Remove LSB support bits libxcrypt: Fix the build with -Os systemd: Update to the latest 242 branch Remove leftovers from the eglinfo removal Update to KERNEL_DEVICETREE variable. ref-manual: Remove documentation for the removed bluez5 distro feature ref-manual: Remove documentation for the removed gnome class Alex Kiernan (1): iputils: Set prefix correctly for usrmerge Alexander Kanavin (23): python3-numpy: update to 1.17.0 images: use 512M of RAM in qemu if 'opengl" is in DISTRO_FEATURES boost: update to 1.71.0 beaglebone: use the default of 256M rather than 512M rpm: resolve a host contamination issue for mono packaging qemu: set default RAM to 256M for all machines mesa: disable gallium swrast driver on x86 x32 ltp: move from python 2.x to python 3.x lsb-release: fix upstream version check libffi: fix upstream version check qemu: switch to '-vga std' emulated hardware from vmware/cirrus for x86/mips ca-certificates: use python 3 for builds libpsl: use python 3 during builds pseudo: use python 3 during builds llvm: use python 3 during builds virglrenderer: update 0.7.0 -> 0.8.0 webkitgtk: use Python 3 for builds webkitgtk: update 2.24.3 -> 2.24.4 default-providers.inc: set mesa as the provider for native(sdk) virtual/libgl libepoxy: enable x11 PACKAGECONFIG for native/nativesdk eglinfo: remove the recipe local.conf.sample: add a commented-out option to enable the gtk UI qemu frontend package.bbclass: allow shell-style wildcards in PRIVATE_LIBS Alistair Francis (2): linux-yocto-dev: Mark qemuriscv64 as compatible qemuriscv: Generate a wic rootFS with a larger filesystem Andre McCurdy (11): bitbake.conf: drop FETCHCMD, GITDIR, etc fetcher over-rides tune-arm1136jf-s.inc: add hardfloat support tune-arm1176jz-s.inc: add support for arm1176jz-s CPU specific tuning tune-ep9312.inc: fix TUNEVALID description packagegroup-core-boot: drop duplicate MACHINE_ESSENTIAL_EXTRA_RDEPENDS, etc rootfs-postcommands.bbclass: add inittab tweak to read_only_rootfs_hook() base-files: nsswitch.conf is glibc specific sqlite3: make CFLAGS consistent across native, nativesdk and target cases sqlite3: explicitly set target endian-ness feature-arm-thumb.inc: fix ARM_THUMB_SUFFIX dropbear: don't create invalid symlinks if ssh, scp, etc are disabled Anuj Mittal (1): libjpeg-turbo: upgrade 2.0.2 -> 2.0.3 Armin Kuster (4): bitbake: tests/fetch: Resolve fetch error in bitbake-selftest gcc-9.2: Security fix for CVE-2019-14250 gcc-8.3: Security fix for CVE-2019-14250 gcc-8.3: remove 8.3 from zeus Bruce Ashfield (11): bluez5: fix build against libc-headers 5.2+ kernel-devsrc: remove python2 dependency perf: change dependencies on python to python3 perf: v5.3+ build fixes devsrc: update arm64 scripts/prepare for kernels 5.3+ linux-libc-headers: update to v5.2 headers linux-yocto: introduce 5.2 recipes qemu: bump linux-yocto preferred version to 5.2 linux-yocto/5.2: update qemuriscv64 SRCREV linux-yocto/5.2: config tweaks and kallsyms fix linux-yocto: drop 5.0 recipes Changhyeok Bae (1): iputils: upgrade to s20190709 Changqing Li (8): libedit: fix failed to get history avahi-ui: support usrmerge bootchart2: support usrmerge systemd-bootchart: support usrmerge bootchart2: switch to add patch from change source in do_install mdadm: skip timeout testcase 11spare-migration dnf.py: installroot support usrmerge man-db: support usrmerge Chen Qi (7): bbtests.py: remove recipe-depends.dot commands.py: fix typo bitbake: cooker.py: remove generation of recipe-depends.dot bind: fix build with python3 PACKAGECONFIG enabled python-testtools: rdepend on doctest module python-numpy: rdepend on multiprocessing module python3: make misc package rdepend on pydoc and pickle modules Christophe PRIOUZEAU (1): common-licenses: update BSD-2-CLAUSE license text Dan Callaghan (1): autotools: always include config_site.d files in CONFIG_SITE Daniel Klauer (1): image.bbclass: Fix debug output for rootfs size Denys Zagorui (1): gzip: add nativesdk support Drew Moseley (1): runqemu: Fix typo in help text. He Zhe (4): ltp: cve/meltdown.c: Fix kernel symbol finding ltp: syscalls: rt_sigwaitinfo01: Fix failure for MIPS arches qemu: Add pkg-config handling for libgcrypt qemu: Fix dependency of PACKAGECONFIG libcurl Jacob Kroon (2): btrfs-tools: Add PACKAGECONFIG knob for controlling python support rm_work: Remove redundant 'after' in addtask statement Jason Wessel (2): serial-getty@.service: Allow device to fast fail if it does not exist Revert "kernel.bbclass: adjust a condition checking" Joshua Watt (9): bitbake: bitbake: respect force flag in runall and runonly oeqa: reproducible: Cleanup reproducible build oeqa: reproducible: Use subTest for packages oeqa: Set LD_LIBRARY_PATH when executing native commands oeqa: reproducible: Record packages in test results oeqa: Enable reproducible build test oeqa: reproducible: Do two clean builds oeqa: reproducible: Preserve sstate mirror for first build resulttool: Add reproducible log extraction Kai Kang (4): avahi: launch avahi-daemon after connman cups: redefine CUPS_SERVERBIN texinfo: fix multilib file confliction sysstat: 12.1.3 -> 12.1.6 Kevin Hao (1): meta-yocto-bsp: Bump to the v5.2 kernel Khem Raj (12): go-1.12: update to 1.12.9 minor release powerpc-tunes: Introduce bigendian tune tune-riscv: Drop littleendian and introduce bigendian tune gcc: Move STANDARD_STARTFILE_PREFIX_1 and STANDARD_STARTFILE_PREFIX_2 back to gcc.c gcc: Fix ldbl-128 support for musl libgpg-error: Fix build with gawk 5.x libgpg-error: Fix build with gawk 5.x musl: Update to latest master gcc-9.2: Fix risc-v dynamic linker relocation python3native, pythonnative: export PYTHON_LIBRARY and PYTHON_INCLUDE_DIR oeqa: Recognise svlogd as another logger python3-manifest.json: Fix typo in summary Mark Asselstine (1): sdkext: use simpler kernel module for devtool test Mark Hatle (1): yocto-check-layer: Allow any case for README file detection Martin Hundebøll (1): initscripts: remove warnings on read-only-rootfs (again) Martin Jansa (3): bitbake: utils: Fix movefile() exception handling with python3 bitbake: fetch2: show warning when renaming the archive with bad checksum failed bitbake: bitbake-user-manual: Improve the example for Removal (Override Style Syntax) Mazliana (1): oeqa/kerneldevelopment: Able to apply a single patch to the Linux kernel source Ming Liu (1): weston: uprev to 7.0.0 Nathan Rossi (30): binutils: Fix mips patch which changes default emulation oeqa/utils/nfs: Add unfs_server function to setup a userspace NFS server scripts/lib/resulttool/report.py: Add more result types gcc-common.inc: Process staging fixme with correct target/native sysroot oeqa/core/tests: Skip test_fail_duplicated_module oeqa/core/tests: Fix test_data module tests Clean up remnants of glibc-initial oe-selftest: Implement console 'keepalive' output dejagnu: Add dejagnu for binutils/gcc test suites binutils-cross-testsuite: Create recipe for test suite execution gcc-runtime: Add do_check task for executing gcc test suites glibc-testsuite: Create a recipe to implement glibc test suite gcc-testsuite.inc: Fix ssh.exp, ensure multilib_flags are populated security_flags.inc: Add overrides for glibc-testsuite to match glibc archiver.bbclass: Do not archive srpm's if there are no package tasks oeqa/core: Implement proper extra result collection and serialization oeqa/core: Rework OETestTag and remove unused OETestFilter oeqa/selftest: Add test run filtering based on test tags oeqa/core/runner.py: Fix OETestTag listing oeqa/selftest/binutils: Create selftest case for binutils test suite oeqa/selftest/gcc: Create selftest case for gcc test suite oeqa/selftest/glibc: Create selftest case for glibc test suite oeqa/selftest/cases/glibc.py: Don't install python3 and pexpect oeqa/selftest: For toolchain tests do not fail if a test failed oeqa/selftest: Toolchain tests suffix "-user" for qemu usermode results oeqa/selftest: Use extraresults on self instead of self.tc oeqa/selftest/context.py: Change -t/-T args to be optional oeqa/core/decorator: Fix super class modifying subclass tags oeqa/selftest/cases/gcc.py: Split into classes for parallelism oeqa/selftest/cases/glibc.py: Rework and tag with toolchain-user/system Neil Armstrong (1): mesa: Add support for the panfrost PACKAGECONFIG Oleksandr Kravchuk (8): libsolv: update to 0.7.6 libevdev: update to 1.8.0 ell: update to 0.22 go-dep: update to 0.5.4 libsecret: 0.19.0 python3-pbr: update to 5.4.3 python3-git: update to 3.0.2 libxcrypt: update to 4.4.8 Peiran Hong (1): diffutils: Added perl to support ptest & Skipped one test case Peter Kjellerstedt (2): libffi: Make it build for MIPS o32 devtool: Avoid failure for recipes with S == WORKDIR and no local files Richard Purdie (11): poky-altcfg: Replace poky-lsb with poky-altcfg ca-certificates: Fix reproducibilty and multilib issue gcc-cross: Clean up fixed-includes systemtap: Use latest and greatest git version for 5.2 kernel support gcc-cross: Fix header file corruption problems poky: Use 5.2 kernel by default binutils/glibc-testsuite: inherit nopackages to fix world builds systemtap: Drop patches merged upstream poky-tiny/poky-altcfg: Use busybox-mdev and systemd oeqa/selftest: Markup 'machine' specific test cases oeqa/selftest: Tweak binutils tests tags as toolchain/user and system Robert Yang (3): ccache: 3.7.1 -> 3.7.3 nfs-utils: 2.3.3 -> 2.4.1 build-compare: 2015.02.10 -> 2019.08.14 Ross Burton (11): libedit: remove FETCHCMD append libnl: upgrade to 3.5.0 libnl: don't patch libnl-3.0.pc systemd: explicitly set the path to nologin core-image-sato: don't use 512MB in qemumips patch: add CVE tags to patches pango: 1.44.6 upgrade pango: add PACKAGECONFIG for libthai support harfbuzz: upgrade to 2.6.1 systemd: ensure reproducible builds by clearly exposing the time epoch support ref-system-requirements: add Debian 10 to supported distribution list Scott Rifenbark (5): ref-manual: Updated the "AS" variable ref-manual: Updates to "Building Your Recipe" dev-manual: Updated "Creating a Team Development Environment" poky-end: Added "pylint3" package to Ubuntu essentials. ref-manual: sync list of image types with source Tim Blechmann (1): icecc: blacklist systemtap Trevor Gamblin (2): lighttpd: drop hard-coded lua version in PACKAGECONFIG libevent: add granularity to ptest log Vijai Kumar K (1): lib/oe/gpg_sign.py: Remove unwanted import Wes Lindauer (1): bitbake: bitbake: cooker: Ensure bbappends are found in stable order Yi Zhao (1): libbsd: upgrade 0.9.1 -> 0.10.0 Yuan Chao (1): man-db: upgrade 2.8.6.1 -> 2.8.7 Zang Ruochen (7): libarchive:upgrade 3.3.3 -> 3.4.0 python3-pip:upgrade 19.2.2 -> 19.2.3 logrotate:upgrade 3.15.0 -> 3.15.1 libogg:upgrade 1.3.3 -> 1.3.4 dtc:upgrade 1.5.0 -> 1.5.1 python3-dbus:upgrade 1.2.8 -> 1.2.10 meson:upgrade 0.51.1 -> 0.51.2 Change-Id: I341e8df65e8196114088a2d6d6eb3a33296188b4 Signed-off-by: Brad Bishop <bradleyb@fuzziesquirrel.com>
Diffstat (limited to 'poky/meta/lib/oeqa/selftest')
-rw-r--r--poky/meta/lib/oeqa/selftest/cases/bbtests.py2
-rw-r--r--poky/meta/lib/oeqa/selftest/cases/binutils.py49
-rw-r--r--poky/meta/lib/oeqa/selftest/cases/gcc.py139
-rw-r--r--poky/meta/lib/oeqa/selftest/cases/glibc.py88
-rw-r--r--poky/meta/lib/oeqa/selftest/cases/kerneldevelopment.py67
-rw-r--r--poky/meta/lib/oeqa/selftest/cases/meta_ide.py2
-rw-r--r--poky/meta/lib/oeqa/selftest/cases/reproducible.py80
-rw-r--r--poky/meta/lib/oeqa/selftest/cases/runqemu.py2
-rw-r--r--poky/meta/lib/oeqa/selftest/context.py21
9 files changed, 419 insertions, 31 deletions
diff --git a/poky/meta/lib/oeqa/selftest/cases/bbtests.py b/poky/meta/lib/oeqa/selftest/cases/bbtests.py
index 0693ba8cf8..8e59bafae3 100644
--- a/poky/meta/lib/oeqa/selftest/cases/bbtests.py
+++ b/poky/meta/lib/oeqa/selftest/cases/bbtests.py
@@ -119,7 +119,7 @@ class BitbakeTests(OESelftestTestCase):
def test_bitbake_g(self):
result = bitbake('-g core-image-minimal')
- for f in ['pn-buildlist', 'recipe-depends.dot', 'task-depends.dot']:
+ for f in ['pn-buildlist', 'task-depends.dot']:
self.addCleanup(os.remove, f)
self.assertTrue('Task dependencies saved to \'task-depends.dot\'' in result.output, msg = "No task dependency \"task-depends.dot\" file was generated for the given task target. bitbake output: %s" % result.output)
self.assertTrue('busybox' in ftools.read_file(os.path.join(self.builddir, 'task-depends.dot')), msg = "No \"busybox\" dependency found in task-depends.dot file.")
diff --git a/poky/meta/lib/oeqa/selftest/cases/binutils.py b/poky/meta/lib/oeqa/selftest/cases/binutils.py
new file mode 100644
index 0000000000..9bc752040f
--- /dev/null
+++ b/poky/meta/lib/oeqa/selftest/cases/binutils.py
@@ -0,0 +1,49 @@
+# SPDX-License-Identifier: MIT
+import os
+import sys
+import re
+import logging
+from oeqa.core.decorator import OETestTag
+from oeqa.selftest.case import OESelftestTestCase
+from oeqa.utils.commands import bitbake, get_bb_var, get_bb_vars
+
+def parse_values(content):
+ for i in content:
+ for v in ["PASS", "FAIL", "XPASS", "XFAIL", "UNRESOLVED", "UNSUPPORTED", "UNTESTED", "ERROR", "WARNING"]:
+ if i.startswith(v + ": "):
+ yield i[len(v) + 2:].strip(), v
+ break
+
+@OETestTag("toolchain-user", "toolchain-system")
+class BinutilsCrossSelfTest(OESelftestTestCase):
+ def test_binutils(self):
+ self.run_binutils("binutils")
+
+ def test_gas(self):
+ self.run_binutils("gas")
+
+ def test_ld(self):
+ self.run_binutils("ld")
+
+ def run_binutils(self, suite):
+ features = []
+ features.append('CHECK_TARGETS = "{0}"'.format(suite))
+ self.write_config("\n".join(features))
+
+ recipe = "binutils-cross-testsuite"
+ bb_vars = get_bb_vars(["B", "TARGET_SYS", "T"], recipe)
+ builddir, target_sys, tdir = bb_vars["B"], bb_vars["TARGET_SYS"], bb_vars["T"]
+
+ bitbake("{0} -c check".format(recipe))
+
+ ptestsuite = "binutils-{}".format(suite) if suite != "binutils" else suite
+ self.extraresults = {"ptestresult.sections" : {ptestsuite : {}}}
+
+ sumspath = os.path.join(builddir, suite, "{0}.sum".format(suite))
+ if not os.path.exists(sumspath):
+ sumspath = os.path.join(builddir, suite, "testsuite", "{0}.sum".format(suite))
+
+ with open(sumspath, "r") as f:
+ for test, result in parse_values(f):
+ self.extraresults["ptestresult.{}.{}".format(ptestsuite, test)] = {"status" : result}
+
diff --git a/poky/meta/lib/oeqa/selftest/cases/gcc.py b/poky/meta/lib/oeqa/selftest/cases/gcc.py
new file mode 100644
index 0000000000..2c25b5904c
--- /dev/null
+++ b/poky/meta/lib/oeqa/selftest/cases/gcc.py
@@ -0,0 +1,139 @@
+# SPDX-License-Identifier: MIT
+import os
+from oeqa.core.decorator import OETestTag
+from oeqa.selftest.case import OESelftestTestCase
+from oeqa.utils.commands import bitbake, get_bb_var, get_bb_vars, runqemu, Command
+
+def parse_values(content):
+ for i in content:
+ for v in ["PASS", "FAIL", "XPASS", "XFAIL", "UNRESOLVED", "UNSUPPORTED", "UNTESTED", "ERROR", "WARNING"]:
+ if i.startswith(v + ": "):
+ yield i[len(v) + 2:].strip(), v
+ break
+
+class GccSelfTestBase(OESelftestTestCase):
+ def check_skip(self, suite):
+ targets = get_bb_var("RUNTIMETARGET", "gcc-runtime").split()
+ if suite not in targets:
+ self.skipTest("Target does not use {0}".format(suite))
+
+ def run_check(self, *suites, ssh = None):
+ targets = set()
+ for s in suites:
+ if s in ["gcc", "g++"]:
+ targets.add("check-gcc")
+ else:
+ targets.add("check-target-{}".format(s))
+
+ # configure ssh target
+ features = []
+ features.append('MAKE_CHECK_TARGETS = "{0}"'.format(" ".join(targets)))
+ if ssh is not None:
+ features.append('TOOLCHAIN_TEST_TARGET = "ssh"')
+ features.append('TOOLCHAIN_TEST_HOST = "{0}"'.format(ssh))
+ features.append('TOOLCHAIN_TEST_HOST_USER = "root"')
+ features.append('TOOLCHAIN_TEST_HOST_PORT = "22"')
+ self.write_config("\n".join(features))
+
+ recipe = "gcc-runtime"
+ bitbake("{} -c check".format(recipe))
+
+ bb_vars = get_bb_vars(["B", "TARGET_SYS"], recipe)
+ builddir, target_sys = bb_vars["B"], bb_vars["TARGET_SYS"]
+
+ self.extraresults = {"ptestresult.sections" : {}}
+ for suite in suites:
+ sumspath = os.path.join(builddir, "gcc", "testsuite", suite, "{0}.sum".format(suite))
+ if not os.path.exists(sumspath): # check in target dirs
+ sumspath = os.path.join(builddir, target_sys, suite, "testsuite", "{0}.sum".format(suite))
+ if not os.path.exists(sumspath): # handle libstdc++-v3 -> libstdc++
+ sumspath = os.path.join(builddir, target_sys, suite, "testsuite", "{0}.sum".format(suite.split("-")[0]))
+
+ ptestsuite = "gcc-{}".format(suite) if suite != "gcc" else suite
+ ptestsuite = ptestsuite + "-user" if ssh is None else ptestsuite
+ self.extraresults["ptestresult.sections"][ptestsuite] = {}
+ with open(sumspath, "r") as f:
+ for test, result in parse_values(f):
+ self.extraresults["ptestresult.{}.{}".format(ptestsuite, test)] = {"status" : result}
+
+ def run_check_emulated(self, *args, **kwargs):
+ # build core-image-minimal with required packages
+ default_installed_packages = ["libgcc", "libstdc++", "libatomic", "libgomp"]
+ features = []
+ features.append('IMAGE_FEATURES += "ssh-server-openssh"')
+ features.append('CORE_IMAGE_EXTRA_INSTALL += "{0}"'.format(" ".join(default_installed_packages)))
+ self.write_config("\n".join(features))
+ bitbake("core-image-minimal")
+
+ # wrap the execution with a qemu instance
+ with runqemu("core-image-minimal", runqemuparams = "nographic") as qemu:
+ # validate that SSH is working
+ status, _ = qemu.run("uname")
+ self.assertEqual(status, 0)
+
+ return self.run_check(*args, ssh=qemu.ip, **kwargs)
+
+@OETestTag("toolchain-user")
+class GccCrossSelfTest(GccSelfTestBase):
+ def test_cross_gcc(self):
+ self.run_check("gcc", "g++")
+
+@OETestTag("toolchain-user")
+class GccLibAtomicSelfTest(GccSelfTestBase):
+ def test_libatomic(self):
+ self.run_check("libatomic")
+
+@OETestTag("toolchain-user")
+class GccLibGompSelfTest(GccSelfTestBase):
+ def test_libgomp(self):
+ self.run_check("libgomp")
+
+@OETestTag("toolchain-user")
+class GccLibStdCxxSelfTest(GccSelfTestBase):
+ def test_libstdcxx(self):
+ self.run_check("libstdc++-v3")
+
+@OETestTag("toolchain-user")
+class GccLibSspSelfTest(GccSelfTestBase):
+ def test_libssp(self):
+ self.check_skip("libssp")
+ self.run_check("libssp")
+
+@OETestTag("toolchain-user")
+class GccLibItmSelfTest(GccSelfTestBase):
+ def test_libitm(self):
+ self.check_skip("libitm")
+ self.run_check("libitm")
+
+@OETestTag("toolchain-system")
+class GccCrossSelfTestSystemEmulated(GccSelfTestBase):
+ def test_cross_gcc(self):
+ self.run_check_emulated("gcc", "g++")
+
+@OETestTag("toolchain-system")
+class GccLibAtomicSelfTestSystemEmulated(GccSelfTestBase):
+ def test_libatomic(self):
+ self.run_check_emulated("libatomic")
+
+@OETestTag("toolchain-system")
+class GccLibGompSelfTestSystemEmulated(GccSelfTestBase):
+ def test_libgomp(self):
+ self.run_check_emulated("libgomp")
+
+@OETestTag("toolchain-system")
+class GccLibStdCxxSelfTestSystemEmulated(GccSelfTestBase):
+ def test_libstdcxx(self):
+ self.run_check_emulated("libstdc++-v3")
+
+@OETestTag("toolchain-system")
+class GccLibSspSelfTestSystemEmulated(GccSelfTestBase):
+ def test_libssp(self):
+ self.check_skip("libssp")
+ self.run_check_emulated("libssp")
+
+@OETestTag("toolchain-system")
+class GccLibItmSelfTestSystemEmulated(GccSelfTestBase):
+ def test_libitm(self):
+ self.check_skip("libitm")
+ self.run_check_emulated("libitm")
+
diff --git a/poky/meta/lib/oeqa/selftest/cases/glibc.py b/poky/meta/lib/oeqa/selftest/cases/glibc.py
new file mode 100644
index 0000000000..2e42485dd6
--- /dev/null
+++ b/poky/meta/lib/oeqa/selftest/cases/glibc.py
@@ -0,0 +1,88 @@
+# SPDX-License-Identifier: MIT
+import os
+import contextlib
+from oeqa.core.decorator import OETestTag
+from oeqa.selftest.case import OESelftestTestCase
+from oeqa.utils.commands import bitbake, get_bb_var, get_bb_vars, runqemu, Command
+from oeqa.utils.nfs import unfs_server
+
+def parse_values(content):
+ for i in content:
+ for v in ["PASS", "FAIL", "XPASS", "XFAIL", "UNRESOLVED", "UNSUPPORTED", "UNTESTED", "ERROR", "WARNING"]:
+ if i.startswith(v + ": "):
+ yield i[len(v) + 2:].strip(), v
+ break
+
+class GlibcSelfTestBase(OESelftestTestCase):
+ def run_check(self, ssh = None):
+ # configure ssh target
+ features = []
+ if ssh is not None:
+ features.append('TOOLCHAIN_TEST_TARGET = "ssh"')
+ features.append('TOOLCHAIN_TEST_HOST = "{0}"'.format(ssh))
+ features.append('TOOLCHAIN_TEST_HOST_USER = "root"')
+ features.append('TOOLCHAIN_TEST_HOST_PORT = "22"')
+ # force single threaded test execution
+ features.append('EGLIBCPARALLELISM_task-check_pn-glibc-testsuite = "PARALLELMFLAGS="-j1""')
+ self.write_config("\n".join(features))
+
+ bitbake("glibc-testsuite -c check")
+
+ builddir = get_bb_var("B", "glibc-testsuite")
+
+ ptestsuite = "glibc-user" if ssh is None else "glibc"
+ self.extraresults = {"ptestresult.sections" : {ptestsuite : {}}}
+ with open(os.path.join(builddir, "tests.sum"), "r") as f:
+ for test, result in parse_values(f):
+ self.extraresults["ptestresult.{}.{}".format(ptestsuite, test)] = {"status" : result}
+
+ def run_check_emulated(self):
+ with contextlib.ExitStack() as s:
+ # use the base work dir, as the nfs mount, since the recipe directory may not exist
+ tmpdir = get_bb_var("BASE_WORKDIR")
+ nfsport, mountport = s.enter_context(unfs_server(tmpdir))
+
+ # build core-image-minimal with required packages
+ default_installed_packages = [
+ "glibc-charmaps",
+ "libgcc",
+ "libstdc++",
+ "libatomic",
+ "libgomp",
+ # "python3",
+ # "python3-pexpect",
+ "nfs-utils",
+ ]
+ features = []
+ features.append('IMAGE_FEATURES += "ssh-server-openssh"')
+ features.append('CORE_IMAGE_EXTRA_INSTALL += "{0}"'.format(" ".join(default_installed_packages)))
+ self.write_config("\n".join(features))
+ bitbake("core-image-minimal")
+
+ # start runqemu
+ qemu = s.enter_context(runqemu("core-image-minimal", runqemuparams = "nographic"))
+
+ # validate that SSH is working
+ status, _ = qemu.run("uname")
+ self.assertEqual(status, 0)
+
+ # setup nfs mount
+ if qemu.run("mkdir -p \"{0}\"".format(tmpdir))[0] != 0:
+ raise Exception("Failed to setup NFS mount directory on target")
+ mountcmd = "mount -o noac,nfsvers=3,port={0},udp,mountport={1} \"{2}:{3}\" \"{3}\"".format(nfsport, mountport, qemu.server_ip, tmpdir)
+ status, output = qemu.run(mountcmd)
+ if status != 0:
+ raise Exception("Failed to setup NFS mount on target ({})".format(repr(output)))
+
+ self.run_check(ssh = qemu.ip)
+
+@OETestTag("toolchain-user")
+class GlibcSelfTest(GlibcSelfTestBase):
+ def test_glibc(self):
+ self.run_check()
+
+@OETestTag("toolchain-system")
+class GlibcSelfTestSystemEmulated(GlibcSelfTestBase):
+ def test_glibc(self):
+ self.run_check_emulated()
+
diff --git a/poky/meta/lib/oeqa/selftest/cases/kerneldevelopment.py b/poky/meta/lib/oeqa/selftest/cases/kerneldevelopment.py
new file mode 100644
index 0000000000..a61876ee61
--- /dev/null
+++ b/poky/meta/lib/oeqa/selftest/cases/kerneldevelopment.py
@@ -0,0 +1,67 @@
+import os
+from oeqa.selftest.case import OESelftestTestCase
+from oeqa.utils.commands import runCmd, get_bb_var
+from oeqa.utils.git import GitRepo
+
+class KernelDev(OESelftestTestCase):
+
+ @classmethod
+ def setUpClass(cls):
+ super(KernelDev, cls).setUpClass()
+ # Create the recipe directory structure inside the created layer
+ cls.layername = 'meta-kerneltest'
+ runCmd('bitbake-layers create-layer %s' % cls.layername)
+ runCmd('mkdir -p %s/recipes-kernel/linux/linux-yocto' % cls.layername)
+ cls.recipes_linuxyocto_dir = os.path.join \
+ (cls.builddir, cls.layername, 'recipes-kernel', 'linux', 'linux-yocto')
+ cls.recipeskernel_dir = os.path.dirname(cls.recipes_linuxyocto_dir)
+ runCmd('bitbake-layers add-layer %s' % cls.layername)
+
+ @classmethod
+ def tearDownClass(cls):
+ runCmd('bitbake-layers remove-layer %s' % cls.layername, ignore_status=True)
+ runCmd('rm -rf %s' % cls.layername)
+ super(KernelDev, cls).tearDownClass()
+
+ def setUp(self):
+ super(KernelDev, self).setUp()
+ self.set_machine_config('MACHINE = "qemux86-64"\n')
+
+ def test_apply_patches(self):
+ """
+ Summary: Able to apply a single patch to the Linux kernel source
+ Expected: The README file should exist and the patch changes should be
+ displayed at the end of the file.
+ Product: Kernel Development
+ Author: Yeoh Ee Peng <ee.peng.yeoh@intel.com>
+ AutomatedBy: Mazliana Mohamad <mazliana.mohamad@intel.com>
+ """
+ runCmd('bitbake virtual/kernel -c patch')
+ kernel_source = get_bb_var('STAGING_KERNEL_DIR')
+ readme = os.path.join(kernel_source, 'README')
+
+ # This test step adds modified file 'README' to git and creates a
+ # patch file '0001-KERNEL_DEV_TEST_CASE.patch' at the same location as file
+ patch_content = 'This is a test to apply a patch to the kernel'
+ with open(readme, 'a+') as f:
+ f.write(patch_content)
+ repo = GitRepo('%s' % kernel_source, is_topdir=True)
+ repo.run_cmd('add %s' % readme)
+ repo.run_cmd(['commit', '-m', 'KERNEL_DEV_TEST_CASE'])
+ repo.run_cmd(['format-patch', '-1'])
+ patch_name = '0001-KERNEL_DEV_TEST_CASE.patch'
+ patchpath = os.path.join(kernel_source, patch_name)
+ runCmd('mv %s %s' % (patchpath, self.recipes_linuxyocto_dir))
+ runCmd('rm %s ' % readme)
+ self.assertFalse(os.path.exists(readme))
+
+ recipe_append = os.path.join(self.recipeskernel_dir, 'linux-yocto_%.bbappend')
+ with open(recipe_append, 'w+') as fh:
+ fh.write('SRC_URI += "file://%s"\n' % patch_name)
+ fh.write('FILESEXTRAPATHS_prepend := "${THISDIR}/${PN}:"')
+
+ runCmd('bitbake virtual/kernel -c clean')
+ runCmd('bitbake virtual/kernel -c patch')
+ self.assertTrue(os.path.exists(readme))
+ result = runCmd('tail -n 1 %s' % readme)
+ self.assertEqual(result.output, patch_content)
diff --git a/poky/meta/lib/oeqa/selftest/cases/meta_ide.py b/poky/meta/lib/oeqa/selftest/cases/meta_ide.py
index f47bc70917..03901a2f32 100644
--- a/poky/meta/lib/oeqa/selftest/cases/meta_ide.py
+++ b/poky/meta/lib/oeqa/selftest/cases/meta_ide.py
@@ -5,9 +5,11 @@
from oeqa.selftest.case import OESelftestTestCase
from oeqa.sdk.utils.sdkbuildproject import SDKBuildProject
from oeqa.utils.commands import bitbake, get_bb_vars, runCmd
+from oeqa.core.decorator import OETestTag
import tempfile
import shutil
+@OETestTag("machine")
class MetaIDE(OESelftestTestCase):
@classmethod
diff --git a/poky/meta/lib/oeqa/selftest/cases/reproducible.py b/poky/meta/lib/oeqa/selftest/cases/reproducible.py
index 6dc83d2847..eee09d3fb2 100644
--- a/poky/meta/lib/oeqa/selftest/cases/reproducible.py
+++ b/poky/meta/lib/oeqa/selftest/cases/reproducible.py
@@ -8,6 +8,7 @@ from oeqa.utils.commands import runCmd, bitbake, get_bb_var, get_bb_vars
import functools
import multiprocessing
import textwrap
+import json
import unittest
MISSING = 'MISSING'
@@ -81,14 +82,12 @@ class ReproducibleTests(OESelftestTestCase):
for v in needed_vars:
setattr(self, v.lower(), bb_vars[v])
- if not hasattr(self.tc, "extraresults"):
- self.tc.extraresults = {}
- self.extras = self.tc.extraresults
-
- self.extras.setdefault('reproducible.rawlogs', {})['log'] = ''
+ self.extrasresults = {}
+ self.extrasresults.setdefault('reproducible.rawlogs', {})['log'] = ''
+ self.extrasresults.setdefault('reproducible', {}).setdefault('files', {})
def append_to_log(self, msg):
- self.extras['reproducible.rawlogs']['log'] += msg
+ self.extrasresults['reproducible.rawlogs']['log'] += msg
def compare_packages(self, reference_dir, test_dir, diffutils_sysroot):
result = PackageCompareResults()
@@ -114,47 +113,70 @@ class ReproducibleTests(OESelftestTestCase):
result.sort()
return result
- @unittest.skip("Reproducible builds do not yet pass")
+ def write_package_list(self, package_class, name, packages):
+ self.extrasresults['reproducible']['files'].setdefault(package_class, {})[name] = [
+ {'reference': p.reference, 'test': p.test} for p in packages]
+
def test_reproducible_builds(self):
capture_vars = ['DEPLOY_DIR_' + c.upper() for c in self.package_classes]
+ # Build native utilities
+ self.write_config('')
+ bitbake("diffutils-native -c addto_recipe_sysroot")
+ diffutils_sysroot = get_bb_var("RECIPE_SYSROOT_NATIVE", "diffutils-native")
+
+ # Reproducible builds should not pull from sstate or mirrors, but
+ # sharing DL_DIR is fine
common_config = textwrap.dedent('''\
INHERIT += "reproducible_build"
PACKAGE_CLASSES = "%s"
+ SSTATE_DIR = "${TMPDIR}/sstate"
''') % (' '.join('package_%s' % c for c in self.package_classes))
- # Do an initial build. It's acceptable for this build to use sstate
- self.write_config(common_config)
- vars_reference = get_bb_vars(capture_vars)
+ # Perform a build.
+ reproducibleA_tmp = os.path.join(self.topdir, 'reproducibleA', 'tmp')
+ if os.path.exists(reproducibleA_tmp):
+ bb.utils.remove(reproducibleA_tmp, recurse=True)
+
+ self.write_config((textwrap.dedent('''\
+ TMPDIR = "%s"
+ ''') % reproducibleA_tmp) + common_config)
+ vars_A = get_bb_vars(capture_vars)
bitbake(' '.join(self.images))
- # Build native utilities
- bitbake("diffutils-native -c addto_recipe_sysroot")
- diffutils_sysroot = get_bb_var("RECIPE_SYSROOT_NATIVE", "diffutils-native")
+ # Perform another build.
+ reproducibleB_tmp = os.path.join(self.topdir, 'reproducibleB', 'tmp')
+ if os.path.exists(reproducibleB_tmp):
+ bb.utils.remove(reproducibleB_tmp, recurse=True)
- # Perform another build. This build should *not* share sstate or pull
- # from any mirrors, but sharing a DL_DIR is fine
- self.write_config(textwrap.dedent('''\
- TMPDIR = "${TOPDIR}/reproducible/tmp"
- SSTATE_DIR = "${TMPDIR}/sstate"
+ self.write_config((textwrap.dedent('''\
SSTATE_MIRROR = ""
- ''') + common_config)
- vars_test = get_bb_vars(capture_vars)
+ TMPDIR = "%s"
+ ''') % reproducibleB_tmp) + common_config)
+ vars_B = get_bb_vars(capture_vars)
bitbake(' '.join(self.images))
+ # NOTE: The temp directories from the reproducible build are purposely
+ # kept after the build so it can be diffed for debugging.
+
for c in self.package_classes:
- package_class = 'package_' + c
+ with self.subTest(package_class=c):
+ package_class = 'package_' + c
+
+ deploy_A = vars_A['DEPLOY_DIR_' + c.upper()]
+ deploy_B = vars_B['DEPLOY_DIR_' + c.upper()]
- deploy_reference = vars_reference['DEPLOY_DIR_' + c.upper()]
- deploy_test = vars_test['DEPLOY_DIR_' + c.upper()]
+ result = self.compare_packages(deploy_A, deploy_B, diffutils_sysroot)
- result = self.compare_packages(deploy_reference, deploy_test, diffutils_sysroot)
+ self.logger.info('Reproducibility summary for %s: %s' % (c, result))
- self.logger.info('Reproducibility summary for %s: %s' % (c, result))
+ self.append_to_log('\n'.join("%s: %s" % (r.status, r.test) for r in result.total))
- self.append_to_log('\n'.join("%s: %s" % (r.status, r.test) for r in result.total))
+ self.write_package_list(package_class, 'missing', result.missing)
+ self.write_package_list(package_class, 'different', result.different)
+ self.write_package_list(package_class, 'same', result.same)
- if result.missing or result.different:
- self.fail("The following %s packages are missing or different: %s" %
- (c, ' '.join(r.test for r in (result.missing + result.different))))
+ if result.missing or result.different:
+ self.fail("The following %s packages are missing or different: %s" %
+ (c, ' '.join(r.test for r in (result.missing + result.different))))
diff --git a/poky/meta/lib/oeqa/selftest/cases/runqemu.py b/poky/meta/lib/oeqa/selftest/cases/runqemu.py
index b88ae306c0..7e676bcb41 100644
--- a/poky/meta/lib/oeqa/selftest/cases/runqemu.py
+++ b/poky/meta/lib/oeqa/selftest/cases/runqemu.py
@@ -8,6 +8,7 @@ import re
import tempfile
import time
import oe.types
+from oeqa.core.decorator import OETestTag
from oeqa.selftest.case import OESelftestTestCase
from oeqa.utils.commands import bitbake, runqemu, get_bb_var, runCmd
@@ -147,6 +148,7 @@ SYSLINUX_TIMEOUT = "10"
# dedicated for MACHINE=qemux86-64 where it test that qemux86-64 will
# bootup various filesystem types, including live image(iso and hddimg)
# where live image was not supported on all qemu architecture.
+@OETestTag("machine")
class QemuTest(OESelftestTestCase):
@classmethod
diff --git a/poky/meta/lib/oeqa/selftest/context.py b/poky/meta/lib/oeqa/selftest/context.py
index d279994ddf..3126ada716 100644
--- a/poky/meta/lib/oeqa/selftest/context.py
+++ b/poky/meta/lib/oeqa/selftest/context.py
@@ -77,7 +77,14 @@ class OESelftestTestContextExecutor(OETestContextExecutor):
parser.add_argument('--machine', required=False, choices=['random', 'all'],
help='Run tests on different machines (random/all).')
-
+
+ parser.add_argument('-t', '--select-tags', dest="select_tags",
+ nargs='*', default=None,
+ help='Filter all (unhidden) tests to any that match any of the specified tags.')
+ parser.add_argument('-T', '--exclude-tags', dest="exclude_tags",
+ nargs='*', default=None,
+ help='Exclude all (unhidden) tests that match any of the specified tags. (exclude applies before select)')
+
parser.set_defaults(func=self.run)
def _get_available_machines(self):
@@ -149,6 +156,18 @@ class OESelftestTestContextExecutor(OETestContextExecutor):
copyfile(self.tc_kwargs['init']['config_paths']['bblayers'],
self.tc_kwargs['init']['config_paths']['bblayers_backup'])
+ def tag_filter(tags):
+ if args.exclude_tags:
+ if any(tag in args.exclude_tags for tag in tags):
+ return True
+ if args.select_tags:
+ if not tags or not any(tag in args.select_tags for tag in tags):
+ return True
+ return False
+
+ if args.select_tags or args.exclude_tags:
+ self.tc_kwargs['load']['tags_filter'] = tag_filter
+
self.tc_kwargs['run']['skips'] = args.skips
self.tc_kwargs['run']['processes'] = args.processes