From 00e122a7b3a839f5ce8b819cb1bfe92cf3781eda Mon Sep 17 00:00:00 2001 From: Brad Bishop Date: Sat, 5 Oct 2019 11:10:57 -0400 Subject: poky: subtree update:81f9e815d3..03d4d9d68f MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Adrian Bunk (1): json-c: Don't --enable-rdrand Alessio Igor Bogani (2): wic: Using the right rootfs size during prepare_rootfs rootfs-postcommands: Avoid use of an hard-coded value Alexander Kanavin (1): binutils: drop UPSTREAM_VERSION_UNKNOWN Alexandre Bard (1): systemd: Expose resolv-conf alternative only when resolved is built Andre McCurdy (1): ffmpeg: enable more verbose build logs André Draszik (4): ruby: drop long-merged CVE patches ruby: configure mis-detects isnan/isinf on musl ruby: fix non-IPv6 support packagegroup: fix a comment regarding PACKAGE_ARCH Bruce Ashfield (6): linux-yocto/5.2: update to v5.2.13 linux-yocto/4.19: update to v4.19.72 linux-yocto/5.2: update to v5.2.14 linux-yocto/5.2: update to v5.2.16 linux-yocto/5.2: update to v5.2.17 yocto-bsps: update to v5.2.17 Böszörményi Zoltán via Openembedded-core (1): classes/image-live.bbclass: Don't hardcode cpio.gz Changqing Li (2): devtool.py: change to do clean before remove-layer devtool.py: fix buildclean test Chen Qi (1): systemd: fix NFS regression Dan Tran (1): unzip: Fix CVE-2019-13232 David Reyna (2): bitbake: toaster: issues in import layer when clicking 'add layer' bitbake: toaster: improve warnings when adding dependency to packages Diego Rondini (2): initramfs-framework: fix var name initramfs-framework: support PARTLABEL option Douglas Royds (1): icecc: Don't use icecc when INHIBIT_DEFAULT_DEPS is set He Zhe (1): ltp: Fix hang of cve test cases Heiko Schocher (1): kernel.fitimage.bbclass: remove ramdisk_ctype Jacob Kroon (1): bitbake: tests/data: Test combinations of _append together with override Joe Slater (1): bash-completion: add image feature Jonathan Marler (1): package: Multiple shlib_providers for the same file should error Joshua Watt (8): classes/reproducible_build: Move SDE deploy to another directory oeqa: Test multiconfig parsing bitbake: cookerdata: Add mc conffiles hashes to cache hash bitbake: hashserve: Add missing import bitbake: siggen: Fix attribute error when hashserver fails bitbake: hashserv: Don't daemonize server process local.conf.sample: Add Hash Equivalence classes/reproducible_build: Create SDE destination Khem Raj (7): musl: Fix riscv64 CAS functions qemuriscv: Do not blacklist clang anymore sdk: Install nativesdk locales for all TCLIBC variants strace: Upgrade to 5.3 packagegroups: All groups are not allarch musl: Fix __riscv_mc* containers to match glibc core-image-sato-sdk-ptest: Remove valgrind ptests for riscv Konrad Scherer (1): gen-lockedsig-cache: Replace glob lookup with hash to filename lookup Lei Maohui (1): bluez5: update patch to fix do_patch error when PATCHTOOL = "patch". Li Zhou (1): shadow: use relaxed usernames for all Limeng (1): u-boot: add CVE patches for u-boot Nathan Rossi (2): oeqa/core/utils/concurrencytest.py: Handle exceptions and details oeqa/core/case.py: Encode binary data of log Niclas Svensson (1): devtool: finish: Keep patches ordered when updating bbappend Otavio Salvador (1): mesa: Add freedreno PACKAGECONFIG option Peter Kjellerstedt (3): systemd: Make it build with hwdb disabled devtool: finish: Add suppport for the --no-clean option lib/oe/lsb: Make sure the distro ID is always lowercased Randy MacLeod (1): ffmpeg: update from 4.2 to 4.2.1 Richard Purdie (17): Revert "meta-extsdk: Either an sstate task is a proper task or it isn't" sstatesig: Fix hash equivlanency locked signature issues oeqa/selftest/signing: Fix for hash equivlance server lib/sstatesig: Fix class inheritance problems populate_sdk_ext: Fix for hash equiv bitbake: runqueue: Fix task migration problems bitbake: siggen: Ensure setscenetasks list is available to worker context bitbake: runqueue: Change task migration behaviour for rerunning setscene tasks bitbake: siggen/runqueue: Fix signature mismatch issues bitbake: siggen: Avoid writing misleading sigdata files bitbake: runqueue: Save unihashes more frequently bitbake: runqueue: Small performance optimisation bitbake: siggen: Remove full path from unitaskhashes keys bitbake: tests/runqueue: Fix hashserve shutdown race base: Improve module import error message sanity.conf: Bump minimum bitbake version bitbake: bitbake: Bump verison 1.43.1 -> 1.43.2 Robert Yang (6): cases/bbtests.py: test_bitbake_g(): Check base-files rather than busybox expect: Fix configure error for nativesdk net-tools: Fix installed-vs-shipped for nativesdk expect: Fix buffer overflow error when build in long path apr: Check for libtoolize rather than libtool lttng-ust: Fix for --enable-python-agent Ross Burton (12): oeqa/selftest/reproducible: test ipkgs too distcc: clean up the UI install logic distcc: use --enable-tcp-insecure instead of --make-me-a-botnet distcc: split into client and server packages json-c: clean up recipe json-c: use GitHub for upstream release checking bitbake: fetch2/git: refactor check for git-lfs command bitbake: tests/fetch: add test case for git-lfs handling python3: move runpy to core pango: fix the failing testiter test case opkg: remove redundant systemd inherit lttng-ust: update patch Signed-off-by Trevor Gamblin (5): python3-subunit: ensure runtime dependencies are present python3-pip: ensure pickle is installed lighttpd: remove fam as a PACKAGECONFIG option tiff: fix CVE-2019-14973 opkg: remove pathfinder PACKAGECONFIG option Wang Quanyang (1): kexec-tools: fix arm kexec failure for __NR_kexec_file_load Yi Zhao (1): python: add tk-lib as runtime dependency for python-tkinter Change-Id: I0570125d49f7e4bc3bbf70508cbfd7e10bdbc032 Signed-off-by: Brad Bishop --- poky/bitbake/lib/bb/tests/data.py | 13 +++++ poky/bitbake/lib/bb/tests/fetch.py | 80 +++++++++++++++++++++++++++ poky/bitbake/lib/bb/tests/runqueue.py | 100 ++++------------------------------ 3 files changed, 103 insertions(+), 90 deletions(-) (limited to 'poky/bitbake/lib/bb/tests') diff --git a/poky/bitbake/lib/bb/tests/data.py b/poky/bitbake/lib/bb/tests/data.py index a9b0bdb099..3e49984c93 100644 --- a/poky/bitbake/lib/bb/tests/data.py +++ b/poky/bitbake/lib/bb/tests/data.py @@ -381,6 +381,19 @@ class TestOverrides(unittest.TestCase): self.d.setVar("OVERRIDES", "foo:bar:some_val") self.assertEqual(self.d.getVar("TEST"), " testvalue5") + def test_append_and_override_1(self): + self.d.setVar("TEST_append", "testvalue2") + self.d.setVar("TEST_bar", "testvalue3") + self.assertEqual(self.d.getVar("TEST"), "testvalue3testvalue2") + + def test_append_and_override_2(self): + self.d.setVar("TEST_append_bar", "testvalue2") + self.assertEqual(self.d.getVar("TEST"), "testvaluetestvalue2") + + def test_append_and_override_3(self): + self.d.setVar("TEST_bar_append", "testvalue2") + self.assertEqual(self.d.getVar("TEST"), "testvalue2") + # Test an override with _ in it based on a real world OE issue def test_underscore_override(self): self.d.setVar("TARGET_ARCH", "x86_64") diff --git a/poky/bitbake/lib/bb/tests/fetch.py b/poky/bitbake/lib/bb/tests/fetch.py index 2ee030546a..a0b656b610 100644 --- a/poky/bitbake/lib/bb/tests/fetch.py +++ b/poky/bitbake/lib/bb/tests/fetch.py @@ -1908,3 +1908,83 @@ class GitShallowTest(FetcherTest): dir = os.listdir(self.unpackdir + "/git/") self.assertIn("fstests.doap", dir) + +class GitLfsTest(FetcherTest): + def setUp(self): + FetcherTest.setUp(self) + + self.gitdir = os.path.join(self.tempdir, 'git') + self.srcdir = os.path.join(self.tempdir, 'gitsource') + + self.d.setVar('WORKDIR', self.tempdir) + self.d.setVar('S', self.gitdir) + self.d.delVar('PREMIRRORS') + self.d.delVar('MIRRORS') + + self.d.setVar('SRCREV', '${AUTOREV}') + self.d.setVar('AUTOREV', '${@bb.fetch2.get_autorev(d)}') + + bb.utils.mkdirhier(self.srcdir) + self.git('init', cwd=self.srcdir) + with open(os.path.join(self.srcdir, '.gitattributes'), 'wt') as attrs: + attrs.write('*.mp3 filter=lfs -text') + self.git(['add', '.gitattributes'], cwd=self.srcdir) + self.git(['commit', '-m', "attributes", '.gitattributes'], cwd=self.srcdir) + + def git(self, cmd, cwd=None): + if isinstance(cmd, str): + cmd = 'git ' + cmd + else: + cmd = ['git'] + cmd + if cwd is None: + cwd = self.gitdir + return bb.process.run(cmd, cwd=cwd)[0] + + def fetch(self, uri=None): + uris = self.d.getVar('SRC_URI').split() + uri = uris[0] + d = self.d + + fetcher = bb.fetch2.Fetch(uris, d) + fetcher.download() + ud = fetcher.ud[uri] + return fetcher, ud + + def test_lfs_enabled(self): + import shutil + + uri = 'git://%s;protocol=file;subdir=${S};lfs=1' % self.srcdir + self.d.setVar('SRC_URI', uri) + + fetcher, ud = self.fetch() + self.assertIsNotNone(ud.method._find_git_lfs) + + # If git-lfs can be found, the unpack should be successful + ud.method._find_git_lfs = lambda d: True + shutil.rmtree(self.gitdir, ignore_errors=True) + fetcher.unpack(self.d.getVar('WORKDIR')) + + # If git-lfs cannot be found, the unpack should throw an error + with self.assertRaises(bb.fetch2.FetchError): + ud.method._find_git_lfs = lambda d: False + shutil.rmtree(self.gitdir, ignore_errors=True) + fetcher.unpack(self.d.getVar('WORKDIR')) + + def test_lfs_disabled(self): + import shutil + + uri = 'git://%s;protocol=file;subdir=${S};lfs=0' % self.srcdir + self.d.setVar('SRC_URI', uri) + + fetcher, ud = self.fetch() + self.assertIsNotNone(ud.method._find_git_lfs) + + # If git-lfs can be found, the unpack should be successful + ud.method._find_git_lfs = lambda d: True + shutil.rmtree(self.gitdir, ignore_errors=True) + fetcher.unpack(self.d.getVar('WORKDIR')) + + # If git-lfs cannot be found, the unpack should be successful + ud.method._find_git_lfs = lambda d: False + shutil.rmtree(self.gitdir, ignore_errors=True) + fetcher.unpack(self.d.getVar('WORKDIR')) diff --git a/poky/bitbake/lib/bb/tests/runqueue.py b/poky/bitbake/lib/bb/tests/runqueue.py index cb4d526f13..5e6439156d 100644 --- a/poky/bitbake/lib/bb/tests/runqueue.py +++ b/poky/bitbake/lib/bb/tests/runqueue.py @@ -12,6 +12,7 @@ import os import tempfile import subprocess import sys +import time # # TODO: @@ -257,6 +258,8 @@ class RunQueueTests(unittest.TestCase): 'a1:package_write_ipk_setscene', 'a1:package_qa_setscene'] self.assertEqual(set(tasks), set(expected)) + self.shutdown(tempdir) + @unittest.skipIf(sys.version_info < (3, 5, 0), 'Python 3.5 or later required') def test_hashserv_double(self): with tempfile.TemporaryDirectory(prefix="runqueuetest") as tempdir: @@ -280,6 +283,7 @@ class RunQueueTests(unittest.TestCase): 'a1:package_write_rpm_setscene', 'b1:package_write_ipk_setscene', 'a1:packagedata_setscene'] self.assertEqual(set(tasks), set(expected)) + self.shutdown(tempdir) @unittest.skipIf(sys.version_info < (3, 5, 0), 'Python 3.5 or later required') def test_hashserv_multiple_setscene(self): @@ -307,97 +311,13 @@ class RunQueueTests(unittest.TestCase): 'e1:package_setscene'] self.assertEqual(set(tasks), set(expected)) for i in expected: - if i in ["e1:package_setscene"]: - self.assertEqual(tasks.count(i), 4, "%s not in task list four times" % i) - else: - self.assertEqual(tasks.count(i), 1, "%s not in task list once" % i) + self.assertEqual(tasks.count(i), 1, "%s not in task list once" % i) - @unittest.skipIf(sys.version_info < (3, 5, 0), 'Python 3.5 or later required') - def test_hashserv_partial_match(self): - # e1:do_package matches initial built but not second hash value - with tempfile.TemporaryDirectory(prefix="runqueuetest") as tempdir: - extraenv = { - "BB_HASHSERVE" : "auto", - "BB_SIGNATURE_HANDLER" : "TestEquivHash" - } - cmd = ["bitbake", "a1", "b1"] - setscenetasks = ['package_write_ipk_setscene', 'package_write_rpm_setscene', 'packagedata_setscene', - 'populate_sysroot_setscene', 'package_qa_setscene'] - sstatevalid = "" - tasks = self.run_bitbakecmd(cmd, tempdir, sstatevalid, extraenv=extraenv, cleanup=True) - expected = ['a1:' + x for x in self.alltasks] + ['b1:' + x for x in self.alltasks] - self.assertEqual(set(tasks), set(expected)) - with open(tempdir + "/stamps/a1.do_install.taint", "w") as f: - f.write("d460a29e-903f-4b76-a96b-3bcc22a65994") - with open(tempdir + "/stamps/b1.do_install.taint", "w") as f: - f.write("ed36d46a-2977-458a-b3de-eef885bc1817") - cmd = ["bitbake", "e1"] - sstatevalid = "e1:do_package:685e69a026b2f029483fdefe6a11e1e06641dd2a0f6f86e27b9b550f8f21229d" - tasks = self.run_bitbakecmd(cmd, tempdir, sstatevalid, extraenv=extraenv, cleanup=True) - expected = ['a1:package', 'a1:install', 'b1:package', 'b1:install', 'a1:populate_sysroot', 'b1:populate_sysroot', - 'a1:package_write_ipk_setscene', 'b1:packagedata_setscene', 'b1:package_write_rpm_setscene', - 'a1:package_write_rpm_setscene', 'b1:package_write_ipk_setscene', 'a1:packagedata_setscene', - 'e1:package_setscene'] + ['e1:' + x for x in self.alltasks] - expected.remove('e1:package') - self.assertEqual(set(tasks), set(expected)) + self.shutdown(tempdir) - @unittest.skipIf(sys.version_info < (3, 5, 0), 'Python 3.5 or later required') - def test_hashserv_partial_match2(self): - # e1:do_package + e1:do_populate_sysroot matches initial built but not second hash value - with tempfile.TemporaryDirectory(prefix="runqueuetest") as tempdir: - extraenv = { - "BB_HASHSERVE" : "auto", - "BB_SIGNATURE_HANDLER" : "TestEquivHash" - } - cmd = ["bitbake", "a1", "b1"] - setscenetasks = ['package_write_ipk_setscene', 'package_write_rpm_setscene', 'packagedata_setscene', - 'populate_sysroot_setscene', 'package_qa_setscene'] - sstatevalid = "" - tasks = self.run_bitbakecmd(cmd, tempdir, sstatevalid, extraenv=extraenv, cleanup=True) - expected = ['a1:' + x for x in self.alltasks] + ['b1:' + x for x in self.alltasks] - self.assertEqual(set(tasks), set(expected)) - with open(tempdir + "/stamps/a1.do_install.taint", "w") as f: - f.write("d460a29e-903f-4b76-a96b-3bcc22a65994") - with open(tempdir + "/stamps/b1.do_install.taint", "w") as f: - f.write("ed36d46a-2977-458a-b3de-eef885bc1817") - cmd = ["bitbake", "e1"] - sstatevalid = "e1:do_package:685e69a026b2f029483fdefe6a11e1e06641dd2a0f6f86e27b9b550f8f21229d e1:do_populate_sysroot:ef7dc0e2dd55d0534e75cba50731ff42f949818b6f29a65d72bc05856e56711d" - tasks = self.run_bitbakecmd(cmd, tempdir, sstatevalid, extraenv=extraenv, cleanup=True) - expected = ['a1:package', 'a1:install', 'b1:package', 'b1:install', 'a1:populate_sysroot', 'b1:populate_sysroot', - 'a1:package_write_ipk_setscene', 'b1:packagedata_setscene', 'b1:package_write_rpm_setscene', - 'a1:package_write_rpm_setscene', 'b1:package_write_ipk_setscene', 'a1:packagedata_setscene', - 'e1:package_setscene', 'e1:populate_sysroot_setscene', 'e1:build', 'e1:package_qa', 'e1:package_write_rpm', 'e1:package_write_ipk', 'e1:packagedata'] - self.assertEqual(set(tasks), set(expected)) - - @unittest.skipIf(sys.version_info < (3, 5, 0), 'Python 3.5 or later required') - def test_hashserv_partial_match3(self): - # e1:do_package is valid for a1 but not after b1 - # In former buggy code, this triggered e1:do_fetch, then e1:do_populate_sysroot to run - # with none of the intermediate tasks which is a serious bug - with tempfile.TemporaryDirectory(prefix="runqueuetest") as tempdir: - extraenv = { - "BB_HASHSERVE" : "auto", - "BB_SIGNATURE_HANDLER" : "TestEquivHash" - } - cmd = ["bitbake", "a1", "b1"] - setscenetasks = ['package_write_ipk_setscene', 'package_write_rpm_setscene', 'packagedata_setscene', - 'populate_sysroot_setscene', 'package_qa_setscene'] - sstatevalid = "" - tasks = self.run_bitbakecmd(cmd, tempdir, sstatevalid, extraenv=extraenv, cleanup=True) - expected = ['a1:' + x for x in self.alltasks] + ['b1:' + x for x in self.alltasks] - self.assertEqual(set(tasks), set(expected)) - with open(tempdir + "/stamps/a1.do_install.taint", "w") as f: - f.write("d460a29e-903f-4b76-a96b-3bcc22a65994") - with open(tempdir + "/stamps/b1.do_install.taint", "w") as f: - f.write("ed36d46a-2977-458a-b3de-eef885bc1817") - cmd = ["bitbake", "e1", "-DD"] - sstatevalid = "e1:do_package:af056eae12a733a6a8c4f4da8c6757e588e13565852c94e2aad4d953a3989c13 e1:do_package:a3677703db82b22d28d57c1820a47851dd780104580863f5bd32e66e003a779d" - tasks = self.run_bitbakecmd(cmd, tempdir, sstatevalid, extraenv=extraenv, cleanup=True, slowtasks="e1:fetch b1:install") - expected = ['a1:package', 'a1:install', 'b1:package', 'b1:install', 'a1:populate_sysroot', 'b1:populate_sysroot', - 'a1:package_write_ipk_setscene', 'b1:packagedata_setscene', 'b1:package_write_rpm_setscene', - 'a1:package_write_rpm_setscene', 'b1:package_write_ipk_setscene', 'a1:packagedata_setscene', - 'e1:package_setscene'] + ['e1:' + x for x in self.alltasks] - expected.remove('e1:package') - self.assertEqual(set(tasks), set(expected)) + def shutdown(self, tempdir): + # Wait for the hashserve socket to disappear else we'll see races with the tempdir cleanup + while os.path.exists(tempdir + "/hashserve.sock"): + time.sleep(0.5) -- cgit v1.2.3