diff options
author | Andrew Geissler <geissonator@yahoo.com> | 2020-06-27 08:28:28 +0300 |
---|---|---|
committer | Andrew Geissler <geissonator@yahoo.com> | 2020-06-27 08:32:13 +0300 |
commit | d25ed3241ddffad58c7a52e45e388e6c48d5123a (patch) | |
tree | b097477c5b9204689d35c06f5761b1767093b338 /poky/documentation/test-manual/test-manual-test-process.xml | |
parent | c87764fefff10735006a31fab72d76c243a3eb40 (diff) | |
download | openbmc-d25ed3241ddffad58c7a52e45e388e6c48d5123a.tar.xz |
poky: subtree update:26ae42ded7..5951cbcabe
Alex Kiernan (1):
recipetool: Fix list concatenation when using edit
Alexander Kanavin (4):
apr-util: make gdbm optional
gobject-introspection: add a patch to fix a build race
icu: merge .inc into main recipe
icu: make filtered data generation optional, serial and off by default
Alexandru N. Onea (3):
bitbake: perforce: add basic progress handler for perforce
bitbake: perforce: add local path handling SRC_URI options
bitbake: bitbake-user-manual: update perforce fetcher docs
Andreas M?ller (1):
meson.bbclass: avoid unexpected operating-system names
Andreas Müller (6):
boost: Add upstream patch to fix build on depending projects
libinput: upgrade 1.15.5 -> 1.15.6
sqlite3: upgrade 3.32.2 -> 3.32.3
desktop-file-utils: upgrade 0.24 -> 0.26
file: upgrade 5.38 -> 5.39
ffmpeg: upgrade 4.2.3 -> 4.3
Andrej Valek (1):
oeqa/runtime/cases/ptest: Make output content path absolute
Andrew Geissler (1):
meson: backport library ordering fix
Armin Kuster (1):
libuv: move from meta-oe to core for bind update
Arthur She (1):
igt-gpu-tools: add new package
Changqing Li (1):
mime.bbclass: fix post install scriptlet error
Chen Qi (1):
systemd-serialgetty: do not use BindsTo
Daniel McGregor (3):
sign_rpm.bbclass: ignore thread count
systemd-conf: Accept MTU from DHCP
buildhistory-collect-srcrevs: sort directories
He Zhe (1):
ltp: Fix copy_file_rang02 for 32-bit arches
Hongxu Jia (1):
libmodulemd: switch branch master -> main
Jacob Kroon (5):
bitbake: lib/bb/utils.py: Do not preserve TERM in the environment
bitbake: bitbake-user-manual: Remove TERM from BB_HASHBASE_WHITELIST example
bitbake.conf: Remove TERM from default BB_HASHBASE_WHITELIST
grub: Remove native version of grub-efi
distro_alias: Remove unused grub-efi distro aliases
Jens Rehsack (1):
u-boot: avoid blind merging all *.cfg
Joe Slater (1):
systemd: fix CVE-2020-13776
Joshua Watt (5):
sstatesig: Account for all dataCaches being passed
bitbake: bitbake: cache: Fix error message with bad multiconfig
wic: Fix error message when reporting invalid offset
classes/archiver: Create patched archive before configuring
bitbake: cache: Bump cache version
Konrad Weihmann (3):
oeqa/runtime: Add OERequirePackage decorator
bitbake: cookerdata: Add BBFILES_DYNAMIC inverse mode
bitbake: bitbake-user-manual: Add BBFILES_DYNAMIC
Mark Morton (2):
New source files and Makefile update for Test Manual
test-manual: Fixed codeblock formatting
Martin Jansa (1):
net-tools: backport a patch from upstream to use the same ifconfig format as debian/ubuntu
Mingli Yu (3):
python3: add the rdepends for python3-misc
python3: add rdepends for python3-idle
python3-dbusmock: add the missing rdepends
Otavio Salvador (2):
systemd: Sync systemd-serialgetty@.service with upstream
mtd-utils: Fix return value of ubiformat
Ovidiu Panait (2):
dbus-test: Remove EXTRA_OECONF_X configs
dbus,dbus-test: Move common parts to dbus.inc
Paul Barker (2):
bitbake: fetch2/gitsm: Mark srcrev as fetched once all submodules are processed
bitbake: fetch2/gitsm: Make need_update() process submodules
Paul Eggleton (5):
graph-tool: switch to argparse
graph-tool: add filter subcommand
dpkg-native: rebase and reinstate fix for "tar: file changed as we read it"
shadow-sysroot: drop unused SRC_URI checksums
devtool: fix typo
Peter Kjellerstedt (1):
relocatable.bbclass: Avoid an exception if an empty pkgconfig dir exist
Pierre-Jean Texier (3):
diffoscope: upgrade 146 -> 147
ell: upgrade 0.31 -> 0.32
curl: upgrade 7.70.0 -> 7.71.0
Rasmus Villemoes (1):
curl: add debug info
Richard Purdie (15):
buildhistory: Add simplistic file move detection
bitbake: bin/bitbake: Update to next series release version
perl: Fix host specific modules problems
sanity.conf: Require bitbake 1.47.0 as the minimum version
patchelf: Upgrade 0.10 -> 0.11
test-manual: Add SPDX license headers
Makefile: Drop obsolete edison/denzil branch conditionals
bitbake: tests/fetch: Switch from git.infradead.org to a YP mirror
pseudo: Fix attr errors due to incorrect library resolution issues
oeqa/selftest/runcmd: Add better debug for thread count mismatch failures
oeqa/utils/command: Improve stdin handling in runCmd
vulkan-headers: Fix upstream branch deletion issue
recipes: Fix Upstream-Status Accepted -> Backport
scripts/install-buildtools: Update to 3.2 M1 buildtools
scripts/install-buildtools: Handle new format checksum files
Robert P. J. Day (1):
python: use official "pypi.org" URLs for HOMEPAGE
Ross Burton (8):
install-buildtools: fail if an error occurs
install-buildtools: remove hardcoded x86-64 architecture
install-buildtools: add option to disable checksum validation
common-licenses: add BSD-2-Clause-Patent
gstreamer1.0-plugins-bad: add support for vdpau
go-binary-native: add binary Go to bootstrap
tcmode-default: use go-binary-native by default
go-native: merge bb/inc and add comment
Ryan Rowe (1):
python3: fix PGO for non-reproducible biniaries
Sakib Sajal (1):
qemu: uprev v4.2.0 -> v5.0.0
Samuli Piippo (2):
cmake: allow chainloading of the toolchain file
perl: use relative paths in the perl wrapper
Steve Sakoman (1):
buildtools-tarball: export OPENSSL_CONF in environment setup
Tanu Kaskinen (1):
pulseaudio: remove unnecessary libltdl copying
Trevor Gamblin (1):
python3-setuptools: patch entrypoints for faster initialization
Tuomas Salokanto (1):
recipetool: create: fix SRCBRANCH not being passed to params
Valentin Longchamp (2):
tools-profile: disable valgrind for powerpc soft-float
valgrind: disable it for powerpc soft-float
Wang Mingyu (5):
powertop: upgrade 2.12 -> 2.13
man-db: upgrade 2.9.2 -> 2.9.3
valgrind: upgrade 3.16.0 -> 3.16.1
man-pages: upgrade 5.06 -> 5.07
harfbuzz: upgrade 2.6.7 -> 2.6.8
Yi Zhao (2):
iptables: fix invalid symbolic link for ip6tables-apply
iptables: split iptables-apply to its own package
Yongxin Liu (1):
linux-firmware: add ice for Intel E800 series driver
Yuki Hoshino (1):
sysvinit-inittab: Add support for tty devices with 10 or more number.
akuster (9):
bind: update to 9.11.19
adt-manual: Add SPDX license headers
bsp-guide: Add SPDX license headers
brief-yoctoprojectsqa: Add SPDX license headers
dev-manual: Add SPDX License headers
kernel-dev: Add SPDX license headers
profile-manual: Add SPDX licence headers
sdk-manual: Add SPDX license headers
toaster-manaul: Add SPDX license headers
haiqing (1):
libpam: Remove option 'obscure' from common-password
hongxu (1):
kmod: add nativesdk support
zangrc (1):
ethtool:upgrade 5.6 -> 5.7
Signed-off-by: Andrew Geissler <geissonator@yahoo.com>
Change-Id: I1190ca17297b1167286cfc06033e8485396c7cce
Diffstat (limited to 'poky/documentation/test-manual/test-manual-test-process.xml')
-rw-r--r-- | poky/documentation/test-manual/test-manual-test-process.xml | 110 |
1 files changed, 110 insertions, 0 deletions
diff --git a/poky/documentation/test-manual/test-manual-test-process.xml b/poky/documentation/test-manual/test-manual-test-process.xml new file mode 100644 index 000000000..6e2157c62 --- /dev/null +++ b/poky/documentation/test-manual/test-manual-test-process.xml @@ -0,0 +1,110 @@ +<!DOCTYPE chapter PUBLIC "-//OASIS//DTD DocBook XML V4.2//EN" +"http://www.oasis-open.org/docbook/xml/4.2/docbookx.dtd" +[<!ENTITY % poky SYSTEM "../poky.ent"> %poky; ] > +<!--SPDX-License-Identifier: CC-BY-2.0-UK--> + +<chapter id='test-manual-test-process'> + +<title>Project Testing and Release Process</title> + <section id='test-daily-devel'> + <title>Day to Day Development</title> + + <para>This section details how the project tests changes, through automation on the + Autobuilder or with the assistance of QA teams, through to making releases.</para> + + <para>The project aims to test changes against our test matrix before those changes are + merged into the master branch. As such, changes are queued up in batches either in the + <filename>master-next</filename> branch in the main trees, or in user trees such as + <filename>ross/mut</filename> in <filename>poky-contrib</filename> (Ross Burton + helps review and test patches and this is his testing tree).</para> + <para>We have two broad categories of test builds, including "full" and "quick". On the + Autobuilder, these can be seen as "a-quick" and "a-full", simply for ease of sorting in + the UI. Use our Autobuilder console view to see where me manage most test-related items, + available at: <link linkend="" + >https://autobuilder.yoctoproject.org/typhoon/#/console</link>.</para> + <para>Builds are triggered manually when the test branches are ready. The builds are + monitored by the SWAT team. For additional information, see <link linkend="" + >https://wiki.yoctoproject.org/wiki/Yocto_Build_Failure_Swat_Team</link>. If + successful, the changes would usually be merged to the <filename>master</filename> + branch. If not successful, someone would respond to the changes on the mailing list + explaining that there was a failure in testing. The choice of quick or full would depend + on the type of changes and the speed with which the result was required.</para> + <para>The Autobuilder does build the <filename>master</filename> branch once daily for + several reasons, in particular, to ensure the current <filename>master</filename> branch + does build, but also to keep <filename>yocto-testresults</filename> (<link linkend="" + >http://git.yoctoproject.org/cgit.cgi/yocto-testresults/</link>), buildhistory + (<link linkend="">http://git.yoctoproject.org/cgit.cgi/poky-buildhistory/</link>), + and our sstate up to date. On the weekend, there is a master-next build instead to + ensure the test results are updated for the less frequently run targets.</para> + <para>Performance builds (buildperf-* targets in the console) are triggered separately every + six hours and automatically push their results to the buildstats repository at: <link + linkend="">http://git.yoctoproject.org/cgit.cgi/yocto-buildstats/</link>. </para> + <para>The 'quick' targets have been selected to be the ones which catch the most failures or + give the most valuable data. We run 'fast' ptests in this case for example but not the + ones which take a long time. The quick target doesn't include *-lsb builds for all + architectures, some world builds and doesn't trigger performance tests or ltp testing. + The full build includes all these things and is slower but more comprehensive.</para> + </section> + + <section id='test-yocto-project-autobuilder-overview'> + <title>Release Builds</title> + + <para>The project typically has two major releases a year with a six month cadence in April + and October. Between these there would be a number of milestone releases (usually four) + with the final one being stablization only along with point releases of our stable + branches.</para> + <para>The build and release process for these project releases is similar to that in <link + linkend="test-daily-devel">Day to Day Development</link>, in that the a-full target + of the Autobuilder is used but in addition the form is configured to generate and + publish artefacts and the milestone number, version, release candidate number and other + information is entered. The box to "generate an email to QA"is also checked.</para> + <para>When the build completes, an email is sent out using the send-qa-email script in the + <filename>yocto-autobuilder-helper</filename> repository to the list of people + configured for that release. Release builds are placed into a directory in <link + linkend="">https://autobuilder.yocto.io/pub/releases</link> on the Autobuilder which + is included in the email. The process from here is more manual and control is + effectively passed to release engineering. The next steps include:<itemizedlist> + <listitem> + <para>QA teams respond to the email saying which tests they plan to run and when + the results will be available.</para> + </listitem> + <listitem> + <para>QA teams run their tests and share their results in the yocto- + testresults-contrib repository, along with a summary of their findings. + </para> + </listitem> + <listitem> + <para>Release engineering prepare the release as per their process. </para> + </listitem> + <listitem> + <para>Test results from the QA teams are included into the release in separate + directories and also uploaded to the yocto-testresults repository alongside + the other test results for the given revision.</para> + </listitem> + <listitem> + <para>The QA report in the final release is regenerated using resulttool to + include the new test results and the test summaries from the teams (as + headers to the generated report).</para> + </listitem> + <listitem> + <para>The release is checked against the release checklist and release readiness + criteria.</para> + </listitem> + <listitem> + <para>A final decision on whether to release is made by the YP TSC who have + final oversight on release readiness.</para> + </listitem> + </itemizedlist></para> + </section> + + + + + + + + +</chapter> +<!-- +vim: expandtab tw=80 ts=4 +--> |