summaryrefslogtreecommitdiff
path: root/poky/bitbake/doc/bitbake-user-manual/bitbake-user-manual-fetching.rst
diff options
context:
space:
mode:
authorAndrew Geissler <geissonator@yahoo.com>2020-09-18 22:11:35 +0300
committerAndrew Geissler <geissonator@yahoo.com>2020-10-06 01:10:26 +0300
commitc9f7865a347606a64696048817b0f09d9c3fcd31 (patch)
tree00db80fae3599061617c0cb052a57302620882ec /poky/bitbake/doc/bitbake-user-manual/bitbake-user-manual-fetching.rst
parentd1a90aa35d35426789d8f4061166a6dd8d27a30e (diff)
downloadopenbmc-c9f7865a347606a64696048817b0f09d9c3fcd31.tar.xz
poky: subtree update:c67f57c09e..c6bc20857c
Adrian Freihofer (2): oe-publish-sdk: fix layers init via ssh oe-publish-sdk: add --keep-orig option Alexander Kanavin (68): meta-selftest: correct the virgl test for 5.8 kernels bison: upgrade 3.6.4 -> 3.7.1 util-linux: upgrade 2.35.2 -> 2.36 python3-numpy: upgrade 1.19.0 -> 1.19.1 python3-setuptools: upgrade 49.3.1 -> 49.6.0 rsync: upgrade 3.2.2 -> 3.2.3 util-linux: merge .inc into .bb acpica: upgrade 20200528 -> 20200717 asciidoc: upgrade 9.0.1 -> 9.0.2 cryptodev: upgrade 1.10 -> 1.11 diffoscope: upgrade 153 -> 156 epiphany: upgrade 3.36.3 -> 3.36.4 font-alias: upgrade 1.0.3 -> 1.0.4 gtk+3: upgrade 3.24.21 -> 3.24.22 libcheck: upgrade 0.15.0 -> 0.15.2 libinput: upgrade 1.16.0 -> 1.16.1 libpipeline: upgrade 1.5.2 -> 1.5.3 libx11: upgrade 1.6.9 -> 1.6.11 linux-firmware: upgrade 20200619 -> 20200721 man-pages: upgrade 5.07 -> 5.08 mc: upgrade 4.8.24 -> 4.8.25 mesa: upgrade 20.1.4 -> 20.1.5 piglit: upgrade to latest revision re2c: upgrade 2.0 -> 2.0.2 sysstat: upgrade 12.2.2 -> 12.4.0 vala: upgrade 0.48.7 -> 0.48.9 bootchart2: update 0.14.8 -> 0.14.9 harfbuzz: convert to meson, enable gobject introspection pango: update 1.44.7 -> 1.46.0 boost: update 1.73.0 -> 1.74.0 xev: update 1.2.3 -> 1.2.4 wpebackend-fdo: update 1.6.1 -> 1.7.1 gpgme: update 1.13.1 -> 1.14.0 libpsl: update 0.21.0 -> 0.21.1. gettext: update 0.20.2 -> 0.21 cmake: update 3.17.3 -> 3.18.1 linux-firmware: update 20200721 -> 20200817 meson: update 0.55.0 -> 0.55.1 systemd-boot: bump version to 246.2 json-glib: inherit upstream-version-is-even packagegroup-core-device-devel: remove oeqa/x32lib: rework to use readelf from the host oeqa/multilib: rework to use readelf from the host oeqa/multilib: un-skip the connman test poky.conf: do not install packagegroup-core-device-devel into qemu images glib-2.0: update 2.64.4 -> 2.64.5 cmake: upgrade 3.18.1 -> 3.18.2 libxcrypt: upgrade 4.4.16 -> 4.4.17 debianutils: upgrade 4.11 -> 4.11.1 enchant2: upgrade 2.2.8 -> 2.2.9 harfbuzz: upgrade 2.7.1 -> 2.7.2 libmpc: upgrade 1.1.0 -> 1.2.0 librepo: upgrade 1.12.0 -> 1.12.1 libuv: upgrade 1.38.1 -> 1.39.0 msmtp: upgrade 1.8.11 -> 1.8.12 ninja: upgrade 1.10.0 -> 1.10.1 p11-kit: upgrade 0.23.20 -> 0.23.21 pango: upgrade 1.46.0 -> 1.46.1 re2c: upgrade 2.0.2 -> 2.0.3 resolvconf: upgrade 1.82 -> 1.83 stress-ng: upgrade 0.11.18 -> 0.11.19 gnu-config: update to latest revision nasm: update 2.15.03 -> 2.15.05 libva-utils: fix upstream version check gnupg: update 2.2.21 -> 2.2.22 libx11: update 1.6.11 -> 1.6.12 mesa: update 20.1.5 -> 20.1.6 xserver-xorg: update 1.20.8 -> 1.20.9 Andrey Zhizhikin (1): insane: check for missing update-alternatives inherit Anibal Limon (1): recipes-kernel: linux-firmware add qcom-venus-{5.2,5.4} packages Aníbal Limón (1): recipes-graphics/xorg-xserver: Add patch to fix segfault when probe Armin Kuster (2): bind: update to 9.11.22 ESV core-image-sato: qemumips use 512 mem Bruce Ashfield (30): linux-yocto/5.4: update to v5.4.59 linux-yocto/5.8: update to v5.8.2 yocto-bsp: update to v5.4.56 yocto-bsp: update to v5.4.58 qemu: bump default reference kernel to v5.8 linux-yocto/5.8: fix perf and virtio_scsi warnings linux-yocto-rt/5.8: fix lttng-modules build linux-yocto/5.8: selftests/bpf: Prevent runqslower from racing on building bpftool linux-yocto/5.8: disable CONFIG_NFS_DISABLE_UDP_SUPPORT poky: set preferred version for linux-yocto to be v5.8 poky-tiny: set preferred version to 5.8 poky: add preferred version for linux-yocto-rt linux-yocto/5.8: update to v5.8.3 linux-yocto/5.4: update to v5.4.60 kernel: config cleanups for 5.8+ linux-yocto/5.4: update to v5.4.61 linux-yocto/5.8: update to v5.8.4 linux-yocto/5.8: disable IKHEADERS in default builds kernel-yocto: allow promotion of configuration warnings to errors kernel-yocto: checksum all modifications to available kernel fragments directories lttng-modules/devupstream: bump to latest 2.12 commits linux-yocto-dev: bump to v5.9+ linux-yocto/5.8: update to v5.8.5 kernel-devsrc: account for HOSTCC and HOSTCXX linux-yocto/config: netfilter: Enable nat for ipv4 and ipv6 linux-yocto/5.8: update to v5.8.8 linux-yocto/5.4: update to v5.4.64 linux-yocto/config: configuration warning cleanup linux-yocto/5.8: update to v5.8.9 linux-yocto/5.4: update to v5.4.65 Changhyeok Bae (2): iw: upgrade 5.4 -> 5.8 iputils: upgrade s20190709 -> s20200821 Chris Laplante (12): bitbake: compat.py: remove file since it no longer actually implements anything bitbake: COW: formatting bitbake: COW: migrate test suite into tests/cow cve-update-db-native: add progress handler cve-check/cve-update-db-native: use lockfile to fix usage under multiconfig cve-update-db-native: use context manager for cve_f cve-check: avoid FileNotFoundError if no do_cve_check task has run bitbake: utils: process_profilelog: use context manager bitbake: utils: fix UnboundLocalError when _print_exception raises cve-update-db-native: be less magical about checking whether the cve-check class is enabled cve-update-db-native: move -journal checking into do_fetch cve-update-db-native: remove unused variable Christophe GUIBOUT (1): initramfs-framework: support kernel cmdline with double quotes Denys Dmytriyenko (2): weston: upgrade 8.0.0 -> 9.0.0 cryptodev: bump 1 commit past 1.11 to fix 5.9-rc1+ Diego Sueiro (2): license_image.bbclass: Create symlink to the image license manifest dir license_image.bbclass: Fix symlink to the image license manifest dir creation Douglas Royds (1): tcmode-default: Drop gcc-cross-initial, gcc-crosssdk-initial references Frazer Clews (1): bitbake: lib: fix most undefined code picked up by pylint Geoff Parker (1): systemd-serialgetty: Replace sed quoting using ' with " to allow var expansion Jacob Kroon (1): gcc10: Don't default back to -fcommon Jean-Francois Dagenais (1): bitbake: siggen: clean_basepath: remove recipe full path when virtual:xyz present Jens Rehsack (1): lttng-modules: backport patches from 2.12.x to fix 5.4.64+ and 5.8.9+ builds Joe Slater (1): pseudo: fix renaming to self Jon Mason (4): cortex-m0plus.inc: change file permissions tune-cortexa55.inc: clean-up ARMv8.2a uses tune-cortexa57-cortexa53.inc: add CRC and set march tune-cortexa*: Cleanups Joshua Watt (8): wic: Add 512 Byte alignment to --offset oeqa: runtime_tests: Extra GPG debugging oeqa: sdk: Capture stderr output oeqa: reproducible: Fix test not producing diffs diffoscope: upgrade 156 -> 158 bitbake: bitbake: Add parsing torture test bitbake: cooker: Block SIGINT in worker processes sphinx: dev-manual: Clarify that virtual providers do not apply to runtime dependencies Kai Kang (1): dhcpcd: 9.1.4 -> 9.2.0 Kevin Hao (1): meta-yocto-bsp: Bump to the v5.8 kernel Khairul Rohaizzat Jamaluddin (1): wic/bootimg-efi: IMAGE_EFI_BOOT_FILES variable added to separate bootimg-efi and bootimg-partition Khem Raj (24): gcc-cross-canadian: Install gcc/g++ wrappers for musl uninative: Upgrade to 2.9 packagegroup-core-tools-profile: Disable lttng-modules for riscv64 lttng-modules: Disable on riscv64 kexec-tools: Fix build with -fno-common on ppc lttng-tools: Do not build for riscv64 util-linux: Allow update alternatives for additional apps lttng-tools: lttng-ust works on riscv64 json-glib: Backport a build fix with clang rpcbind: Use update-alternatives for rpcinfo go: Upgrade to 1.15 major release weston-init: Redefine weston service and add socket activation option musl: Upgrade to latest master libucontext: Recognise riscv32 architecture linuxloader.bbclass: Define riscv32 ldso for musl populate_sdk_ext: Do not assume local.conf will always exist weston: plane_add_prop() calls break musl atomic modesetting weston-init: Enable RDP screen share weston-init: Do not use fbdev backend weston-init: Select drm/fbdev backends for qemu machines oeqa/weston: Fix tests to run with systemd core-image-weston: Bump qemu memory to 512M go: Update to 1.15.2 minor release bind: Inherit update-alternatives Mark Hatle (6): package_tar.bbclass: Sync to the other package_* classes kernel.bbclass: Remove do_install[prefunc] no longer needed buildhistory.bbclass: Rework to use read_subpackage_metadata kernel.bbclass: Move away from calling package_get_auto_pr package.bbclass: hash equivalency and pr service bitbake: process.py: Handle SystemExit exception to eliminate backtrace Mark Morton (1): sphinx: test-manual code block, link, and format update Martin Jansa (7): devtool: expand SRC_URI when guessing recipe update mode image-artifact-names: introduce new bbclass and move some variables into it kernel.bbclass: use bash variables like imageType, base_name without {} kernel.bbclass: eliminate (initramfs_)symlink_name variables kernel.bbclass: use camelCase notation for bash variables in do_deploy *-initramfs: don't use .rootfs IMAGE_NAME_SUFFIX bitbake.conf: use ${TCMODE}-${TCLIBC} directory for CACHE Matt Madison (1): image.bbclass: fix REPRODUCIBLE_TIMESTAMP_ROOTFS reference Michael Gloff (2): sysvinit rc: Use PSPLASH_FIFO_DIR for progress fifo sysvinit: Remove ${B} assignment Michael Tretter (1): devtool: deploy-target: Fix size calculation for hard links Ming Liu (2): systemd: split systemd specific udev rules into its own package libubootenv: inherit uboot-config Mingli Yu (3): qemu: always define unknown_lock_type qemu: override DEBUG_BUILD bison: remove the parallel build patch Naveen Saini (1): lib/oe/recipeutils.py: add support for BBFILES_DYNAMIC Nicolas Dechesne (73): linux-libc-headers: kernel headers are installed in STAGING_KERNEL_BUILDDIR bitbake: sphinx: add initial build infrastructure bitbake: sphinx: initial sphinx support bitbake: sphinx: bitbake-user-manual: use builtin sphinx glossary bitbake: sphinx: switch to readthedocs theme bitbake: sphinx: override theme CSS bitbake: sphinx: fixup for links bitbake: sphinx: fix links inside notes bitbake: sphinx: fixes all remaining warnings bitbake: sphinx: Makefile.sphinx: add clean and publish targets bitbake: sphinx: tweak html output a bit bitbake: sphinx: add SPDX headers bitbake: sphinx: index: move the boilerplate at the end of the page bitbake: sphinx: conf: enable extlinks extension bitbake: sphinx: add releases page bitbake: sphinx: bitbake-user-manual: insert additional blank line after title bitbake: sphinx: last manual round of fixes/improvements bitbake: sphinx: update style for important, caution and warnings bitbake: sphinx: remove leading '/' bitbake: sphinx: theme_override: properly set font for verbatim text bitbake: bitbake-user-manual: fix bad links sphinx: add initial build infrastructure sphinx: initial sphinx support sphinx: ref-variables: use builtin sphinx glossary sphinx: overview-manual: add figures sphinx: switch to readthedocs theme sphinx: Add SPDX license headers sphinx: add CSS theme override sphinx: bsp-guide: add figures sphinx: add Yocto project logo sphinx: conf: update copyright sphinx: conf: add substitutions/global variables sphinx: add boilerplate file sphinx: add boilerplate to manuals sphinx: ref-manual: add revision history table sphinx: add a general index sphinx: conf.py: enable sphinx.ext.autosectionlabel sphinx: ref-manual: use builtin glossary for the Terms section sphinx: fix internal links sphinx: ref-manual: fix typo sphinx: fix custom term links sphinx: manual updates for some links sphinx: dev-manual add figures sphinx: kernel-dev: add figures sphinx: profile-manual: add figures sphinx: fix up bold text for informalexample container sphinx: ref-manual: add figures sphinx: sdk-manual: add figures sphinx: test-manual: add figures sphinx: toaster-manual: add figures sphinx: add links for Yocto project website sphinx: fix links when the link text should be displayed sphinx: add links to terms in the BitBake glossary sphinx: add links to section in the Bitbake manual sphinx: setup extlink for docs.yoctoproject.org sphinx: enable intersphinx extension sphinx: insert blank below between title and toc sphinx: fix up terms related to kernel-fitimage sphinx: conf: a few rendering tweaks sphinx: makefile: add publish target sphinx: conf: include CSS/JS files, the proper way sphinx: convert 'what I wish I'd known' sphinx: convert 'transitioning to a custom environment' sphinx: ref-manual: fix heading for oe-init-build-env sphinx: brief-yoctoprojectqs: fix up all remaining rendering issues sphinx: Makefile.sphinx improvements sphinx: convert bsp-guide sphinx: remove leading '/' sphinx: update style for important, caution and warnings sphinx: profile-manual: convert profile-manual sphinx: theme_override: properly set font for verbatim text sphinx: theme_override: add tying-it-together admonition sphinx: conf: exclude adt-manual/*.rst Oleksandr Kravchuk (1): ell: update to 0.33 Ovidiu Panait (1): libxml2: Fix CVE-2020-24977 Peter A. Bigot (2): bluez5: fix builds that require ell support timezone: include leap second data in tzdata-core Peter Bergin (1): systemd: avoid failing if no udev rules provided Pierre-Jean Texier (2): libubootenv: upgrade 0.3 -> 0.3.1 diffoscope: upgrade 158 -> 160 Quentin Schulz (16): sphinx: brief-yoctoprojectqs: remove redundant welcome sphinx: brief-yoctoprojectqs: fix ambiguous note for cyclone5 example sphinx: brief-yoctoprojectqs: add missing boilerplate sphinx: overview-manual: add link to AUH how-to section sphinx: overview-manual: fix bitbake basic explanation sphinx: brief-yoctoprojectqs: add note on branch consistency between layers sphinx: what-i-wish-id-known: update "don't be fooled by doc search results" sphinx: overview-manual: remove highlight in bold section sphinx: replace special quotes with single and double quotes sphinx: fix incorrect indentations sphinx: brief-yoctoprojectqs: put other distros note after Ubuntu-specific packages sphinx: fix a few typos or missing/too many words sphinx: "highlight" some variables, tasks or files sphinx: fix or add missing links and remove mention of Eclipse workflow ref-manual: examples: hello-autotools: upgrade to 2.10 ref-manual: examples: libxpm: add relative path to .inc Rahul Kumar (1): systemd-serialgetty: Fix sed expression quoting Rasmus Villemoes (1): kernel.bbclass: run do_symlink_kernsrc before do_patch Richard Purdie (74): nativesdk-sdk-provides-dummy: Add /bin/sh bitbake: fetch2/wget: Remove buffering parameter bitbake: cooker: Ensure parse_quit thread is closed down bitbake: cooker: Explictly shut down the sync thread bitbake: fetch2: Drop cups.org from wget status checks bitbake: build/msg: Cleanup verbose option handling bitbake: cooker/cookerdata/main: Improve loglevel handling bitbake: cookerdata: Ensure UI options are updated to the server bitbake: cooker/cookerdata: Ensure UI event log is updated from commandline bitbake: cooker: Defer configuration init to after UI connection bitbake: server/process: Move the socket code to server process only bitbake: main/server/process: Drop configuration object passing bitbake: cooker: Ensure BB_ORIGENV is updated by changes to configuration.env bitbake: server/process: Log extra threads at exit bitbake: server/process: Add bitbake-server and exec() a new server process bitbake: runqueue: Don't use sys.argv bitbake: cooker: Ensure cooker's enviroment is updated on updateConfig connman-gnome/matchbox-desktop: Remove file:// globbing selftest/recipetool: Drop globbing SRC_URI test, no longer supported local.conf.sample: Document memory resident bitbake bitbake: fetch2: Drop globbing supprt in file:// SRC_URIs bitbake: server/process: Use sys.executable for bitbake-server bitbake: process: Avoid bb.utils.timeout bitbake: utils: Drop broken timeout function bitbake: server/process: Fix typo in code causing tracebacks oeqa/selftest: Apply patch to fix cpio build with -fno-common runqemu: Show an error for conflicting graphics options lttng: Move platform logic to dedicated inc file patchelf: upgrade 0.11 -> 0.12 build-appliance/packagegroup-core-base-utils: Replace dhcp-client/dhcp-server with dhcpcd/kea selftest/prservice: Improve test failure message iputils: Adapt ${PN}-tftpd package dependency to PACKAGECONFIG bitbake: process/knotty: Improve early exception handling bitbake: cooker/cookerdata: Use BBHandledException, not sys.exit() bitbake: cookerdata: Fix exception raise statements bitbake: process: Avoid printing binary strings for leftover processes bitbake: server/process: Ensure logging is flushed bitbake: server/process: Don't show tracebacks if the lockfile is removed bitbake: cooker: Ensure parser replacement calls parser final_cleanup bitbake: cooker: Assign a name to the sync thread to aid debugging bitbake: server/process: Ensure we don't keep looping if some other server is started bitbake: server/process: Prefix the log data with pid/time information bitbake: server/process: Note when commands complete in logs bitbake: cooker: Ensure parser is cleaned up runqemu: Add a hook to allow it to renice bitbake: cooker: Avoid parser deadlocks bitbake: cooker: Ensure parser worker signal handlers are default selftest/signing: Ensure build path relocation is safe oeqa/concurrencytest: Improve builddir path manipulations bitbake: cooker/command: Fix disconnection handling bitbake: tinfoil: Ensure sockets don't leak even when exceptions occur bitbake: tests/fetch: Move away from problematic freedesktop.org urls bitbake: sphinx: Enhance the sphinx experience/nagivation with: bitbake: sphinx: theme_override: Use bold for emphasis text Revert "qemu: always define unknown_lock_type" Revert "core-image-sato: qemumips use 512 mem" sphinx: Organize top level docs sphinx: releases.rst: Add index/links to docs for previous releases sphinx: boilerplate.rst: Drop versions notes as we have better navigation now sphinx: boilerplate.rst: Sphinx puts the copyright elsewhere sphinx: history: Move revision history to its own section sphinx: manuals: Move boilerplate after toctree sphinx: Add support for multiple docs version sphinx: index.rst: Fix links sphinx: ref-system-requirements: Improve formatting of the notes sections, merging them sphinx: ref-manual links fixes and many other cleanups to import sphinx: dev-manual: Various URL, code block and other fixes to imported data sphinx: sdk-manual: Various URL, code block and other fixes to imported data sphinx: kernel-dev: Various URL, code block and other fixes to imported data sphinx: theme_override: Use bold for emphasis text sphinx: ref-tasks: Add populate_sdk_ext task definition sphinx: ref-manual/migration: Split each release into its own file sphinx: overview-manual: Various URL, code block and other fixes to imported data build-appliance-image: Update to master head revision Robert Yang (3): bitbake: cooker.py: Save prioritized BBFILES to BBFILES_PRIORITIZED bitbake: utils.py: get_file_layer(): Exit the loop when file is matched bitbake: utils.py: get_file_layer(): Improve performance Ross Burton (25): package.bbclass: explode the RPROVIDES so we don't think the versions are provides elfutils: silence a new QA warning insane: improve gnu-hash-style warning gdk-pixbuf: add tests PACKAGECONFIG debianutils: change SRC_URI to use snapshot.debian.org insane: only load real files as ELF autoconf: consolidate SRC_URI autoconf: consolidate DEPENDS kea: no need to depend on kea-native kea: don't use PACKAGECONFIG inappropriately kea: bump to 1.7.10 help2man: rewrite recipe local.conf.sample.extended: remove help2man reference curl: add vendors to CVE_PRODUCT to exclude false positives harfbuzz: update patch status harfbuzz: fix a build race around hb-version.h cmake: whitelist CVE-2016-10642 ncurses: remove config.cache qemu: fix CVE-2020-14364 cve-update-db-native: remove unused import cve-update-db-native: add more logging when fetching cve-update-db-native: use fetch task alsa-plugins: improve .la removal sato-screenshot: improve .la removal buildhistory-diff: use BUILDDIR to know where buildhistory is Saul Wold (1): gnupg: uprev 2.2.22 -> 2.2.23 Stacy Gaikovaia (2): bison: uprev from 3.7.1 to 3.7.2 valgrind: fix memcheck vgtests remove fullpath-after flags Steve Sakoman (1): xinput-calibrator: change SRC_URI to branch with libinput support Sumit Garg (1): insane: fix gnu-hash-style check TeohJayShen (1): oeqa/runtime: add test for matchbox-terminal Tim Orling (1): sphinx: toaster-manual: fix vars, links, code blocks Vijai Kumar K (2): image_types_wic: Add ASSUME_PROVIDED to WICVARS wic: misc: Add /bin to the list of searchpaths Yanfei Xu (1): kernel-yocto: only replace leading -I in include paths Yi Zhao (1): glib-networking: add ptest Zhixiong Chi (1): gnutls: CVE-2020-24659 akuster (8): log4cplus: move meta-oe pkg to core kea: Move from meta-networking maintainers.inc: Add me as kea & log4plus maintainer. dhcpcd: Move from meta-network as OE-Core needs a client maintainers.inc: Add me as dhcpcd maintainer dhcp: remove from core bind: Add 9.16.x bind: 9.11 remove hongxu (1): sysstat: fix installed-vs-shipped QA Issue in systemd zangrc (4): libcap:upgrade 2.42 -> 2.43 libcap-ng:upgrade 0.7.10 -> 0.7.11 libgpg-error:upgrade 1.38 -> 1.39 at-spi2-core:upgrade 2.36.0 -> 2.36.1 Signed-off-by: Andrew Geissler <geissonator@yahoo.com> Change-Id: I5542f5eea751a2641342e945725fd687cd74bebe
Diffstat (limited to 'poky/bitbake/doc/bitbake-user-manual/bitbake-user-manual-fetching.rst')
-rw-r--r--poky/bitbake/doc/bitbake-user-manual/bitbake-user-manual-fetching.rst652
1 files changed, 652 insertions, 0 deletions
diff --git a/poky/bitbake/doc/bitbake-user-manual/bitbake-user-manual-fetching.rst b/poky/bitbake/doc/bitbake-user-manual/bitbake-user-manual-fetching.rst
new file mode 100644
index 000000000..f62ddffe8
--- /dev/null
+++ b/poky/bitbake/doc/bitbake-user-manual/bitbake-user-manual-fetching.rst
@@ -0,0 +1,652 @@
+.. SPDX-License-Identifier: CC-BY-2.5
+
+=====================
+File Download Support
+=====================
+
+|
+
+BitBake's fetch module is a standalone piece of library code that deals
+with the intricacies of downloading source code and files from remote
+systems. Fetching source code is one of the cornerstones of building
+software. As such, this module forms an important part of BitBake.
+
+The current fetch module is called "fetch2" and refers to the fact that
+it is the second major version of the API. The original version is
+obsolete and has been removed from the codebase. Thus, in all cases,
+"fetch" refers to "fetch2" in this manual.
+
+The Download (Fetch)
+====================
+
+BitBake takes several steps when fetching source code or files. The
+fetcher codebase deals with two distinct processes in order: obtaining
+the files from somewhere (cached or otherwise) and then unpacking those
+files into a specific location and perhaps in a specific way. Getting
+and unpacking the files is often optionally followed by patching.
+Patching, however, is not covered by this module.
+
+The code to execute the first part of this process, a fetch, looks
+something like the following: ::
+
+ src_uri = (d.getVar('SRC_URI') or "").split()
+ fetcher = bb.fetch2.Fetch(src_uri, d)
+ fetcher.download()
+
+This code sets up an instance of the fetch class. The instance uses a
+space-separated list of URLs from the :term:`SRC_URI`
+variable and then calls the ``download`` method to download the files.
+
+The instantiation of the fetch class is usually followed by: ::
+
+ rootdir = l.getVar('WORKDIR')
+ fetcher.unpack(rootdir)
+
+This code unpacks the downloaded files to the specified by ``WORKDIR``.
+
+.. note::
+
+ For convenience, the naming in these examples matches the variables
+ used by OpenEmbedded. If you want to see the above code in action,
+ examine the OpenEmbedded class file ``base.bbclass``
+ .
+
+The ``SRC_URI`` and ``WORKDIR`` variables are not hardcoded into the
+fetcher, since those fetcher methods can be (and are) called with
+different variable names. In OpenEmbedded for example, the shared state
+(sstate) code uses the fetch module to fetch the sstate files.
+
+When the ``download()`` method is called, BitBake tries to resolve the
+URLs by looking for source files in a specific search order:
+
+- *Pre-mirror Sites:* BitBake first uses pre-mirrors to try and find
+ source files. These locations are defined using the
+ :term:`PREMIRRORS` variable.
+
+- *Source URI:* If pre-mirrors fail, BitBake uses the original URL (e.g
+ from ``SRC_URI``).
+
+- *Mirror Sites:* If fetch failures occur, BitBake next uses mirror
+ locations as defined by the :term:`MIRRORS` variable.
+
+For each URL passed to the fetcher, the fetcher calls the submodule that
+handles that particular URL type. This behavior can be the source of
+some confusion when you are providing URLs for the ``SRC_URI`` variable.
+Consider the following two URLs: ::
+
+ http://git.yoctoproject.org/git/poky;protocol=git
+ git://git.yoctoproject.org/git/poky;protocol=http
+
+In the former case, the URL is passed to the ``wget`` fetcher, which does not
+understand "git". Therefore, the latter case is the correct form since the Git
+fetcher does know how to use HTTP as a transport.
+
+Here are some examples that show commonly used mirror definitions: ::
+
+ PREMIRRORS ?= "\
+ bzr://.*/.\* http://somemirror.org/sources/ \\n \
+ cvs://.*/.\* http://somemirror.org/sources/ \\n \
+ git://.*/.\* http://somemirror.org/sources/ \\n \
+ hg://.*/.\* http://somemirror.org/sources/ \\n \
+ osc://.*/.\* http://somemirror.org/sources/ \\n \
+ p4://.*/.\* http://somemirror.org/sources/ \\n \
+ svn://.*/.\* http://somemirror.org/sources/ \\n"
+
+ MIRRORS =+ "\
+ ftp://.*/.\* http://somemirror.org/sources/ \\n \
+ http://.*/.\* http://somemirror.org/sources/ \\n \
+ https://.*/.\* http://somemirror.org/sources/ \\n"
+
+It is useful to note that BitBake
+supports cross-URLs. It is possible to mirror a Git repository on an
+HTTP server as a tarball. This is what the ``git://`` mapping in the
+previous example does.
+
+Since network accesses are slow, BitBake maintains a cache of files
+downloaded from the network. Any source files that are not local (i.e.
+downloaded from the Internet) are placed into the download directory,
+which is specified by the :term:`DL_DIR` variable.
+
+File integrity is of key importance for reproducing builds. For
+non-local archive downloads, the fetcher code can verify SHA-256 and MD5
+checksums to ensure the archives have been downloaded correctly. You can
+specify these checksums by using the ``SRC_URI`` variable with the
+appropriate varflags as follows: ::
+
+ SRC_URI[md5sum] = "value"
+ SRC_URI[sha256sum] = "value"
+
+You can also specify the checksums as
+parameters on the ``SRC_URI`` as shown below: ::
+
+ SRC_URI = "http://example.com/foobar.tar.bz2;md5sum=4a8e0f237e961fd7785d19d07fdb994d"
+
+If multiple URIs exist, you can specify the checksums either directly as
+in the previous example, or you can name the URLs. The following syntax
+shows how you name the URIs: ::
+
+ SRC_URI = "http://example.com/foobar.tar.bz2;name=foo"
+ SRC_URI[foo.md5sum] = 4a8e0f237e961fd7785d19d07fdb994d
+
+After a file has been downloaded and
+has had its checksum checked, a ".done" stamp is placed in ``DL_DIR``.
+BitBake uses this stamp during subsequent builds to avoid downloading or
+comparing a checksum for the file again.
+
+.. note::
+
+ It is assumed that local storage is safe from data corruption. If
+ this were not the case, there would be bigger issues to worry about.
+
+If :term:`BB_STRICT_CHECKSUM` is set, any
+download without a checksum triggers an error message. The
+:term:`BB_NO_NETWORK` variable can be used to
+make any attempted network access a fatal error, which is useful for
+checking that mirrors are complete as well as other things.
+
+.. _bb-the-unpack:
+
+The Unpack
+==========
+
+The unpack process usually immediately follows the download. For all
+URLs except Git URLs, BitBake uses the common ``unpack`` method.
+
+A number of parameters exist that you can specify within the URL to
+govern the behavior of the unpack stage:
+
+- *unpack:* Controls whether the URL components are unpacked. If set to
+ "1", which is the default, the components are unpacked. If set to
+ "0", the unpack stage leaves the file alone. This parameter is useful
+ when you want an archive to be copied in and not be unpacked.
+
+- *dos:* Applies to ``.zip`` and ``.jar`` files and specifies whether
+ to use DOS line ending conversion on text files.
+
+- *basepath:* Instructs the unpack stage to strip the specified
+ directories from the source path when unpacking.
+
+- *subdir:* Unpacks the specific URL to the specified subdirectory
+ within the root directory.
+
+The unpack call automatically decompresses and extracts files with ".Z",
+".z", ".gz", ".xz", ".zip", ".jar", ".ipk", ".rpm". ".srpm", ".deb" and
+".bz2" extensions as well as various combinations of tarball extensions.
+
+As mentioned, the Git fetcher has its own unpack method that is
+optimized to work with Git trees. Basically, this method works by
+cloning the tree into the final directory. The process is completed
+using references so that there is only one central copy of the Git
+metadata needed.
+
+.. _bb-fetchers:
+
+Fetchers
+========
+
+As mentioned earlier, the URL prefix determines which fetcher submodule
+BitBake uses. Each submodule can support different URL parameters, which
+are described in the following sections.
+
+.. _local-file-fetcher:
+
+Local file fetcher (``file://``)
+--------------------------------
+
+This submodule handles URLs that begin with ``file://``. The filename
+you specify within the URL can be either an absolute or relative path to
+a file. If the filename is relative, the contents of the
+:term:`FILESPATH` variable is used in the same way
+``PATH`` is used to find executables. If the file cannot be found, it is
+assumed that it is available in :term:`DL_DIR` by the
+time the ``download()`` method is called.
+
+If you specify a directory, the entire directory is unpacked.
+
+Here are a couple of example URLs, the first relative and the second
+absolute: ::
+
+ SRC_URI = "file://relativefile.patch"
+ SRC_URI = "file:///Users/ich/very_important_software"
+
+.. _http-ftp-fetcher:
+
+HTTP/FTP wget fetcher (``http://``, ``ftp://``, ``https://``)
+-------------------------------------------------------------
+
+This fetcher obtains files from web and FTP servers. Internally, the
+fetcher uses the wget utility.
+
+The executable and parameters used are specified by the
+``FETCHCMD_wget`` variable, which defaults to sensible values. The
+fetcher supports a parameter "downloadfilename" that allows the name of
+the downloaded file to be specified. Specifying the name of the
+downloaded file is useful for avoiding collisions in
+:term:`DL_DIR` when dealing with multiple files that
+have the same name.
+
+Some example URLs are as follows: ::
+
+ SRC_URI = "http://oe.handhelds.org/not_there.aac"
+ SRC_URI = "ftp://oe.handhelds.org/not_there_as_well.aac"
+ SRC_URI = "ftp://you@oe.handhelds.org/home/you/secret.plan"
+
+.. note::
+
+ Because URL parameters are delimited by semi-colons, this can
+ introduce ambiguity when parsing URLs that also contain semi-colons,
+ for example:
+ ::
+
+ SRC_URI = "http://abc123.org/git/?p=gcc/gcc.git;a=snapshot;h=a5dd47"
+
+
+ Such URLs should should be modified by replacing semi-colons with '&'
+ characters:
+ ::
+
+ SRC_URI = "http://abc123.org/git/?p=gcc/gcc.git&a=snapshot&h=a5dd47"
+
+
+ In most cases this should work. Treating semi-colons and '&' in
+ queries identically is recommended by the World Wide Web Consortium
+ (W3C). Note that due to the nature of the URL, you may have to
+ specify the name of the downloaded file as well:
+ ::
+
+ SRC_URI = "http://abc123.org/git/?p=gcc/gcc.git&a=snapshot&h=a5dd47;downloadfilename=myfile.bz2"
+
+
+.. _cvs-fetcher:
+
+CVS fetcher (``(cvs://``)
+-------------------------
+
+This submodule handles checking out files from the CVS version control
+system. You can configure it using a number of different variables:
+
+- :term:`FETCHCMD_cvs <FETCHCMD>`: The name of the executable to use when running
+ the ``cvs`` command. This name is usually "cvs".
+
+- :term:`SRCDATE`: The date to use when fetching the CVS source code. A
+ special value of "now" causes the checkout to be updated on every
+ build.
+
+- :term:`CVSDIR`: Specifies where a temporary
+ checkout is saved. The location is often ``DL_DIR/cvs``.
+
+- CVS_PROXY_HOST: The name to use as a "proxy=" parameter to the
+ ``cvs`` command.
+
+- CVS_PROXY_PORT: The port number to use as a "proxyport="
+ parameter to the ``cvs`` command.
+
+As well as the standard username and password URL syntax, you can also
+configure the fetcher with various URL parameters:
+
+The supported parameters are as follows:
+
+- *"method":* The protocol over which to communicate with the CVS
+ server. By default, this protocol is "pserver". If "method" is set to
+ "ext", BitBake examines the "rsh" parameter and sets ``CVS_RSH``. You
+ can use "dir" for local directories.
+
+- *"module":* Specifies the module to check out. You must supply this
+ parameter.
+
+- *"tag":* Describes which CVS TAG should be used for the checkout. By
+ default, the TAG is empty.
+
+- *"date":* Specifies a date. If no "date" is specified, the
+ :term:`SRCDATE` of the configuration is used to
+ checkout a specific date. The special value of "now" causes the
+ checkout to be updated on every build.
+
+- *"localdir":* Used to rename the module. Effectively, you are
+ renaming the output directory to which the module is unpacked. You
+ are forcing the module into a special directory relative to
+ :term:`CVSDIR`.
+
+- *"rsh":* Used in conjunction with the "method" parameter.
+
+- *"scmdata":* Causes the CVS metadata to be maintained in the tarball
+ the fetcher creates when set to "keep". The tarball is expanded into
+ the work directory. By default, the CVS metadata is removed.
+
+- *"fullpath":* Controls whether the resulting checkout is at the
+ module level, which is the default, or is at deeper paths.
+
+- *"norecurse":* Causes the fetcher to only checkout the specified
+ directory with no recurse into any subdirectories.
+
+- *"port":* The port to which the CVS server connects.
+
+Some example URLs are as follows: ::
+
+ SRC_URI = "cvs://CVSROOT;module=mymodule;tag=some-version;method=ext"
+ SRC_URI = "cvs://CVSROOT;module=mymodule;date=20060126;localdir=usethat"
+
+.. _svn-fetcher:
+
+Subversion (SVN) Fetcher (``svn://``)
+-------------------------------------
+
+This fetcher submodule fetches code from the Subversion source control
+system. The executable used is specified by ``FETCHCMD_svn``, which
+defaults to "svn". The fetcher's temporary working directory is set by
+:term:`SVNDIR`, which is usually ``DL_DIR/svn``.
+
+The supported parameters are as follows:
+
+- *"module":* The name of the svn module to checkout. You must provide
+ this parameter. You can think of this parameter as the top-level
+ directory of the repository data you want.
+
+- *"path_spec":* A specific directory in which to checkout the
+ specified svn module.
+
+- *"protocol":* The protocol to use, which defaults to "svn". If
+ "protocol" is set to "svn+ssh", the "ssh" parameter is also used.
+
+- *"rev":* The revision of the source code to checkout.
+
+- *"scmdata":* Causes the “.svn” directories to be available during
+ compile-time when set to "keep". By default, these directories are
+ removed.
+
+- *"ssh":* An optional parameter used when "protocol" is set to
+ "svn+ssh". You can use this parameter to specify the ssh program used
+ by svn.
+
+- *"transportuser":* When required, sets the username for the
+ transport. By default, this parameter is empty. The transport
+ username is different than the username used in the main URL, which
+ is passed to the subversion command.
+
+Following are three examples using svn: ::
+
+ SRC_URI = "svn://myrepos/proj1;module=vip;protocol=http;rev=667"
+ SRC_URI = "svn://myrepos/proj1;module=opie;protocol=svn+ssh"
+ SRC_URI = "svn://myrepos/proj1;module=trunk;protocol=http;path_spec=${MY_DIR}/proj1"
+
+.. _git-fetcher:
+
+Git Fetcher (``git://``)
+------------------------
+
+This fetcher submodule fetches code from the Git source control system.
+The fetcher works by creating a bare clone of the remote into
+:term:`GITDIR`, which is usually ``DL_DIR/git2``. This
+bare clone is then cloned into the work directory during the unpack
+stage when a specific tree is checked out. This is done using alternates
+and by reference to minimize the amount of duplicate data on the disk
+and make the unpack process fast. The executable used can be set with
+``FETCHCMD_git``.
+
+This fetcher supports the following parameters:
+
+- *"protocol":* The protocol used to fetch the files. The default is
+ "git" when a hostname is set. If a hostname is not set, the Git
+ protocol is "file". You can also use "http", "https", "ssh" and
+ "rsync".
+
+- *"nocheckout":* Tells the fetcher to not checkout source code when
+ unpacking when set to "1". Set this option for the URL where there is
+ a custom routine to checkout code. The default is "0".
+
+- *"rebaseable":* Indicates that the upstream Git repository can be
+ rebased. You should set this parameter to "1" if revisions can become
+ detached from branches. In this case, the source mirror tarball is
+ done per revision, which has a loss of efficiency. Rebasing the
+ upstream Git repository could cause the current revision to disappear
+ from the upstream repository. This option reminds the fetcher to
+ preserve the local cache carefully for future use. The default value
+ for this parameter is "0".
+
+- *"nobranch":* Tells the fetcher to not check the SHA validation for
+ the branch when set to "1". The default is "0". Set this option for
+ the recipe that refers to the commit that is valid for a tag instead
+ of the branch.
+
+- *"bareclone":* Tells the fetcher to clone a bare clone into the
+ destination directory without checking out a working tree. Only the
+ raw Git metadata is provided. This parameter implies the "nocheckout"
+ parameter as well.
+
+- *"branch":* The branch(es) of the Git tree to clone. If unset, this
+ is assumed to be "master". The number of branch parameters much match
+ the number of name parameters.
+
+- *"rev":* The revision to use for the checkout. The default is
+ "master".
+
+- *"tag":* Specifies a tag to use for the checkout. To correctly
+ resolve tags, BitBake must access the network. For that reason, tags
+ are often not used. As far as Git is concerned, the "tag" parameter
+ behaves effectively the same as the "rev" parameter.
+
+- *"subpath":* Limits the checkout to a specific subpath of the tree.
+ By default, the whole tree is checked out.
+
+- *"destsuffix":* The name of the path in which to place the checkout.
+ By default, the path is ``git/``.
+
+- *"usehead":* Enables local ``git://`` URLs to use the current branch
+ HEAD as the revision for use with ``AUTOREV``. The "usehead"
+ parameter implies no branch and only works when the transfer protocol
+ is ``file://``.
+
+Here are some example URLs: ::
+
+ SRC_URI = "git://git.oe.handhelds.org/git/vip.git;tag=version-1"
+ SRC_URI = "git://git.oe.handhelds.org/git/vip.git;protocol=http"
+
+.. _gitsm-fetcher:
+
+Git Submodule Fetcher (``gitsm://``)
+------------------------------------
+
+This fetcher submodule inherits from the :ref:`Git
+fetcher<bitbake-user-manual/bitbake-user-manual-fetching:git fetcher
+(\`\`git://\`\`)>` and extends that fetcher's behavior by fetching a
+repository's submodules. :term:`SRC_URI` is passed to the Git fetcher as
+described in the :ref:`bitbake-user-manual/bitbake-user-manual-fetching:git
+fetcher (\`\`git://\`\`)` section.
+
+.. note::
+
+ You must clean a recipe when switching between '``git://``' and
+ '``gitsm://``' URLs.
+
+ The Git Submodules fetcher is not a complete fetcher implementation.
+ The fetcher has known issues where it does not use the normal source
+ mirroring infrastructure properly. Further, the submodule sources it
+ fetches are not visible to the licensing and source archiving
+ infrastructures.
+
+.. _clearcase-fetcher:
+
+ClearCase Fetcher (``ccrc://``)
+-------------------------------
+
+This fetcher submodule fetches code from a
+`ClearCase <http://en.wikipedia.org/wiki/Rational_ClearCase>`__
+repository.
+
+To use this fetcher, make sure your recipe has proper
+:term:`SRC_URI`, :term:`SRCREV`, and
+:term:`PV` settings. Here is an example: ::
+
+ SRC_URI = "ccrc://cc.example.org/ccrc;vob=/example_vob;module=/example_module"
+ SRCREV = "EXAMPLE_CLEARCASE_TAG"
+ PV = "${@d.getVar("SRCREV", False).replace("/", "+")}"
+
+The fetcher uses the ``rcleartool`` or
+``cleartool`` remote client, depending on which one is available.
+
+Following are options for the ``SRC_URI`` statement:
+
+- *vob*: The name, which must include the prepending "/" character,
+ of the ClearCase VOB. This option is required.
+
+- *module*: The module, which must include the prepending "/"
+ character, in the selected VOB.
+
+ .. note::
+
+ The module and vob options are combined to create the load rule in the
+ view config spec. As an example, consider the vob and module values from
+ the SRC_URI statement at the start of this section. Combining those values
+ results in the following: ::
+
+ load /example_vob/example_module
+
+- *proto*: The protocol, which can be either ``http`` or ``https``.
+
+By default, the fetcher creates a configuration specification. If you
+want this specification written to an area other than the default, use
+the ``CCASE_CUSTOM_CONFIG_SPEC`` variable in your recipe to define where
+the specification is written.
+
+.. note::
+
+ the SRCREV loses its functionality if you specify this variable. However,
+ SRCREV is still used to label the archive after a fetch even though it does
+ not define what is fetched.
+
+Here are a couple of other behaviors worth mentioning:
+
+- When using ``cleartool``, the login of ``cleartool`` is handled by
+ the system. The login require no special steps.
+
+- In order to use ``rcleartool`` with authenticated users, an
+ "rcleartool login" is necessary before using the fetcher.
+
+.. _perforce-fetcher:
+
+Perforce Fetcher (``p4://``)
+----------------------------
+
+This fetcher submodule fetches code from the
+`Perforce <https://www.perforce.com/>`__ source control system. The
+executable used is specified by ``FETCHCMD_p4``, which defaults to "p4".
+The fetcher's temporary working directory is set by
+:term:`P4DIR`, which defaults to "DL_DIR/p4".
+The fetcher does not make use of a perforce client, instead it
+relies on ``p4 files`` to retrieve a list of
+files and ``p4 print`` to transfer the content
+of those files locally.
+
+To use this fetcher, make sure your recipe has proper
+:term:`SRC_URI`, :term:`SRCREV`, and
+:term:`PV` values. The p4 executable is able to use the
+config file defined by your system's ``P4CONFIG`` environment variable
+in order to define the Perforce server URL and port, username, and
+password if you do not wish to keep those values in a recipe itself. If
+you choose not to use ``P4CONFIG``, or to explicitly set variables that
+``P4CONFIG`` can contain, you can specify the ``P4PORT`` value, which is
+the server's URL and port number, and you can specify a username and
+password directly in your recipe within ``SRC_URI``.
+
+Here is an example that relies on ``P4CONFIG`` to specify the server URL
+and port, username, and password, and fetches the Head Revision: ::
+
+ SRC_URI = "p4://example-depot/main/source/..."
+ SRCREV = "${AUTOREV}"
+ PV = "p4-${SRCPV}"
+ S = "${WORKDIR}/p4"
+
+Here is an example that specifies the server URL and port, username, and
+password, and fetches a Revision based on a Label: ::
+
+ P4PORT = "tcp:p4server.example.net:1666"
+ SRC_URI = "p4://user:passwd@example-depot/main/source/..."
+ SRCREV = "release-1.0"
+ PV = "p4-${SRCPV}"
+ S = "${WORKDIR}/p4"
+
+.. note::
+
+ You should always set S to "${WORKDIR}/p4" in your recipe.
+
+By default, the fetcher strips the depot location from the local file paths. In
+the above example, the content of ``example-depot/main/source/`` will be placed
+in ``${WORKDIR}/p4``. For situations where preserving parts of the remote depot
+paths locally is desirable, the fetcher supports two parameters:
+
+- *"module":*
+ The top-level depot location or directory to fetch. The value of this
+ parameter can also point to a single file within the depot, in which case
+ the local file path will include the module path.
+- *"remotepath":*
+ When used with the value "``keep``", the fetcher will mirror the full depot
+ paths locally for the specified location, even in combination with the
+ ``module`` parameter.
+
+Here is an example use of the the ``module`` parameter: ::
+
+ SRC_URI = "p4://user:passwd@example-depot/main;module=source/..."
+
+In this case, the content of the top-level directory ``source/`` will be fetched
+to ``${P4DIR}``, including the directory itself. The top-level directory will
+be accesible at ``${P4DIR}/source/``.
+
+Here is an example use of the the ``remotepath`` parameter: ::
+
+ SRC_URI = "p4://user:passwd@example-depot/main;module=source/...;remotepath=keep"
+
+In this case, the content of the top-level directory ``source/`` will be fetched
+to ``${P4DIR}``, but the complete depot paths will be mirrored locally. The
+top-level directory will be accessible at
+``${P4DIR}/example-depot/main/source/``.
+
+.. _repo-fetcher:
+
+Repo Fetcher (``repo://``)
+--------------------------
+
+This fetcher submodule fetches code from ``google-repo`` source control
+system. The fetcher works by initiating and syncing sources of the
+repository into :term:`REPODIR`, which is usually
+``${DL_DIR}/repo``.
+
+This fetcher supports the following parameters:
+
+- *"protocol":* Protocol to fetch the repository manifest (default:
+ git).
+
+- *"branch":* Branch or tag of repository to get (default: master).
+
+- *"manifest":* Name of the manifest file (default: ``default.xml``).
+
+Here are some example URLs: ::
+
+ SRC_URI = "repo://REPOROOT;protocol=git;branch=some_branch;manifest=my_manifest.xml"
+ SRC_URI = "repo://REPOROOT;protocol=file;branch=some_branch;manifest=my_manifest.xml"
+
+Other Fetchers
+--------------
+
+Fetch submodules also exist for the following:
+
+- Bazaar (``bzr://``)
+
+- Mercurial (``hg://``)
+
+- npm (``npm://``)
+
+- OSC (``osc://``)
+
+- Secure FTP (``sftp://``)
+
+- Secure Shell (``ssh://``)
+
+- Trees using Git Annex (``gitannex://``)
+
+No documentation currently exists for these lesser used fetcher
+submodules. However, you might find the code helpful and readable.
+
+Auto Revisions
+==============
+
+We need to document ``AUTOREV`` and ``SRCREV_FORMAT`` here.