diff options
Diffstat (limited to 'poky/bitbake/doc/bitbake-user-manual/bitbake-user-manual-fetching.xml')
-rw-r--r-- | poky/bitbake/doc/bitbake-user-manual/bitbake-user-manual-fetching.xml | 857 |
1 files changed, 857 insertions, 0 deletions
diff --git a/poky/bitbake/doc/bitbake-user-manual/bitbake-user-manual-fetching.xml b/poky/bitbake/doc/bitbake-user-manual/bitbake-user-manual-fetching.xml new file mode 100644 index 000000000..29ae486a7 --- /dev/null +++ b/poky/bitbake/doc/bitbake-user-manual/bitbake-user-manual-fetching.xml @@ -0,0 +1,857 @@ +<!DOCTYPE chapter PUBLIC "-//OASIS//DTD DocBook XML V4.2//EN" +"http://www.oasis-open.org/docbook/xml/4.2/docbookx.dtd"> + +<chapter> +<title>File Download Support</title> + + <para> + BitBake's fetch module is a standalone piece of library code + that deals with the intricacies of downloading source code + and files from remote systems. + Fetching source code is one of the cornerstones of building software. + As such, this module forms an important part of BitBake. + </para> + + <para> + The current fetch module is called "fetch2" and refers to the + fact that it is the second major version of the API. + The original version is obsolete and has been removed from the codebase. + Thus, in all cases, "fetch" refers to "fetch2" in this + manual. + </para> + + <section id='the-download-fetch'> + <title>The Download (Fetch)</title> + + <para> + BitBake takes several steps when fetching source code or files. + The fetcher codebase deals with two distinct processes in order: + obtaining the files from somewhere (cached or otherwise) + and then unpacking those files into a specific location and + perhaps in a specific way. + Getting and unpacking the files is often optionally followed + by patching. + Patching, however, is not covered by this module. + </para> + + <para> + The code to execute the first part of this process, a fetch, + looks something like the following: + <literallayout class='monospaced'> + src_uri = (d.getVar('SRC_URI') or "").split() + fetcher = bb.fetch2.Fetch(src_uri, d) + fetcher.download() + </literallayout> + This code sets up an instance of the fetch class. + The instance uses a space-separated list of URLs from the + <link linkend='var-SRC_URI'><filename>SRC_URI</filename></link> + variable and then calls the <filename>download</filename> + method to download the files. + </para> + + <para> + The instantiation of the fetch class is usually followed by: + <literallayout class='monospaced'> + rootdir = l.getVar('WORKDIR') + fetcher.unpack(rootdir) + </literallayout> + This code unpacks the downloaded files to the + specified by <filename>WORKDIR</filename>. + <note> + For convenience, the naming in these examples matches + the variables used by OpenEmbedded. + If you want to see the above code in action, examine + the OpenEmbedded class file <filename>base.bbclass</filename>. + </note> + The <filename>SRC_URI</filename> and <filename>WORKDIR</filename> + variables are not hardcoded into the fetcher, since those fetcher + methods can be (and are) called with different variable names. + In OpenEmbedded for example, the shared state (sstate) code uses + the fetch module to fetch the sstate files. + </para> + + <para> + When the <filename>download()</filename> method is called, + BitBake tries to resolve the URLs by looking for source files + in a specific search order: + <itemizedlist> + <listitem><para><emphasis>Pre-mirror Sites:</emphasis> + BitBake first uses pre-mirrors to try and find source files. + These locations are defined using the + <link linkend='var-PREMIRRORS'><filename>PREMIRRORS</filename></link> + variable. + </para></listitem> + <listitem><para><emphasis>Source URI:</emphasis> + If pre-mirrors fail, BitBake uses the original URL (e.g from + <filename>SRC_URI</filename>). + </para></listitem> + <listitem><para><emphasis>Mirror Sites:</emphasis> + If fetch failures occur, BitBake next uses mirror locations as + defined by the + <link linkend='var-MIRRORS'><filename>MIRRORS</filename></link> + variable. + </para></listitem> + </itemizedlist> + </para> + + <para> + For each URL passed to the fetcher, the fetcher + calls the submodule that handles that particular URL type. + This behavior can be the source of some confusion when you + are providing URLs for the <filename>SRC_URI</filename> + variable. + Consider the following two URLs: + <literallayout class='monospaced'> + http://git.yoctoproject.org/git/poky;protocol=git + git://git.yoctoproject.org/git/poky;protocol=http + </literallayout> + In the former case, the URL is passed to the + <filename>wget</filename> fetcher, which does not + understand "git". + Therefore, the latter case is the correct form since the + Git fetcher does know how to use HTTP as a transport. + </para> + + <para> + Here are some examples that show commonly used mirror + definitions: + <literallayout class='monospaced'> + PREMIRRORS ?= "\ + bzr://.*/.* http://somemirror.org/sources/ \n \ + cvs://.*/.* http://somemirror.org/sources/ \n \ + git://.*/.* http://somemirror.org/sources/ \n \ + hg://.*/.* http://somemirror.org/sources/ \n \ + osc://.*/.* http://somemirror.org/sources/ \n \ + p4://.*/.* http://somemirror.org/sources/ \n \ + svn://.*/.* http://somemirror.org/sources/ \n" + + MIRRORS =+ "\ + ftp://.*/.* http://somemirror.org/sources/ \n \ + http://.*/.* http://somemirror.org/sources/ \n \ + https://.*/.* http://somemirror.org/sources/ \n" + </literallayout> + It is useful to note that BitBake supports + cross-URLs. + It is possible to mirror a Git repository on an HTTP + server as a tarball. + This is what the <filename>git://</filename> mapping in + the previous example does. + </para> + + <para> + Since network accesses are slow, Bitbake maintains a + cache of files downloaded from the network. + Any source files that are not local (i.e. + downloaded from the Internet) are placed into the download + directory, which is specified by the + <link linkend='var-DL_DIR'><filename>DL_DIR</filename></link> + variable. + </para> + + <para> + File integrity is of key importance for reproducing builds. + For non-local archive downloads, the fetcher code can verify + SHA-256 and MD5 checksums to ensure the archives have been + downloaded correctly. + You can specify these checksums by using the + <filename>SRC_URI</filename> variable with the appropriate + varflags as follows: + <literallayout class='monospaced'> + SRC_URI[md5sum] = "<replaceable>value</replaceable>" + SRC_URI[sha256sum] = "<replaceable>value</replaceable>" + </literallayout> + You can also specify the checksums as parameters on the + <filename>SRC_URI</filename> as shown below: + <literallayout class='monospaced'> + SRC_URI = "http://example.com/foobar.tar.bz2;md5sum=4a8e0f237e961fd7785d19d07fdb994d" + </literallayout> + If multiple URIs exist, you can specify the checksums either + directly as in the previous example, or you can name the URLs. + The following syntax shows how you name the URIs: + <literallayout class='monospaced'> + SRC_URI = "http://example.com/foobar.tar.bz2;name=foo" + SRC_URI[foo.md5sum] = 4a8e0f237e961fd7785d19d07fdb994d + </literallayout> + After a file has been downloaded and has had its checksum checked, + a ".done" stamp is placed in <filename>DL_DIR</filename>. + BitBake uses this stamp during subsequent builds to avoid + downloading or comparing a checksum for the file again. + <note> + It is assumed that local storage is safe from data corruption. + If this were not the case, there would be bigger issues to worry about. + </note> + </para> + + <para> + If + <link linkend='var-BB_STRICT_CHECKSUM'><filename>BB_STRICT_CHECKSUM</filename></link> + is set, any download without a checksum triggers an + error message. + The + <link linkend='var-BB_NO_NETWORK'><filename>BB_NO_NETWORK</filename></link> + variable can be used to make any attempted network access a fatal + error, which is useful for checking that mirrors are complete + as well as other things. + </para> + </section> + + <section id='bb-the-unpack'> + <title>The Unpack</title> + + <para> + The unpack process usually immediately follows the download. + For all URLs except Git URLs, BitBake uses the common + <filename>unpack</filename> method. + </para> + + <para> + A number of parameters exist that you can specify within the + URL to govern the behavior of the unpack stage: + <itemizedlist> + <listitem><para><emphasis>unpack:</emphasis> + Controls whether the URL components are unpacked. + If set to "1", which is the default, the components + are unpacked. + If set to "0", the unpack stage leaves the file alone. + This parameter is useful when you want an archive to be + copied in and not be unpacked. + </para></listitem> + <listitem><para><emphasis>dos:</emphasis> + Applies to <filename>.zip</filename> and + <filename>.jar</filename> files and specifies whether to + use DOS line ending conversion on text files. + </para></listitem> + <listitem><para><emphasis>basepath:</emphasis> + Instructs the unpack stage to strip the specified + directories from the source path when unpacking. + </para></listitem> + <listitem><para><emphasis>subdir:</emphasis> + Unpacks the specific URL to the specified subdirectory + within the root directory. + </para></listitem> + </itemizedlist> + The unpack call automatically decompresses and extracts files + with ".Z", ".z", ".gz", ".xz", ".zip", ".jar", ".ipk", ".rpm". + ".srpm", ".deb" and ".bz2" extensions as well as various combinations + of tarball extensions. + </para> + + <para> + As mentioned, the Git fetcher has its own unpack method that + is optimized to work with Git trees. + Basically, this method works by cloning the tree into the final + directory. + The process is completed using references so that there is + only one central copy of the Git metadata needed. + </para> + </section> + + <section id='bb-fetchers'> + <title>Fetchers</title> + + <para> + As mentioned earlier, the URL prefix determines which + fetcher submodule BitBake uses. + Each submodule can support different URL parameters, + which are described in the following sections. + </para> + + <section id='local-file-fetcher'> + <title>Local file fetcher (<filename>file://</filename>)</title> + + <para> + This submodule handles URLs that begin with + <filename>file://</filename>. + The filename you specify within the URL can be + either an absolute or relative path to a file. + If the filename is relative, the contents of the + <link linkend='var-FILESPATH'><filename>FILESPATH</filename></link> + variable is used in the same way + <filename>PATH</filename> is used to find executables. + If the file cannot be found, it is assumed that it is available in + <link linkend='var-DL_DIR'><filename>DL_DIR</filename></link> + by the time the <filename>download()</filename> method is called. + </para> + + <para> + If you specify a directory, the entire directory is + unpacked. + </para> + + <para> + Here are a couple of example URLs, the first relative and + the second absolute: + <literallayout class='monospaced'> + SRC_URI = "file://relativefile.patch" + SRC_URI = "file:///Users/ich/very_important_software" + </literallayout> + </para> + </section> + + <section id='http-ftp-fetcher'> + <title>HTTP/FTP wget fetcher (<filename>http://</filename>, <filename>ftp://</filename>, <filename>https://</filename>)</title> + + <para> + This fetcher obtains files from web and FTP servers. + Internally, the fetcher uses the wget utility. + </para> + + <para> + The executable and parameters used are specified by the + <filename>FETCHCMD_wget</filename> variable, which defaults + to sensible values. + The fetcher supports a parameter "downloadfilename" that + allows the name of the downloaded file to be specified. + Specifying the name of the downloaded file is useful + for avoiding collisions in + <link linkend='var-DL_DIR'><filename>DL_DIR</filename></link> + when dealing with multiple files that have the same name. + </para> + + <para> + Some example URLs are as follows: + <literallayout class='monospaced'> + SRC_URI = "http://oe.handhelds.org/not_there.aac" + SRC_URI = "ftp://oe.handhelds.org/not_there_as_well.aac" + SRC_URI = "ftp://you@oe.handhelds.org/home/you/secret.plan" + </literallayout> + </para> + <note> + Because URL parameters are delimited by semi-colons, this can + introduce ambiguity when parsing URLs that also contain semi-colons, + for example: + <literallayout class='monospaced'> + SRC_URI = "http://abc123.org/git/?p=gcc/gcc.git;a=snapshot;h=a5dd47" + </literallayout> + Such URLs should should be modified by replacing semi-colons with '&' characters: + <literallayout class='monospaced'> + SRC_URI = "http://abc123.org/git/?p=gcc/gcc.git&a=snapshot&h=a5dd47" + </literallayout> + In most cases this should work. Treating semi-colons and '&' in queries + identically is recommended by the World Wide Web Consortium (W3C). + Note that due to the nature of the URL, you may have to specify the name + of the downloaded file as well: + <literallayout class='monospaced'> + SRC_URI = "http://abc123.org/git/?p=gcc/gcc.git&a=snapshot&h=a5dd47;downloadfilename=myfile.bz2" + </literallayout> + </note> + </section> + + <section id='cvs-fetcher'> + <title>CVS fetcher (<filename>(cvs://</filename>)</title> + + <para> + This submodule handles checking out files from the + CVS version control system. + You can configure it using a number of different variables: + <itemizedlist> + <listitem><para><emphasis><filename>FETCHCMD_cvs</filename>:</emphasis> + The name of the executable to use when running + the <filename>cvs</filename> command. + This name is usually "cvs". + </para></listitem> + <listitem><para><emphasis><filename>SRCDATE</filename>:</emphasis> + The date to use when fetching the CVS source code. + A special value of "now" causes the checkout to + be updated on every build. + </para></listitem> + <listitem><para><emphasis><link linkend='var-CVSDIR'><filename>CVSDIR</filename></link>:</emphasis> + Specifies where a temporary checkout is saved. + The location is often <filename>DL_DIR/cvs</filename>. + </para></listitem> + <listitem><para><emphasis><filename>CVS_PROXY_HOST</filename>:</emphasis> + The name to use as a "proxy=" parameter to the + <filename>cvs</filename> command. + </para></listitem> + <listitem><para><emphasis><filename>CVS_PROXY_PORT</filename>:</emphasis> + The port number to use as a "proxyport=" parameter to + the <filename>cvs</filename> command. + </para></listitem> + </itemizedlist> + As well as the standard username and password URL syntax, + you can also configure the fetcher with various URL parameters: + </para> + + <para> + The supported parameters are as follows: + <itemizedlist> + <listitem><para><emphasis>"method":</emphasis> + The protocol over which to communicate with the CVS + server. + By default, this protocol is "pserver". + If "method" is set to "ext", BitBake examines the + "rsh" parameter and sets <filename>CVS_RSH</filename>. + You can use "dir" for local directories. + </para></listitem> + <listitem><para><emphasis>"module":</emphasis> + Specifies the module to check out. + You must supply this parameter. + </para></listitem> + <listitem><para><emphasis>"tag":</emphasis> + Describes which CVS TAG should be used for + the checkout. + By default, the TAG is empty. + </para></listitem> + <listitem><para><emphasis>"date":</emphasis> + Specifies a date. + If no "date" is specified, the + <link linkend='var-SRCDATE'><filename>SRCDATE</filename></link> + of the configuration is used to checkout a specific date. + The special value of "now" causes the checkout to be + updated on every build. + </para></listitem> + <listitem><para><emphasis>"localdir":</emphasis> + Used to rename the module. + Effectively, you are renaming the output directory + to which the module is unpacked. + You are forcing the module into a special + directory relative to + <link linkend='var-CVSDIR'><filename>CVSDIR</filename></link>. + </para></listitem> + <listitem><para><emphasis>"rsh"</emphasis> + Used in conjunction with the "method" parameter. + </para></listitem> + <listitem><para><emphasis>"scmdata":</emphasis> + Causes the CVS metadata to be maintained in the tarball + the fetcher creates when set to "keep". + The tarball is expanded into the work directory. + By default, the CVS metadata is removed. + </para></listitem> + <listitem><para><emphasis>"fullpath":</emphasis> + Controls whether the resulting checkout is at the + module level, which is the default, or is at deeper + paths. + </para></listitem> + <listitem><para><emphasis>"norecurse":</emphasis> + Causes the fetcher to only checkout the specified + directory with no recurse into any subdirectories. + </para></listitem> + <listitem><para><emphasis>"port":</emphasis> + The port to which the CVS server connects. + </para></listitem> + </itemizedlist> + Some example URLs are as follows: + <literallayout class='monospaced'> + SRC_URI = "cvs://CVSROOT;module=mymodule;tag=some-version;method=ext" + SRC_URI = "cvs://CVSROOT;module=mymodule;date=20060126;localdir=usethat" + </literallayout> + </para> + </section> + + <section id='svn-fetcher'> + <title>Subversion (SVN) Fetcher (<filename>svn://</filename>)</title> + + <para> + This fetcher submodule fetches code from the + Subversion source control system. + The executable used is specified by + <filename>FETCHCMD_svn</filename>, which defaults + to "svn". + The fetcher's temporary working directory is set by + <link linkend='var-SVNDIR'><filename>SVNDIR</filename></link>, + which is usually <filename>DL_DIR/svn</filename>. + </para> + + <para> + The supported parameters are as follows: + <itemizedlist> + <listitem><para><emphasis>"module":</emphasis> + The name of the svn module to checkout. + You must provide this parameter. + You can think of this parameter as the top-level + directory of the repository data you want. + </para></listitem> + <listitem><para><emphasis>"path_spec":</emphasis> + A specific directory in which to checkout the + specified svn module. + </para></listitem> + <listitem><para><emphasis>"protocol":</emphasis> + The protocol to use, which defaults to "svn". + If "protocol" is set to "svn+ssh", the "ssh" + parameter is also used. + </para></listitem> + <listitem><para><emphasis>"rev":</emphasis> + The revision of the source code to checkout. + </para></listitem> + <listitem><para><emphasis>"scmdata":</emphasis> + Causes the “.svn” directories to be available during + compile-time when set to "keep". + By default, these directories are removed. + </para></listitem> + <listitem><para><emphasis>"ssh":</emphasis> + An optional parameter used when "protocol" is set + to "svn+ssh". + You can use this parameter to specify the ssh + program used by svn. + </para></listitem> + <listitem><para><emphasis>"transportuser":</emphasis> + When required, sets the username for the transport. + By default, this parameter is empty. + The transport username is different than the username + used in the main URL, which is passed to the subversion + command. + </para></listitem> + </itemizedlist> + Following are three examples using svn: + <literallayout class='monospaced'> + SRC_URI = "svn://myrepos/proj1;module=vip;protocol=http;rev=667" + SRC_URI = "svn://myrepos/proj1;module=opie;protocol=svn+ssh" + SRC_URI = "svn://myrepos/proj1;module=trunk;protocol=http;path_spec=${MY_DIR}/proj1" + </literallayout> + </para> + </section> + + <section id='git-fetcher'> + <title>Git Fetcher (<filename>git://</filename>)</title> + + <para> + This fetcher submodule fetches code from the Git + source control system. + The fetcher works by creating a bare clone of the + remote into + <link linkend='var-GITDIR'><filename>GITDIR</filename></link>, + which is usually <filename>DL_DIR/git2</filename>. + This bare clone is then cloned into the work directory during the + unpack stage when a specific tree is checked out. + This is done using alternates and by reference to + minimize the amount of duplicate data on the disk and + make the unpack process fast. + The executable used can be set with + <filename>FETCHCMD_git</filename>. + </para> + + <para> + This fetcher supports the following parameters: + <itemizedlist> + <listitem><para><emphasis>"protocol":</emphasis> + The protocol used to fetch the files. + The default is "git" when a hostname is set. + If a hostname is not set, the Git protocol is "file". + You can also use "http", "https", "ssh" and "rsync". + </para></listitem> + <listitem><para><emphasis>"nocheckout":</emphasis> + Tells the fetcher to not checkout source code when + unpacking when set to "1". + Set this option for the URL where there is a custom + routine to checkout code. + The default is "0". + </para></listitem> + <listitem><para><emphasis>"rebaseable":</emphasis> + Indicates that the upstream Git repository can be rebased. + You should set this parameter to "1" if + revisions can become detached from branches. + In this case, the source mirror tarball is done per + revision, which has a loss of efficiency. + Rebasing the upstream Git repository could cause the + current revision to disappear from the upstream repository. + This option reminds the fetcher to preserve the local cache + carefully for future use. + The default value for this parameter is "0". + </para></listitem> + <listitem><para><emphasis>"nobranch":</emphasis> + Tells the fetcher to not check the SHA validation + for the branch when set to "1". + The default is "0". + Set this option for the recipe that refers to + the commit that is valid for a tag instead of + the branch. + </para></listitem> + <listitem><para><emphasis>"bareclone":</emphasis> + Tells the fetcher to clone a bare clone into the + destination directory without checking out a working tree. + Only the raw Git metadata is provided. + This parameter implies the "nocheckout" parameter as well. + </para></listitem> + <listitem><para><emphasis>"branch":</emphasis> + The branch(es) of the Git tree to clone. + If unset, this is assumed to be "master". + The number of branch parameters much match the number of + name parameters. + </para></listitem> + <listitem><para><emphasis>"rev":</emphasis> + The revision to use for the checkout. + The default is "master". + </para></listitem> + <listitem><para><emphasis>"tag":</emphasis> + Specifies a tag to use for the checkout. + To correctly resolve tags, BitBake must access the + network. + For that reason, tags are often not used. + As far as Git is concerned, the "tag" parameter behaves + effectively the same as the "rev" parameter. + </para></listitem> + <listitem><para><emphasis>"subpath":</emphasis> + Limits the checkout to a specific subpath of the tree. + By default, the whole tree is checked out. + </para></listitem> + <listitem><para><emphasis>"destsuffix":</emphasis> + The name of the path in which to place the checkout. + By default, the path is <filename>git/</filename>. + </para></listitem> + </itemizedlist> + Here are some example URLs: + <literallayout class='monospaced'> + SRC_URI = "git://git.oe.handhelds.org/git/vip.git;tag=version-1" + SRC_URI = "git://git.oe.handhelds.org/git/vip.git;protocol=http" + </literallayout> + </para> + </section> + + <section id='gitsm-fetcher'> + <title>Git Submodule Fetcher (<filename>gitsm://</filename>)</title> + + <para> + This fetcher submodule inherits from the + <link linkend='git-fetcher'>Git fetcher</link> and extends + that fetcher's behavior by fetching a repository's submodules. + <link linkend='var-SRC_URI'><filename>SRC_URI</filename></link> + is passed to the Git fetcher as described in the + "<link linkend='git-fetcher'>Git Fetcher (<filename>git://</filename>)</link>" + section. + <note> + <title>Notes and Warnings</title> + <para> + You must clean a recipe when switching between + '<filename>git://</filename>' and + '<filename>gitsm://</filename>' URLs. + </para> + + <para> + The Git Submodules fetcher is not a complete fetcher + implementation. + The fetcher has known issues where it does not use the + normal source mirroring infrastructure properly. Further, + the submodule sources it fetches are not visible to the + licensing and source archiving infrastructures. + </para> + </note> + </para> + </section> + + <section id='clearcase-fetcher'> + <title>ClearCase Fetcher (<filename>ccrc://</filename>)</title> + + <para> + This fetcher submodule fetches code from a + <ulink url='http://en.wikipedia.org/wiki/Rational_ClearCase'>ClearCase</ulink> + repository. + </para> + + <para> + To use this fetcher, make sure your recipe has proper + <link linkend='var-SRC_URI'><filename>SRC_URI</filename></link>, + <link linkend='var-SRCREV'><filename>SRCREV</filename></link>, and + <link linkend='var-PV'><filename>PV</filename></link> settings. + Here is an example: + <literallayout class='monospaced'> + SRC_URI = "ccrc://cc.example.org/ccrc;vob=/example_vob;module=/example_module" + SRCREV = "EXAMPLE_CLEARCASE_TAG" + PV = "${@d.getVar("SRCREV", False).replace("/", "+")}" + </literallayout> + The fetcher uses the <filename>rcleartool</filename> or + <filename>cleartool</filename> remote client, depending on + which one is available. + </para> + + <para> + Following are options for the <filename>SRC_URI</filename> + statement: + <itemizedlist> + <listitem><para><emphasis><filename>vob</filename></emphasis>: + The name, which must include the + prepending "/" character, of the ClearCase VOB. + This option is required. + </para></listitem> + <listitem><para><emphasis><filename>module</filename></emphasis>: + The module, which must include the + prepending "/" character, in the selected VOB. + <note> + The <filename>module</filename> and <filename>vob</filename> + options are combined to create the <filename>load</filename> rule in + the view config spec. + As an example, consider the <filename>vob</filename> and + <filename>module</filename> values from the + <filename>SRC_URI</filename> statement at the start of this section. + Combining those values results in the following: + <literallayout class='monospaced'> + load /example_vob/example_module + </literallayout> + </note> + </para></listitem> + <listitem><para><emphasis><filename>proto</filename></emphasis>: + The protocol, which can be either <filename>http</filename> or + <filename>https</filename>. + </para></listitem> + </itemizedlist> + </para> + + <para> + By default, the fetcher creates a configuration specification. + If you want this specification written to an area other than the default, + use the <filename>CCASE_CUSTOM_CONFIG_SPEC</filename> variable + in your recipe to define where the specification is written. + <note> + the <filename>SRCREV</filename> loses its functionality if you + specify this variable. + However, <filename>SRCREV</filename> is still used to label the + archive after a fetch even though it does not define what is + fetched. + </note> + </para> + + <para> + Here are a couple of other behaviors worth mentioning: + <itemizedlist> + <listitem><para> + When using <filename>cleartool</filename>, the login of + <filename>cleartool</filename> is handled by the system. + The login require no special steps. + </para></listitem> + <listitem><para> + In order to use <filename>rcleartool</filename> with authenticated + users, an "rcleartool login" is necessary before using the fetcher. + </para></listitem> + </itemizedlist> + </para> + </section> + + <section id='perforce-fetcher'> + <title>Perforce Fetcher (<filename>p4://</filename>)</title> + + <para> + This fetcher submodule fetches code from the + <ulink url='https://www.perforce.com/'>Perforce</ulink> + source control system. + The executable used is specified by + <filename>FETCHCMD_p4</filename>, which defaults + to "p4". + The fetcher's temporary working directory is set by + <link linkend='var-P4DIR'><filename>P4DIR</filename></link>, + which defaults to "DL_DIR/p4". + </para> + + <para> + To use this fetcher, make sure your recipe has proper + <link linkend='var-SRC_URI'><filename>SRC_URI</filename></link>, + <link linkend='var-SRCREV'><filename>SRCREV</filename></link>, and + <link linkend='var-PV'><filename>PV</filename></link> values. + The p4 executable is able to use the config file defined by your + system's <filename>P4CONFIG</filename> environment variable in + order to define the Perforce server URL and port, username, and + password if you do not wish to keep those values in a recipe + itself. + If you choose not to use <filename>P4CONFIG</filename>, + or to explicitly set variables that <filename>P4CONFIG</filename> + can contain, you can specify the <filename>P4PORT</filename> value, + which is the server's URL and port number, and you can + specify a username and password directly in your recipe within + <filename>SRC_URI</filename>. + </para> + + <para> + Here is an example that relies on <filename>P4CONFIG</filename> + to specify the server URL and port, username, and password, and + fetches the Head Revision: + <literallayout class='monospaced'> + SRC_URI = "p4://example-depot/main/source/..." + SRCREV = "${AUTOREV}" + PV = "p4-${SRCPV}" + S = "${WORKDIR}/p4" + </literallayout> + </para> + + <para> + Here is an example that specifies the server URL and port, + username, and password, and fetches a Revision based on a Label: + <literallayout class='monospaced'> + P4PORT = "tcp:p4server.example.net:1666" + SRC_URI = "p4://user:passwd@example-depot/main/source/..." + SRCREV = "release-1.0" + PV = "p4-${SRCPV}" + S = "${WORKDIR}/p4" + </literallayout> + <note> + You should always set <filename>S</filename> + to <filename>"${WORKDIR}/p4"</filename> in your recipe. + </note> + </para> + </section> + + <section id='repo-fetcher'> + <title>Repo Fetcher (<filename>repo://</filename>)</title> + + <para> + This fetcher submodule fetches code from + <filename>google-repo</filename> source control system. + The fetcher works by initiating and syncing sources of the + repository into + <link linkend='var-REPODIR'><filename>REPODIR</filename></link>, + which is usually + <link linkend='var-DL_DIR'><filename>DL_DIR</filename></link><filename>/repo</filename>. + </para> + + <para> + This fetcher supports the following parameters: + <itemizedlist> + <listitem><para> + <emphasis>"protocol":</emphasis> + Protocol to fetch the repository manifest (default: git). + </para></listitem> + <listitem><para> + <emphasis>"branch":</emphasis> + Branch or tag of repository to get (default: master). + </para></listitem> + <listitem><para> + <emphasis>"manifest":</emphasis> + Name of the manifest file (default: <filename>default.xml</filename>). + </para></listitem> + </itemizedlist> + Here are some example URLs: + <literallayout class='monospaced'> + SRC_URI = "repo://REPOROOT;protocol=git;branch=some_branch;manifest=my_manifest.xml" + SRC_URI = "repo://REPOROOT;protocol=file;branch=some_branch;manifest=my_manifest.xml" + </literallayout> + </para> + </section> + + <section id='other-fetchers'> + <title>Other Fetchers</title> + + <para> + Fetch submodules also exist for the following: + <itemizedlist> + <listitem><para> + Bazaar (<filename>bzr://</filename>) + </para></listitem> + <listitem><para> + Trees using Git Annex (<filename>gitannex://</filename>) + </para></listitem> + <listitem><para> + Secure FTP (<filename>sftp://</filename>) + </para></listitem> + <listitem><para> + Secure Shell (<filename>ssh://</filename>) + </para></listitem> + <listitem><para> + OSC (<filename>osc://</filename>) + </para></listitem> + <listitem><para> + Mercurial (<filename>hg://</filename>) + </para></listitem> + </itemizedlist> + No documentation currently exists for these lesser used + fetcher submodules. + However, you might find the code helpful and readable. + </para> + </section> + </section> + + <section id='auto-revisions'> + <title>Auto Revisions</title> + + <para> + We need to document <filename>AUTOREV</filename> and + <filename>SRCREV_FORMAT</filename> here. + </para> + </section> +</chapter> |