summaryrefslogtreecommitdiff
path: root/scripts/package
AgeCommit message (Collapse)AuthorFilesLines
2019-07-17kbuild: create *.mod with full directory path and remove MODVERDIRMasahiro Yamada1-1/+1
While descending directories, Kbuild produces objects for modules, but do not link final *.ko files; it is done in the modpost. To keep track of modules, Kbuild creates a *.mod file in $(MODVERDIR) for every module it is building. Some post-processing steps read the necessary information from *.mod files. This avoids descending into directories again. This mechanism was introduced in 2003 or so. Later, commit 551559e13af1 ("kbuild: implement modules.order") added modules.order. So, we can simply read it out to know all the modules with directory paths. This is easier than parsing the first line of *.mod files. $(MODVERDIR) has a flat directory structure, that is, *.mod files are named only with base names. This is based on the assumption that the module name is unique across the tree. This assumption is really fragile. Stephen Rothwell reported a race condition caused by a module name conflict: https://lkml.org/lkml/2019/5/13/991 In parallel building, two different threads could write to the same $(MODVERDIR)/*.mod simultaneously. Non-unique module names are the source of all kind of troubles, hence commit 3a48a91901c5 ("kbuild: check uniqueness of module names") introduced a new checker script. However, it is still fragile in the build system point of view because this race happens before scripts/modules-check.sh is invoked. If it happens again, the modpost will emit unclear error messages. To fix this issue completely, create *.mod with full directory path so that two threads never attempt to write to the same file. $(MODVERDIR) is no longer needed. Since modules with directory paths are listed in modules.order, Kbuild is still able to find *.mod files without additional descending. I also killed cmd_secanalysis; scripts/mod/sumversion.c computes MD4 hash for modules with MODULE_VERSION(). When CONFIG_DEBUG_SECTION_MISMATCH=y, it occurs not only in the modpost stage, but also during directory descending, where sumversion.c may parse stale *.mod files. It would emit 'No such file or directory' warning when an object consisting a module is renamed, or when a single-obj module is turned into a multi-obj module or vice versa. Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com> Acked-by: Nicolas Pitre <nico@fluxnic.net>
2019-07-17builddeb: generate multi-arch friendly linux-libc-dev packageCedric Hombourger2-0/+6
Debian-based distributions place libc header files in a machine specific directory (/usr/include/<libc-machine>) instead of /usr/include/asm to support installation of the linux-libc-dev package from multiple architectures. Move headers installed by "make headers_install" accordingly using Debian's tuple from dpkg-architecture (stored in debian/arch). Signed-off-by: Cedric Hombourger <Cedric_Hombourger@mentor.com> Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
2019-07-09kbuild: do not create wrappers for header-test-yMasahiro Yamada1-1/+1
header-test-y does not work with headers in sub-directories. For example, you may want to write a Makefile, like this: include/linux/Kbuild: header-test-y += mtd/nand.h This entry will create a wrapper include/linux/mtd/nand.hdrtest.c with the following content: #include "mtd/nand.h" To make this work, we need to add $(srctree)/include/linux to the header search path. It would be tedious to add ccflags-y. Instead, we could change the *.hdrtest.c rule to wrap: #include "nand.h" This works for in-tree build since #include "..." searches in the relative path from the header with this directive. For O=... build, we need to add $(srctree)/include/linux/mtd to the header search path, which will be even more tedious. After all, I thought it would be handier to compile headers directly without creating wrappers. I added a new build rule to compile %.h into %.h.s The target is %.h.s instead of %.h.o because it is slightly faster. Also, as for GCC, an empty assembly is smaller than an empty object. I wrote the build rule: $(CC) $(c_flags) -S -o $@ -x c /dev/null -include $< instead of: $(CC) $(c_flags) -S -o $@ -x c $< Both work fine with GCC, but the latter is bad for Clang. This comes down to the difference in the -Wunused-function policy. GCC does not warn about unused 'static inline' functions at all. Clang does not warn about the ones in included headers, but does about the ones in the source. So, we should handle headers as headers, not as source files. In fact, this has been hidden since commit abb2ea7dfd82 ("compiler, clang: suppress warning for unused static inline functions"), but we should not rely on that. Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com> Acked-by: Jani Nikula <jani.nikula@intel.com> Tested-by: Jani Nikula <jani.nikula@intel.com>
2019-06-15kbuild: deb-pkg: do not run headers_checkMasahiro Yamada1-1/+1
It is absolutely fine to add extra sanity checks in package scripts, but it is not necessary to do so. This is already covered by the daily compile-testing (0day bot etc.) because headers_check is run as a part of the normal build process when CONFIG_HEADERS_CHECK=y. Replace it with the newly-added "make headers". Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
2019-06-04kbuild: tar-pkg: enable communication with jobserverTrevor Bourget1-1/+1
The buildtar script might want to invoke a make, so tell the parent make to pass the jobserver token pipe to the subcommand by prefixing the command with a +. This addresses the issue seen here: /bin/sh ../scripts/package/buildtar tar-pkg make[3]: warning: jobserver unavailable: using -j1. Add '+' to parent make rule. See https://www.gnu.org/software/make/manual/html_node/Job-Slots.html for more information. Signed-off-by: Trevor Bourget <tgb.kernel@gmail.com> Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
2019-05-21treewide: Add SPDX license identifier - Makefile/KconfigThomas Gleixner1-0/+1
Add SPDX license identifiers to all Make/Kconfig files which: - Have no license information of any form These files fall under the project license, GPL v2 only. The resulting SPDX license identifier is: GPL-2.0-only Signed-off-by: Thomas Gleixner <tglx@linutronix.de> Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
2019-03-17kbuild: deb-pkg: avoid implicit effectsArseny Maslennikov1-1/+4
* The man page for dpkg-source(1) notes: > -b, --build directory [format-specific-parameters] > Build a source package (--build since dpkg 1.17.14). > <...> > > dpkg-source will build the source package with the first > format found in this ordered list: the format indicated > with the --format command line option, the format > indicated in debian/source/format, “1.0”. The fallback > to “1.0” is deprecated and will be removed at some point > in the future, you should always document the desired > source format in debian/source/format. See section > SOURCE PACKAGE FORMATS for an extensive description of > the various source package formats. Thus it would be more foolproof to explicitly use 1.0 (as we always did) than to rely on dpkg-source's defaults. * In a similar vein, debian/rules is not made executable by mkdebian, and dpkg-source warns about that but still silently fixes the file. Let's be explicit once again. Signed-off-by: Arseny Maslennikov <ar@cs.msu.ru> Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
2019-03-13kbuild: pkg: grep include/config/auto.conf instead of $KCONFIG_CONFIGMasahiro Yamada3-3/+3
This will be a little more efficient since unset CONFIG options are stripped away from auto.conf, and we can hard-code the path to auto.conf since it is never overridden. include/config/kernel.release is generated before %pkg is run. So, it is guaranteed auto.conf is up-to-date. Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
2019-03-13kbuild: deb-pkg: introduce is_enabled and if_enabled_echo to builddebMasahiro Yamada1-13/+19
I think is_enabled() and if_enable_echo() in scripts/package/mkdebian are useful. builddeb also has many repetitive greps over the kernel config, so I borrowed the idea to clean it up. Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
2019-03-13kbuild: deb-pkg: add CONFIG_ prefix to kernel config optionsMasahiro Yamada1-17/+17
This might be a kind of bike-shed, but I personally prefer grep'able code. I often do 'git grep CONFIG_FOO' instead of 'git grep FOO' when I want to know where that CONFIG option is used. This makes code longer, but I hope this is acceptable level. Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
2019-03-13deb-pkg: generate correct build dependenciesRiku Voipio1-1/+3
bison/flex is now needed always for building for kconfig. Some build dependencies depend on kernel configuration, enable them as needed: - libelf-dev when UNWINDER_ORC is set - libssl-dev for SYSTEM_TRUSTED_KEYRING Since the libssl-dev is needed for extract_cert binary, denote with :native to install the libssl-dev for the build machines architecture, rather than for the architecture of the kernel being built. Tested-by: Manivannan Sadhasivam <manivannan.sadhasivam@linaro.org> Signed-off-by: Riku Voipio <riku.voipio@linaro.org> Reviewed-by: Ben Hutchings <ben@decadent.org.uk> Acked-by: maximilian attems <maks@stro.at> [masahiro.yamada: change 'flex' to 'flex | flex:native' ] Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
2019-02-28kbuild: [bin]deb-pkg: add DPKG_FLAGS variableKacper Kołodziej1-2/+2
DPKG_FLAGS variable lets user to add more flags to dpkg-buildpackage command in deb-pkg and bindeb-pkg. Signed-off-by: Kacper Kołodziej <kacper@kolodziej.it> Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
2019-02-20kbuild: pkg: use -f $(srctree)/Makefile to recurse to top MakefileMasahiro Yamada4-10/+12
'$(MAKE) KBUILD_SRC=' changes the working directory back and forth between objtree and srctree. It is better to recurse to the top-level Makefile directly. Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
2018-12-01kbuild: remove redundant 'set -e' from cmd_* definesMasahiro Yamada1-1/+0
These three cmd_* are invoked in the $(call cmd,*) form. Now that 'set -e' moved to the 'cmd' macro, they do not need to explicitly give 'set -e'. Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
2018-11-11builddeb: Fix inclusion of dtbs in debian packageRob Herring1-2/+2
Commit 37c8a5fafa3b ("kbuild: consolidate Devicetree dtb build rules") moved the location of 'dtbs_install' target which caused dtbs to not be installed when building debian package with 'bindeb-pkg' target. Update the builddeb script to use the same logic that determines if there's a 'dtbs_install' target which is presence of the arch dts directory. Also, use CONFIG_OF_EARLY_FLATTREE instead of CONFIG_OF as that's a better indication of whether we are building dtbs. This commit will also have the side effect of installing dtbs on any arch that has dts files. Previously, it was dependent on whether the arch defined 'dtbs_install'. Fixes: 37c8a5fafa3b ("kbuild: consolidate Devicetree dtb build rules") Reported-by: Nuno Gonçalves <nunojpg@gmail.com> Signed-off-by: Rob Herring <robh@kernel.org> Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
2018-11-11kbuild: deb-pkg: fix too low build version numberMasahiro Yamada1-2/+5
Since commit b41d920acff8 ("kbuild: deb-pkg: split generating packaging and build"), the build version of the kernel contained in a deb package is too low by 1. Prior to the bad commit, the kernel was built first, then the number in .version file was read out, and written into the debian control file. Now, the debian control file is created before the kernel is actually compiled, which is causing the version number mismatch. Let the mkdebian script pass KBUILD_BUILD_VERSION=${revision} to require the build system to use the specified version number. Fixes: b41d920acff8 ("kbuild: deb-pkg: split generating packaging and build") Reported-by: Doug Smythies <dsmythies@telus.net> Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com> Tested-by: Doug Smythies <dsmythies@telus.net>
2018-11-05kbuild: deb-pkg: fix bindeb-pkg breakage when O= is usedMasahiro Yamada1-1/+1
Ard Biesheuvel reports bindeb-pkg with O= option is broken in the following way: ... LD [M] sound/soc/rockchip/snd-soc-rk3399-gru-sound.ko LD [M] sound/soc/rockchip/snd-soc-rockchip-pcm.ko LD [M] sound/soc/rockchip/snd-soc-rockchip-rt5645.ko LD [M] sound/soc/rockchip/snd-soc-rockchip-spdif.ko LD [M] sound/soc/sh/rcar/snd-soc-rcar.ko fakeroot -u debian/rules binary make KERNELRELEASE=4.19.0-12677-g19beffaf7a99-dirty ARCH=arm64 KBUILD_SRC= intdeb-pkg /bin/bash /home/ard/linux/scripts/package/builddeb Makefile:600: include/config/auto.conf: No such file or directory *** *** Configuration file ".config" not found! *** *** Please run some configurator (e.g. "make oldconfig" or *** "make menuconfig" or "make xconfig"). *** make[12]: *** [syncconfig] Error 1 make[11]: *** [syncconfig] Error 2 make[10]: *** [include/config/auto.conf] Error 2 make[9]: *** [__sub-make] Error 2 ... Prior to commit 80463f1b7bf9 ("kbuild: add --include-dir flag only for out-of-tree build"), both srctree and objtree were added to --include-dir redundantly, and the wrong code '$MAKE image_name' was working by relying on that. Now, the potential issue that had previously been hidden just showed up. '$MAKE image_name' recurses to the generated $(objtree)/Makefile and ends up with running in srctree, which is incorrect. It should be invoked with '-f $srctree/Makefile' (or KBUILD_SRC=) to be executed in objtree. Fixes: 80463f1b7bf9 ("kbuild: add --include-dir flag only for out-of-tree build") Reported-by: Ard Biesheuvel <ard.biesheuvel@linaro.org> Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com> Tested-by: Ard Biesheuvel <ard.biesheuvel@linaro.org>
2018-11-05kbuild: rpm-pkg: fix binrpm-pkg breakage when O= is usedMasahiro Yamada1-5/+6
Zhenzhong Duan reported that running 'make O=/build/kernel binrpm-pkg' failed with the following errors: Running 'make O=/build/kernel binrpm-pkg' failed with below two errors. Makefile:600: include/config/auto.conf: No such file or directory + cp make -C /mnt/root/kernel O=/build/kernel image_name make -f /mnt/root/kernel/Makefile ... cp: invalid option -- 'C' Try 'cp --help' for more information. Prior to commit 80463f1b7bf9 ("kbuild: add --include-dir flag only for out-of-tree build"), both srctree and objtree were added to --include-dir redundantly, and the wrong code 'make image_name' was working by relying on that. Now, the potential issue that had previously been hidden just showed up. 'make image_name' recurses to the generated $(objtree)/Makefile and ends up with running in srctree, which is incorrect. It should be invoked with '-f $srctree/Makefile' (or KBUILD_SRC=) to be executed in objtree. Fixes: 80463f1b7bf9 ("kbuild: add --include-dir flag only for out-of-tree build") Reported-by: Zhenzhong Duan <zhenzhong.duan@oracle.com> Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
2018-07-28builddeb: Add automatic support for sh{3,4}{,eb} architecturesBen Hutchings1-3/+10
Different generations of the SH architecture are not very compatible, so there are/were separate Debian ports for SH3 and SH4. Move the fallback out of the "case" statement, so that it will also be used in case we find some SH architecture version without a known mapping. Signed-off-by: Ben Hutchings <ben@decadent.org.uk> Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
2018-07-28builddeb: Add automatic support for riscv* architecturesBen Hutchings1-1/+1
Debian currently only defines "riscv64", but it seems safe to assume that any 32-bit port will now be called "riscv32", also matching $UTS_MACHINE. Signed-off-by: Ben Hutchings <ben@decadent.org.uk> Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
2018-07-28builddeb: Add automatic support for m68k architectureBen Hutchings1-1/+1
Signed-off-by: Ben Hutchings <ben@decadent.org.uk> Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
2018-07-28builddeb: Add automatic support for or1k architectureBen Hutchings1-0/+2
Signed-off-by: Ben Hutchings <ben@decadent.org.uk> Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
2018-07-28builddeb: Add automatic support for sparc64 architectureBen Hutchings1-1/+1
We currently label 64-bit kernel packages as sparc (32-bit), mostly because it was officially supported while sparc64 was not. Now neither is officially supported, so label these packages as sparc64. Signed-off-by: Ben Hutchings <ben@decadent.org.uk> Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
2018-07-28builddeb: Add automatic support for mips{,64}r6{,el} architecturesBen Hutchings1-1/+3
MIPS R6 is not fully backward-compatible, so Debian has separate architecture names for userland built for R6. Label kernel packages accordingly. Signed-off-by: Ben Hutchings <ben@decadent.org.uk> Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
2018-07-28builddeb: Add automatic support for mips64el architectureBen Hutchings1-1/+6
We currently label 64-bit little-endian kernel packages as mipsel (32-bit little-endian), mostly it was officially supported while mips64el (64-bit little-endian) was not. Now both are officially supported, so label these packages as mips64el. Signed-off-by: Ben Hutchings <ben@decadent.org.uk> Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
2018-07-28builddeb: Add automatic support for ppc64 and powerpcspe architecturesBen Hutchings1-1/+6
We currently label 64-bit big-endian kernel packages as powerpc (32-bit), mostly because it was officially supported while ppc64 (64-bit big-endian) was not. Now neither is officially supported, so label these packages as ppc64. Debian also has a powerpcspe (32-bit with SPE) architecture. Label packages with a suitable configuration as powerpcspe. Signed-off-by: Ben Hutchings <ben@decadent.org.uk> Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
2018-07-28builddeb: Introduce functions to simplify kconfig tests in set_debarchBen Hutchings1-9/+17
We now have many repetitive greps over the kernel config. Refactor them into functions. Signed-off-by: Ben Hutchings <ben@decadent.org.uk> Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
2018-07-28builddeb: Drop check for 32-bit s390Ben Hutchings1-1/+1
s390 now only supports 64-bit configurations. Signed-off-by: Ben Hutchings <ben@decadent.org.uk> Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
2018-07-28builddeb: Change architecture detection fallback to use dpkg-architectureBen Hutchings1-2/+2
We currently use dpkg --print-architecture, which reports the architecture of the build machine. We can make a better guess than this by asking dpkg-architecture what the host architecture, i.e. the default architecture for building packages, is. This is sensitive to environment variables such as CC and DEB_HOST_ARCH, which should already be set in a cross-build environment. Signed-off-by: Ben Hutchings <ben@decadent.org.uk> Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
2018-07-28builddeb: Skip architecture detection when KBUILD_DEBARCH is setBen Hutchings1-3/+6
If KBUILD_DEBARCH is set then we will not use the result of architecture detection, and we may also warn unnecessarily. Move the check for KBUILD_DEBARCH further up to avoid this. Signed-off-by: Ben Hutchings <ben@decadent.org.uk> Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
2018-07-19kbuild: buildtar: add arm64 supportOlof Johansson1-0/+8
Make 'make tar-pkg' work on arm64. Cc: Will Deacon <will.deacon@arm.com> Cc: Catalin Marinas <catalin.marinas@arm.com> Signed-off-by: Olof Johansson <olof@lixom.net> Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
2018-07-19kbuild: buildtar: remove crufty vax piecesOlof Johansson1-4/+0
ARCH=vax isn't in mainline; it can be added back if/when it shows up. Signed-off-by: Olof Johansson <olof@lixom.net> Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
2018-05-17kbuild: deb-pkg improve maintainer address generationRiku Voipio1-14/+13
There is multiple issues with the genaration of maintainer string It uses DEBEMAIL and EMAIL enviroment variables, which may contain angle brackets, creating invalid maintainer strings. The documented KBUILD_BUILD_USER and KBUILD_BUILD_HOST variables are not used. Undocumented and uncommon NAME variable is used. Refactor the Maintainer string to: - use EMAIL or DEBEMAIL directly if they are in form "name <user@host>" - use KBUILD_BUILD_USER and KBUILD_BUILD_HOST if set before falling back to autodetection - no longer use NAME variable or the useless Anonymous string The logic is switched from multiline if/then/fi statements to compact shell variable substition commands. Reported-by: Mathieu Malaterre <malat@debian.org> Signed-off-by: Riku Voipio <riku.voipio@linaro.org> Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
2018-04-13kbuild: rpm-pkg: use kernel-install as a fallback for new-kernel-pkgJavier Martinez Canillas1-0/+2
The new-kernel-pkg script is only present when grubby is installed, but it may not always be the case. So if the script isn't present, attempt to use the kernel-install script as a fallback instead. Signed-off-by: Javier Martinez Canillas <javierm@redhat.com> Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
2018-04-07kbuild: deb-pkg: split generating packaging and buildRiku Voipio3-244/+200
Move debian/ directory generation out of builddeb to a new script, mkdebian. The package build commands are kept in builddeb, which is now an internal command called from debian/rules. With these changes in place, we can now use dpkg-buildpackage from deb-pkg and bindeb-pkg removing need for handrolled source/changes generation. This patch is based on the criticism of the current state of builddeb discussed on: https://patchwork.kernel.org/patch/9656403/ Signed-off-by: Riku Voipio <riku.voipio@linaro.org> Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
2018-03-27kbuild: rpm-pkg: Support GNU tar >= 1.29Jason Gunthorpe1-1/+1
There is a change in how command line parsing is done in this version. Excludes and includes are now ordered with the file list. Since the spec file puts the file list before the exclude list it means newer tar ignores the excludes and packs all the build output into the kernel-devel RPM resulting in a huge package. Simple argument re-ordering fixes the problem. Signed-off-by: Jason Gunthorpe <jgg@mellanox.com> Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
2018-03-27builddeb: Fix header package regarding dtc source linksJan Kiszka1-1/+1
Since d5d332d3f7e8, a couple of links in scripts/dtc/include-prefixes are additionally required in order to build device trees with the header package. Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com> Reviewed-by: Riku Voipio <riku.voipio@linaro.org> Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
2017-12-12kbuild: pkg: make out-of-tree rpm/deb-pkg build immediately failMasahiro Yamada1-3/+7
We do not support out-of-tree building of rpm-pkg / deb-pkg. If O= is given, the build should be terminated, but the "false" command is not effective since it is not the last command in the cmd_src_tar. Then, rpm-pkg / deb-pkg tries to continue building, and fails for a different reason. Set -e option so that the "false" terminates the building immediately. I also put the error messages to stderr, and made it stand out more. For example, "make O=foo rpm-pkg" will fail as follows: /bin/bash ../scripts/package/mkspec >./kernel.spec TAR kernel-4.15.0_rc2+.tar.gz ERROR: Building source tarball is not possible outside the kernel source tree. Don't set KBUILD_OUTPUT, or use the binrpm-pkg or bindeb-pkg target instead. ../scripts/package/Makefile:53: recipe for target 'rpm-pkg' failed Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com> Tested-by: Jim Davis <jim.epost@gmail.com>
2017-12-12scripts/package: snap-pkg targetPaolo Pisati2-0/+30
Following in footsteps of other targets like 'deb-pkg, 'rpm-pkg' and 'tar-pkg', this patch adds a 'snap-pkg' target for the creation of a Linux kernel snap package using the kbuild infrastructure. A snap, in its general form, is a self contained, sandboxed, universal package and it is intended to work across multiple distributions and/or devices. A snap package is distributed as a single compressed squashfs filesystem. A kernel snap is a snap package carrying the Linux kernel, kernel modules, accessory files (DTBs, System.map, etc) and a manifesto file. The purpose of a kernel snap is to carry the Linux kernel during the creation of a system image, eg. Ubuntu Core, and its subsequent upgrades. For more information on snap packages: https://snapcraft.io/docs/ Signed-off-by: Paolo Pisati <paolo.pisati@canonical.com> Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
2017-11-23kbuild: rpm: prompt to use "rpm-pkg" if "rpm" target is usedMasahiro Yamada1-1/+1
The "rpm" has been kept for backward compatibility since pre-git era. I am planning to remove it after the Linux 4.18 release. Annouce the end of the support, prompting to use "rpm-pkg" instead. If you use "rpm", it will work like "rpm-pkg", but warning messages will be displayed as follows: WARNING: "rpm" target will be removed after Linux 4.18 Please use "rpm-pkg" instead. Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
2017-11-23kbuild: pkg: use --transform option to prefix paths in tarMasahiro Yamada1-3/+2
For rpm-pkg and deb-pkg, a source tar file is created. All paths in the archive must be prefixed with the base name of the tar so that everything is contained in the directory when you extract it. Currently, scripts/package/Makefile uses a symlink for that, and removes it after the tar is created. If you terminate the build during the tar creation, the symlink is left over. Then, at the next package build, you will see a warning like follows: ln: '.' and 'kernel-4.14.0+/.' are the same file It is possible to fix it by adding -n (--no-dereference) option to the "ln" command, but a cleaner way is to use --transform option of "tar" command. This option is GNU extension, but it should not hurt to use it in the Linux build system. The 'S' flag is needed to exclude symlinks from the path fixup. Without it, symlinks in the kernel are broken. Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
2017-11-18Merge tag 'kbuild-misc-v4.15' of ↵Linus Torvalds3-151/+131
git://git.kernel.org/pub/scm/linux/kernel/git/masahiroy/linux-kbuild Pull Kbuild misc updates from Masahiro Yamada: - Clean up and fix RPM package build - Fix a warning in DEB package build - Improve coccicheck script - Improve some semantic patches * tag 'kbuild-misc-v4.15' of git://git.kernel.org/pub/scm/linux/kernel/git/masahiroy/linux-kbuild: docs: dev-tools: coccinelle: delete out of date wiki reference coccinelle: orplus: reorganize to improve performance coccinelle: use exists to improve efficiency builddeb: Pass the kernel:debarch substvar to dpkg-genchanges Coccinelle: use false positive annotation coccinelle: fix verbose message about .cocci file being run coccinelle: grep Options and Requires fields more precisely Coccinelle: make DEBUG_FILE option more useful coccinelle: api: detect identical chip data arrays coccinelle: Improve setup_timer.cocci matching Coccinelle: setup_timer: improve messages from setup_timer kbuild: rpm-pkg: do not force -jN in submake kbuild: rpm-pkg: keep spec file until make mrproper kbuild: rpm-pkg: fix jobserver unavailable warning kbuild: rpm-pkg: replace $RPM_BUILD_ROOT with %{buildroot} kbuild: rpm-pkg: fix build error when CONFIG_MODULES is disabled kbuild: rpm-pkg: refactor mkspec with here doc kbuild: rpm-pkg: clean up mkspec kbuild: rpm-pkg: install vmlinux.bz2 unconditionally kbuild: rpm-pkg: remove ppc64 specific image handling
2017-11-14builddeb: Pass the kernel:debarch substvar to dpkg-genchangesSven Joachim1-2/+2
At the end of "make bindeb-pkg" I noticed the following warning: dpkg-genchanges: warning: unknown substitution variable ${kernel:debarch} It turns out that since dpkg version 1.19.0 dpkg-genchanges honors substitution variables in the Description field, while earlier versions silently left them alone, see https://bugs.debian.org/856547. The result is an incomplete description of the linux-headers package in the generated .changes file. Fix it by passing the kernel:debarch substitution variable to dpkg-genchanges. Signed-off-by: Sven Joachim <svenjoac@gmx.de> Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
2017-11-14kbuild: rpm-pkg: do not force -jN in submakeMasahiro Yamada1-1/+2
The spec file always passes %{?_smp_mflags}, but we have two problems here. [1] "make -jN rpm-pkg" emits the following warning message: make[2]: warning: -jN forced in submake: disabling jobserver mode. [2] We can not specify the number of jobs that run in parallel. Whether we give -jN or not from the top Makefile, the spec file always passes ${?_smp_mflags} to the build commands. ${?_smp_mflags} will be useful when we run rpmbuild by hand. When we invoke it from Makefile, -jN is propagated down to submake; it should not be overridden because we want to respect the number of jobs given by the user. Set _smp_mflags to empty string in this case. Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
2017-11-14kbuild: rpm-pkg: keep spec file until make mrproperMasahiro Yamada1-2/+2
If build fails during (bin)rpm-pkg, the spec file is not cleaned by anyone until the next successful build of the package. We do not have to immediately delete the spec file in case somebody may want to take a look at it. Instead, make them ignored by git, and cleaned up by make mrproper. Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
2017-11-14kbuild: rpm-pkg: fix jobserver unavailable warningMasahiro Yamada1-2/+2
If "make rpm-pkg" or "make binrpm-pkg" is run with -j[jobs] option, the following warning message is displayed. warning: jobserver unavailable: using -j1. Add '+' to parent make rule. Follow the suggestion. Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
2017-11-14kbuild: rpm-pkg: replace $RPM_BUILD_ROOT with %{buildroot}Masahiro Yamada1-16/+16
$RPM_BUILD_ROOT must be escaped to prevent shell from expanding it when generating the spec file. %{build_root} is more readable than \$RPM_BUILD_ROOT. Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
2017-11-14kbuild: rpm-pkg: fix build error when CONFIG_MODULES is disabledMasahiro Yamada1-25/+32
When CONFIG_MODULES is disabled, make rpm-pkg / binrpm-pkg fails with the following message: The present kernel configuration has modules disabled. Type 'make config' and enable loadable module support. Then build a kernel with module support enabled. Do not install modules in the case. Also, omit the devel package. Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
2017-11-14kbuild: rpm-pkg: refactor mkspec with here docMasahiro Yamada1-113/+105
The repeat of echo is unreadable. The here-document is a well-known device for such scripts. One difficulty is we have a bunch of PREBUILT conditionals that would split the here-document. My idea is to add "$S" annotatation to lines only for the source package spec file, then post-process it by sed. I hope it will make our life easier than repeat of "cat <<EOF ..." I confirmed this commit still produced the same (bin)kernel.spec as before. Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
2017-11-02License cleanup: add SPDX GPL-2.0 license identifier to files with no licenseGreg Kroah-Hartman1-0/+1
Many source files in the tree are missing licensing information, which makes it harder for compliance tools to determine the correct license. By default all files without license information are under the default license of the kernel, which is GPL version 2. Update the files which contain no license information with the 'GPL-2.0' SPDX license identifier. The SPDX identifier is a legally binding shorthand, which can be used instead of the full boiler plate text. This patch is based on work done by Thomas Gleixner and Kate Stewart and Philippe Ombredanne. How this work was done: Patches were generated and checked against linux-4.14-rc6 for a subset of the use cases: - file had no licensing information it it. - file was a */uapi/* one with no licensing information in it, - file was a */uapi/* one with existing licensing information, Further patches will be generated in subsequent months to fix up cases where non-standard license headers were used, and references to license had to be inferred by heuristics based on keywords. The analysis to determine which SPDX License Identifier to be applied to a file was done in a spreadsheet of side by side results from of the output of two independent scanners (ScanCode & Windriver) producing SPDX tag:value files created by Philippe Ombredanne. Philippe prepared the base worksheet, and did an initial spot review of a few 1000 files. The 4.13 kernel was the starting point of the analysis with 60,537 files assessed. Kate Stewart did a file by file comparison of the scanner results in the spreadsheet to determine which SPDX license identifier(s) to be applied to the file. She confirmed any determination that was not immediately clear with lawyers working with the Linux Foundation. Criteria used to select files for SPDX license identifier tagging was: - Files considered eligible had to be source code files. - Make and config files were included as candidates if they contained >5 lines of source - File already had some variant of a license header in it (even if <5 lines). All documentation files were explicitly excluded. The following heuristics were used to determine which SPDX license identifiers to apply. - when both scanners couldn't find any license traces, file was considered to have no license information in it, and the top level COPYING file license applied. For non */uapi/* files that summary was: SPDX license identifier # files ---------------------------------------------------|------- GPL-2.0 11139 and resulted in the first patch in this series. If that file was a */uapi/* path one, it was "GPL-2.0 WITH Linux-syscall-note" otherwise it was "GPL-2.0". Results of that was: SPDX license identifier # files ---------------------------------------------------|------- GPL-2.0 WITH Linux-syscall-note 930 and resulted in the second patch in this series. - if a file had some form of licensing information in it, and was one of the */uapi/* ones, it was denoted with the Linux-syscall-note if any GPL family license was found in the file or had no licensing in it (per prior point). Results summary: SPDX license identifier # files ---------------------------------------------------|------ GPL-2.0 WITH Linux-syscall-note 270 GPL-2.0+ WITH Linux-syscall-note 169 ((GPL-2.0 WITH Linux-syscall-note) OR BSD-2-Clause) 21 ((GPL-2.0 WITH Linux-syscall-note) OR BSD-3-Clause) 17 LGPL-2.1+ WITH Linux-syscall-note 15 GPL-1.0+ WITH Linux-syscall-note 14 ((GPL-2.0+ WITH Linux-syscall-note) OR BSD-3-Clause) 5 LGPL-2.0+ WITH Linux-syscall-note 4 LGPL-2.1 WITH Linux-syscall-note 3 ((GPL-2.0 WITH Linux-syscall-note) OR MIT) 3 ((GPL-2.0 WITH Linux-syscall-note) AND MIT) 1 and that resulted in the third patch in this series. - when the two scanners agreed on the detected license(s), that became the concluded license(s). - when there was disagreement between the two scanners (one detected a license but the other didn't, or they both detected different licenses) a manual inspection of the file occurred. - In most cases a manual inspection of the information in the file resulted in a clear resolution of the license that should apply (and which scanner probably needed to revisit its heuristics). - When it was not immediately clear, the license identifier was confirmed with lawyers working with the Linux Foundation. - If there was any question as to the appropriate license identifier, the file was flagged for further research and to be revisited later in time. In total, over 70 hours of logged manual review was done on the spreadsheet to determine the SPDX license identifiers to apply to the source files by Kate, Philippe, Thomas and, in some cases, confirmation by lawyers working with the Linux Foundation. Kate also obtained a third independent scan of the 4.13 code base from FOSSology, and compared selected files where the other two scanners disagreed against that SPDX file, to see if there was new insights. The Windriver scanner is based on an older version of FOSSology in part, so they are related. Thomas did random spot checks in about 500 files from the spreadsheets for the uapi headers and agreed with SPDX license identifier in the files he inspected. For the non-uapi files Thomas did random spot checks in about 15000 files. In initial set of patches against 4.14-rc6, 3 files were found to have copy/paste license identifier errors, and have been fixed to reflect the correct identifier. Additionally Philippe spent 10 hours this week doing a detailed manual inspection and review of the 12,461 patched files from the initial patch version early this week with: - a full scancode scan run, collecting the matched texts, detected license ids and scores - reviewing anything where there was a license detected (about 500+ files) to ensure that the applied SPDX license was correct - reviewing anything where there was no detection but the patch license was not GPL-2.0 WITH Linux-syscall-note to ensure that the applied SPDX license was correct This produced a worksheet with 20 files needing minor correction. This worksheet was then exported into 3 different .csv files for the different types of files to be modified. These .csv files were then reviewed by Greg. Thomas wrote a script to parse the csv files and add the proper SPDX tag to the file, in the format that the file expected. This script was further refined by Greg based on the output to detect more types of files automatically and to distinguish between header and source .c files (which need different comment types.) Finally Greg ran the script using the .csv files to generate the patches. Reviewed-by: Kate Stewart <kstewart@linuxfoundation.org> Reviewed-by: Philippe Ombredanne <pombredanne@nexb.com> Reviewed-by: Thomas Gleixner <tglx@linutronix.de> Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>