From 7e2d619411cbcb5ee6c6e97a53dcba605d276f44 Mon Sep 17 00:00:00 2001 From: Tales Aparecida Date: Sat, 13 Aug 2022 01:20:52 -0300 Subject: Documentation: kunit: fix trivial typo Missing closing block-quote Signed-off-by: Tales Aparecida Reviewed-by: David Gow Reviewed-by: Sadiya Kazi Signed-off-by: Shuah Khan --- Documentation/dev-tools/kunit/run_wrapper.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) (limited to 'Documentation/dev-tools') diff --git a/Documentation/dev-tools/kunit/run_wrapper.rst b/Documentation/dev-tools/kunit/run_wrapper.rst index cce203138fb7..db1e867820e7 100644 --- a/Documentation/dev-tools/kunit/run_wrapper.rst +++ b/Documentation/dev-tools/kunit/run_wrapper.rst @@ -30,7 +30,7 @@ We may want to use the following options: .. code-block:: - ./tools/testing/kunit/kunit.py run --timeout=30 --jobs=`nproc --all + ./tools/testing/kunit/kunit.py run --timeout=30 --jobs=`nproc --all` - ``--timeout`` sets a maximum amount of time for tests to run. - ``--jobs`` sets the number of threads to build the kernel. -- cgit v1.2.3 From 8ae4826363cedc84b6b4e18952a49793586fa92b Mon Sep 17 00:00:00 2001 From: Tales Aparecida Date: Sat, 13 Aug 2022 01:20:53 -0300 Subject: Documentation: Kunit: Fix inconsistent titles Use the same wording when citing and describing Kunit parts. Signed-off-by: Tales Aparecida Reviewed-by: David Gow Reviewed-by: Sadiya Kazi Signed-off-by: Shuah Khan --- Documentation/dev-tools/kunit/architecture.rst | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) (limited to 'Documentation/dev-tools') diff --git a/Documentation/dev-tools/kunit/architecture.rst b/Documentation/dev-tools/kunit/architecture.rst index cf9e6e3eeae4..8efe792bdcb9 100644 --- a/Documentation/dev-tools/kunit/architecture.rst +++ b/Documentation/dev-tools/kunit/architecture.rst @@ -6,8 +6,8 @@ KUnit Architecture The KUnit architecture can be divided into two parts: -- Kernel testing library -- kunit_tool (Command line test harness) +- `In-Kernel Testing Framework`_ +- `kunit_tool (Command Line Test Harness)`_ In-Kernel Testing Framework =========================== -- cgit v1.2.3 From 43ca52a910d15026b474160c2b625db5be687f51 Mon Sep 17 00:00:00 2001 From: Tales Aparecida Date: Sat, 13 Aug 2022 01:20:54 -0300 Subject: Documentation: KUnit: Fix non-uml anchor The section was rewritten but its anchor got left behind. Fix the anchor and add some references to running on QEMU. Signed-off-by: Tales Aparecida Reviewed-by: David Gow Signed-off-by: Shuah Khan --- Documentation/dev-tools/kunit/faq.rst | 6 +++--- Documentation/dev-tools/kunit/run_wrapper.rst | 2 ++ Documentation/dev-tools/kunit/usage.rst | 4 ++-- 3 files changed, 7 insertions(+), 5 deletions(-) (limited to 'Documentation/dev-tools') diff --git a/Documentation/dev-tools/kunit/faq.rst b/Documentation/dev-tools/kunit/faq.rst index 172e239791a8..f1b4cef68ced 100644 --- a/Documentation/dev-tools/kunit/faq.rst +++ b/Documentation/dev-tools/kunit/faq.rst @@ -31,7 +31,8 @@ For the most part, the KUnit core framework (what we use to write the tests) can compile to any architecture. It compiles like just another part of the kernel and runs when the kernel boots, or when built as a module, when the module is loaded. However, there is infrastructure, like the KUnit Wrapper -(``tools/testing/kunit/kunit.py``) that does not support other architectures. +(``tools/testing/kunit/kunit.py``) that might not support some architectures +(see :ref:`kunit-on-qemu`). In short, yes, you can run KUnit on other architectures, but it might require more work than using KUnit on UML. @@ -95,8 +96,7 @@ things to try. seeing. When tests are built-in, they will execute when the kernel boots, and modules will automatically execute associated tests when loaded. Test results can be collected from ``/sys/kernel/debug/kunit//results``, and - can be parsed with ``kunit.py parse``. For more details, see "KUnit on - non-UML architectures" in Documentation/dev-tools/kunit/usage.rst. + can be parsed with ``kunit.py parse``. For more details, see :ref:`kunit-on-qemu`. If none of the above tricks help, you are always welcome to email any issues to kunit-dev@googlegroups.com. diff --git a/Documentation/dev-tools/kunit/run_wrapper.rst b/Documentation/dev-tools/kunit/run_wrapper.rst index db1e867820e7..a1070def284f 100644 --- a/Documentation/dev-tools/kunit/run_wrapper.rst +++ b/Documentation/dev-tools/kunit/run_wrapper.rst @@ -165,6 +165,8 @@ example: if we only want to run KUnit resource tests, use: This uses the standard glob format with wildcard characters. +.. _kunit-on-qemu: + Run Tests on qemu ================= diff --git a/Documentation/dev-tools/kunit/usage.rst b/Documentation/dev-tools/kunit/usage.rst index 44158eecb51e..2737863ef365 100644 --- a/Documentation/dev-tools/kunit/usage.rst +++ b/Documentation/dev-tools/kunit/usage.rst @@ -165,6 +165,8 @@ built as a module). For more information, see Documentation/dev-tools/kunit/api/test.rst. +.. _kunit-on-non-uml: + Writing Tests For Other Architectures ------------------------------------- @@ -544,8 +546,6 @@ By reusing the same ``cases`` array from above, we can write the test as a {} }; -.. _kunit-on-non-uml: - Exiting Early on Failed Expectations ------------------------------------ -- cgit v1.2.3 From 3920c725cbff5c6db69d7077716987a93b0e6a9a Mon Sep 17 00:00:00 2001 From: Tales Aparecida Date: Sat, 13 Aug 2022 01:20:55 -0300 Subject: Documentation: Kunit: Add ref for other kinds of tests Add an organic link to the "other kinds of tests" in the index page Signed-off-by: Tales Aparecida Reviewed-by: David Gow Signed-off-by: Shuah Khan --- Documentation/dev-tools/kunit/faq.rst | 2 ++ Documentation/dev-tools/kunit/index.rst | 2 ++ 2 files changed, 4 insertions(+) (limited to 'Documentation/dev-tools') diff --git a/Documentation/dev-tools/kunit/faq.rst b/Documentation/dev-tools/kunit/faq.rst index f1b4cef68ced..fae426f2634a 100644 --- a/Documentation/dev-tools/kunit/faq.rst +++ b/Documentation/dev-tools/kunit/faq.rst @@ -39,6 +39,8 @@ more work than using KUnit on UML. For more information, see :ref:`kunit-on-non-uml`. +.. _kinds-of-tests: + What is the difference between a unit test and other kinds of tests? ==================================================================== Most existing tests for the Linux kernel would be categorized as an integration diff --git a/Documentation/dev-tools/kunit/index.rst b/Documentation/dev-tools/kunit/index.rst index 595205348d2d..bc91ad7b8961 100644 --- a/Documentation/dev-tools/kunit/index.rst +++ b/Documentation/dev-tools/kunit/index.rst @@ -95,6 +95,8 @@ Unit Testing Advantages - Improves code quality. - Encourages writing testable code. +Read also :ref:`kinds-of-tests`. + How do I use it? ================ -- cgit v1.2.3 From 2327f7e956bab3e25f472e8c9d0aa6380970a91a Mon Sep 17 00:00:00 2001 From: Tales Aparecida Date: Sun, 21 Aug 2022 23:26:39 -0300 Subject: Documentation: KUnit: remove duplicated docs for kunit_tool Delete "kunit-tool.rst" to remove repeated info from KUnit docs. "What is kunit_tool?" was integrated into index.rst, the remaining sections were moved into run_wrapper.rst and renamed as follows: "What is a .kunitconfig?" -> "Creating a ``.kunitconfig`` file" "Getting Started with kunit_tool" -> "Running tests with kunit_tool" "Configuring, Building, and Running Tests" -> "Configuring, building, and running tests" "Running Tests on QEMU" -> "Running tests on QEMU" "Parsing Test Results" -> "Parsing test results" "Filtering Tests" -> "Filtering tests" "Other Useful Options" -> "Running command-line arguments" Signed-off-by: Tales Aparecida Reviewed-by: Sadiya Kazi Reviewed-by: David Gow Signed-off-by: Shuah Khan --- Documentation/dev-tools/kunit/index.rst | 3 - Documentation/dev-tools/kunit/kunit-tool.rst | 232 -------------------------- Documentation/dev-tools/kunit/run_wrapper.rst | 32 ++-- Documentation/dev-tools/kunit/start.rst | 2 - 4 files changed, 16 insertions(+), 253 deletions(-) delete mode 100644 Documentation/dev-tools/kunit/kunit-tool.rst (limited to 'Documentation/dev-tools') diff --git a/Documentation/dev-tools/kunit/index.rst b/Documentation/dev-tools/kunit/index.rst index bc91ad7b8961..d7187282ba28 100644 --- a/Documentation/dev-tools/kunit/index.rst +++ b/Documentation/dev-tools/kunit/index.rst @@ -13,7 +13,6 @@ KUnit - Linux Kernel Unit Testing run_wrapper run_manual usage - kunit-tool api/index style faq @@ -109,7 +108,5 @@ How do I use it? examples. * Documentation/dev-tools/kunit/api/index.rst - KUnit APIs used for testing. -* Documentation/dev-tools/kunit/kunit-tool.rst - kunit_tool helper - script. * Documentation/dev-tools/kunit/faq.rst - KUnit common questions and answers. diff --git a/Documentation/dev-tools/kunit/kunit-tool.rst b/Documentation/dev-tools/kunit/kunit-tool.rst deleted file mode 100644 index ae52e0f489f9..000000000000 --- a/Documentation/dev-tools/kunit/kunit-tool.rst +++ /dev/null @@ -1,232 +0,0 @@ -.. SPDX-License-Identifier: GPL-2.0 - -================= -kunit_tool How-To -================= - -What is kunit_tool? -=================== - -kunit_tool is a script (``tools/testing/kunit/kunit.py``) that aids in building -the Linux kernel as UML (`User Mode Linux -`_), running KUnit tests, parsing -the test results and displaying them in a user friendly manner. - -kunit_tool addresses the problem of being able to run tests without needing a -virtual machine or actual hardware with User Mode Linux. User Mode Linux is a -Linux architecture, like ARM or x86; however, unlike other architectures it -compiles the kernel as a standalone Linux executable that can be run like any -other program directly inside of a host operating system. To be clear, it does -not require any virtualization support: it is just a regular program. - -What is a .kunitconfig? -======================= - -It's just a defconfig that kunit_tool looks for in the build directory -(``.kunit`` by default). kunit_tool uses it to generate a .config as you might -expect. In addition, it verifies that the generated .config contains the CONFIG -options in the .kunitconfig; the reason it does this is so that it is easy to -be sure that a CONFIG that enables a test actually ends up in the .config. - -It's also possible to pass a separate .kunitconfig fragment to kunit_tool, -which is useful if you have several different groups of tests you wish -to run independently, or if you want to use pre-defined test configs for -certain subsystems. - -Getting Started with kunit_tool -=============================== - -If a kunitconfig is present at the root directory, all you have to do is: - -.. code-block:: bash - - ./tools/testing/kunit/kunit.py run - -However, you most likely want to use it with the following options: - -.. code-block:: bash - - ./tools/testing/kunit/kunit.py run --timeout=30 --jobs=`nproc --all` - -- ``--timeout`` sets a maximum amount of time to allow tests to run. -- ``--jobs`` sets the number of threads to use to build the kernel. - -.. note:: - This command will work even without a .kunitconfig file: if no - .kunitconfig is present, a default one will be used instead. - -If you wish to use a different .kunitconfig file (such as one provided for -testing a particular subsystem), you can pass it as an option. - -.. code-block:: bash - - ./tools/testing/kunit/kunit.py run --kunitconfig=fs/ext4/.kunitconfig - -For a list of all the flags supported by kunit_tool, you can run: - -.. code-block:: bash - - ./tools/testing/kunit/kunit.py run --help - -Configuring, Building, and Running Tests -======================================== - -It's also possible to run just parts of the KUnit build process independently, -which is useful if you want to make manual changes to part of the process. - -A .config can be generated from a .kunitconfig by using the ``config`` argument -when running kunit_tool: - -.. code-block:: bash - - ./tools/testing/kunit/kunit.py config - -Similarly, if you just want to build a KUnit kernel from the current .config, -you can use the ``build`` argument: - -.. code-block:: bash - - ./tools/testing/kunit/kunit.py build - -And, if you already have a built UML kernel with built-in KUnit tests, you can -run the kernel and display the test results with the ``exec`` argument: - -.. code-block:: bash - - ./tools/testing/kunit/kunit.py exec - -The ``run`` command which is discussed above is equivalent to running all three -of these in sequence. - -All of these commands accept a number of optional command-line arguments. The -``--help`` flag will give a complete list of these, or keep reading this page -for a guide to some of the more useful ones. - -Parsing Test Results -==================== - -KUnit tests output their results in TAP (Test Anything Protocol) format. -kunit_tool will, when running tests, parse this output and print a summary -which is much more pleasant to read. If you wish to look at the raw test -results in TAP format, you can pass the ``--raw_output`` argument. - -.. code-block:: bash - - ./tools/testing/kunit/kunit.py run --raw_output - -The raw output from test runs may contain other, non-KUnit kernel log -lines. You can see just KUnit output with ``--raw_output=kunit``: - -.. code-block:: bash - - ./tools/testing/kunit/kunit.py run --raw_output=kunit - -If you have KUnit results in their raw TAP format, you can parse them and print -the human-readable summary with the ``parse`` command for kunit_tool. This -accepts a filename for an argument, or will read from standard input. - -.. code-block:: bash - - # Reading from a file - ./tools/testing/kunit/kunit.py parse /var/log/dmesg - # Reading from stdin - dmesg | ./tools/testing/kunit/kunit.py parse - -This is very useful if you wish to run tests in a configuration not supported -by kunit_tool (such as on real hardware, or an unsupported architecture). - -Filtering Tests -=============== - -It's possible to run only a subset of the tests built into a kernel by passing -a filter to the ``exec`` or ``run`` commands. For example, if you only wanted -to run KUnit resource tests, you could use: - -.. code-block:: bash - - ./tools/testing/kunit/kunit.py run 'kunit-resource*' - -This uses the standard glob format for wildcards. - -Running Tests on QEMU -===================== - -kunit_tool supports running tests on QEMU as well as via UML (as mentioned -elsewhere). The default way of running tests on QEMU requires two flags: - -``--arch`` - Selects a collection of configs (Kconfig as well as QEMU configs - options, etc) that allow KUnit tests to be run on the specified - architecture in a minimal way; this is usually not much slower than - using UML. The architecture argument is the same as the name of the - option passed to the ``ARCH`` variable used by Kbuild. Not all - architectures are currently supported by this flag, but can be handled - by the ``--qemu_config`` discussed later. If ``um`` is passed (or this - this flag is ignored) the tests will run via UML. Non-UML architectures, - e.g. i386, x86_64, arm, um, etc. Non-UML run on QEMU. - -``--cross_compile`` - Specifies the use of a toolchain by Kbuild. The argument passed here is - the same passed to the ``CROSS_COMPILE`` variable used by Kbuild. As a - reminder this will be the prefix for the toolchain binaries such as gcc - for example ``sparc64-linux-gnu-`` if you have the sparc toolchain - installed on your system, or - ``$HOME/toolchains/microblaze/gcc-9.2.0-nolibc/microblaze-linux/bin/microblaze-linux-`` - if you have downloaded the microblaze toolchain from the 0-day website - to a directory in your home directory called ``toolchains``. - -In many cases it is likely that you may want to run an architecture which is -not supported by the ``--arch`` flag, or you may want to just run KUnit tests -on QEMU using a non-default configuration. For this use case, you can write -your own QemuConfig. These QemuConfigs are written in Python. They must have an -import line ``from ..qemu_config import QemuArchParams`` at the top of the file -and the file must contain a variable called ``QEMU_ARCH`` that has an instance -of ``QemuArchParams`` assigned to it. An example can be seen in -``tools/testing/kunit/qemu_configs/x86_64.py``. - -Once you have a QemuConfig you can pass it into kunit_tool using the -``--qemu_config`` flag; when used this flag replaces the ``--arch`` flag. If we -were to do this with the ``x86_64.py`` example from above, the invocation would -look something like this: - -.. code-block:: bash - - ./tools/testing/kunit/kunit.py run \ - --timeout=60 \ - --jobs=12 \ - --qemu_config=./tools/testing/kunit/qemu_configs/x86_64.py - -Other Useful Options -==================== - -kunit_tool has a number of other command-line arguments which can be useful -when adapting it to fit your environment or needs. - -Some of the more useful ones are: - -``--help`` - Lists all of the available options. Note that different commands - (``config``, ``build``, ``run``, etc) will have different supported - options. Place ``--help`` before the command to list common options, - and after the command for options specific to that command. - -``--build_dir`` - Specifies the build directory that kunit_tool will use. This is where - the .kunitconfig file is located, as well as where the .config and - compiled kernel will be placed. Defaults to ``.kunit``. - -``--make_options`` - Specifies additional options to pass to ``make`` when compiling a - kernel (with the ``build`` or ``run`` commands). For example, to enable - compiler warnings, you can pass ``--make_options W=1``. - -``--alltests`` - Builds a UML kernel with all config options enabled using ``make - allyesconfig``. This allows you to run as many tests as is possible, - but is very slow and prone to breakage as new options are added or - modified. In most cases, enabling all tests which have satisfied - dependencies by adding ``CONFIG_KUNIT_ALL_TESTS=1`` to your - .kunitconfig is preferable. - -There are several other options (and new ones are often added), so do check -``--help`` if you're looking for something not mentioned here. diff --git a/Documentation/dev-tools/kunit/run_wrapper.rst b/Documentation/dev-tools/kunit/run_wrapper.rst index a1070def284f..518cf87ea732 100644 --- a/Documentation/dev-tools/kunit/run_wrapper.rst +++ b/Documentation/dev-tools/kunit/run_wrapper.rst @@ -1,8 +1,8 @@ .. SPDX-License-Identifier: GPL-2.0 -========================= -Run Tests with kunit_tool -========================= +============================= +Running tests with kunit_tool +============================= We can either run KUnit tests using kunit_tool or can run tests manually, and then use kunit_tool to parse the results. To run tests @@ -58,8 +58,8 @@ To view kunit_tool flags (optional command-line arguments), run: ./tools/testing/kunit/kunit.py run --help -Create a ``.kunitconfig`` File -=============================== +Creating a ``.kunitconfig`` file +================================ If we want to run a specific set of tests (rather than those listed in the KUnit ``defconfig``), we can provide Kconfig options in the @@ -98,8 +98,8 @@ have not included the options dependencies. The build dir needs to be set for ``make menuconfig`` to work, therefore by default use ``make O=.kunit menuconfig``. -Configure, Build, and Run Tests -=============================== +Configuring, building, and running tests +======================================== If we want to make manual changes to the KUnit build process, we can run part of the KUnit build process independently. @@ -125,11 +125,11 @@ argument: ./tools/testing/kunit/kunit.py exec -The ``run`` command discussed in section: **Run Tests with kunit_tool**, +The ``run`` command discussed in section: **Running tests with kunit_tool**, is equivalent to running the above three commands in sequence. -Parse Test Results -================== +Parsing test results +==================== KUnit tests output displays results in TAP (Test Anything Protocol) format. When running tests, kunit_tool parses this output and prints @@ -152,8 +152,8 @@ standard input. # Reading from stdin dmesg | ./tools/testing/kunit/kunit.py parse -Run Selected Test Suites -======================== +Filtering tests +=============== By passing a bash style glob filter to the ``exec`` or ``run`` commands, we can run a subset of the tests built into a kernel . For @@ -167,8 +167,8 @@ This uses the standard glob format with wildcard characters. .. _kunit-on-qemu: -Run Tests on qemu -================= +Running tests on QEMU +===================== kunit_tool supports running tests on qemu as well as via UML. To run tests on qemu, by default it requires two flags: @@ -231,8 +231,8 @@ as --jobs=12 \ --qemu_config=./tools/testing/kunit/qemu_configs/x86_64.py -Command-Line Arguments -====================== +Running command-line arguments +============================== kunit_tool has a number of other command-line arguments which can be useful for our test environment. Below are the most commonly used diff --git a/Documentation/dev-tools/kunit/start.rst b/Documentation/dev-tools/kunit/start.rst index 867a4bba6bf6..e730df1f468e 100644 --- a/Documentation/dev-tools/kunit/start.rst +++ b/Documentation/dev-tools/kunit/start.rst @@ -254,7 +254,5 @@ Next Steps examples. * Documentation/dev-tools/kunit/api/index.rst - KUnit APIs used for testing. -* Documentation/dev-tools/kunit/kunit-tool.rst - kunit_tool helper - script. * Documentation/dev-tools/kunit/faq.rst - KUnit common questions and answers. -- cgit v1.2.3 From 5ceb9a2558593658276854770f0d4e84fbcddf14 Mon Sep 17 00:00:00 2001 From: Tales Aparecida Date: Sun, 21 Aug 2022 23:26:40 -0300 Subject: Documentation: KUnit: avoid repeating "kunit.py run" in start.rst MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Combine two sections mentioning "kunit.py run" to streamline the getting-started guide. Update "kunit.py run" expected output in the guide and run_wrapper. Signed-off-by: Tales Aparecida Reviewed-by: Maíra Canal Reviewed-by: Sadiya Kazi Reviewed-by: David Gow Signed-off-by: Shuah Khan --- Documentation/dev-tools/kunit/run_wrapper.rst | 2 +- Documentation/dev-tools/kunit/start.rst | 38 +++++++++++---------------- 2 files changed, 16 insertions(+), 24 deletions(-) (limited to 'Documentation/dev-tools') diff --git a/Documentation/dev-tools/kunit/run_wrapper.rst b/Documentation/dev-tools/kunit/run_wrapper.rst index 518cf87ea732..6b33caf6c8ab 100644 --- a/Documentation/dev-tools/kunit/run_wrapper.rst +++ b/Documentation/dev-tools/kunit/run_wrapper.rst @@ -22,7 +22,7 @@ We should see the following: .. code-block:: - Generating .config... + Configuring KUnit Kernel ... Building KUnit kernel... Starting KUnit kernel... diff --git a/Documentation/dev-tools/kunit/start.rst b/Documentation/dev-tools/kunit/start.rst index e730df1f468e..2e31350a85e1 100644 --- a/Documentation/dev-tools/kunit/start.rst +++ b/Documentation/dev-tools/kunit/start.rst @@ -19,7 +19,21 @@ can run kunit_tool: ./tools/testing/kunit/kunit.py run -For more information on this wrapper, see: +If everything worked correctly, you should see the following: + +.. code-block:: + + Configuring KUnit Kernel ... + Building KUnit Kernel ... + Starting KUnit Kernel ... + +The tests will pass or fail. + +.. note :: + Because it is building a lot of sources for the first time, + the ``Building KUnit Kernel`` step may take a while. + +For detailed information on this wrapper, see: Documentation/dev-tools/kunit/run_wrapper.rst. Creating a ``.kunitconfig`` @@ -74,28 +88,6 @@ you if you have not included dependencies for the options used. tools like ``make menuconfig O=.kunit``. As long as its a superset of ``.kunitconfig``, kunit.py won't overwrite your changes. -Running Tests (KUnit Wrapper) ------------------------------ -1. To make sure that everything is set up correctly, invoke the Python - wrapper from your kernel repository: - -.. code-block:: bash - - ./tools/testing/kunit/kunit.py run - -If everything worked correctly, you should see the following: - -.. code-block:: - - Generating .config ... - Building KUnit Kernel ... - Starting KUnit Kernel ... - -The tests will pass or fail. - -.. note :: - Because it is building a lot of sources for the first time, the - ``Building KUnit kernel`` may take a while. Running Tests without the KUnit Wrapper ======================================= -- cgit v1.2.3 From c5b6c4ecbd95c8e56ef1350ab091e57e401b01fe Mon Sep 17 00:00:00 2001 From: Tales Aparecida Date: Sun, 21 Aug 2022 23:26:41 -0300 Subject: Documentation: KUnit: add note about mrproper in start.rst The "Getting Started" guide should be beginner-friendly, therefore add a note about the requirement of a clean source tree when running kunit_tool for the first time, and its related error. Signed-off-by: Tales Aparecida Reviewed-by: David Gow Signed-off-by: Shuah Khan --- Documentation/dev-tools/kunit/start.rst | 16 ++++++++++++++++ 1 file changed, 16 insertions(+) (limited to 'Documentation/dev-tools') diff --git a/Documentation/dev-tools/kunit/start.rst b/Documentation/dev-tools/kunit/start.rst index 2e31350a85e1..9beec7d6ac4b 100644 --- a/Documentation/dev-tools/kunit/start.rst +++ b/Documentation/dev-tools/kunit/start.rst @@ -19,6 +19,22 @@ can run kunit_tool: ./tools/testing/kunit/kunit.py run +.. note :: + You may see the following error: + "The source tree is not clean, please run 'make ARCH=um mrproper'" + + This happens because internally kunit.py specifies ``.kunit`` + (default option) as the build directory in the command ``make O=output/dir`` + through the argument ``--build_dir``. Hence, before starting an + out-of-tree build, the source tree must be clean. + + There is also the same caveat mentioned in the "Build directory for + the kernel" section of the :doc:`admin-guide `, + that is, its use, it must be used for all invocations of ``make``. + The good news is that it can indeed be solved by running + ``make ARCH=um mrproper``, just be aware that this will delete the + current configuration and all generated files. + If everything worked correctly, you should see the following: .. code-block:: -- cgit v1.2.3 From 7a79b7df4e5122ae3b5b5f2bd5b52ecb1295398a Mon Sep 17 00:00:00 2001 From: Tales Aparecida Date: Sun, 21 Aug 2022 23:26:42 -0300 Subject: Documentation: KUnit: Reword start guide for selecting tests MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Reword "Creating a ``.kunitconfig``" into "Selecting which tests to run" covering the current alternatives for editing configs and glob-filtering Signed-off-by: Tales Aparecida Reviewed-by: Maíra Canal Reviewed-by:Sadiya Kazi Reviewed-by: David Gow Signed-off-by: Shuah Khan --- Documentation/dev-tools/kunit/start.rst | 90 +++++++++++++++++++++++---------- 1 file changed, 63 insertions(+), 27 deletions(-) (limited to 'Documentation/dev-tools') diff --git a/Documentation/dev-tools/kunit/start.rst b/Documentation/dev-tools/kunit/start.rst index 9beec7d6ac4b..adf782507999 100644 --- a/Documentation/dev-tools/kunit/start.rst +++ b/Documentation/dev-tools/kunit/start.rst @@ -52,27 +52,20 @@ The tests will pass or fail. For detailed information on this wrapper, see: Documentation/dev-tools/kunit/run_wrapper.rst. -Creating a ``.kunitconfig`` ---------------------------- - -By default, kunit_tool runs a selection of tests. However, you can specify which -unit tests to run by creating a ``.kunitconfig`` file with kernel config options -that enable only a specific set of tests and their dependencies. -The ``.kunitconfig`` file contains a list of kconfig options which are required -to run the desired targets. The ``.kunitconfig`` also contains any other test -specific config options, such as test dependencies. For example: the -``FAT_FS`` tests - ``FAT_KUNIT_TEST``, depends on -``FAT_FS``. ``FAT_FS`` can be enabled by selecting either ``MSDOS_FS`` -or ``VFAT_FS``. To run ``FAT_KUNIT_TEST``, the ``.kunitconfig`` has: +Selecting which tests to run +---------------------------- -.. code-block:: none +By default, kunit_tool runs all tests reachable with minimal configuration, +that is, using default values for most of the kconfig options. However, +you can select which tests to run by: - CONFIG_KUNIT=y - CONFIG_MSDOS_FS=y - CONFIG_FAT_KUNIT_TEST=y +- `Customizing Kconfig`_ used to compile the kernel, or +- `Filtering tests by name`_ to select specifically which compiled tests to run. -1. A good starting point for the ``.kunitconfig`` is the KUnit default config. - You can generate it by running: +Customizing Kconfig +~~~~~~~~~~~~~~~~~~~ +A good starting point for the ``.kunitconfig`` is the KUnit default config. +If you didn't run ``kunit.py run`` yet, you can generate it by running: .. code-block:: bash @@ -84,27 +77,70 @@ or ``VFAT_FS``. To run ``FAT_KUNIT_TEST``, the ``.kunitconfig`` has: ``.kunitconfig`` lives in the ``--build_dir`` used by kunit.py, which is ``.kunit`` by default. -.. note :: +Before running the tests, kunit_tool ensures that all config options +set in ``.kunitconfig`` are set in the kernel ``.config``. It will warn +you if you have not included dependencies for the options used. + +There are many ways to customize the configurations: + +a. Edit ``.kunit/.kunitconfig``. The file should contain the list of kconfig + options required to run the desired tests, including their dependencies. You may want to remove CONFIG_KUNIT_ALL_TESTS from the ``.kunitconfig`` as it will enable a number of additional tests that you may not want. + If you need to run on an architecture other than UML see :ref:`kunit-on-qemu`. -2. You can then add any other Kconfig options, for example: +b. Enable additional kconfig options on top of ``.kunit/.kunitconfig``. + For example, to include the kernel's linked-list test you can run:: -.. code-block:: none + ./tools/testing/kunit/kunit.py run \ + --kconfig_add CONFIG_LIST_KUNIT_TEST=y - CONFIG_LIST_KUNIT_TEST=y +c. Provide the path of one or more .kunitconfig files from the tree. + For example, to run only ``FAT_FS`` and ``EXT4`` tests you can run:: -Before running the tests, kunit_tool ensures that all config options -set in ``.kunitconfig`` are set in the kernel ``.config``. It will warn -you if you have not included dependencies for the options used. + ./tools/testing/kunit/kunit.py run \ + --kunitconfig ./fs/fat/.kunitconfig \ + --kunitconfig ./fs/ext4/.kunitconfig -.. note :: - If you change the ``.kunitconfig``, kunit.py will trigger a rebuild of the +d. If you change the ``.kunitconfig``, kunit.py will trigger a rebuild of the ``.config`` file. But you can edit the ``.config`` file directly or with tools like ``make menuconfig O=.kunit``. As long as its a superset of ``.kunitconfig``, kunit.py won't overwrite your changes. +.. note :: + + To save a .kunitconfig after finding a satisfactory configuration:: + + make savedefconfig O=.kunit + cp .kunit/defconfig .kunit/.kunitconfig + +Filtering tests by name +~~~~~~~~~~~~~~~~~~~~~~~ +If you want to be more specific than Kconfig can provide, it is also possible +to select which tests to execute at boot-time by passing a glob filter +(read instructions regarding the pattern in the manpage :manpage:`glob(7)`). +If there is a ``"."`` (period) in the filter, it will be interpreted as a +separator between the name of the test suite and the test case, +otherwise, it will be interpreted as the name of the test suite. +For example, let's assume we are using the default config: + +a. inform the name of a test suite, like ``"kunit_executor_test"``, + to run every test case it contains:: + + ./tools/testing/kunit/kunit.py run "kunit_executor_test" + +b. inform the name of a test case prefixed by its test suite, + like ``"example.example_simple_test"``, to run specifically that test case:: + + ./tools/testing/kunit/kunit.py run "example.example_simple_test" + +c. use wildcard characters (``*?[``) to run any test case that matches the pattern, + like ``"*.*64*"`` to run test cases containing ``"64"`` in the name inside + any test suite:: + + ./tools/testing/kunit/kunit.py run "*.*64*" + Running Tests without the KUnit Wrapper ======================================= If you do not want to use the KUnit Wrapper (for example: you want code -- cgit v1.2.3 From deff8d2955aa045d7053a7770cdae9874af6f474 Mon Sep 17 00:00:00 2001 From: Tales Aparecida Date: Sun, 21 Aug 2022 23:26:43 -0300 Subject: Documentation: KUnit: add intro to the getting-started page MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Describe the objective of the Getting Started page, which should be a brief and beginner-friendly walkthrough for running and writing tests, showing the reader where to find detailed instructions in other pages. Signed-off-by: Tales Aparecida Reviewed-by: Maíra Canal Reviewed-by:Sadiya Kazi Reviewed-by: David Gow Signed-off-by: Shuah Khan --- Documentation/dev-tools/kunit/start.rst | 4 ++++ 1 file changed, 4 insertions(+) (limited to 'Documentation/dev-tools') diff --git a/Documentation/dev-tools/kunit/start.rst b/Documentation/dev-tools/kunit/start.rst index adf782507999..75fd05286396 100644 --- a/Documentation/dev-tools/kunit/start.rst +++ b/Documentation/dev-tools/kunit/start.rst @@ -4,6 +4,10 @@ Getting Started =============== +This page contains an overview of the kunit_tool and KUnit framework, +teaching how to run existing tests and then how to write a simple test case, +and covers common problems users face when using KUnit for the first time. + Installing Dependencies ======================= KUnit has the same dependencies as the Linux kernel. As long as you can -- cgit v1.2.3 From 79c18e37c4e645560f3952bf177a10b43e94bf88 Mon Sep 17 00:00:00 2001 From: Tales Aparecida Date: Sun, 21 Aug 2022 23:26:44 -0300 Subject: Documentation: KUnit: update links in the index page Replace out-of-date external links with references to the kernel documentation, replacing TAP webpage for the more appropriate KTAP documentation and the UML webpage by its documentation. Signed-off-by: Tales Aparecida Reviewed-by: David Gow Signed-off-by: Shuah Khan --- Documentation/dev-tools/kunit/index.rst | 13 +++++++------ 1 file changed, 7 insertions(+), 6 deletions(-) (limited to 'Documentation/dev-tools') diff --git a/Documentation/dev-tools/kunit/index.rst b/Documentation/dev-tools/kunit/index.rst index d7187282ba28..f5d13f1d37be 100644 --- a/Documentation/dev-tools/kunit/index.rst +++ b/Documentation/dev-tools/kunit/index.rst @@ -28,10 +28,10 @@ KUnit (Kernel unit testing framework) provides a common framework for unit tests within the Linux kernel. Using KUnit, you can define groups of test cases called test suites. The tests either run on kernel boot if built-in, or load as a module. KUnit automatically flags and reports -failed test cases in the kernel log. The test results appear in `TAP -(Test Anything Protocol) format `_. It is inspired by -JUnit, Python’s unittest.mock, and GoogleTest/GoogleMock (C++ unit testing -framework). +failed test cases in the kernel log. The test results appear in +:doc:`KTAP (Kernel - Test Anything Protocol) format`. +It is inspired by JUnit, Python’s unittest.mock, and GoogleTest/GoogleMock +(C++ unit testing framework). KUnit tests are part of the kernel, written in the C (programming) language, and test parts of the Kernel implementation (example: a C @@ -45,8 +45,9 @@ internal system functionality. KUnit runs in kernel space and is not restricted to things exposed to user-space. In addition, KUnit has kunit_tool, a script (``tools/testing/kunit/kunit.py``) -that configures the Linux kernel, runs KUnit tests under QEMU or UML (`User Mode -Linux `_), parses the test results and +that configures the Linux kernel, runs KUnit tests under QEMU or UML +(:doc:`User Mode Linux `), +parses the test results and displays them in a user friendly manner. Features -- cgit v1.2.3 From 4e37057387cca749b7fbc8c77e3d86605117fffd Mon Sep 17 00:00:00 2001 From: Khalid Masum Date: Wed, 28 Sep 2022 10:00:58 +0600 Subject: Documentation: Kunit: Use full path to .kunitconfig The numbered list contains full path to every files that need to be modified or created in order to implement misc-example kunit test. Except for .kunitconfig. Which might make a newcommer confused about where the file exists. Since there are multiple .kunitconfig files. Fix this by using the full path to .kunitconfig. Signed-off-by: Khalid Masum Reviewed-by: David Gow Signed-off-by: Shuah Khan --- Documentation/dev-tools/kunit/start.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) (limited to 'Documentation/dev-tools') diff --git a/Documentation/dev-tools/kunit/start.rst b/Documentation/dev-tools/kunit/start.rst index 75fd05286396..f4f504f1fb15 100644 --- a/Documentation/dev-tools/kunit/start.rst +++ b/Documentation/dev-tools/kunit/start.rst @@ -265,7 +265,7 @@ Now we are ready to write the test cases. obj-$(CONFIG_MISC_EXAMPLE_TEST) += example_test.o -4. Add the following lines to ``.kunitconfig``: +4. Add the following lines to ``.kunit/.kunitconfig``: .. code-block:: none -- cgit v1.2.3