From a693396fd569c0a33e4b0f82398fbd6410a7beec Mon Sep 17 00:00:00 2001 From: Frank Rowand Date: Thu, 10 Feb 2022 17:36:30 -0600 Subject: Documentation: dev-tools: clarify KTAP specification wording Add the spec version to the title line. Explain likely source of "Unknown lines". "Unknown lines" in nested tests are optionally indented. Add "Unknown lines" items to differences between TAP & KTAP list Convert "Major differences between TAP and KTAP" from a bullet list to a table. The bullet list was being formatted as a single paragraph. Reviewed-by: Tim Bird Reviewed-by: David Gow Reviewed-by: Shuah Khan Signed-off-by: Frank Rowand Reviewed-by: Brendan Higgins Link: https://lore.kernel.org/r/20220210233630.3304495-1-frowand.list@gmail.com Signed-off-by: Jonathan Corbet --- Documentation/dev-tools/ktap.rst | 31 +++++++++++++++++++++++-------- 1 file changed, 23 insertions(+), 8 deletions(-) (limited to 'Documentation/dev-tools') diff --git a/Documentation/dev-tools/ktap.rst b/Documentation/dev-tools/ktap.rst index 878530cb9c27..d7fe05de40b4 100644 --- a/Documentation/dev-tools/ktap.rst +++ b/Documentation/dev-tools/ktap.rst @@ -1,8 +1,8 @@ .. SPDX-License-Identifier: GPL-2.0 -======================================== -The Kernel Test Anything Protocol (KTAP) -======================================== +=================================================== +The Kernel Test Anything Protocol (KTAP), version 1 +=================================================== TAP, or the Test Anything Protocol is a format for specifying test results used by a number of projects. It's website and specification are found at this `link @@ -174,6 +174,13 @@ There may be lines within KTAP output that do not follow the format of one of the four formats for lines described above. This is allowed, however, they will not influence the status of the tests. +This is an important difference from TAP. Kernel tests may print messages +to the system console or a log file. Both of these destinations may contain +messages either from unrelated kernel or userspace activity, or kernel +messages from non-test code that is invoked by the test. The kernel code +invoked by the test likely is not aware that a test is in progress and +thus can not print the message as a diagnostic message. + Nested tests ------------ @@ -186,10 +193,13 @@ starting with another KTAP version line and test plan, and end with the overall result. If one of the subtests fail, for example, the parent test should also fail. -Additionally, all result lines in a subtest should be indented. One level of +Additionally, all lines in a subtest should be indented. One level of indentation is two spaces: " ". The indentation should begin at the version line and should end before the parent test's result line. +"Unknown lines" are not considered to be lines in a subtest and thus are +allowed to be either indented or not indented. + An example of a test with two nested subtests: .. code-block:: @@ -224,10 +234,15 @@ An example format with multiple levels of nested testing: Major differences between TAP and KTAP -------------------------------------- -Note the major differences between the TAP and KTAP specification: -- yaml and json are not recommended in diagnostic messages -- TODO directive not recognized -- KTAP allows for an arbitrary number of tests to be nested +================================================== ========= =============== +Feature TAP KTAP +================================================== ========= =============== +yaml and json in diagnosic message ok not recommended +TODO directive ok not recognized +allows an arbitrary number of tests to be nested no yes +"Unknown lines" are in category of "Anything else" yes no +"Unknown lines" are incorrect allowed +================================================== ========= =============== The TAP14 specification does permit nested tests, but instead of using another nested version line, uses a line of the form -- cgit v1.2.3 From ff1368763bb488783820c6ba68d73c03c562a9b7 Mon Sep 17 00:00:00 2001 From: Randy Dunlap Date: Sun, 30 Jan 2022 16:36:37 -0800 Subject: Docs: ktap: add code-block type Fix multiple "code-block::" warnings by adding "none" as the type of code-block. Mends these warnings: Documentation/dev-tools/ktap.rst:71: WARNING: Error in "code-block" directive: 1 argument(s) required, 0 supplied. Documentation/dev-tools/ktap.rst:120: WARNING: Error in "code-block" directive: 1 argument(s) required, 0 supplied. Documentation/dev-tools/ktap.rst:126: WARNING: Error in "code-block" directive: 1 argument(s) required, 0 supplied. Documentation/dev-tools/ktap.rst:132: WARNING: Error in "code-block" directive: 1 argument(s) required, 0 supplied. Documentation/dev-tools/ktap.rst:139: WARNING: Error in "code-block" directive: 1 argument(s) required, 0 supplied. Documentation/dev-tools/ktap.rst:145: WARNING: Error in "code-block" directive: 1 argument(s) required, 0 supplied. Documentation/dev-tools/ktap.rst:195: WARNING: Error in "code-block" directive: 1 argument(s) required, 0 supplied. Documentation/dev-tools/ktap.rst:208: WARNING: Error in "code-block" directive: 1 argument(s) required, 0 supplied. Documentation/dev-tools/ktap.rst:238: WARNING: Error in "code-block" directive: 1 argument(s) required, 0 supplied. Fixes: a32fa6b2e8b4 ("Documentation: dev-tools: Add KTAP specification") Signed-off-by: Randy Dunlap Cc: Rae Moar Cc: David Gow Cc: Jonathan Corbet Cc: linux-doc@vger.kernel.org Reviewed-by: Frank Rowand Reviewed-by: David Gow Link: https://lore.kernel.org/r/20220131003637.14274-1-rdunlap@infradead.org Signed-off-by: Jonathan Corbet --- Documentation/dev-tools/ktap.rst | 18 +++++++++--------- 1 file changed, 9 insertions(+), 9 deletions(-) (limited to 'Documentation/dev-tools') diff --git a/Documentation/dev-tools/ktap.rst b/Documentation/dev-tools/ktap.rst index d7fe05de40b4..5ee735c6687f 100644 --- a/Documentation/dev-tools/ktap.rst +++ b/Documentation/dev-tools/ktap.rst @@ -68,7 +68,7 @@ Test case result lines Test case result lines indicate the final status of a test. They are required and must have the format: -.. code-block:: +.. code-block:: none [][ # [] []] @@ -117,32 +117,32 @@ separator. Example result lines include: -.. code-block:: +.. code-block:: none ok 1 test_case_name The test "test_case_name" passed. -.. code-block:: +.. code-block:: none not ok 1 test_case_name The test "test_case_name" failed. -.. code-block:: +.. code-block:: none ok 1 test # SKIP necessary dependency unavailable The test "test" was SKIPPED with the diagnostic message "necessary dependency unavailable". -.. code-block:: +.. code-block:: none not ok 1 test # TIMEOUT 30 seconds The test "test" timed out, with diagnostic data "30 seconds". -.. code-block:: +.. code-block:: none ok 5 check return code # rcode=0 @@ -202,7 +202,7 @@ allowed to be either indented or not indented. An example of a test with two nested subtests: -.. code-block:: +.. code-block:: none KTAP version 1 1..1 @@ -215,7 +215,7 @@ An example of a test with two nested subtests: An example format with multiple levels of nested testing: -.. code-block:: +.. code-block:: none KTAP version 1 1..2 @@ -250,7 +250,7 @@ nested version line, uses a line of the form Example KTAP output -------------------- -.. code-block:: +.. code-block:: none KTAP version 1 1..1 -- cgit v1.2.3