From a66d733da8010c732979041cd602cfceab7f587b Mon Sep 17 00:00:00 2001 From: Miguel Ojeda Date: Tue, 18 Jul 2023 07:27:51 +0200 Subject: rust: support running Rust documentation tests as KUnit ones Rust has documentation tests: these are typically examples of usage of any item (e.g. function, struct, module...). They are very convenient because they are just written alongside the documentation. For instance: /// Sums two numbers. /// /// ``` /// assert_eq!(mymod::f(10, 20), 30); /// ``` pub fn f(a: i32, b: i32) -> i32 { a + b } In userspace, the tests are collected and run via `rustdoc`. Using the tool as-is would be useful already, since it allows to compile-test most tests (thus enforcing they are kept in sync with the code they document) and run those that do not depend on in-kernel APIs. However, by transforming the tests into a KUnit test suite, they can also be run inside the kernel. Moreover, the tests get to be compiled as other Rust kernel objects instead of targeting userspace. On top of that, the integration with KUnit means the Rust support gets to reuse the existing testing facilities. For instance, the kernel log would look like: KTAP version 1 1..1 KTAP version 1 # Subtest: rust_doctests_kernel 1..59 # rust_doctest_kernel_build_assert_rs_0.location: rust/kernel/build_assert.rs:13 ok 1 rust_doctest_kernel_build_assert_rs_0 # rust_doctest_kernel_build_assert_rs_1.location: rust/kernel/build_assert.rs:56 ok 2 rust_doctest_kernel_build_assert_rs_1 # rust_doctest_kernel_init_rs_0.location: rust/kernel/init.rs:122 ok 3 rust_doctest_kernel_init_rs_0 ... # rust_doctest_kernel_types_rs_2.location: rust/kernel/types.rs:150 ok 59 rust_doctest_kernel_types_rs_2 # rust_doctests_kernel: pass:59 fail:0 skip:0 total:59 # Totals: pass:59 fail:0 skip:0 total:59 ok 1 rust_doctests_kernel Therefore, add support for running Rust documentation tests in KUnit. Some other notes about the current implementation and support follow. The transformation is performed by a couple scripts written as Rust hostprogs. Tests using the `?` operator are also supported as usual, e.g.: /// ``` /// # use kernel::{spawn_work_item, workqueue}; /// spawn_work_item!(workqueue::system(), || pr_info!("x"))?; /// # Ok::<(), Error>(()) /// ``` The tests are also compiled with Clippy under `CLIPPY=1`, just like normal code, thus also benefitting from extra linting. The names of the tests are currently automatically generated. This allows to reduce the burden for documentation writers, while keeping them fairly stable for bisection. This is an improvement over the `rustdoc`-generated names, which include the line number; but ideally we would like to get `rustdoc` to provide the Rust item path and a number (for multiple examples in a single documented Rust item). In order for developers to easily see from which original line a failed doctests came from, a KTAP diagnostic line is printed to the log, containing the location (file and line) of the original test (i.e. instead of the location in the generated Rust file): # rust_doctest_kernel_types_rs_2.location: rust/kernel/types.rs:150 This line follows the syntax for declaring test metadata in the proposed KTAP v2 spec [1], which may be used for the proposed KUnit test attributes API [2]. Thus hopefully this will make migration easier later on (suggested by David [3]). The original line in that test attribute is figured out by providing an anchor (suggested by Boqun [4]). The original file is found by walking the filesystem, checking directory prefixes to reduce the amount of combinations to check, and it is only done once per file. Ambiguities are detected and reported. A notable difference from KUnit C tests is that the Rust tests appear to assert using the usual `assert!` and `assert_eq!` macros from the Rust standard library (`core`). We provide a custom version that forwards the call to KUnit instead. Importantly, these macros do not require passing context, unlike the KUnit C ones (i.e. `struct kunit *`). This makes them easier to use, and readers of the documentation do not need to care about which testing framework is used. In addition, it may allow us to test third-party code more easily in the future. However, a current limitation is that KUnit does not support assertions in other tasks. Thus we presently simply print an error to the kernel log if an assertion actually failed. This should be revisited to properly fail the test, perhaps saving the context somewhere else, or letting KUnit handle it. Link: https://lore.kernel.org/lkml/20230420205734.1288498-1-rmoar@google.com/ [1] Link: https://lore.kernel.org/linux-kselftest/20230707210947.1208717-1-rmoar@google.com/ [2] Link: https://lore.kernel.org/rust-for-linux/CABVgOSkOLO-8v6kdAGpmYnZUb+LKOX0CtYCo-Bge7r_2YTuXDQ@mail.gmail.com/ [3] Link: https://lore.kernel.org/rust-for-linux/ZIps86MbJF%2FiGIzd@boqun-archlinux/ [4] Signed-off-by: Miguel Ojeda Reviewed-by: David Gow Signed-off-by: Shuah Khan --- lib/Kconfig.debug | 13 +++++++++++++ 1 file changed, 13 insertions(+) (limited to 'lib') diff --git a/lib/Kconfig.debug b/lib/Kconfig.debug index fbc89baf7de6..550cb967b668 100644 --- a/lib/Kconfig.debug +++ b/lib/Kconfig.debug @@ -3010,6 +3010,19 @@ config RUST_BUILD_ASSERT_ALLOW If unsure, say N. +config RUST_KERNEL_DOCTESTS + bool "Doctests for the `kernel` crate" if !KUNIT_ALL_TESTS + depends on RUST && KUNIT=y + default KUNIT_ALL_TESTS + help + This builds the documentation tests of the `kernel` crate + as KUnit tests. + + For more information on KUnit and unit tests in general, + please refer to the KUnit documentation in Documentation/dev-tools/kunit/. + + If unsure, say N. + endmenu # "Rust" endmenu # Kernel hacking -- cgit v1.2.3 From 39e92cb1e4a1f6a12097ea2aa9e9ca6f2d2f8a83 Mon Sep 17 00:00:00 2001 From: Rae Moar Date: Tue, 25 Jul 2023 21:25:12 +0000 Subject: kunit: Add test attributes API structure Add the basic structure of the test attribute API to KUnit, which can be used to save and access test associated data. Add attributes.c and attributes.h to hold associated structs and functions for the API. Create a struct that holds a variety of associated helper functions for each test attribute. These helper functions will be used to get the attribute value, convert the value to a string, and filter based on the value. This struct is flexible by design to allow for attributes of numerous types and contexts. Add a method to print test attributes in the format of "# [.]: ". Example for a suite: "# speed: slow" Example for a test case: "# test_case.speed: very_slow" Use this method to report attributes in the KTAP output (KTAP spec: https://docs.kernel.org/dev-tools/ktap.html) and _list_tests output when kernel's new kunit.action=list_attr option is used. Note this is derivative of the kunit.action=list option. In test.h, add fields and associated helper functions to test cases and suites to hold user-inputted test attributes. Reviewed-by: David Gow Signed-off-by: Rae Moar Signed-off-by: Shuah Khan --- include/kunit/attributes.h | 19 +++++++++++ include/kunit/test.h | 33 +++++++++++++++++++ lib/kunit/Makefile | 3 +- lib/kunit/attributes.c | 80 ++++++++++++++++++++++++++++++++++++++++++++++ lib/kunit/executor.c | 21 +++++++++--- lib/kunit/test.c | 17 ++++++---- 6 files changed, 161 insertions(+), 12 deletions(-) create mode 100644 include/kunit/attributes.h create mode 100644 lib/kunit/attributes.c (limited to 'lib') diff --git a/include/kunit/attributes.h b/include/kunit/attributes.h new file mode 100644 index 000000000000..9fcd184cce36 --- /dev/null +++ b/include/kunit/attributes.h @@ -0,0 +1,19 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* + * KUnit API to save and access test attributes + * + * Copyright (C) 2023, Google LLC. + * Author: Rae Moar + */ + +#ifndef _KUNIT_ATTRIBUTES_H +#define _KUNIT_ATTRIBUTES_H + +/* + * Print all test attributes for a test case or suite. + * Output format for test cases: "# .: " + * Output format for test suites: "# : " + */ +void kunit_print_attr(void *test_or_suite, bool is_test, unsigned int test_level); + +#endif /* _KUNIT_ATTRIBUTES_H */ diff --git a/include/kunit/test.h b/include/kunit/test.h index 23120d50499e..1fc9155988e9 100644 --- a/include/kunit/test.h +++ b/include/kunit/test.h @@ -63,12 +63,16 @@ enum kunit_status { KUNIT_SKIPPED, }; +/* Holds attributes for each test case and suite */ +struct kunit_attributes {}; + /** * struct kunit_case - represents an individual test case. * * @run_case: the function representing the actual test case. * @name: the name of the test case. * @generate_params: the generator function for parameterized tests. + * @attr: the attributes associated with the test * * A test case is a function with the signature, * ``void (*)(struct kunit *)`` @@ -104,6 +108,7 @@ struct kunit_case { void (*run_case)(struct kunit *test); const char *name; const void* (*generate_params)(const void *prev, char *desc); + struct kunit_attributes attr; /* private: internal use only. */ enum kunit_status status; @@ -133,6 +138,18 @@ static inline char *kunit_status_to_ok_not_ok(enum kunit_status status) */ #define KUNIT_CASE(test_name) { .run_case = test_name, .name = #test_name } +/** + * KUNIT_CASE_ATTR - A helper for creating a &struct kunit_case + * with attributes + * + * @test_name: a reference to a test case function. + * @attributes: a reference to a struct kunit_attributes object containing + * test attributes + */ +#define KUNIT_CASE_ATTR(test_name, attributes) \ + { .run_case = test_name, .name = #test_name, \ + .attr = attributes } + /** * KUNIT_CASE_PARAM - A helper for creation a parameterized &struct kunit_case * @@ -154,6 +171,20 @@ static inline char *kunit_status_to_ok_not_ok(enum kunit_status status) { .run_case = test_name, .name = #test_name, \ .generate_params = gen_params } +/** + * KUNIT_CASE_PARAM_ATTR - A helper for creating a parameterized &struct + * kunit_case with attributes + * + * @test_name: a reference to a test case function. + * @gen_params: a reference to a parameter generator function. + * @attributes: a reference to a struct kunit_attributes object containing + * test attributes + */ +#define KUNIT_CASE_PARAM_ATTR(test_name, gen_params, attributes) \ + { .run_case = test_name, .name = #test_name, \ + .generate_params = gen_params, \ + .attr = attributes } + /** * struct kunit_suite - describes a related collection of &struct kunit_case * @@ -163,6 +194,7 @@ static inline char *kunit_status_to_ok_not_ok(enum kunit_status status) * @init: called before every test case. * @exit: called after every test case. * @test_cases: a null terminated array of test cases. + * @attr: the attributes associated with the test suite * * A kunit_suite is a collection of related &struct kunit_case s, such that * @init is called before every test case and @exit is called after every @@ -182,6 +214,7 @@ struct kunit_suite { int (*init)(struct kunit *test); void (*exit)(struct kunit *test); struct kunit_case *test_cases; + struct kunit_attributes attr; /* private: internal use only */ char status_comment[KUNIT_STATUS_COMMENT_SIZE]; diff --git a/lib/kunit/Makefile b/lib/kunit/Makefile index cb417f504996..46f75f23dfe4 100644 --- a/lib/kunit/Makefile +++ b/lib/kunit/Makefile @@ -6,7 +6,8 @@ kunit-objs += test.o \ string-stream.o \ assert.o \ try-catch.o \ - executor.o + executor.o \ + attributes.o ifeq ($(CONFIG_KUNIT_DEBUGFS),y) kunit-objs += debugfs.o diff --git a/lib/kunit/attributes.c b/lib/kunit/attributes.c new file mode 100644 index 000000000000..9bda5a5f4030 --- /dev/null +++ b/lib/kunit/attributes.c @@ -0,0 +1,80 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * KUnit API to save and access test attributes + * + * Copyright (C) 2023, Google LLC. + * Author: Rae Moar + */ + +#include +#include + +/* Options for printing attributes: + * PRINT_ALWAYS - attribute is printed for every test case and suite if set + * PRINT_SUITE - attribute is printed for every suite if set but not for test cases + * PRINT_NEVER - attribute is never printed + */ +enum print_ops { + PRINT_ALWAYS, + PRINT_SUITE, + PRINT_NEVER, +}; + +/** + * struct kunit_attr - represents a test attribute and holds flexible + * helper functions to interact with attribute. + * + * @name: name of test attribute, eg. speed + * @get_attr: function to return attribute value given a test + * @to_string: function to return string representation of given + * attribute value + * @filter: function to indicate whether a given attribute value passes a + * filter + */ +struct kunit_attr { + const char *name; + void *(*get_attr)(void *test_or_suite, bool is_test); + const char *(*to_string)(void *attr, bool *to_free); + int (*filter)(void *attr, const char *input, int *err); + void *attr_default; + enum print_ops print; +}; + +/* List of all Test Attributes */ + +static struct kunit_attr kunit_attr_list[] = {}; + +/* Helper Functions to Access Attributes */ + +void kunit_print_attr(void *test_or_suite, bool is_test, unsigned int test_level) +{ + int i; + bool to_free; + void *attr; + const char *attr_name, *attr_str; + struct kunit_suite *suite = is_test ? NULL : test_or_suite; + struct kunit_case *test = is_test ? test_or_suite : NULL; + + for (i = 0; i < ARRAY_SIZE(kunit_attr_list); i++) { + if (kunit_attr_list[i].print == PRINT_NEVER || + (test && kunit_attr_list[i].print == PRINT_SUITE)) + continue; + attr = kunit_attr_list[i].get_attr(test_or_suite, is_test); + if (attr) { + attr_name = kunit_attr_list[i].name; + attr_str = kunit_attr_list[i].to_string(attr, &to_free); + if (test) { + kunit_log(KERN_INFO, test, "%*s# %s.%s: %s", + KUNIT_INDENT_LEN * test_level, "", test->name, + attr_name, attr_str); + } else { + kunit_log(KERN_INFO, suite, "%*s# %s: %s", + KUNIT_INDENT_LEN * test_level, "", attr_name, attr_str); + } + + /* Free to_string of attribute if needed */ + if (to_free) + kfree(attr_str); + } + } +} diff --git a/lib/kunit/executor.c b/lib/kunit/executor.c index 74982b83707c..12e38a48a5cc 100644 --- a/lib/kunit/executor.c +++ b/lib/kunit/executor.c @@ -2,6 +2,7 @@ #include #include +#include #include #include @@ -24,7 +25,8 @@ module_param_named(action, action_param, charp, 0); MODULE_PARM_DESC(action, "Changes KUnit executor behavior, valid values are:\n" ": run the tests like normal\n" - "'list' to list test names instead of running them.\n"); + "'list' to list test names instead of running them.\n" + "'list_attr' to list test names and attributes instead of running them.\n"); /* glob_match() needs NULL terminated strings, so we need a copy of filter_glob_param. */ struct kunit_test_filter { @@ -172,7 +174,7 @@ static void kunit_exec_run_tests(struct suite_set *suite_set) __kunit_test_suites_init(suite_set->start, num_suites); } -static void kunit_exec_list_tests(struct suite_set *suite_set) +static void kunit_exec_list_tests(struct suite_set *suite_set, bool include_attr) { struct kunit_suite * const *suites; struct kunit_case *test_case; @@ -180,10 +182,19 @@ static void kunit_exec_list_tests(struct suite_set *suite_set) /* Hack: print a ktap header so kunit.py can find the start of KUnit output. */ pr_info("KTAP version 1\n"); - for (suites = suite_set->start; suites < suite_set->end; suites++) + for (suites = suite_set->start; suites < suite_set->end; suites++) { + /* Print suite name and suite attributes */ + pr_info("%s\n", (*suites)->name); + if (include_attr) + kunit_print_attr((void *)(*suites), false, 0); + + /* Print test case name and attributes in suite */ kunit_suite_for_each_test_case((*suites), test_case) { pr_info("%s.%s\n", (*suites)->name, test_case->name); + if (include_attr) + kunit_print_attr((void *)test_case, true, 0); } + } } int kunit_run_all_tests(void) @@ -206,7 +217,9 @@ int kunit_run_all_tests(void) if (!action_param) kunit_exec_run_tests(&suite_set); else if (strcmp(action_param, "list") == 0) - kunit_exec_list_tests(&suite_set); + kunit_exec_list_tests(&suite_set, false); + else if (strcmp(action_param, "list_attr") == 0) + kunit_exec_list_tests(&suite_set, true); else pr_err("kunit executor: unknown action '%s'\n", action_param); diff --git a/lib/kunit/test.c b/lib/kunit/test.c index 84e4666555c9..9ee55139ecd1 100644 --- a/lib/kunit/test.c +++ b/lib/kunit/test.c @@ -9,6 +9,7 @@ #include #include #include +#include #include #include #include @@ -168,6 +169,13 @@ size_t kunit_suite_num_test_cases(struct kunit_suite *suite) } EXPORT_SYMBOL_GPL(kunit_suite_num_test_cases); +/* Currently supported test levels */ +enum { + KUNIT_LEVEL_SUITE = 0, + KUNIT_LEVEL_CASE, + KUNIT_LEVEL_CASE_PARAM, +}; + static void kunit_print_suite_start(struct kunit_suite *suite) { /* @@ -181,17 +189,11 @@ static void kunit_print_suite_start(struct kunit_suite *suite) pr_info(KUNIT_SUBTEST_INDENT "KTAP version 1\n"); pr_info(KUNIT_SUBTEST_INDENT "# Subtest: %s\n", suite->name); + kunit_print_attr((void *)suite, false, KUNIT_LEVEL_CASE); pr_info(KUNIT_SUBTEST_INDENT "1..%zd\n", kunit_suite_num_test_cases(suite)); } -/* Currently supported test levels */ -enum { - KUNIT_LEVEL_SUITE = 0, - KUNIT_LEVEL_CASE, - KUNIT_LEVEL_CASE_PARAM, -}; - static void kunit_print_ok_not_ok(struct kunit *test, unsigned int test_level, enum kunit_status status, @@ -651,6 +653,7 @@ int kunit_run_tests(struct kunit_suite *suite) } } + kunit_print_attr((void *)test_case, true, KUNIT_LEVEL_CASE); kunit_print_test_stats(&test, param_stats); -- cgit v1.2.3 From 02c2d0c2a84172c3c7ec0229c61db55d23dd4730 Mon Sep 17 00:00:00 2001 From: Rae Moar Date: Tue, 25 Jul 2023 21:25:13 +0000 Subject: kunit: Add speed attribute Add speed attribute to the test attribute API. This attribute will allow users to mark tests with a category of speed. Currently the categories of speed proposed are: normal, slow, and very_slow (outlined in enum kunit_speed). These are outlined in the enum kunit_speed. The assumed default speed for tests is "normal". This indicates that the test takes a relatively trivial amount of time (less than 1 second), regardless of the machine it is running on. Any test slower than this could be marked as "slow" or "very_slow". Add the macro KUNIT_CASE_SLOW to set a test as slow, as this is likely a common use of the attributes API. Add an example of marking a slow test to kunit-example-test.c. Reviewed-by: David Gow Signed-off-by: Rae Moar Signed-off-by: Shuah Khan --- include/kunit/test.h | 32 +++++++++++++++++++++++++++++- lib/kunit/attributes.c | 44 +++++++++++++++++++++++++++++++++++++++++- lib/kunit/kunit-example-test.c | 9 +++++++++ 3 files changed, 83 insertions(+), 2 deletions(-) (limited to 'lib') diff --git a/include/kunit/test.h b/include/kunit/test.h index 1fc9155988e9..ed5f5000a095 100644 --- a/include/kunit/test.h +++ b/include/kunit/test.h @@ -63,8 +63,27 @@ enum kunit_status { KUNIT_SKIPPED, }; +/* Attribute struct/enum definitions */ + +/* + * Speed Attribute is stored as an enum and separated into categories of + * speed: very_slowm, slow, and normal. These speeds are relative to + * other KUnit tests. + * + * Note: unset speed attribute acts as default of KUNIT_SPEED_NORMAL. + */ +enum kunit_speed { + KUNIT_SPEED_UNSET, + KUNIT_SPEED_VERY_SLOW, + KUNIT_SPEED_SLOW, + KUNIT_SPEED_NORMAL, + KUNIT_SPEED_MAX = KUNIT_SPEED_NORMAL, +}; + /* Holds attributes for each test case and suite */ -struct kunit_attributes {}; +struct kunit_attributes { + enum kunit_speed speed; +}; /** * struct kunit_case - represents an individual test case. @@ -150,6 +169,17 @@ static inline char *kunit_status_to_ok_not_ok(enum kunit_status status) { .run_case = test_name, .name = #test_name, \ .attr = attributes } +/** + * KUNIT_CASE_SLOW - A helper for creating a &struct kunit_case + * with the slow attribute + * + * @test_name: a reference to a test case function. + */ + +#define KUNIT_CASE_SLOW(test_name) \ + { .run_case = test_name, .name = #test_name, \ + .attr.speed = KUNIT_SPEED_SLOW } + /** * KUNIT_CASE_PARAM - A helper for creation a parameterized &struct kunit_case * diff --git a/lib/kunit/attributes.c b/lib/kunit/attributes.c index 9bda5a5f4030..ffd0d692b334 100644 --- a/lib/kunit/attributes.c +++ b/lib/kunit/attributes.c @@ -40,9 +40,51 @@ struct kunit_attr { enum print_ops print; }; +/* String Lists for enum Attributes */ + +static const char * const speed_str_list[] = {"unset", "very_slow", "slow", "normal"}; + +/* To String Methods */ + +static const char *attr_enum_to_string(void *attr, const char * const str_list[], bool *to_free) +{ + long val = (long)attr; + + *to_free = false; + if (!val) + return NULL; + return str_list[val]; +} + +static const char *attr_speed_to_string(void *attr, bool *to_free) +{ + return attr_enum_to_string(attr, speed_str_list, to_free); +} + +/* Get Attribute Methods */ + +static void *attr_speed_get(void *test_or_suite, bool is_test) +{ + struct kunit_suite *suite = is_test ? NULL : test_or_suite; + struct kunit_case *test = is_test ? test_or_suite : NULL; + + if (test) + return ((void *) test->attr.speed); + else + return ((void *) suite->attr.speed); +} + /* List of all Test Attributes */ -static struct kunit_attr kunit_attr_list[] = {}; +static struct kunit_attr kunit_attr_list[] = { + { + .name = "speed", + .get_attr = attr_speed_get, + .to_string = attr_speed_to_string, + .attr_default = (void *)KUNIT_SPEED_NORMAL, + .print = PRINT_ALWAYS, + }, +}; /* Helper Functions to Access Attributes */ diff --git a/lib/kunit/kunit-example-test.c b/lib/kunit/kunit-example-test.c index b69b689ea850..01a769f35e1d 100644 --- a/lib/kunit/kunit-example-test.c +++ b/lib/kunit/kunit-example-test.c @@ -220,6 +220,14 @@ static void example_params_test(struct kunit *test) KUNIT_EXPECT_EQ(test, param->value % param->value, 0); } +/* + * This test should always pass. Can be used to practice filtering attributes. + */ +static void example_slow_test(struct kunit *test) +{ + KUNIT_EXPECT_EQ(test, 1 + 1, 2); +} + /* * Here we make a list of all the test cases we want to add to the test suite * below. @@ -237,6 +245,7 @@ static struct kunit_case example_test_cases[] = { KUNIT_CASE(example_all_expect_macros_test), KUNIT_CASE(example_static_stub_test), KUNIT_CASE_PARAM(example_params_test, example_gen_params), + KUNIT_CASE_SLOW(example_slow_test), {} }; -- cgit v1.2.3 From a00a72709175a4d53096301a8792b8171d1223e5 Mon Sep 17 00:00:00 2001 From: Rae Moar Date: Tue, 25 Jul 2023 21:25:14 +0000 Subject: kunit: Add module attribute Add module attribute to the test attribute API. This attribute stores the module name associated with the test using KBUILD_MODNAME. The name of a test suite and the module name often do not match. A reference to the module name associated with the suite could be extremely helpful in running tests as modules without needing to check the codebase. This attribute will be printed for each suite. Reviewed-by: David Gow Signed-off-by: Rae Moar Signed-off-by: Shuah Khan --- include/kunit/test.h | 13 ++++++++----- lib/kunit/attributes.c | 25 +++++++++++++++++++++++++ 2 files changed, 33 insertions(+), 5 deletions(-) (limited to 'lib') diff --git a/include/kunit/test.h b/include/kunit/test.h index ed5f5000a095..011e0d6bb506 100644 --- a/include/kunit/test.h +++ b/include/kunit/test.h @@ -131,6 +131,7 @@ struct kunit_case { /* private: internal use only. */ enum kunit_status status; + char *module_name; char *log; }; @@ -155,7 +156,9 @@ static inline char *kunit_status_to_ok_not_ok(enum kunit_status status) * &struct kunit_case object from it. See the documentation for * &struct kunit_case for an example on how to use it. */ -#define KUNIT_CASE(test_name) { .run_case = test_name, .name = #test_name } +#define KUNIT_CASE(test_name) \ + { .run_case = test_name, .name = #test_name, \ + .module_name = KBUILD_MODNAME} /** * KUNIT_CASE_ATTR - A helper for creating a &struct kunit_case @@ -167,7 +170,7 @@ static inline char *kunit_status_to_ok_not_ok(enum kunit_status status) */ #define KUNIT_CASE_ATTR(test_name, attributes) \ { .run_case = test_name, .name = #test_name, \ - .attr = attributes } + .attr = attributes, .module_name = KBUILD_MODNAME} /** * KUNIT_CASE_SLOW - A helper for creating a &struct kunit_case @@ -178,7 +181,7 @@ static inline char *kunit_status_to_ok_not_ok(enum kunit_status status) #define KUNIT_CASE_SLOW(test_name) \ { .run_case = test_name, .name = #test_name, \ - .attr.speed = KUNIT_SPEED_SLOW } + .attr.speed = KUNIT_SPEED_SLOW, .module_name = KBUILD_MODNAME} /** * KUNIT_CASE_PARAM - A helper for creation a parameterized &struct kunit_case @@ -199,7 +202,7 @@ static inline char *kunit_status_to_ok_not_ok(enum kunit_status status) */ #define KUNIT_CASE_PARAM(test_name, gen_params) \ { .run_case = test_name, .name = #test_name, \ - .generate_params = gen_params } + .generate_params = gen_params, .module_name = KBUILD_MODNAME} /** * KUNIT_CASE_PARAM_ATTR - A helper for creating a parameterized &struct @@ -213,7 +216,7 @@ static inline char *kunit_status_to_ok_not_ok(enum kunit_status status) #define KUNIT_CASE_PARAM_ATTR(test_name, gen_params, attributes) \ { .run_case = test_name, .name = #test_name, \ .generate_params = gen_params, \ - .attr = attributes } + .attr = attributes, .module_name = KBUILD_MODNAME} /** * struct kunit_suite - describes a related collection of &struct kunit_case diff --git a/lib/kunit/attributes.c b/lib/kunit/attributes.c index ffd0d692b334..9dce4f4d726c 100644 --- a/lib/kunit/attributes.c +++ b/lib/kunit/attributes.c @@ -61,6 +61,12 @@ static const char *attr_speed_to_string(void *attr, bool *to_free) return attr_enum_to_string(attr, speed_str_list, to_free); } +static const char *attr_string_to_string(void *attr, bool *to_free) +{ + *to_free = false; + return (char *) attr; +} + /* Get Attribute Methods */ static void *attr_speed_get(void *test_or_suite, bool is_test) @@ -74,6 +80,18 @@ static void *attr_speed_get(void *test_or_suite, bool is_test) return ((void *) suite->attr.speed); } +static void *attr_module_get(void *test_or_suite, bool is_test) +{ + struct kunit_suite *suite = is_test ? NULL : test_or_suite; + struct kunit_case *test = is_test ? test_or_suite : NULL; + + // Suites get their module attribute from their first test_case + if (test) + return ((void *) test->module_name); + else + return ((void *) suite->test_cases[0].module_name); +} + /* List of all Test Attributes */ static struct kunit_attr kunit_attr_list[] = { @@ -84,6 +102,13 @@ static struct kunit_attr kunit_attr_list[] = { .attr_default = (void *)KUNIT_SPEED_NORMAL, .print = PRINT_ALWAYS, }, + { + .name = "module", + .get_attr = attr_module_get, + .to_string = attr_string_to_string, + .attr_default = (void *)"", + .print = PRINT_SUITE, + } }; /* Helper Functions to Access Attributes */ -- cgit v1.2.3 From 529534e8cba3e60f843a682e2a3149612b30d608 Mon Sep 17 00:00:00 2001 From: Rae Moar Date: Tue, 25 Jul 2023 21:25:15 +0000 Subject: kunit: Add ability to filter attributes Add filtering of test attributes. Users can filter tests using the module_param called "filter". Filters are imputed in the format: Example: kunit.filter="speed>slow" Operations include: >, <, >=, <=, !=, and =. These operations will act the same for attributes of the same type but may not between types. Note multiple filters can be inputted by separating them with a comma. Example: kunit.filter="speed=slow, module!=example" Since both suites and test cases can have attributes, there may be conflicts. The process of filtering follows these rules: - Filtering always operates at a per-test level. - If a test has an attribute set, then the test's value is filtered on. - Otherwise, the value falls back to the suite's value. - If neither are set, the attribute has a global "default" value, which is used. Filtered tests will not be run or show in output. The tests can instead be skipped using the configurable option "kunit.filter_action=skip". Note the default settings for running tests remains unfiltered. Finally, add "filter" methods for the speed and module attributes to parse and compare attribute values. Note this filtering functionality will be added to kunit.py in the next patch. Reviewed-by: David Gow Signed-off-by: Rae Moar Signed-off-by: Shuah Khan --- include/kunit/attributes.h | 31 ++++++ lib/kunit/attributes.c | 271 +++++++++++++++++++++++++++++++++++++++++++++ lib/kunit/executor.c | 93 ++++++++++++---- lib/kunit/executor_test.c | 12 +- lib/kunit/test.c | 10 +- 5 files changed, 389 insertions(+), 28 deletions(-) (limited to 'lib') diff --git a/include/kunit/attributes.h b/include/kunit/attributes.h index 9fcd184cce36..bc76a0b786d2 100644 --- a/include/kunit/attributes.h +++ b/include/kunit/attributes.h @@ -9,6 +9,20 @@ #ifndef _KUNIT_ATTRIBUTES_H #define _KUNIT_ATTRIBUTES_H +/* + * struct kunit_attr_filter - representation of attributes filter with the + * attribute object and string input + */ +struct kunit_attr_filter { + struct kunit_attr *attr; + char *input; +}; + +/* + * Returns the name of the filter's attribute. + */ +const char *kunit_attr_filter_name(struct kunit_attr_filter filter); + /* * Print all test attributes for a test case or suite. * Output format for test cases: "# .: " @@ -16,4 +30,21 @@ */ void kunit_print_attr(void *test_or_suite, bool is_test, unsigned int test_level); +/* + * Returns the number of fitlers in input. + */ +int kunit_get_filter_count(char *input); + +/* + * Parse attributes filter input and return an objects containing the + * attribute object and the string input of the next filter. + */ +struct kunit_attr_filter kunit_next_attr_filter(char **filters, int *err); + +/* + * Returns a copy of the suite containing only tests that pass the filter. + */ +struct kunit_suite *kunit_filter_attr_tests(const struct kunit_suite *const suite, + struct kunit_attr_filter filter, char *action, int *err); + #endif /* _KUNIT_ATTRIBUTES_H */ diff --git a/lib/kunit/attributes.c b/lib/kunit/attributes.c index 9dce4f4d726c..d37c40c0ce4f 100644 --- a/lib/kunit/attributes.c +++ b/lib/kunit/attributes.c @@ -67,6 +67,104 @@ static const char *attr_string_to_string(void *attr, bool *to_free) return (char *) attr; } +/* Filter Methods */ + +static const char op_list[] = "<>!="; + +/* + * Returns whether the inputted integer value matches the filter given + * by the operation string and inputted integer. + */ +static int int_filter(long val, const char *op, int input, int *err) +{ + if (!strncmp(op, "<=", 2)) + return (val <= input); + else if (!strncmp(op, ">=", 2)) + return (val >= input); + else if (!strncmp(op, "!=", 2)) + return (val != input); + else if (!strncmp(op, ">", 1)) + return (val > input); + else if (!strncmp(op, "<", 1)) + return (val < input); + else if (!strncmp(op, "=", 1)) + return (val == input); + *err = -EINVAL; + pr_err("kunit executor: invalid filter operation: %s\n", op); + return false; +} + +/* + * Returns whether the inputted enum value "attr" matches the filter given + * by the input string. Note: the str_list includes the corresponding string + * list to the enum values. + */ +static int attr_enum_filter(void *attr, const char *input, int *err, + const char * const str_list[], int max) +{ + int i, j, input_int; + long test_val = (long)attr; + const char *input_val = NULL; + + for (i = 0; input[i]; i++) { + if (!strchr(op_list, input[i])) { + input_val = input + i; + break; + } + } + + if (!input_val) { + *err = -EINVAL; + pr_err("kunit executor: filter value not found: %s\n", input); + return false; + } + + for (j = 0; j <= max; j++) { + if (!strcmp(input_val, str_list[j])) + input_int = j; + } + + if (!input_int) { + *err = -EINVAL; + pr_err("kunit executor: invalid filter input: %s\n", input); + return false; + } + + return int_filter(test_val, input, input_int, err); +} + +static int attr_speed_filter(void *attr, const char *input, int *err) +{ + return attr_enum_filter(attr, input, err, speed_str_list, KUNIT_SPEED_MAX); +} + +/* + * Returns whether the inputted string value (attr) matches the filter given + * by the input string. + */ +static int attr_string_filter(void *attr, const char *input, int *err) +{ + char *str = attr; + + if (!strncmp(input, "<", 1)) { + *err = -EINVAL; + pr_err("kunit executor: invalid filter input: %s\n", input); + return false; + } else if (!strncmp(input, ">", 1)) { + *err = -EINVAL; + pr_err("kunit executor: invalid filter input: %s\n", input); + return false; + } else if (!strncmp(input, "!=", 2)) { + return (strcmp(input + 2, str) != 0); + } else if (!strncmp(input, "=", 1)) { + return (strcmp(input + 1, str) == 0); + } + *err = -EINVAL; + pr_err("kunit executor: invalid filter operation: %s\n", input); + return false; +} + + /* Get Attribute Methods */ static void *attr_speed_get(void *test_or_suite, bool is_test) @@ -99,6 +197,7 @@ static struct kunit_attr kunit_attr_list[] = { .name = "speed", .get_attr = attr_speed_get, .to_string = attr_speed_to_string, + .filter = attr_speed_filter, .attr_default = (void *)KUNIT_SPEED_NORMAL, .print = PRINT_ALWAYS, }, @@ -106,6 +205,7 @@ static struct kunit_attr kunit_attr_list[] = { .name = "module", .get_attr = attr_module_get, .to_string = attr_string_to_string, + .filter = attr_string_filter, .attr_default = (void *)"", .print = PRINT_SUITE, } @@ -113,6 +213,11 @@ static struct kunit_attr kunit_attr_list[] = { /* Helper Functions to Access Attributes */ +const char *kunit_attr_filter_name(struct kunit_attr_filter filter) +{ + return filter.attr->name; +} + void kunit_print_attr(void *test_or_suite, bool is_test, unsigned int test_level) { int i; @@ -145,3 +250,169 @@ void kunit_print_attr(void *test_or_suite, bool is_test, unsigned int test_level } } } + +/* Helper Functions to Filter Attributes */ + +int kunit_get_filter_count(char *input) +{ + int i, comma_index, count = 0; + + for (i = 0; input[i]; i++) { + if (input[i] == ',') { + if ((i - comma_index) > 1) + count++; + comma_index = i; + } + } + if ((i - comma_index) > 0) + count++; + return count; +} + +struct kunit_attr_filter kunit_next_attr_filter(char **filters, int *err) +{ + struct kunit_attr_filter filter = {}; + int i, j, comma_index, new_start_index; + int op_index = -1, attr_index = -1; + char op; + char *input = *filters; + + /* Parse input until operation */ + for (i = 0; input[i]; i++) { + if (op_index < 0 && strchr(op_list, input[i])) { + op_index = i; + } else if (!comma_index && input[i] == ',') { + comma_index = i; + } else if (comma_index && input[i] != ' ') { + new_start_index = i; + break; + } + } + + if (op_index <= 0) { + *err = -EINVAL; + pr_err("kunit executor: filter operation not found: %s\n", input); + return filter; + } + + /* Temporarily set operator to \0 character. */ + op = input[op_index]; + input[op_index] = '\0'; + + /* Find associated kunit_attr object */ + for (j = 0; j < ARRAY_SIZE(kunit_attr_list); j++) { + if (!strcmp(input, kunit_attr_list[j].name)) { + attr_index = j; + break; + } + } + + input[op_index] = op; + + if (attr_index < 0) { + *err = -EINVAL; + pr_err("kunit executor: attribute not found: %s\n", input); + } else { + filter.attr = &kunit_attr_list[attr_index]; + } + + if (comma_index) { + input[comma_index] = '\0'; + filter.input = input + op_index; + input = input + new_start_index; + } else { + filter.input = input + op_index; + input = NULL; + } + + *filters = input; + + return filter; +} + +struct kunit_suite *kunit_filter_attr_tests(const struct kunit_suite *const suite, + struct kunit_attr_filter filter, char *action, int *err) +{ + int n = 0; + struct kunit_case *filtered, *test_case; + struct kunit_suite *copy; + void *suite_val, *test_val; + bool suite_result, test_result, default_result, result; + + /* Allocate memory for new copy of suite and list of test cases */ + copy = kmemdup(suite, sizeof(*copy), GFP_KERNEL); + if (!copy) + return ERR_PTR(-ENOMEM); + + kunit_suite_for_each_test_case(suite, test_case) { n++; } + + filtered = kcalloc(n + 1, sizeof(*filtered), GFP_KERNEL); + if (!filtered) { + kfree(copy); + return ERR_PTR(-ENOMEM); + } + + n = 0; + + /* Save filtering result on default value */ + default_result = filter.attr->filter(filter.attr->attr_default, filter.input, err); + if (*err) { + kfree(copy); + kfree(filtered); + return NULL; + } + + /* Save suite attribute value and filtering result on that value */ + suite_val = filter.attr->get_attr((void *)suite, false); + suite_result = filter.attr->filter(suite_val, filter.input, err); + if (*err) { + kfree(copy); + kfree(filtered); + return NULL; + } + + /* For each test case, save test case if passes filtering. */ + kunit_suite_for_each_test_case(suite, test_case) { + test_val = filter.attr->get_attr((void *) test_case, true); + test_result = filter.attr->filter(filter.attr->get_attr(test_case, true), + filter.input, err); + if (*err) { + kfree(copy); + kfree(filtered); + return NULL; + } + + /* + * If attribute value of test case is set, filter on that value. + * If not, filter on suite value if set. If not, filter on + * default value. + */ + result = false; + if (test_val) { + if (test_result) + result = true; + } else if (suite_val) { + if (suite_result) + result = true; + } else if (default_result) { + result = true; + } + + if (result) { + filtered[n++] = *test_case; + } else if (action && strcmp(action, "skip") == 0) { + test_case->status = KUNIT_SKIPPED; + filtered[n++] = *test_case; + } + } + + if (n == 0) { + kfree(copy); + kfree(filtered); + return NULL; + } + + copy->test_cases = filtered; + + return copy; +} diff --git a/lib/kunit/executor.c b/lib/kunit/executor.c index 12e38a48a5cc..483f7b7873a7 100644 --- a/lib/kunit/executor.c +++ b/lib/kunit/executor.c @@ -17,6 +17,8 @@ extern struct kunit_suite * const __kunit_suites_end[]; static char *filter_glob_param; static char *action_param; +static char *filter_param; +static char *filter_action_param; module_param_named(filter_glob, filter_glob_param, charp, 0); MODULE_PARM_DESC(filter_glob, @@ -27,15 +29,23 @@ MODULE_PARM_DESC(action, ": run the tests like normal\n" "'list' to list test names instead of running them.\n" "'list_attr' to list test names and attributes instead of running them.\n"); +module_param_named(filter, filter_param, charp, 0); +MODULE_PARM_DESC(filter, + "Filter which KUnit test suites/tests run at boot-time using attributes, e.g. speed>slow"); +module_param_named(filter_action, filter_action_param, charp, 0); +MODULE_PARM_DESC(filter_action, + "Changes behavior of filtered tests using attributes, valid values are:\n" + ": do not run filtered tests as normal\n" + "'skip': skip all filtered tests instead so tests will appear in output\n"); /* glob_match() needs NULL terminated strings, so we need a copy of filter_glob_param. */ -struct kunit_test_filter { +struct kunit_glob_filter { char *suite_glob; char *test_glob; }; /* Split "suite_glob.test_glob" into two. Assumes filter_glob is not empty. */ -static void kunit_parse_filter_glob(struct kunit_test_filter *parsed, +static void kunit_parse_glob_filter(struct kunit_glob_filter *parsed, const char *filter_glob) { const int len = strlen(filter_glob); @@ -57,7 +67,7 @@ static void kunit_parse_filter_glob(struct kunit_test_filter *parsed, /* Create a copy of suite with only tests that match test_glob. */ static struct kunit_suite * -kunit_filter_tests(const struct kunit_suite *const suite, const char *test_glob) +kunit_filter_glob_tests(const struct kunit_suite *const suite, const char *test_glob) { int n = 0; struct kunit_case *filtered, *test_case; @@ -111,12 +121,15 @@ static void kunit_free_suite_set(struct suite_set suite_set) static struct suite_set kunit_filter_suites(const struct suite_set *suite_set, const char *filter_glob, + char *filters, + char *filter_action, int *err) { - int i; - struct kunit_suite **copy, *filtered_suite; + int i, j, k, filter_count; + struct kunit_suite **copy, *filtered_suite, *new_filtered_suite; struct suite_set filtered; - struct kunit_test_filter filter; + struct kunit_glob_filter parsed_glob; + struct kunit_attr_filter *parsed_filters; const size_t max = suite_set->end - suite_set->start; @@ -127,17 +140,52 @@ static struct suite_set kunit_filter_suites(const struct suite_set *suite_set, return filtered; } - kunit_parse_filter_glob(&filter, filter_glob); + if (filter_glob) + kunit_parse_glob_filter(&parsed_glob, filter_glob); - for (i = 0; &suite_set->start[i] != suite_set->end; i++) { - if (!glob_match(filter.suite_glob, suite_set->start[i]->name)) - continue; - - filtered_suite = kunit_filter_tests(suite_set->start[i], filter.test_glob); - if (IS_ERR(filtered_suite)) { - *err = PTR_ERR(filtered_suite); + /* Parse attribute filters */ + if (filters) { + filter_count = kunit_get_filter_count(filters); + parsed_filters = kcalloc(filter_count + 1, sizeof(*parsed_filters), GFP_KERNEL); + for (j = 0; j < filter_count; j++) + parsed_filters[j] = kunit_next_attr_filter(&filters, err); + if (*err) return filtered; + } + + for (i = 0; &suite_set->start[i] != suite_set->end; i++) { + filtered_suite = suite_set->start[i]; + if (filter_glob) { + if (!glob_match(parsed_glob.suite_glob, filtered_suite->name)) + continue; + filtered_suite = kunit_filter_glob_tests(filtered_suite, + parsed_glob.test_glob); + if (IS_ERR(filtered_suite)) { + *err = PTR_ERR(filtered_suite); + return filtered; + } } + if (filter_count) { + for (k = 0; k < filter_count; k++) { + new_filtered_suite = kunit_filter_attr_tests(filtered_suite, + parsed_filters[k], filter_action, err); + + /* Free previous copy of suite */ + if (k > 0 || filter_glob) + kfree(filtered_suite); + filtered_suite = new_filtered_suite; + + if (*err) + return filtered; + if (IS_ERR(filtered_suite)) { + *err = PTR_ERR(filtered_suite); + return filtered; + } + if (!filtered_suite) + break; + } + } + if (!filtered_suite) continue; @@ -145,8 +193,14 @@ static struct suite_set kunit_filter_suites(const struct suite_set *suite_set, } filtered.end = copy; - kfree(filter.suite_glob); - kfree(filter.test_glob); + if (filter_glob) { + kfree(parsed_glob.suite_glob); + kfree(parsed_glob.test_glob); + } + + if (filter_count) + kfree(parsed_filters); + return filtered; } @@ -206,8 +260,9 @@ int kunit_run_all_tests(void) goto out; } - if (filter_glob_param) { - suite_set = kunit_filter_suites(&suite_set, filter_glob_param, &err); + if (filter_glob_param || filter_param) { + suite_set = kunit_filter_suites(&suite_set, filter_glob_param, + filter_param, filter_action_param, &err); if (err) { pr_err("kunit executor: error filtering suites: %d\n", err); goto out; @@ -223,7 +278,7 @@ int kunit_run_all_tests(void) else pr_err("kunit executor: unknown action '%s'\n", action_param); - if (filter_glob_param) { /* a copy was made of each suite */ + if (filter_glob_param || filter_param) { /* a copy was made of each suite */ kunit_free_suite_set(suite_set); } diff --git a/lib/kunit/executor_test.c b/lib/kunit/executor_test.c index ce6749af374d..d7ab069324b5 100644 --- a/lib/kunit/executor_test.c +++ b/lib/kunit/executor_test.c @@ -24,15 +24,15 @@ static struct kunit_case dummy_test_cases[] = { static void parse_filter_test(struct kunit *test) { - struct kunit_test_filter filter = {NULL, NULL}; + struct kunit_glob_filter filter = {NULL, NULL}; - kunit_parse_filter_glob(&filter, "suite"); + kunit_parse_glob_filter(&filter, "suite"); KUNIT_EXPECT_STREQ(test, filter.suite_glob, "suite"); KUNIT_EXPECT_FALSE(test, filter.test_glob); kfree(filter.suite_glob); kfree(filter.test_glob); - kunit_parse_filter_glob(&filter, "suite.test"); + kunit_parse_glob_filter(&filter, "suite.test"); KUNIT_EXPECT_STREQ(test, filter.suite_glob, "suite"); KUNIT_EXPECT_STREQ(test, filter.test_glob, "test"); kfree(filter.suite_glob); @@ -50,7 +50,7 @@ static void filter_suites_test(struct kunit *test) subsuite[1] = alloc_fake_suite(test, "suite2", dummy_test_cases); /* Want: suite1, suite2, NULL -> suite2, NULL */ - got = kunit_filter_suites(&suite_set, "suite2", &err); + got = kunit_filter_suites(&suite_set, "suite2", NULL, NULL, &err); KUNIT_ASSERT_NOT_ERR_OR_NULL(test, got.start); KUNIT_ASSERT_EQ(test, err, 0); kfree_at_end(test, got.start); @@ -74,7 +74,7 @@ static void filter_suites_test_glob_test(struct kunit *test) subsuite[1] = alloc_fake_suite(test, "suite2", dummy_test_cases); /* Want: suite1, suite2, NULL -> suite2 (just test1), NULL */ - got = kunit_filter_suites(&suite_set, "suite2.test2", &err); + got = kunit_filter_suites(&suite_set, "suite2.test2", NULL, NULL, &err); KUNIT_ASSERT_NOT_ERR_OR_NULL(test, got.start); KUNIT_ASSERT_EQ(test, err, 0); kfree_at_end(test, got.start); @@ -100,7 +100,7 @@ static void filter_suites_to_empty_test(struct kunit *test) subsuite[0] = alloc_fake_suite(test, "suite1", dummy_test_cases); subsuite[1] = alloc_fake_suite(test, "suite2", dummy_test_cases); - got = kunit_filter_suites(&suite_set, "not_found", &err); + got = kunit_filter_suites(&suite_set, "not_found", NULL, NULL, &err); KUNIT_ASSERT_EQ(test, err, 0); kfree_at_end(test, got.start); /* just in case */ diff --git a/lib/kunit/test.c b/lib/kunit/test.c index 9ee55139ecd1..cb9797fa6303 100644 --- a/lib/kunit/test.c +++ b/lib/kunit/test.c @@ -613,18 +613,22 @@ int kunit_run_tests(struct kunit_suite *suite) kunit_suite_for_each_test_case(suite, test_case) { struct kunit test = { .param_value = NULL, .param_index = 0 }; struct kunit_result_stats param_stats = { 0 }; - test_case->status = KUNIT_SKIPPED; kunit_init_test(&test, test_case->name, test_case->log); - - if (!test_case->generate_params) { + if (test_case->status == KUNIT_SKIPPED) { + /* Test marked as skip */ + test.status = KUNIT_SKIPPED; + kunit_update_stats(¶m_stats, test.status); + } else if (!test_case->generate_params) { /* Non-parameterised test. */ + test_case->status = KUNIT_SKIPPED; kunit_run_case_catch_errors(suite, test_case, &test); kunit_update_stats(¶m_stats, test.status); } else { /* Get initial param. */ param_desc[0] = '\0'; test.param_value = test_case->generate_params(NULL, param_desc); + test_case->status = KUNIT_SKIPPED; kunit_log(KERN_INFO, &test, KUNIT_SUBTEST_INDENT KUNIT_SUBTEST_INDENT "KTAP version 1\n"); kunit_log(KERN_INFO, &test, KUNIT_SUBTEST_INDENT KUNIT_SUBTEST_INDENT -- cgit v1.2.3 From d055c6a2cc162fa3fe23d0a88a279baf46abe85d Mon Sep 17 00:00:00 2001 From: Rae Moar Date: Tue, 25 Jul 2023 21:25:17 +0000 Subject: kunit: memcpy: Mark tests as slow using test attributes Mark slow memcpy KUnit tests using test attributes. Tests marked as slow are as follows: memcpy_large_test, memmove_test, memmove_large_test, and memmove_overlap_test. These tests were the slowest of the memcpy tests and relatively slower to most other KUnit tests. Most of these tests are already skipped when CONFIG_MEMCPY_SLOW_KUNIT_TEST is not enabled. These tests can now be filtered using the KUnit test attribute filtering feature. Example: --filter "speed>slow". This will run only the tests that have speeds faster than slow. The slow attribute will also be outputted in KTAP. Note: This patch is intended to replace the use of CONFIG_MEMCPY_SLOW_KUNIT_TEST and to potentially deprecate this feature. This patch does not remove the config option but does add a note to the config definition commenting on this future shift. Reviewed-by: David Gow Acked-by: Kees Cook Signed-off-by: Rae Moar Signed-off-by: Shuah Khan --- lib/Kconfig.debug | 3 +++ lib/memcpy_kunit.c | 8 ++++---- 2 files changed, 7 insertions(+), 4 deletions(-) (limited to 'lib') diff --git a/lib/Kconfig.debug b/lib/Kconfig.debug index 550cb967b668..1b3894e861f2 100644 --- a/lib/Kconfig.debug +++ b/lib/Kconfig.debug @@ -2701,6 +2701,9 @@ config MEMCPY_SLOW_KUNIT_TEST and bit ranges. These can be very slow, so they are split out as a separate config, in case they need to be disabled. + Note this config option will be replaced by the use of KUnit test + attributes. + config IS_SIGNED_TYPE_KUNIT_TEST tristate "Test is_signed_type() macro" if !KUNIT_ALL_TESTS depends on KUNIT diff --git a/lib/memcpy_kunit.c b/lib/memcpy_kunit.c index 887926f04731..440aee705ccc 100644 --- a/lib/memcpy_kunit.c +++ b/lib/memcpy_kunit.c @@ -551,10 +551,10 @@ static void strtomem_test(struct kunit *test) static struct kunit_case memcpy_test_cases[] = { KUNIT_CASE(memset_test), KUNIT_CASE(memcpy_test), - KUNIT_CASE(memcpy_large_test), - KUNIT_CASE(memmove_test), - KUNIT_CASE(memmove_large_test), - KUNIT_CASE(memmove_overlap_test), + KUNIT_CASE_SLOW(memcpy_large_test), + KUNIT_CASE_SLOW(memmove_test), + KUNIT_CASE_SLOW(memmove_large_test), + KUNIT_CASE_SLOW(memmove_overlap_test), KUNIT_CASE(strtomem_test), {} }; -- cgit v1.2.3 From 76066f93f1df27657eb937b7c9c091e3a6abf4db Mon Sep 17 00:00:00 2001 From: Rae Moar Date: Tue, 25 Jul 2023 21:25:19 +0000 Subject: kunit: add tests for filtering attributes Add four tests to executor_test.c to test behavior of filtering attributes. - parse_filter_attr_test - to test the parsing of inputted filters - filter_attr_test - to test the filtering procedure on attributes - filter_attr_empty_test - to test the behavior when all tests are filtered out - filter_attr_skip_test - to test the configurable filter_action=skip option Reviewed-by: David Gow Signed-off-by: Rae Moar Signed-off-by: Shuah Khan --- lib/kunit/executor_test.c | 116 ++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 116 insertions(+) (limited to 'lib') diff --git a/lib/kunit/executor_test.c b/lib/kunit/executor_test.c index d7ab069324b5..01280cb8d451 100644 --- a/lib/kunit/executor_test.c +++ b/lib/kunit/executor_test.c @@ -7,6 +7,7 @@ */ #include +#include static void kfree_at_end(struct kunit *test, const void *to_free); static struct kunit_suite *alloc_fake_suite(struct kunit *test, @@ -108,11 +109,126 @@ static void filter_suites_to_empty_test(struct kunit *test) "should be empty to indicate no match"); } +static void parse_filter_attr_test(struct kunit *test) +{ + int j, filter_count; + struct kunit_attr_filter *parsed_filters; + char *filters = "speed>slow, module!=example"; + int err = 0; + + filter_count = kunit_get_filter_count(filters); + KUNIT_EXPECT_EQ(test, filter_count, 2); + + parsed_filters = kunit_kcalloc(test, filter_count + 1, sizeof(*parsed_filters), + GFP_KERNEL); + for (j = 0; j < filter_count; j++) { + parsed_filters[j] = kunit_next_attr_filter(&filters, &err); + KUNIT_ASSERT_EQ_MSG(test, err, 0, "failed to parse filter '%s'", filters[j]); + } + + KUNIT_EXPECT_STREQ(test, kunit_attr_filter_name(parsed_filters[0]), "speed"); + KUNIT_EXPECT_STREQ(test, parsed_filters[0].input, ">slow"); + + KUNIT_EXPECT_STREQ(test, kunit_attr_filter_name(parsed_filters[1]), "module"); + KUNIT_EXPECT_STREQ(test, parsed_filters[1].input, "!=example"); +} + +static struct kunit_case dummy_attr_test_cases[] = { + /* .run_case is not important, just needs to be non-NULL */ + { .name = "slow", .run_case = dummy_test, .module_name = "dummy", + .attr.speed = KUNIT_SPEED_SLOW }, + { .name = "normal", .run_case = dummy_test, .module_name = "dummy" }, + {}, +}; + +static void filter_attr_test(struct kunit *test) +{ + struct kunit_suite *subsuite[3] = {NULL, NULL}; + struct suite_set suite_set = {.start = subsuite, .end = &subsuite[2]}; + struct suite_set got; + int err = 0; + + subsuite[0] = alloc_fake_suite(test, "normal_suite", dummy_attr_test_cases); + subsuite[1] = alloc_fake_suite(test, "slow_suite", dummy_attr_test_cases); + subsuite[1]->attr.speed = KUNIT_SPEED_SLOW; // Set suite attribute + + /* + * Want: normal_suite(slow, normal), slow_suite(slow, normal), + * NULL -> normal_suite(normal), NULL + * + * The normal test in slow_suite is filtered out because the speed + * attribute is unset and thus, the filtering is based on the parent attribute + * of slow. + */ + got = kunit_filter_suites(&suite_set, NULL, "speed>slow", NULL, &err); + KUNIT_ASSERT_NOT_ERR_OR_NULL(test, got.start); + KUNIT_ASSERT_EQ(test, err, 0); + kfree_at_end(test, got.start); + + /* Validate we just have normal_suite */ + KUNIT_ASSERT_NOT_ERR_OR_NULL(test, got.start[0]); + KUNIT_EXPECT_STREQ(test, got.start[0]->name, "normal_suite"); + KUNIT_ASSERT_EQ(test, got.end - got.start, 1); + + /* Now validate we just have normal test case */ + KUNIT_ASSERT_NOT_ERR_OR_NULL(test, got.start[0]->test_cases); + KUNIT_EXPECT_STREQ(test, got.start[0]->test_cases[0].name, "normal"); + KUNIT_EXPECT_FALSE(test, got.start[0]->test_cases[1].name); +} + +static void filter_attr_empty_test(struct kunit *test) +{ + struct kunit_suite *subsuite[3] = {NULL, NULL}; + struct suite_set suite_set = {.start = subsuite, .end = &subsuite[2]}; + struct suite_set got; + int err = 0; + + subsuite[0] = alloc_fake_suite(test, "suite1", dummy_attr_test_cases); + subsuite[1] = alloc_fake_suite(test, "suite2", dummy_attr_test_cases); + + got = kunit_filter_suites(&suite_set, NULL, "module!=dummy", NULL, &err); + KUNIT_ASSERT_EQ(test, err, 0); + kfree_at_end(test, got.start); /* just in case */ + + KUNIT_EXPECT_PTR_EQ_MSG(test, got.start, got.end, + "should be empty to indicate no match"); +} + +static void filter_attr_skip_test(struct kunit *test) +{ + struct kunit_suite *subsuite[2] = {NULL}; + struct suite_set suite_set = {.start = subsuite, .end = &subsuite[1]}; + struct suite_set got; + int err = 0; + + subsuite[0] = alloc_fake_suite(test, "suite", dummy_attr_test_cases); + + /* Want: suite(slow, normal), NULL -> suite(slow with SKIP, normal), NULL */ + got = kunit_filter_suites(&suite_set, NULL, "speed>slow", "skip", &err); + KUNIT_ASSERT_NOT_ERR_OR_NULL(test, got.start); + KUNIT_ASSERT_EQ(test, err, 0); + kfree_at_end(test, got.start); + + /* Validate we have both the slow and normal test */ + KUNIT_ASSERT_NOT_ERR_OR_NULL(test, got.start[0]->test_cases); + KUNIT_ASSERT_EQ(test, kunit_suite_num_test_cases(got.start[0]), 2); + KUNIT_EXPECT_STREQ(test, got.start[0]->test_cases[0].name, "slow"); + KUNIT_EXPECT_STREQ(test, got.start[0]->test_cases[1].name, "normal"); + + /* Now ensure slow is skipped and normal is not */ + KUNIT_EXPECT_EQ(test, got.start[0]->test_cases[0].status, KUNIT_SKIPPED); + KUNIT_EXPECT_FALSE(test, got.start[0]->test_cases[1].status); +} + static struct kunit_case executor_test_cases[] = { KUNIT_CASE(parse_filter_test), KUNIT_CASE(filter_suites_test), KUNIT_CASE(filter_suites_test_glob_test), KUNIT_CASE(filter_suites_to_empty_test), + KUNIT_CASE(parse_filter_attr_test), + KUNIT_CASE(filter_attr_test), + KUNIT_CASE(filter_attr_empty_test), + KUNIT_CASE(filter_attr_skip_test), {} }; -- cgit v1.2.3 From 5a175d369c702ce08c9feb630125c9fc7a9e1370 Mon Sep 17 00:00:00 2001 From: Ruan Jinjie Date: Sat, 29 Jul 2023 09:00:03 +0800 Subject: kunit: fix wild-memory-access bug in kunit_filter_suites() As for kunit_filter_suites(), When the filters arg = NULL, such as the call of kunit_filter_suites(&suite_set, "suite2", NULL, NULL, &err) in filter_suites_test() tese case in kunit, both filter_count and parsed_filters will not be initialized. So it's possible to enter kunit_filter_attr_tests(), and the use of uninitialized parsed_filters will cause below wild-memory-access. RIP: 0010:kunit_filter_suites+0x780/0xa40 Code: fe ff ff e8 42 87 4d ff 41 83 c6 01 49 83 c5 10 49 89 dc 44 39 74 24 50 0f 8e 81 fe ff ff e8 27 87 4d ff 4c 89 e8 48 c1 e8 03 <66> 42 83 3c 38 00 0f 85 af 01 00 00 49 8b 75 00 49 8b 55 08 4c 89 RSP: 0000:ff1100010743fc38 EFLAGS: 00010203 RAX: 03fc4400041d0ff1 RBX: ff1100010389a900 RCX: ffffffff9f940ad9 RDX: ff11000107429740 RSI: 0000000000000000 RDI: ff110001037ec920 RBP: ff1100010743fd50 R08: 0000000000000000 R09: ffe21c0020e87f1e R10: 0000000000000003 R11: 0000000000032001 R12: ff110001037ec800 R13: 1fe2200020e87f8c R14: 0000000000000000 R15: dffffc0000000000 FS: 0000000000000000(0000) GS:ff1100011b000000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: ff11000115201000 CR3: 0000000113066001 CR4: 0000000000771ef0 DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400 PKRU: 55555554 Call Trace: ? die_addr+0x3c/0xa0 ? exc_general_protection+0x148/0x220 ? asm_exc_general_protection+0x26/0x30 ? kunit_filter_suites+0x779/0xa40 ? kunit_filter_suites+0x780/0xa40 ? kunit_filter_suites+0x779/0xa40 ? __pfx_kunit_filter_suites+0x10/0x10 ? __pfx_kfree+0x10/0x10 ? kunit_add_action_or_reset+0x3d/0x50 filter_suites_test+0x1b7/0x440 ? __pfx_filter_suites_test+0x10/0x10 ? __pfx___schedule+0x10/0x10 ? try_to_wake_up+0xa8e/0x1210 ? _raw_spin_lock_irqsave+0x86/0xe0 ? __pfx__raw_spin_lock_irqsave+0x10/0x10 ? set_cpus_allowed_ptr+0x7c/0xb0 kunit_try_run_case+0x119/0x270 ? __kthread_parkme+0xdc/0x160 ? __pfx_kunit_try_run_case+0x10/0x10 kunit_generic_run_threadfn_adapter+0x4e/0xa0 ? __pfx_kunit_generic_run_threadfn_adapter+0x10/0x10 kthread+0x2c7/0x3c0 ? __pfx_kthread+0x10/0x10 ret_from_fork+0x2c/0x70 ? __pfx_kthread+0x10/0x10 ret_from_fork_asm+0x1b/0x30 Modules linked in: Dumping ftrace buffer: (ftrace buffer empty) ---[ end trace 0000000000000000 ]--- RIP: 0010:kunit_filter_suites+0x780/0xa40 Code: fe ff ff e8 42 87 4d ff 41 83 c6 01 49 83 c5 10 49 89 dc 44 39 74 24 50 0f 8e 81 fe ff ff e8 27 87 4d ff 4c 89 e8 48 c1 e8 03 <66> 42 83 3c 38 00 0f 85 af 01 00 00 49 8b 75 00 49 8b 55 08 4c 89 RSP: 0000:ff1100010743fc38 EFLAGS: 00010203 RAX: 03fc4400041d0ff1 RBX: ff1100010389a900 RCX: ffffffff9f940ad9 RDX: ff11000107429740 RSI: 0000000000000000 RDI: ff110001037ec920 RBP: ff1100010743fd50 R08: 0000000000000000 R09: ffe21c0020e87f1e R10: 0000000000000003 R11: 0000000000032001 R12: ff110001037ec800 R13: 1fe2200020e87f8c R14: 0000000000000000 R15: dffffc0000000000 FS: 0000000000000000(0000) GS:ff1100011b000000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: ff11000115201000 CR3: 0000000113066001 CR4: 0000000000771ef0 DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400 PKRU: 55555554 Kernel panic - not syncing: Fatal exception Dumping ftrace buffer: (ftrace buffer empty) Kernel Offset: 0x1da00000 from 0xffffffff81000000 (relocation range: 0xffffffff80000000-0xffffffffbfffffff) Rebooting in 1 seconds.. Fixes: 529534e8cba3 ("kunit: Add ability to filter attributes") Signed-off-by: Ruan Jinjie Reviewed-by: David Gow Tested-by: Guenter Roeck Signed-off-by: Shuah Khan --- lib/kunit/executor.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) (limited to 'lib') diff --git a/lib/kunit/executor.c b/lib/kunit/executor.c index 483f7b7873a7..5b5bed1efb93 100644 --- a/lib/kunit/executor.c +++ b/lib/kunit/executor.c @@ -125,7 +125,8 @@ static struct suite_set kunit_filter_suites(const struct suite_set *suite_set, char *filter_action, int *err) { - int i, j, k, filter_count; + int i, j, k; + int filter_count = 0; struct kunit_suite **copy, *filtered_suite, *new_filtered_suite; struct suite_set filtered; struct kunit_glob_filter parsed_glob; -- cgit v1.2.3 From abbf73816b6f5f4268fbfb3b3505003c2356d4a9 Mon Sep 17 00:00:00 2001 From: Ruan Jinjie Date: Tue, 1 Aug 2023 15:37:00 +0800 Subject: kunit: fix possible memory leak in kunit_filter_suites() Inject fault while probing drm_kunit_helpers.ko, if one of kunit_next_attr_filter(), kunit_filter_glob_tests() and kunit_filter_attr_tests() fails, parsed_filters, parsed_glob.suite_glob/test_glob alloced in kunit_parse_glob_filter() is leaked. And the filtered_suite->test_cases alloced in kunit_filter_glob_tests() or kunit_filter_attr_tests() may also be leaked. unreferenced object 0xff110001067e4800 (size 1024): comm "kunit_try_catch", pid 96, jiffies 4294671796 (age 763.547s) hex dump (first 32 bytes): 73 75 69 74 65 32 00 00 00 00 00 00 00 00 00 00 suite2.......... 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ backtrace: [<00000000116e8eba>] __kmalloc_node_track_caller+0x4e/0x140 [<00000000e2f9cce9>] kmemdup+0x2c/0x60 [<000000002a36710b>] kunit_filter_suites+0x3e4/0xa50 [<0000000045779fb9>] filter_suites_test+0x1b7/0x440 [<00000000cd1104a7>] kunit_try_run_case+0x119/0x270 [<00000000c654c917>] kunit_generic_run_threadfn_adapter+0x4e/0xa0 [<00000000d195ac13>] kthread+0x2c7/0x3c0 [<00000000b79c1ee9>] ret_from_fork+0x2c/0x70 [<000000001167f7e6>] ret_from_fork_asm+0x1b/0x30 unreferenced object 0xff11000105d79b00 (size 192): comm "kunit_try_catch", pid 96, jiffies 4294671796 (age 763.547s) hex dump (first 32 bytes): f0 e1 5a 88 ff ff ff ff 60 59 bb 8a ff ff ff ff ..Z.....`Y...... 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ backtrace: [<000000000d6e4891>] __kmalloc+0x4d/0x140 [<000000006afe50bd>] kunit_filter_suites+0x424/0xa50 [<0000000045779fb9>] filter_suites_test+0x1b7/0x440 [<00000000cd1104a7>] kunit_try_run_case+0x119/0x270 [<00000000c654c917>] kunit_generic_run_threadfn_adapter+0x4e/0xa0 [<00000000d195ac13>] kthread+0x2c7/0x3c0 [<00000000b79c1ee9>] ret_from_fork+0x2c/0x70 [<000000001167f7e6>] ret_from_fork_asm+0x1b/0x30 unreferenced object 0xff110001067e6000 (size 1024): comm "kunit_try_catch", pid 98, jiffies 4294671798 (age 763.545s) hex dump (first 32 bytes): 73 75 69 74 65 32 00 00 00 00 00 00 00 00 00 00 suite2.......... 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ backtrace: [<00000000116e8eba>] __kmalloc_node_track_caller+0x4e/0x140 [<00000000e2f9cce9>] kmemdup+0x2c/0x60 [<000000002a36710b>] kunit_filter_suites+0x3e4/0xa50 [<00000000f452f130>] filter_suites_test_glob_test+0x1b7/0x660 [<00000000cd1104a7>] kunit_try_run_case+0x119/0x270 [<00000000c654c917>] kunit_generic_run_threadfn_adapter+0x4e/0xa0 [<00000000d195ac13>] kthread+0x2c7/0x3c0 [<00000000b79c1ee9>] ret_from_fork+0x2c/0x70 [<000000001167f7e6>] ret_from_fork_asm+0x1b/0x30 unreferenced object 0xff11000103f3a800 (size 96): comm "kunit_try_catch", pid 98, jiffies 4294671798 (age 763.545s) hex dump (first 32 bytes): f0 e1 5a 88 ff ff ff ff 40 39 bb 8a ff ff ff ff ..Z.....@9...... 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ backtrace: [<000000000d6e4891>] __kmalloc+0x4d/0x140 [<000000006afe50bd>] kunit_filter_suites+0x424/0xa50 [<00000000f452f130>] filter_suites_test_glob_test+0x1b7/0x660 [<00000000cd1104a7>] kunit_try_run_case+0x119/0x270 [<00000000c654c917>] kunit_generic_run_threadfn_adapter+0x4e/0xa0 [<00000000d195ac13>] kthread+0x2c7/0x3c0 [<00000000b79c1ee9>] ret_from_fork+0x2c/0x70 [<000000001167f7e6>] ret_from_fork_asm+0x1b/0x30 unreferenced object 0xff11000101a72ac0 (size 16): comm "kunit_try_catch", pid 104, jiffies 4294671814 (age 763.529s) hex dump (first 16 bytes): 00 00 00 00 00 00 00 00 e0 2a a7 01 01 00 11 ff .........*...... backtrace: [<000000000d6e4891>] __kmalloc+0x4d/0x140 [<00000000c7b724e7>] kunit_filter_suites+0x108/0xa50 [<00000000bad5427d>] filter_attr_test+0x1e9/0x6a0 [<00000000cd1104a7>] kunit_try_run_case+0x119/0x270 [<00000000c654c917>] kunit_generic_run_threadfn_adapter+0x4e/0xa0 [<00000000d195ac13>] kthread+0x2c7/0x3c0 [<00000000b79c1ee9>] ret_from_fork+0x2c/0x70 [<000000001167f7e6>] ret_from_fork_asm+0x1b/0x30 unreferenced object 0xff11000103caf880 (size 32): comm "kunit_try_catch", pid 104, jiffies 4294671814 (age 763.547s) hex dump (first 32 bytes): 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ backtrace: [<000000000d6e4891>] __kmalloc+0x4d/0x140 [<00000000c47b0f75>] kunit_filter_suites+0x189/0xa50 [<00000000bad5427d>] filter_attr_test+0x1e9/0x6a0 [<00000000cd1104a7>] kunit_try_run_case+0x119/0x270 [<00000000c654c917>] kunit_generic_run_threadfn_adapter+0x4e/0xa0 [<00000000d195ac13>] kthread+0x2c7/0x3c0 [<00000000b79c1ee9>] ret_from_fork+0x2c/0x70 [<000000001167f7e6>] ret_from_fork_asm+0x1b/0x30 unreferenced object 0xff11000101a72ae0 (size 16): comm "kunit_try_catch", pid 106, jiffies 4294671823 (age 763.538s) hex dump (first 16 bytes): 00 00 00 00 00 00 00 00 00 2b a7 01 01 00 11 ff .........+...... backtrace: [<000000000d6e4891>] __kmalloc+0x4d/0x140 [<00000000c7b724e7>] kunit_filter_suites+0x108/0xa50 [<0000000096255c51>] filter_attr_empty_test+0x1b0/0x310 [<00000000cd1104a7>] kunit_try_run_case+0x119/0x270 [<00000000c654c917>] kunit_generic_run_threadfn_adapter+0x4e/0xa0 [<00000000d195ac13>] kthread+0x2c7/0x3c0 [<00000000b79c1ee9>] ret_from_fork+0x2c/0x70 [<000000001167f7e6>] ret_from_fork_asm+0x1b/0x30 unreferenced object 0xff11000103caf9c0 (size 32): comm "kunit_try_catch", pid 106, jiffies 4294671823 (age 763.538s) hex dump (first 32 bytes): 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ backtrace: [<000000000d6e4891>] __kmalloc+0x4d/0x140 [<00000000c47b0f75>] kunit_filter_suites+0x189/0xa50 [<0000000096255c51>] filter_attr_empty_test+0x1b0/0x310 [<00000000cd1104a7>] kunit_try_run_case+0x119/0x270 [<00000000c654c917>] kunit_generic_run_threadfn_adapter+0x4e/0xa0 [<00000000d195ac13>] kthread+0x2c7/0x3c0 [<00000000b79c1ee9>] ret_from_fork+0x2c/0x70 [<000000001167f7e6>] ret_from_fork_asm+0x1b/0x30 unreferenced object 0xff11000101a72b00 (size 16): comm "kunit_try_catch", pid 108, jiffies 4294671832 (age 763.529s) hex dump (first 16 bytes): 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ backtrace: [<000000000d6e4891>] __kmalloc+0x4d/0x140 [<00000000c47b0f75>] kunit_filter_suites+0x189/0xa50 [<00000000881258cc>] filter_attr_skip_test+0x148/0x770 [<00000000cd1104a7>] kunit_try_run_case+0x119/0x270 [<00000000c654c917>] kunit_generic_run_threadfn_adapter+0x4e/0xa0 [<00000000d195ac13>] kthread+0x2c7/0x3c0 [<00000000b79c1ee9>] ret_from_fork+0x2c/0x70 [<000000001167f7e6>] ret_from_fork_asm+0x1b/0x30 Fixes: 5d31f71efcb6 ("kunit: add kunit.filter_glob cmdline option to filter suites") Fixes: 529534e8cba3 ("kunit: Add ability to filter attributes") Signed-off-by: Ruan Jinjie Reviewed-by: Rae Moar Reviewed-by: David Gow Signed-off-by: Shuah Khan --- lib/kunit/executor.c | 17 ++++++++++++----- 1 file changed, 12 insertions(+), 5 deletions(-) (limited to 'lib') diff --git a/lib/kunit/executor.c b/lib/kunit/executor.c index 5b5bed1efb93..481901d245d0 100644 --- a/lib/kunit/executor.c +++ b/lib/kunit/executor.c @@ -151,7 +151,7 @@ static struct suite_set kunit_filter_suites(const struct suite_set *suite_set, for (j = 0; j < filter_count; j++) parsed_filters[j] = kunit_next_attr_filter(&filters, err); if (*err) - return filtered; + goto err; } for (i = 0; &suite_set->start[i] != suite_set->end; i++) { @@ -163,7 +163,7 @@ static struct suite_set kunit_filter_suites(const struct suite_set *suite_set, parsed_glob.test_glob); if (IS_ERR(filtered_suite)) { *err = PTR_ERR(filtered_suite); - return filtered; + goto err; } } if (filter_count) { @@ -172,15 +172,18 @@ static struct suite_set kunit_filter_suites(const struct suite_set *suite_set, parsed_filters[k], filter_action, err); /* Free previous copy of suite */ - if (k > 0 || filter_glob) + if (k > 0 || filter_glob) { + kfree(filtered_suite->test_cases); kfree(filtered_suite); + } + filtered_suite = new_filtered_suite; if (*err) - return filtered; + goto err; if (IS_ERR(filtered_suite)) { *err = PTR_ERR(filtered_suite); - return filtered; + goto err; } if (!filtered_suite) break; @@ -194,6 +197,10 @@ static struct suite_set kunit_filter_suites(const struct suite_set *suite_set, } filtered.end = copy; +err: + if (*err) + kfree(copy); + if (filter_glob) { kfree(parsed_glob.suite_glob); kfree(parsed_glob.test_glob); -- cgit v1.2.3 From 1c9fd080dffe5e5ad763527fbc2aa3f6f8c653e9 Mon Sep 17 00:00:00 2001 From: Rae Moar Date: Thu, 3 Aug 2023 19:36:35 +0000 Subject: kunit: fix uninitialized variables bug in attributes filtering Fix smatch warnings regarding uninitialized variables in the filtering patch of the new KUnit Attributes feature. Fixes: 529534e8cba3 ("kunit: Add ability to filter attributes") Reported-by: kernel test robot Reported-by: Dan Carpenter Closes: https://lore.kernel.org/r/202307270610.s0w4NKEn-lkp@intel.com/ Signed-off-by: Rae Moar Reviewed-by: David Gow Signed-off-by: Shuah Khan --- lib/kunit/attributes.c | 40 +++++++++++++++++----------------------- lib/kunit/executor.c | 18 +++++++++++------- lib/kunit/executor_test.c | 2 +- 3 files changed, 29 insertions(+), 31 deletions(-) (limited to 'lib') diff --git a/lib/kunit/attributes.c b/lib/kunit/attributes.c index d37c40c0ce4f..5e3034b6be99 100644 --- a/lib/kunit/attributes.c +++ b/lib/kunit/attributes.c @@ -102,7 +102,7 @@ static int int_filter(long val, const char *op, int input, int *err) static int attr_enum_filter(void *attr, const char *input, int *err, const char * const str_list[], int max) { - int i, j, input_int; + int i, j, input_int = -1; long test_val = (long)attr; const char *input_val = NULL; @@ -124,7 +124,7 @@ static int attr_enum_filter(void *attr, const char *input, int *err, input_int = j; } - if (!input_int) { + if (input_int < 0) { *err = -EINVAL; pr_err("kunit executor: invalid filter input: %s\n", input); return false; @@ -186,8 +186,10 @@ static void *attr_module_get(void *test_or_suite, bool is_test) // Suites get their module attribute from their first test_case if (test) return ((void *) test->module_name); - else + else if (kunit_suite_num_test_cases(suite) > 0) return ((void *) suite->test_cases[0].module_name); + else + return (void *) ""; } /* List of all Test Attributes */ @@ -221,7 +223,7 @@ const char *kunit_attr_filter_name(struct kunit_attr_filter filter) void kunit_print_attr(void *test_or_suite, bool is_test, unsigned int test_level) { int i; - bool to_free; + bool to_free = false; void *attr; const char *attr_name, *attr_str; struct kunit_suite *suite = is_test ? NULL : test_or_suite; @@ -255,7 +257,7 @@ void kunit_print_attr(void *test_or_suite, bool is_test, unsigned int test_level int kunit_get_filter_count(char *input) { - int i, comma_index, count = 0; + int i, comma_index = 0, count = 0; for (i = 0; input[i]; i++) { if (input[i] == ',') { @@ -272,7 +274,7 @@ int kunit_get_filter_count(char *input) struct kunit_attr_filter kunit_next_attr_filter(char **filters, int *err) { struct kunit_attr_filter filter = {}; - int i, j, comma_index, new_start_index; + int i, j, comma_index = 0, new_start_index = 0; int op_index = -1, attr_index = -1; char op; char *input = *filters; @@ -316,7 +318,7 @@ struct kunit_attr_filter kunit_next_attr_filter(char **filters, int *err) filter.attr = &kunit_attr_list[attr_index]; } - if (comma_index) { + if (comma_index > 0) { input[comma_index] = '\0'; filter.input = input + op_index; input = input + new_start_index; @@ -356,31 +358,22 @@ struct kunit_suite *kunit_filter_attr_tests(const struct kunit_suite *const suit /* Save filtering result on default value */ default_result = filter.attr->filter(filter.attr->attr_default, filter.input, err); - if (*err) { - kfree(copy); - kfree(filtered); - return NULL; - } + if (*err) + goto err; /* Save suite attribute value and filtering result on that value */ suite_val = filter.attr->get_attr((void *)suite, false); suite_result = filter.attr->filter(suite_val, filter.input, err); - if (*err) { - kfree(copy); - kfree(filtered); - return NULL; - } + if (*err) + goto err; /* For each test case, save test case if passes filtering. */ kunit_suite_for_each_test_case(suite, test_case) { test_val = filter.attr->get_attr((void *) test_case, true); test_result = filter.attr->filter(filter.attr->get_attr(test_case, true), filter.input, err); - if (*err) { - kfree(copy); - kfree(filtered); - return NULL; - } + if (*err) + goto err; /* * If attribute value of test case is set, filter on that value. @@ -406,7 +399,8 @@ struct kunit_suite *kunit_filter_attr_tests(const struct kunit_suite *const suit } } - if (n == 0) { +err: + if (n == 0 || *err) { kfree(copy); kfree(filtered); return NULL; diff --git a/lib/kunit/executor.c b/lib/kunit/executor.c index 481901d245d0..dc295150c4e5 100644 --- a/lib/kunit/executor.c +++ b/lib/kunit/executor.c @@ -127,19 +127,18 @@ static struct suite_set kunit_filter_suites(const struct suite_set *suite_set, { int i, j, k; int filter_count = 0; - struct kunit_suite **copy, *filtered_suite, *new_filtered_suite; - struct suite_set filtered; + struct kunit_suite **copy, **copy_start, *filtered_suite, *new_filtered_suite; + struct suite_set filtered = {NULL, NULL}; struct kunit_glob_filter parsed_glob; - struct kunit_attr_filter *parsed_filters; + struct kunit_attr_filter *parsed_filters = NULL; const size_t max = suite_set->end - suite_set->start; copy = kmalloc_array(max, sizeof(*filtered.start), GFP_KERNEL); - filtered.start = copy; if (!copy) { /* won't be able to run anything, return an empty set */ - filtered.end = copy; return filtered; } + copy_start = copy; if (filter_glob) kunit_parse_glob_filter(&parsed_glob, filter_glob); @@ -147,7 +146,11 @@ static struct suite_set kunit_filter_suites(const struct suite_set *suite_set, /* Parse attribute filters */ if (filters) { filter_count = kunit_get_filter_count(filters); - parsed_filters = kcalloc(filter_count + 1, sizeof(*parsed_filters), GFP_KERNEL); + parsed_filters = kcalloc(filter_count, sizeof(*parsed_filters), GFP_KERNEL); + if (!parsed_filters) { + kfree(copy); + return filtered; + } for (j = 0; j < filter_count; j++) parsed_filters[j] = kunit_next_attr_filter(&filters, err); if (*err) @@ -166,7 +169,7 @@ static struct suite_set kunit_filter_suites(const struct suite_set *suite_set, goto err; } } - if (filter_count) { + if (filter_count > 0 && parsed_filters != NULL) { for (k = 0; k < filter_count; k++) { new_filtered_suite = kunit_filter_attr_tests(filtered_suite, parsed_filters[k], filter_action, err); @@ -195,6 +198,7 @@ static struct suite_set kunit_filter_suites(const struct suite_set *suite_set, *copy++ = filtered_suite; } + filtered.start = copy_start; filtered.end = copy; err: diff --git a/lib/kunit/executor_test.c b/lib/kunit/executor_test.c index 01280cb8d451..3e0a1c99cb4e 100644 --- a/lib/kunit/executor_test.c +++ b/lib/kunit/executor_test.c @@ -119,7 +119,7 @@ static void parse_filter_attr_test(struct kunit *test) filter_count = kunit_get_filter_count(filters); KUNIT_EXPECT_EQ(test, filter_count, 2); - parsed_filters = kunit_kcalloc(test, filter_count + 1, sizeof(*parsed_filters), + parsed_filters = kunit_kcalloc(test, filter_count, sizeof(*parsed_filters), GFP_KERNEL); for (j = 0; j < filter_count; j++) { parsed_filters[j] = kunit_next_attr_filter(&filters, &err); -- cgit v1.2.3 From c95e7c05c139b1a8a51d368bde57cf20ce931a98 Mon Sep 17 00:00:00 2001 From: Janusz Krzysztofik Date: Mon, 7 Aug 2023 12:23:54 +0200 Subject: kunit: Report the count of test suites in a module According to KTAP specification[1], results should always start from a header that provides a TAP protocol version, followed by a test plan with a count of items to be executed. That pattern should be followed at each nesting level. In the current implementation of the top-most, i.e., test suite level, those rules apply only for test suites built into the kernel, executed and reported on boot. Results submitted to dmesg from kunit test modules loaded later are missing those top-level headers. As a consequence, if a kunit test module provides more than one test suite then, without the top level test plan, external tools that are parsing dmesg for kunit test output are not able to tell how many test suites should be expected and whether to continue parsing after complete output from the first test suite is collected. Submit the top-level headers also from the kunit test module notifier initialization callback. v3: Fix new name of a structure moved to kunit namespace not updated in executor_test functions (lkp@intel.com). v2: Use kunit_exec_run_tests() (Mauro, Rae), but prevent it from emitting the headers when called on load of non-test modules. [1] https://docs.kernel.org/dev-tools/ktap.html# Signed-off-by: Janusz Krzysztofik Cc: Mauro Carvalho Chehab Cc: Rae Moar Reviewed-by: Rae Moar Reviewed-by: David Gow Signed-off-by: Shuah Khan --- include/kunit/test.h | 8 ++++++++ lib/kunit/executor.c | 42 +++++++++++++++++++++++------------------- lib/kunit/executor_test.c | 36 ++++++++++++++++++++++++------------ lib/kunit/test.c | 6 +++++- 4 files changed, 60 insertions(+), 32 deletions(-) (limited to 'lib') diff --git a/include/kunit/test.h b/include/kunit/test.h index 011e0d6bb506..3d002e6b252f 100644 --- a/include/kunit/test.h +++ b/include/kunit/test.h @@ -256,6 +256,12 @@ struct kunit_suite { int suite_init_err; }; +/* Stores an array of suites, end points one past the end */ +struct kunit_suite_set { + struct kunit_suite * const *start; + struct kunit_suite * const *end; +}; + /** * struct kunit - represents a running instance of a test. * @@ -317,6 +323,8 @@ int __kunit_test_suites_init(struct kunit_suite * const * const suites, int num_ void __kunit_test_suites_exit(struct kunit_suite **suites, int num_suites); +void kunit_exec_run_tests(struct kunit_suite_set *suite_set, bool builtin); + #if IS_BUILTIN(CONFIG_KUNIT) int kunit_run_all_tests(void); #else diff --git a/lib/kunit/executor.c b/lib/kunit/executor.c index dc295150c4e5..5ef90c334eb0 100644 --- a/lib/kunit/executor.c +++ b/lib/kunit/executor.c @@ -104,13 +104,7 @@ kunit_filter_glob_tests(const struct kunit_suite *const suite, const char *test_ static char *kunit_shutdown; core_param(kunit_shutdown, kunit_shutdown, charp, 0644); -/* Stores an array of suites, end points one past the end */ -struct suite_set { - struct kunit_suite * const *start; - struct kunit_suite * const *end; -}; - -static void kunit_free_suite_set(struct suite_set suite_set) +static void kunit_free_suite_set(struct kunit_suite_set suite_set) { struct kunit_suite * const *suites; @@ -119,16 +113,17 @@ static void kunit_free_suite_set(struct suite_set suite_set) kfree(suite_set.start); } -static struct suite_set kunit_filter_suites(const struct suite_set *suite_set, - const char *filter_glob, - char *filters, - char *filter_action, - int *err) +static struct kunit_suite_set +kunit_filter_suites(const struct kunit_suite_set *suite_set, + const char *filter_glob, + char *filters, + char *filter_action, + int *err) { int i, j, k; int filter_count = 0; struct kunit_suite **copy, **copy_start, *filtered_suite, *new_filtered_suite; - struct suite_set filtered = {NULL, NULL}; + struct kunit_suite_set filtered = {NULL, NULL}; struct kunit_glob_filter parsed_glob; struct kunit_attr_filter *parsed_filters = NULL; @@ -230,17 +225,24 @@ static void kunit_handle_shutdown(void) } -static void kunit_exec_run_tests(struct suite_set *suite_set) +#endif + +void kunit_exec_run_tests(struct kunit_suite_set *suite_set, bool builtin) { size_t num_suites = suite_set->end - suite_set->start; - pr_info("KTAP version 1\n"); - pr_info("1..%zu\n", num_suites); + if (builtin || num_suites) { + pr_info("KTAP version 1\n"); + pr_info("1..%zu\n", num_suites); + } __kunit_test_suites_init(suite_set->start, num_suites); } -static void kunit_exec_list_tests(struct suite_set *suite_set, bool include_attr) +#if IS_BUILTIN(CONFIG_KUNIT) + +static void kunit_exec_list_tests(struct kunit_suite_set *suite_set, + bool include_attr) { struct kunit_suite * const *suites; struct kunit_case *test_case; @@ -265,7 +267,9 @@ static void kunit_exec_list_tests(struct suite_set *suite_set, bool include_attr int kunit_run_all_tests(void) { - struct suite_set suite_set = {__kunit_suites_start, __kunit_suites_end}; + struct kunit_suite_set suite_set = { + __kunit_suites_start, __kunit_suites_end, + }; int err = 0; if (!kunit_enabled()) { pr_info("kunit: disabled\n"); @@ -282,7 +286,7 @@ int kunit_run_all_tests(void) } if (!action_param) - kunit_exec_run_tests(&suite_set); + kunit_exec_run_tests(&suite_set, true); else if (strcmp(action_param, "list") == 0) kunit_exec_list_tests(&suite_set, false); else if (strcmp(action_param, "list_attr") == 0) diff --git a/lib/kunit/executor_test.c b/lib/kunit/executor_test.c index 3e0a1c99cb4e..4084071d0eb5 100644 --- a/lib/kunit/executor_test.c +++ b/lib/kunit/executor_test.c @@ -43,8 +43,10 @@ static void parse_filter_test(struct kunit *test) static void filter_suites_test(struct kunit *test) { struct kunit_suite *subsuite[3] = {NULL, NULL}; - struct suite_set suite_set = {.start = subsuite, .end = &subsuite[2]}; - struct suite_set got; + struct kunit_suite_set suite_set = { + .start = subsuite, .end = &subsuite[2], + }; + struct kunit_suite_set got; int err = 0; subsuite[0] = alloc_fake_suite(test, "suite1", dummy_test_cases); @@ -67,8 +69,10 @@ static void filter_suites_test(struct kunit *test) static void filter_suites_test_glob_test(struct kunit *test) { struct kunit_suite *subsuite[3] = {NULL, NULL}; - struct suite_set suite_set = {.start = subsuite, .end = &subsuite[2]}; - struct suite_set got; + struct kunit_suite_set suite_set = { + .start = subsuite, .end = &subsuite[2], + }; + struct kunit_suite_set got; int err = 0; subsuite[0] = alloc_fake_suite(test, "suite1", dummy_test_cases); @@ -94,8 +98,10 @@ static void filter_suites_test_glob_test(struct kunit *test) static void filter_suites_to_empty_test(struct kunit *test) { struct kunit_suite *subsuite[3] = {NULL, NULL}; - struct suite_set suite_set = {.start = subsuite, .end = &subsuite[2]}; - struct suite_set got; + struct kunit_suite_set suite_set = { + .start = subsuite, .end = &subsuite[2], + }; + struct kunit_suite_set got; int err = 0; subsuite[0] = alloc_fake_suite(test, "suite1", dummy_test_cases); @@ -144,8 +150,10 @@ static struct kunit_case dummy_attr_test_cases[] = { static void filter_attr_test(struct kunit *test) { struct kunit_suite *subsuite[3] = {NULL, NULL}; - struct suite_set suite_set = {.start = subsuite, .end = &subsuite[2]}; - struct suite_set got; + struct kunit_suite_set suite_set = { + .start = subsuite, .end = &subsuite[2], + }; + struct kunit_suite_set got; int err = 0; subsuite[0] = alloc_fake_suite(test, "normal_suite", dummy_attr_test_cases); @@ -179,8 +187,10 @@ static void filter_attr_test(struct kunit *test) static void filter_attr_empty_test(struct kunit *test) { struct kunit_suite *subsuite[3] = {NULL, NULL}; - struct suite_set suite_set = {.start = subsuite, .end = &subsuite[2]}; - struct suite_set got; + struct kunit_suite_set suite_set = { + .start = subsuite, .end = &subsuite[2], + }; + struct kunit_suite_set got; int err = 0; subsuite[0] = alloc_fake_suite(test, "suite1", dummy_attr_test_cases); @@ -197,8 +207,10 @@ static void filter_attr_empty_test(struct kunit *test) static void filter_attr_skip_test(struct kunit *test) { struct kunit_suite *subsuite[2] = {NULL}; - struct suite_set suite_set = {.start = subsuite, .end = &subsuite[1]}; - struct suite_set got; + struct kunit_suite_set suite_set = { + .start = subsuite, .end = &subsuite[1], + }; + struct kunit_suite_set got; int err = 0; subsuite[0] = alloc_fake_suite(test, "suite", dummy_attr_test_cases); diff --git a/lib/kunit/test.c b/lib/kunit/test.c index cb9797fa6303..8b2808068497 100644 --- a/lib/kunit/test.c +++ b/lib/kunit/test.c @@ -736,7 +736,11 @@ EXPORT_SYMBOL_GPL(__kunit_test_suites_exit); #ifdef CONFIG_MODULES static void kunit_module_init(struct module *mod) { - __kunit_test_suites_init(mod->kunit_suites, mod->num_kunit_suites); + struct kunit_suite_set suite_set = { + mod->kunit_suites, mod->kunit_suites + mod->num_kunit_suites, + }; + + kunit_exec_run_tests(&suite_set, false); } static void kunit_module_exit(struct module *mod) -- cgit v1.2.3 From 18258c60f8a74b9c39b593d118f13f3265d44cd6 Mon Sep 17 00:00:00 2001 From: Janusz Krzysztofik Date: Mon, 7 Aug 2023 12:23:55 +0200 Subject: kunit: Make 'list' action available to kunit test modules Results from kunit tests reported via dmesg may be interleaved with other kernel messages. When parsing dmesg for modular kunit results in real time, external tools, e.g., Intel GPU tools (IGT), may want to insert their own test name markers into dmesg at the start of each test, before any kernel message related to that test appears there, so existing upper level test result parsers have no doubt which test to blame for a specific kernel message. Unfortunately, kunit reports names of tests only at their completion (with the exeption of a not standarized "# Subtest: " header above a test plan of each test suite or parametrized test). External tools could be able to insert their own "start of the test" markers with test names included if they new those names in advance. Test names could be learned from a list if provided by a kunit test module. There exists a feature of listing kunit tests without actually executing them, but it is now limited to configurations with the kunit module built in and covers only built-in tests, already available at boot time. Moreover, switching from list to normal mode requires reboot. If that feature was also available when kunit is built as a module, userspace could load the module with action=list parameter, load some kunit test modules they are interested in and learn about the list of tests provided by those modules, then unload them, reload the kunit module in normal mode and execute the tests with their lists already known. Extend kunit module notifier initialization callback with a processing path for only listing the tests provided by a module if the kunit action parameter is set to "list" or "list_attr". For user convenience, make the kunit.action parameter visible in sysfs. v2: Don't use a different format, use kunit_exec_list_tests() (Rae), - refresh on top of new attributes patches, handle newly introduced kunit.action=list_attr case (Rae). Signed-off-by: Janusz Krzysztofik Cc: Rae Moar Reviewed-by: David Gow Signed-off-by: Shuah Khan --- include/kunit/test.h | 2 ++ lib/kunit/executor.c | 28 +++++++++++++++++----------- lib/kunit/test.c | 18 +++++++++++++++--- 3 files changed, 34 insertions(+), 14 deletions(-) (limited to 'lib') diff --git a/include/kunit/test.h b/include/kunit/test.h index 3d002e6b252f..6a338a3267ed 100644 --- a/include/kunit/test.h +++ b/include/kunit/test.h @@ -309,6 +309,7 @@ static inline void kunit_set_failure(struct kunit *test) } bool kunit_enabled(void); +const char *kunit_action(void); void kunit_init_test(struct kunit *test, const char *name, char *log); @@ -324,6 +325,7 @@ int __kunit_test_suites_init(struct kunit_suite * const * const suites, int num_ void __kunit_test_suites_exit(struct kunit_suite **suites, int num_suites); void kunit_exec_run_tests(struct kunit_suite_set *suite_set, bool builtin); +void kunit_exec_list_tests(struct kunit_suite_set *suite_set, bool include_attr); #if IS_BUILTIN(CONFIG_KUNIT) int kunit_run_all_tests(void); diff --git a/lib/kunit/executor.c b/lib/kunit/executor.c index 5ef90c334eb0..e877c1f1e75c 100644 --- a/lib/kunit/executor.c +++ b/lib/kunit/executor.c @@ -13,22 +13,29 @@ extern struct kunit_suite * const __kunit_suites_start[]; extern struct kunit_suite * const __kunit_suites_end[]; +static char *action_param; + +module_param_named(action, action_param, charp, 0400); +MODULE_PARM_DESC(action, + "Changes KUnit executor behavior, valid values are:\n" + ": run the tests like normal\n" + "'list' to list test names instead of running them.\n" + "'list_attr' to list test names and attributes instead of running them.\n"); + +const char *kunit_action(void) +{ + return action_param; +} + #if IS_BUILTIN(CONFIG_KUNIT) static char *filter_glob_param; -static char *action_param; static char *filter_param; static char *filter_action_param; module_param_named(filter_glob, filter_glob_param, charp, 0); MODULE_PARM_DESC(filter_glob, "Filter which KUnit test suites/tests run at boot-time, e.g. list* or list*.*del_test"); -module_param_named(action, action_param, charp, 0); -MODULE_PARM_DESC(action, - "Changes KUnit executor behavior, valid values are:\n" - ": run the tests like normal\n" - "'list' to list test names instead of running them.\n" - "'list_attr' to list test names and attributes instead of running them.\n"); module_param_named(filter, filter_param, charp, 0); MODULE_PARM_DESC(filter, "Filter which KUnit test suites/tests run at boot-time using attributes, e.g. speed>slow"); @@ -239,10 +246,7 @@ void kunit_exec_run_tests(struct kunit_suite_set *suite_set, bool builtin) __kunit_test_suites_init(suite_set->start, num_suites); } -#if IS_BUILTIN(CONFIG_KUNIT) - -static void kunit_exec_list_tests(struct kunit_suite_set *suite_set, - bool include_attr) +void kunit_exec_list_tests(struct kunit_suite_set *suite_set, bool include_attr) { struct kunit_suite * const *suites; struct kunit_case *test_case; @@ -265,6 +269,8 @@ static void kunit_exec_list_tests(struct kunit_suite_set *suite_set, } } +#if IS_BUILTIN(CONFIG_KUNIT) + int kunit_run_all_tests(void) { struct kunit_suite_set suite_set = { diff --git a/lib/kunit/test.c b/lib/kunit/test.c index 8b2808068497..5232a4373782 100644 --- a/lib/kunit/test.c +++ b/lib/kunit/test.c @@ -739,13 +739,25 @@ static void kunit_module_init(struct module *mod) struct kunit_suite_set suite_set = { mod->kunit_suites, mod->kunit_suites + mod->num_kunit_suites, }; - - kunit_exec_run_tests(&suite_set, false); + const char *action = kunit_action(); + + if (!action) + kunit_exec_run_tests(&suite_set, false); + else if (!strcmp(action, "list")) + kunit_exec_list_tests(&suite_set, false); + else if (!strcmp(action, "list_attr")) + kunit_exec_list_tests(&suite_set, true); + else + pr_err("kunit: unknown action '%s'\n", action); } static void kunit_module_exit(struct module *mod) { - __kunit_test_suites_exit(mod->kunit_suites, mod->num_kunit_suites); + const char *action = kunit_action(); + + if (!action) + __kunit_test_suites_exit(mod->kunit_suites, + mod->num_kunit_suites); } static int kunit_module_notify(struct notifier_block *nb, unsigned long val, -- cgit v1.2.3 From b67abaad4d25b5d9364a1d4f6bc18286ebaaa013 Mon Sep 17 00:00:00 2001 From: Janusz Krzysztofik Date: Mon, 7 Aug 2023 12:23:56 +0200 Subject: kunit: Allow kunit test modules to use test filtering External tools, e.g., Intel GPU tools (IGT), support execution of individual selftests provided by kernel modules. That could be also applicable to kunit test modules if they provided test filtering. But test filtering is now possible only when kunit code is built into the kernel. Moreover, a filter can be specified only at boot time, then reboot is required each time a different filter is needed. Build the test filtering code also when kunit is configured as a module, expose test filtering functions to other kunit source files, and use them in kunit module notifier callback functions. Userspace can then reload the kunit module with a value of the filter_glob parameter tuned to a specific kunit test module every time it wants to limit the scope of tests executed on that module load. Make the kunit.filter* parameters visible in sysfs for user convenience. v5: Refresh on tpp of attributes filtering fix v4: Refresh on top of newly applied attributes patches and changes introdced by new versions of other patches submitted in series with this one. v3: Fix CONFIG_GLOB, required by filtering functions, not selected when building as a module (lkp@intel.com). v2: Fix new name of a structure moved to kunit namespace not updated across all uses (lkp@intel.com). Signed-off-by: Janusz Krzysztofik Reviewed-by: David Gow Signed-off-by: Shuah Khan --- include/kunit/test.h | 11 +++++++++ lib/kunit/Kconfig | 2 +- lib/kunit/executor.c | 63 ++++++++++++++++++++++++++++++---------------------- lib/kunit/test.c | 17 ++++++++++++++ 4 files changed, 66 insertions(+), 27 deletions(-) (limited to 'lib') diff --git a/include/kunit/test.h b/include/kunit/test.h index 6a338a3267ed..d33114097d0d 100644 --- a/include/kunit/test.h +++ b/include/kunit/test.h @@ -310,6 +310,9 @@ static inline void kunit_set_failure(struct kunit *test) bool kunit_enabled(void); const char *kunit_action(void); +const char *kunit_filter_glob(void); +char *kunit_filter(void); +char *kunit_filter_action(void); void kunit_init_test(struct kunit *test, const char *name, char *log); @@ -320,6 +323,14 @@ size_t kunit_suite_num_test_cases(struct kunit_suite *suite); unsigned int kunit_test_case_num(struct kunit_suite *suite, struct kunit_case *test_case); +struct kunit_suite_set +kunit_filter_suites(const struct kunit_suite_set *suite_set, + const char *filter_glob, + char *filters, + char *filter_action, + int *err); +void kunit_free_suite_set(struct kunit_suite_set suite_set); + int __kunit_test_suites_init(struct kunit_suite * const * const suites, int num_suites); void __kunit_test_suites_exit(struct kunit_suite **suites, int num_suites); diff --git a/lib/kunit/Kconfig b/lib/kunit/Kconfig index 626719b95bad..68a6daec0aef 100644 --- a/lib/kunit/Kconfig +++ b/lib/kunit/Kconfig @@ -4,7 +4,7 @@ menuconfig KUNIT tristate "KUnit - Enable support for unit tests" - select GLOB if KUNIT=y + select GLOB help Enables support for kernel unit tests (KUnit), a lightweight unit testing and mocking framework for the Linux kernel. These tests are diff --git a/lib/kunit/executor.c b/lib/kunit/executor.c index e877c1f1e75c..5181aa2e760b 100644 --- a/lib/kunit/executor.c +++ b/lib/kunit/executor.c @@ -27,24 +27,37 @@ const char *kunit_action(void) return action_param; } -#if IS_BUILTIN(CONFIG_KUNIT) - static char *filter_glob_param; static char *filter_param; static char *filter_action_param; -module_param_named(filter_glob, filter_glob_param, charp, 0); +module_param_named(filter_glob, filter_glob_param, charp, 0400); MODULE_PARM_DESC(filter_glob, "Filter which KUnit test suites/tests run at boot-time, e.g. list* or list*.*del_test"); -module_param_named(filter, filter_param, charp, 0); +module_param_named(filter, filter_param, charp, 0400); MODULE_PARM_DESC(filter, "Filter which KUnit test suites/tests run at boot-time using attributes, e.g. speed>slow"); -module_param_named(filter_action, filter_action_param, charp, 0); +module_param_named(filter_action, filter_action_param, charp, 0400); MODULE_PARM_DESC(filter_action, "Changes behavior of filtered tests using attributes, valid values are:\n" ": do not run filtered tests as normal\n" "'skip': skip all filtered tests instead so tests will appear in output\n"); +const char *kunit_filter_glob(void) +{ + return filter_glob_param; +} + +char *kunit_filter(void) +{ + return filter_param; +} + +char *kunit_filter_action(void) +{ + return filter_action_param; +} + /* glob_match() needs NULL terminated strings, so we need a copy of filter_glob_param. */ struct kunit_glob_filter { char *suite_glob; @@ -108,10 +121,7 @@ kunit_filter_glob_tests(const struct kunit_suite *const suite, const char *test_ return copy; } -static char *kunit_shutdown; -core_param(kunit_shutdown, kunit_shutdown, charp, 0644); - -static void kunit_free_suite_set(struct kunit_suite_set suite_set) +void kunit_free_suite_set(struct kunit_suite_set suite_set) { struct kunit_suite * const *suites; @@ -120,7 +130,7 @@ static void kunit_free_suite_set(struct kunit_suite_set suite_set) kfree(suite_set.start); } -static struct kunit_suite_set +struct kunit_suite_set kunit_filter_suites(const struct kunit_suite_set *suite_set, const char *filter_glob, char *filters, @@ -218,22 +228,6 @@ err: return filtered; } -static void kunit_handle_shutdown(void) -{ - if (!kunit_shutdown) - return; - - if (!strcmp(kunit_shutdown, "poweroff")) - kernel_power_off(); - else if (!strcmp(kunit_shutdown, "halt")) - kernel_halt(); - else if (!strcmp(kunit_shutdown, "reboot")) - kernel_restart(NULL); - -} - -#endif - void kunit_exec_run_tests(struct kunit_suite_set *suite_set, bool builtin) { size_t num_suites = suite_set->end - suite_set->start; @@ -271,6 +265,23 @@ void kunit_exec_list_tests(struct kunit_suite_set *suite_set, bool include_attr) #if IS_BUILTIN(CONFIG_KUNIT) +static char *kunit_shutdown; +core_param(kunit_shutdown, kunit_shutdown, charp, 0644); + +static void kunit_handle_shutdown(void) +{ + if (!kunit_shutdown) + return; + + if (!strcmp(kunit_shutdown, "poweroff")) + kernel_power_off(); + else if (!strcmp(kunit_shutdown, "halt")) + kernel_halt(); + else if (!strcmp(kunit_shutdown, "reboot")) + kernel_restart(NULL); + +} + int kunit_run_all_tests(void) { struct kunit_suite_set suite_set = { diff --git a/lib/kunit/test.c b/lib/kunit/test.c index 5232a4373782..49698a168437 100644 --- a/lib/kunit/test.c +++ b/lib/kunit/test.c @@ -740,6 +740,17 @@ static void kunit_module_init(struct module *mod) mod->kunit_suites, mod->kunit_suites + mod->num_kunit_suites, }; const char *action = kunit_action(); + int err = 0; + + suite_set = kunit_filter_suites(&suite_set, + kunit_filter_glob() ?: "*.*", + kunit_filter(), kunit_filter_action(), + &err); + if (err) + pr_err("kunit module: error filtering suites: %d\n", err); + + mod->kunit_suites = (struct kunit_suite **)suite_set.start; + mod->num_kunit_suites = suite_set.end - suite_set.start; if (!action) kunit_exec_run_tests(&suite_set, false); @@ -753,11 +764,17 @@ static void kunit_module_init(struct module *mod) static void kunit_module_exit(struct module *mod) { + struct kunit_suite_set suite_set = { + mod->kunit_suites, mod->kunit_suites + mod->num_kunit_suites, + }; const char *action = kunit_action(); if (!action) __kunit_test_suites_exit(mod->kunit_suites, mod->num_kunit_suites); + + if (suite_set.start) + kunit_free_suite_set(suite_set); } static int kunit_module_notify(struct notifier_block *nb, unsigned long val, -- cgit v1.2.3 From 25e324bc9cf2ee956eec1db384c39c1a17b7c44a Mon Sep 17 00:00:00 2001 From: Rae Moar Date: Thu, 17 Aug 2023 19:14:51 +0000 Subject: kunit: fix struct kunit_attr header Add parameter descriptions to struct kunit_attr header for the parameters attr_default and print. Fixes: 39e92cb1e4a1 ("kunit: Add test attributes API structure") Reported-by: kernel test robot Closes: https://lore.kernel.org/oe-kbuild-all/202308180127.VD7YRPGa-lkp@intel.com/ Signed-off-by: Rae Moar Reviewed-by: David Gow Signed-off-by: Shuah Khan --- lib/kunit/attributes.c | 2 ++ 1 file changed, 2 insertions(+) (limited to 'lib') diff --git a/lib/kunit/attributes.c b/lib/kunit/attributes.c index 5e3034b6be99..1b512f7e1838 100644 --- a/lib/kunit/attributes.c +++ b/lib/kunit/attributes.c @@ -30,6 +30,8 @@ enum print_ops { * attribute value * @filter: function to indicate whether a given attribute value passes a * filter + * @attr_default: default attribute value used during filtering + * @print: value of enum print_ops to indicate when to print attribute */ struct kunit_attr { const char *name; -- cgit v1.2.3