diff options
author | Simon Glass <sjg@chromium.org> | 2022-10-30 04:47:13 +0300 |
---|---|---|
committer | Simon Glass <sjg@chromium.org> | 2022-11-08 02:24:30 +0300 |
commit | d1b46595700b063faaec3e33f5754642e68b3d8f (patch) | |
tree | d63e52f47c82a554fc26f3bcfe5b3d7064532175 /doc/develop | |
parent | 6580b618306a24b6ae2974318922c40588ab0284 (diff) | |
download | u-boot-d1b46595700b063faaec3e33f5754642e68b3d8f.tar.xz |
test: Add a way to detect a test that breaks another
When running unit tests, some may have side effects which cause a
subsequent test to break. This can sometimes be seen when using 'ut dm'
or similar.
Add a new argument which allows a particular (failing) test to be run
immediately after a certain number of tests have run. This allows the
test causing the failure to be determined.
Update the documentation also.
Signed-off-by: Simon Glass <sjg@chromium.org>
Diffstat (limited to 'doc/develop')
-rw-r--r-- | doc/develop/tests_sandbox.rst | 69 |
1 files changed, 69 insertions, 0 deletions
diff --git a/doc/develop/tests_sandbox.rst b/doc/develop/tests_sandbox.rst index 8e42a32afb..bfd3bdb927 100644 --- a/doc/develop/tests_sandbox.rst +++ b/doc/develop/tests_sandbox.rst @@ -143,6 +143,75 @@ For example:: Test dm_test_rtc_reset failed 3 times +Isolating a test that breaks another +------------------------------------ + +When running unit tests, some may have side effects which cause a subsequent +test to break. This can sometimes be seen when using 'ut dm' or similar. + +You can use the `-I` argument to the `ut` command to isolate this problem. +First use `ut info` to see how many tests there are, then use a binary search to +home in on the problem. Note that you might need to restart U-Boot after each +iteration, so the `-c` argument to U-Boot is useful. + +For example, let's stay that dm_test_host() is failing:: + + => ut dm + ... + Test: dm_test_get_stats: core.c + Test: dm_test_get_stats: core.c (flat tree) + Test: dm_test_host: host.c + test/dm/host.c:71, dm_test_host(): 0 == ut_check_delta(mem_start): Expected 0x0 (0), got 0xffffcbb0 (-13392) + Test: dm_test_host: host.c (flat tree) + Test <NULL> failed 1 times + Test: dm_test_host_dup: host.c + Test: dm_test_host_dup: host.c (flat tree) + ... + +You can then tell U-Boot to run the failing test at different points in the +sequence: + + => ut info + Test suites: 21 + Total tests: 645 + +:: + + $ ./u-boot -T -c "ut dm -I300:dm_test_host" + ... + Test: dm_test_pinctrl_single: pinmux.c (flat tree) + Test: dm_test_host: host.c + test/dm/host.c:71, dm_test_host(): 0 == ut_check_delta(mem_start): Expected 0x0 (0), got 0xfffffdb0 (-592) + Test: dm_test_host: host.c (flat tree) + Test dm_test_host failed 1 times (position 300) + Failures: 4 + +So it happened before position 300. Trying 150 shows it failing, so we try 75:: + + $ ./u-boot -T -c "ut dm -I75:dm_test_host" + ... + Test: dm_test_autoprobe: core.c + Test: dm_test_autoprobe: core.c (flat tree) + Test: dm_test_host: host.c + Test: dm_test_host: host.c (flat tree) + Failures: 0 + +That succeeds, so we try 120, etc. until eventually we can figure out that the +problem first happens at position 82. + + $ ./u-boot -T -c "ut dm -I82:dm_test_host" + ... + Test: dm_test_blk_flags: blk.c + Test: dm_test_blk_flags: blk.c (flat tree) + Test: dm_test_host: host.c + test/dm/host.c:71, dm_test_host(): 0 == ut_check_delta(mem_start): Expected 0x0 (0), got 0xffffc960 (-13984) + Test: dm_test_host: host.c (flat tree) + Test dm_test_host failed 1 times (position 82) + Failures: 1 + +From this we can deduce that `dm_test_blk_flags()` causes the problem with +`dm_test_host()`. + Running sandbox_spl tests directly ---------------------------------- |