From: Harinder Singh <sharinder@google.com>
To: davidgow@google.com, brendanhiggins@google.com, shuah@kernel.org,
corbet@lwn.net
Cc: linux-kselftest@vger.kernel.org, kunit-dev@googlegroups.com,
linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org,
tim.bird@sony.com, elver@google.com,
Harinder Singh <sharinder@google.com>
Subject: [PATCH v6 3/7] Documentation: KUnit: Added KUnit Architecture
Date: Fri, 17 Dec 2021 04:49:07 +0000 [thread overview]
Message-ID: <20211217044911.798817-4-sharinder@google.com> (raw)
In-Reply-To: <20211217044911.798817-1-sharinder@google.com>
Describe the components of KUnit and how the kernel mode parts
interact with kunit_tool.
Signed-off-by: Harinder Singh <sharinder@google.com>
---
.../dev-tools/kunit/architecture.rst | 204 ++++++++++++++++++
Documentation/dev-tools/kunit/index.rst | 2 +
.../kunit/kunit_suitememorydiagram.svg | 81 +++++++
Documentation/dev-tools/kunit/start.rst | 1 +
4 files changed, 288 insertions(+)
create mode 100644 Documentation/dev-tools/kunit/architecture.rst
create mode 100644 Documentation/dev-tools/kunit/kunit_suitememorydiagram.svg
diff --git a/Documentation/dev-tools/kunit/architecture.rst b/Documentation/dev-tools/kunit/architecture.rst
new file mode 100644
index 000000000000..aa2cea821e25
--- /dev/null
+++ b/Documentation/dev-tools/kunit/architecture.rst
@@ -0,0 +1,204 @@
+.. SPDX-License-Identifier: GPL-2.0
+
+==================
+KUnit Architecture
+==================
+
+The KUnit architecture can be divided into two parts:
+
+- Kernel testing library
+- kunit_tool (Command line test harness)
+
+In-Kernel Testing Framework
+===========================
+
+The kernel testing library supports KUnit tests written in C using
+KUnit. KUnit tests are kernel code. KUnit does several things:
+
+- Organizes tests
+- Reports test results
+- Provides test utilities
+
+Test Cases
+----------
+
+The fundamental unit in KUnit is the test case. The KUnit test cases are
+grouped into KUnit suites. A KUnit test case is a function with type
+signature ``void (*)(struct kunit *test)``.
+These test case functions are wrapped in a struct called
+``struct kunit_case``. For code, see:
+
+.. kernel-doc:: include/kunit/test.h
+ :identifiers: kunit_case
+
+.. note:
+ ``generate_params`` is optional for non-parameterized tests.
+
+Each KUnit test case gets a ``struct kunit`` context
+object passed to it that tracks a running test. The KUnit assertion
+macros and other KUnit utilities use the ``struct kunit`` context
+object. As an exception, there are two fields:
+
+- ``->priv``: The setup functions can use it to store arbitrary test
+ user data.
+
+- ``->param_value``: It contains the parameter value which can be
+ retrieved in the parameterized tests.
+
+Test Suites
+-----------
+
+A KUnit suite includes a collection of test cases. The KUnit suites
+are represented by the ``struct kunit_suite``. For example:
+
+.. code-block:: c
+
+ static struct kunit_case example_test_cases[] = {
+ KUNIT_CASE(example_test_foo),
+ KUNIT_CASE(example_test_bar),
+ KUNIT_CASE(example_test_baz),
+ {}
+ };
+
+ static struct kunit_suite example_test_suite = {
+ .name = "example",
+ .init = example_test_init,
+ .exit = example_test_exit,
+ .test_cases = example_test_cases,
+ };
+ kunit_test_suite(example_test_suite);
+
+In the above example, the test suite ``example_test_suite``, runs the
+test cases ``example_test_foo``, ``example_test_bar``, and
+``example_test_baz``. Before running the test, the ``example_test_init``
+is called and after running the test, ``example_test_exit`` is called.
+The ``kunit_test_suite(example_test_suite)`` registers the test suite
+with the KUnit test framework.
+
+Executor
+--------
+
+The KUnit executor can list and run built-in KUnit tests on boot.
+The Test suites are stored in a linker section
+called ``.kunit_test_suites``. For code, see:
+https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/include/asm-generic/vmlinux.lds.h?h=v5.15#n945.
+The linker section consists of an array of pointers to
+``struct kunit_suite``, and is populated by the ``kunit_test_suites()``
+macro. To run all tests compiled into the kernel, the KUnit executor
+iterates over the linker section array.
+
+.. kernel-figure:: kunit_suitememorydiagram.svg
+ :alt: KUnit Suite Memory
+
+ KUnit Suite Memory Diagram
+
+On the kernel boot, the KUnit executor uses the start and end addresses
+of this section to iterate over and run all tests. For code, see:
+https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/lib/kunit/executor.c
+
+When built as a module, the ``kunit_test_suites()`` macro defines a
+``module_init()`` function, which runs all the tests in the compilation
+unit instead of utilizing the executor.
+
+In KUnit tests, some error classes do not affect other tests
+or parts of the kernel, each KUnit case executes in a separate thread
+context. For code, see:
+https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/lib/kunit/try-catch.c?h=v5.15#n58
+
+Assertion Macros
+----------------
+
+KUnit tests verify state using expectations/assertions.
+All expectations/assertions are formatted as:
+``KUNIT_{EXPECT|ASSERT}_<op>[_MSG](kunit, property[, message])``
+
+- ``{EXPECT|ASSERT}`` determines whether the check is an assertion or an
+ expectation.
+
+ - For an expectation, if the check fails, marks the test as failed
+ and logs the failure.
+
+ - An assertion, on failure, causes the test case to terminate
+ immediately.
+
+ - Assertions call function:
+ ``void __noreturn kunit_abort(struct kunit *)``.
+
+ - ``kunit_abort`` calls function:
+ ``void __noreturn kunit_try_catch_throw(struct kunit_try_catch *try_catch)``.
+
+ - ``kunit_try_catch_throw`` calls function:
+ ``void complete_and_exit(struct completion *, long) __noreturn;``
+ and terminates the special thread context.
+
+- ``<op>`` denotes a check with options: ``TRUE`` (supplied property
+ has the boolean value “true”), ``EQ`` (two supplied properties are
+ equal), ``NOT_ERR_OR_NULL`` (supplied pointer is not null and does not
+ contain an “err” value).
+
+- ``[_MSG]`` prints a custom message on failure.
+
+Test Result Reporting
+---------------------
+KUnit prints test results in KTAP format. KTAP is based on TAP14, see:
+https://github.com/isaacs/testanything.github.io/blob/tap14/tap-version-14-specification.md.
+KTAP (yet to be standardized format) works with KUnit and Kselftest.
+The KUnit executor prints KTAP results to dmesg, and debugfs
+(if configured).
+
+Parameterized Tests
+-------------------
+
+Each KUnit parameterized test is associated with a collection of
+parameters. The test is invoked multiple times, once for each parameter
+value and the parameter is stored in the ``param_value`` field.
+The test case includes a ``KUNIT_CASE_PARAM()`` macro that accepts a
+generator function.
+The generator function is passed the previous parameter and returns the next
+parameter. It also provides a macro to generate common-case generators based on
+arrays.
+
+For code, see:
+
+.. kernel-doc:: include/kunit/test.h
+ :identifiers: KUNIT_ARRAY_PARAM
+
+
+kunit_tool (Command Line Test Harness)
+======================================
+
+kunit_tool is a Python script ``(tools/testing/kunit/kunit.py)``
+that can be used to configure, build, exec, parse and run (runs other
+commands in order) test results. You can either run KUnit tests using
+kunit_tool or can include KUnit in kernel and parse manually.
+
+- ``configure`` command generates the kernel ``.config`` from a
+ ``.kunitconfig`` file (and any architecture-specific options).
+ For some architectures, additional config options are specified in the
+ ``qemu_config`` Python script
+ (For example: ``tools/testing/kunit/qemu_configs/powerpc.py``).
+ It parses both the existing ``.config`` and the ``.kunitconfig`` files
+ and ensures that ``.config`` is a superset of ``.kunitconfig``.
+ If this is not the case, it will combine the two and run
+ ``make olddefconfig`` to regenerate the ``.config`` file. It then
+ verifies that ``.config`` is now a superset. This checks if all
+ Kconfig dependencies are correctly specified in ``.kunitconfig``.
+ ``kunit_config.py`` includes the parsing Kconfigs code. The code which
+ runs ``make olddefconfig`` is a part of ``kunit_kernel.py``. You can
+ invoke this command via: ``./tools/testing/kunit/kunit.py config`` and
+ generate a ``.config`` file.
+- ``build`` runs ``make`` on the kernel tree with required options
+ (depends on the architecture and some options, for example: build_dir)
+ and reports any errors.
+ To build a KUnit kernel from the current ``.config``, you can use the
+ ``build`` argument: ``./tools/testing/kunit/kunit.py build``.
+- ``exec`` command executes kernel results either directly (using
+ User-mode Linux configuration), or via an emulator such
+ as QEMU. It reads results from the log via standard
+ output (stdout), and passes them to ``parse`` to be parsed.
+ If you already have built a kernel with built-in KUnit tests,
+ you can run the kernel and display the test results with the ``exec``
+ argument: ``./tools/testing/kunit/kunit.py exec``.
+- ``parse`` extracts the KTAP output from a kernel log, parses
+ the test results, and prints a summary. For failed tests, any
+ diagnostic output will be included.
diff --git a/Documentation/dev-tools/kunit/index.rst b/Documentation/dev-tools/kunit/index.rst
index 55d2444b0745..50d3ef9359dd 100644
--- a/Documentation/dev-tools/kunit/index.rst
+++ b/Documentation/dev-tools/kunit/index.rst
@@ -9,6 +9,7 @@ KUnit - Linux Kernel Unit Testing
:caption: Contents:
start
+ architecture
usage
kunit-tool
api/index
@@ -96,6 +97,7 @@ How do I use it?
================
* Documentation/dev-tools/kunit/start.rst - for KUnit new users.
+* Documentation/dev-tools/kunit/architecture.rst - KUnit architecture.
* Documentation/dev-tools/kunit/usage.rst - KUnit features.
* Documentation/dev-tools/kunit/tips.rst - best practices with
examples.
diff --git a/Documentation/dev-tools/kunit/kunit_suitememorydiagram.svg b/Documentation/dev-tools/kunit/kunit_suitememorydiagram.svg
new file mode 100644
index 000000000000..cf8fddc27500
--- /dev/null
+++ b/Documentation/dev-tools/kunit/kunit_suitememorydiagram.svg
@@ -0,0 +1,81 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<svg width="796.93" height="555.73" version="1.1" viewBox="0 0 796.93 555.73" xmlns="http://www.w3.org/2000/svg">
+ <g transform="translate(-13.724 -17.943)">
+ <g fill="#dad4d4" fill-opacity=".91765" stroke="#1a1a1a">
+ <rect x="323.56" y="18.443" width="115.75" height="41.331"/>
+ <rect x="323.56" y="463.09" width="115.75" height="41.331"/>
+ <rect x="323.56" y="531.84" width="115.75" height="41.331"/>
+ <rect x="323.56" y="88.931" width="115.75" height="74.231"/>
+ </g>
+ <g>
+ <rect x="323.56" y="421.76" width="115.75" height="41.331" fill="#b9dbc6" stroke="#1a1a1a"/>
+ <text x="328.00888" y="446.61826" fill="#000000" font-family="sans-serif" font-size="16px" style="line-height:1.25" xml:space="preserve"><tspan x="328.00888" y="446.61826" font-family="monospace" font-size="16px">kunit_suite</tspan></text>
+ </g>
+ <g transform="translate(0 -258.6)">
+ <rect x="323.56" y="421.76" width="115.75" height="41.331" fill="#b9dbc6" stroke="#1a1a1a"/>
+ <text x="328.00888" y="446.61826" fill="#000000" font-family="sans-serif" font-size="16px" style="line-height:1.25" xml:space="preserve"><tspan x="328.00888" y="446.61826" font-family="monospace" font-size="16px">kunit_suite</tspan></text>
+ </g>
+ <g transform="translate(0 -217.27)">
+ <rect x="323.56" y="421.76" width="115.75" height="41.331" fill="#b9dbc6" stroke="#1a1a1a"/>
+ <text x="328.00888" y="446.61826" fill="#000000" font-family="sans-serif" font-size="16px" style="line-height:1.25" xml:space="preserve"><tspan x="328.00888" y="446.61826" font-family="monospace" font-size="16px">kunit_suite</tspan></text>
+ </g>
+ <g transform="translate(0 -175.94)">
+ <rect x="323.56" y="421.76" width="115.75" height="41.331" fill="#b9dbc6" stroke="#1a1a1a"/>
+ <text x="328.00888" y="446.61826" fill="#000000" font-family="sans-serif" font-size="16px" style="line-height:1.25" xml:space="preserve"><tspan x="328.00888" y="446.61826" font-family="monospace" font-size="16px">kunit_suite</tspan></text>
+ </g>
+ <g transform="translate(0 -134.61)">
+ <rect x="323.56" y="421.76" width="115.75" height="41.331" fill="#b9dbc6" stroke="#1a1a1a"/>
+ <text x="328.00888" y="446.61826" fill="#000000" font-family="sans-serif" font-size="16px" style="line-height:1.25" xml:space="preserve"><tspan x="328.00888" y="446.61826" font-family="monospace" font-size="16px">kunit_suite</tspan></text>
+ </g>
+ <g transform="translate(0 -41.331)">
+ <rect x="323.56" y="421.76" width="115.75" height="41.331" fill="#b9dbc6" stroke="#1a1a1a"/>
+ <text x="328.00888" y="446.61826" fill="#000000" font-family="sans-serif" font-size="16px" style="line-height:1.25" xml:space="preserve"><tspan x="328.00888" y="446.61826" font-family="monospace" font-size="16px">kunit_suite</tspan></text>
+ </g>
+ <g transform="translate(3.4459e-5 -.71088)">
+ <rect x="502.19" y="143.16" width="201.13" height="41.331" fill="#dad4d4" fill-opacity=".91765" stroke="#1a1a1a"/>
+ <text x="512.02319" y="168.02026" font-family="sans-serif" font-size="16px" style="line-height:1.25" xml:space="preserve"><tspan x="512.02319" y="168.02026" font-family="monospace">_kunit_suites_start</tspan></text>
+ </g>
+ <g transform="translate(3.0518e-5 -3.1753)">
+ <rect x="502.19" y="445.69" width="201.13" height="41.331" fill="#dad4d4" fill-opacity=".91765" stroke="#1a1a1a"/>
+ <text x="521.61694" y="470.54846" font-family="sans-serif" font-size="16px" style="line-height:1.25" xml:space="preserve"><tspan x="521.61694" y="470.54846" font-family="monospace">_kunit_suites_end</tspan></text>
+ </g>
+ <rect x="14.224" y="277.78" width="134.47" height="41.331" fill="#dad4d4" fill-opacity=".91765" stroke="#1a1a1a"/>
+ <text x="32.062176" y="304.41287" font-family="sans-serif" font-size="16px" style="line-height:1.25" xml:space="preserve"><tspan x="32.062176" y="304.41287" font-family="monospace">.init.data</tspan></text>
+ <g transform="translate(217.98 145.12)" stroke="#1a1a1a">
+ <circle cx="149.97" cy="373.01" r="3.4012"/>
+ <circle cx="163.46" cy="373.01" r="3.4012"/>
+ <circle cx="176.95" cy="373.01" r="3.4012"/>
+ </g>
+ <g transform="translate(217.98 -298.66)" stroke="#1a1a1a">
+ <circle cx="149.97" cy="373.01" r="3.4012"/>
+ <circle cx="163.46" cy="373.01" r="3.4012"/>
+ <circle cx="176.95" cy="373.01" r="3.4012"/>
+ </g>
+ <g stroke="#1a1a1a">
+ <rect x="323.56" y="328.49" width="115.75" height="51.549" fill="#b9dbc6"/>
+ <g transform="translate(217.98 -18.75)">
+ <circle cx="149.97" cy="373.01" r="3.4012"/>
+ <circle cx="163.46" cy="373.01" r="3.4012"/>
+ <circle cx="176.95" cy="373.01" r="3.4012"/>
+ </g>
+ </g>
+ <g transform="scale(1.0933 .9147)" stroke-width="32.937" aria-label="{">
+ <path d="m275.49 545.57c-35.836-8.432-47.43-24.769-47.957-64.821v-88.536c-0.527-44.795-10.54-57.97-49.538-67.456 38.998-10.013 49.011-23.715 49.538-67.983v-88.536c0.527-40.052 12.121-56.389 47.957-64.821v-5.797c-65.348 0-85.901 17.391-86.955 73.253v93.806c-0.527 36.89-10.013 50.065-44.795 59.551 34.782 10.013 44.268 23.188 44.795 60.078v93.279c1.581 56.389 21.607 73.78 86.955 73.78z"/>
+ </g>
+ <g transform="scale(1.1071 .90325)" stroke-width="14.44" aria-label="{">
+ <path d="m461.46 443.55c-15.711-3.6967-20.794-10.859-21.025-28.418v-38.815c-0.23104-19.639-4.6209-25.415-21.718-29.574 17.097-4.3898 21.487-10.397 21.718-29.805v-38.815c0.23105-17.559 5.314-24.722 21.025-28.418v-2.5415c-28.649 0-37.66 7.6244-38.122 32.115v41.126c-0.23105 16.173-4.3898 21.949-19.639 26.108 15.249 4.3898 19.408 10.166 19.639 26.339v40.895c0.69313 24.722 9.4728 32.346 38.122 32.346z"/>
+ </g>
+ <path d="m449.55 161.84v2.5h49.504v-2.5z" color="#000000" style="-inkscape-stroke:none"/>
+ <g fill-rule="evenodd">
+ <path d="m443.78 163.09 8.65-5v10z" color="#000000" stroke-width="1pt" style="-inkscape-stroke:none"/>
+ <path d="m453.1 156.94-10.648 6.1543 0.99804 0.57812 9.6504 5.5781zm-1.334 2.3125v7.6856l-6.6504-3.8438z" color="#000000" style="-inkscape-stroke:none"/>
+ </g>
+ <path d="m449.55 461.91v2.5h49.504v-2.5z" color="#000000" style="-inkscape-stroke:none"/>
+ <g fill-rule="evenodd">
+ <path d="m443.78 463.16 8.65-5v10z" color="#000000" stroke-width="1pt" style="-inkscape-stroke:none"/>
+ <path d="m453.1 457-10.648 6.1562 0.99804 0.57617 9.6504 5.5781zm-1.334 2.3125v7.6856l-6.6504-3.8438z" color="#000000" style="-inkscape-stroke:none"/>
+ </g>
+ <rect x="515.64" y="223.9" width="294.52" height="178.49" fill="#dad4d4" fill-opacity=".91765" stroke="#1a1a1a"/>
+ <text x="523.33319" y="262.52542" font-family="monospace" font-size="14.667px" style="line-height:1.25" xml:space="preserve"><tspan x="523.33319" y="262.52542"><tspan fill="#008000" font-family="monospace" font-size="14.667px" font-weight="bold">struct</tspan> kunit_suite {</tspan><tspan x="523.33319" y="280.8588"><tspan fill="#008000" font-family="monospace" font-size="14.667px" font-weight="bold"> const char</tspan> name[<tspan fill="#ff00ff" font-size="14.667px">256</tspan>];</tspan><tspan x="523.33319" y="299.19217"> <tspan fill="#008000" font-family="monospace" font-size="14.667px" font-weight="bold">int</tspan> (*init)(<tspan fill="#008000" font-family="monospace" font-size="14.667px" font-weight="bold">struct</tspan> kunit *);</tspan><tspan x="523.33319" y="317.52554"> <tspan fill="#008000" font-family="monospace" font-size="14.667px" font-weight="bold">void</tspan> (*exit)(<tspan fill="#008000" font-family="monospace" font-size="14.667px" font-weight="bold">struct</tspan> kunit *);</tspan><tspan x="523.33319" y="335.85892"> <tspan fill="#008000" font-family="monospace" font-size="14.667px" font-weight="bold">struct</tspan> kunit_case *test_cases;</tspan><tspan x="523.33319" y="354.19229"> ...</tspan><tspan x="523.33319" y="372.52567">};</tspan></text>
+ </g>
+</svg>
diff --git a/Documentation/dev-tools/kunit/start.rst b/Documentation/dev-tools/kunit/start.rst
index 55f8df1abd40..5dd2c88fa2bd 100644
--- a/Documentation/dev-tools/kunit/start.rst
+++ b/Documentation/dev-tools/kunit/start.rst
@@ -240,6 +240,7 @@ Congrats! You just wrote your first KUnit test.
Next Steps
==========
+* Documentation/dev-tools/kunit/architecture.rst - KUnit architecture.
* Documentation/dev-tools/kunit/usage.rst - KUnit features.
* Documentation/dev-tools/kunit/tips.rst - best practices with
examples.
--
2.34.1.173.g76aa8bc2d0-goog
next prev parent reply other threads:[~2021-12-17 4:49 UTC|newest]
Thread overview: 12+ messages / expand[flat|nested] mbox.gz Atom feed top
2021-12-17 4:49 [PATCH v6 0/7] Documentation: KUnit: Rework KUnit documentation Harinder Singh
2021-12-17 4:49 ` [PATCH v6 1/7] Documentation: KUnit: Rewrite main page Harinder Singh
2021-12-17 4:49 ` [PATCH v6 2/7] Documentation: KUnit: Rewrite getting started Harinder Singh
2021-12-17 4:49 ` Harinder Singh [this message]
2021-12-17 10:50 ` [PATCH v6 3/7] Documentation: KUnit: Added KUnit Architecture Marco Elver
2021-12-23 19:48 ` Jonathan Corbet
2021-12-17 4:49 ` [PATCH v6 4/7] Documentation: kunit: Reorganize documentation related to running tests Harinder Singh
2021-12-17 4:49 ` [PATCH v6 5/7] Documentation: KUnit: Rework writing page to focus on writing tests Harinder Singh
2021-12-17 4:49 ` [PATCH v6 6/7] Documentation: KUnit: Restyle Test Style and Nomenclature page Harinder Singh
2021-12-17 4:49 ` [PATCH v6 7/7] Documentation: KUnit: Restyled Frequently Asked Questions Harinder Singh
2021-12-17 5:09 ` [PATCH v6 0/7] Documentation: KUnit: Rework KUnit documentation Brendan Higgins
2021-12-23 19:49 ` Jonathan Corbet
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20211217044911.798817-4-sharinder@google.com \
--to=sharinder@google.com \
--cc=brendanhiggins@google.com \
--cc=corbet@lwn.net \
--cc=davidgow@google.com \
--cc=elver@google.com \
--cc=kunit-dev@googlegroups.com \
--cc=linux-doc@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-kselftest@vger.kernel.org \
--cc=shuah@kernel.org \
--cc=tim.bird@sony.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.