scripts: Add HTML build report script

[grahamroff-dev]

This change adds a script to generate an HTML report, invoked via west, providing a consolidated and user-friendly view of various build artifacts and metrics. This includes footprint information (as both tables and plots), configuration tracing, sys init levels, an interactive devicetree browser, and more.

Invoke using west build -t buildreport

An example of the HTML output is available at https://grahamroff-dev.github.io/zephyr-buildreport/index.html

There is inconsistency in naming convention for the Zephyr scripts, some use underscores (like ram_report) and many do not (like traceconfig, initlevels, debugserver). I chose "buildreport" (sounded more like a report on the build as opposed to "build_report" which sounds a bit like building a report) but any other suggestions on script naming are welcome.

I added the documentation to the "Optimization Tools" section as that is where the memory report tools are documented. The buildreport covers a lot more than just optimization though, so suggestions welcome if there is a better place.

[pdgendt]

This should not be a single commit, changes to the existing scripts check_init_priorities.py/edtlib.py/plot.py should each be a separate commit.

[mbolivar]

Alas, this is an abstraction violation we can't allow in edtlib. See the "general biased advice" at top of the file.

[grahamroff-dev]

I acknowledge this goes against the abstraction advice, however that advice seems somewhat arbitrary ;)
Any suggestion on how to get a list of all the properties in a node? The current edtlib purposefully leaves out valid properties like #address-cells, leaving a consumer (the build report in this case) unable to get a complete list of properties. As an abstraction layer this is broken.

[grahamroff-dev]

There are two problems I am trying to get around. First, in Node._init_prop a bunch of properties are simply skipped even though they appear in the devicetree:

        # Skip properties that start with '#', like '#size-cells', and mapping
        # properties like 'gpio-map'/'interrupt-map'
        if name[0] == "#" or name.endswith("-map"):
            return

Then in Node._check_undeclared_props the validation also allows a number of properties that will not appear in the Node.props list:

        wl = {"compatible", "status", "ranges", "phandle",
              "interrupt-parent", "interrupts-extended", "device_type"}

        for prop_name in self._node.props:
            # Allow a few special properties to not be declared in the binding
            if (prop_name.endswith("-controller")
                or prop_name.startswith("#")
                or prop_name in wl):
                continue

If I just iterate over Node.props then the table view will be missing a lot of important properties.

[grahamroff-dev]

@mbolivar I removed the change to edtlib by reparsing the dts file using dtlib directly, then pulling in the nodes and bindings that match.

[nordicjm]

how funny, this sounds like the same issue I hit yesterday on defaults in bindings being wholly ignored, #104478 @mbolivar there is some issue there

[pdgendt]

This should not be a single commit, changes to the existing scripts check_init_priorities.py/edtlib.py/plot.py should each be a separate commit.

Your last update did something wrong, the first commit now deletes the existing scripts, and later commits re-introduce them with changes...

You first need to update the existing scripts (not delete!), and add your new stuff after.

[grahamroff-dev]

This should not be a single commit, changes to the existing scripts check_init_priorities.py/edtlib.py/plot.py should each be a separate commit.

Your last update did something wrong, the first commit now deletes the existing scripts, and later commits re-introduce them with changes...

You first need to update the existing scripts (not delete!), and add your new stuff after.

Sorry, messed up the rework of the commit history. Fixed.

[tejlmand]

Invoke using west build -t buildreport

Doesn't work.
Did you forget to include some files in the commit ?

[tejlmand]

plotly is part of requirements-extra.

Perhaps better to refer there instead, or in addition.

zephyr/scripts/requirements-extras.txt

Lines 9 to 10 in 69f3a44

	# used by scripts/footprint/plot.py for generating plots of size reports
	plotly

[grahamroff-dev]

Thanks. I updated to handle missing plotly and just skip generating the plots, also updated the import error to suggest looking at requirements-extra.

[grahamroff-dev]

Invoke using west build -t buildreport

Doesn't work. Did you forget to include some files in the commit ?

@tejlmand, I updated with missing cmake script changes, sorry about that, please try again.

[sonarqubecloud]

Quality Gate passed

Issues
1 New issue
0 Accepted issues

Measures
1 Security Hotspot
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud

[pdgendt]

I have to say, this looks impressive. The report output is nice, and provides a lot of info in an easy accessible way.

I do have a question regarding maintainership: Are you/Qualcomm going to oversee this? Should we create an area/subarea in the MAINTAINERS.yml file for this?

[tejlmand]

Nice tool.

Some minor nits.

One idea, comments from everyone are more than welcome.
When running ninja puncover, https://docs.zephyrproject.org/latest/develop/optimizations/tools.html#build-target-puncover
then the target will not only launch a webserver, but also open your default browser and connect to the page.

For consistency with puncover behavior, should west html_buildreport default open the generated page(s) in the user's default webbrowser ?
(this suggestion is not blocking the review)

[tejlmand]

Other commands, such as ram_report, rom_report, initlevels, etc produces textual output directly.

Commands not doing so or relies on dedicated program indicates this is their target name, such as pahole and puncover.

I would like buildreport to reflect in its name that this is not a textual output.

For example html_report / html_buildreport.

[grahamroff-dev]

Makes sense. There are number of locations whose name should be consistent. Currently the PR uses:

• west command: west build -t buildreport
• Script location: scripts/buildreport
• Script name: buildreport.py
• Output location: build/report

Based on suggestions so far I will update the west command to west build -t html_report and the default output location to build/html/report.

What are your thoughts on the script location and name? Would this be preferred:

• Script location: scripts/html_report
• Script name: html_report.py

[grahamroff-dev]

Another naming idea could be "dashboard":

• west command: west build -t dashboard or west build -t html_dashboard
• Script location: scripts/dashboard
• Script name: dashboard.py
• Output location: build/html/dashboard

[tejlmand]

are those extra dots really neccesary ?

It's annoying that I cannot just double click on line and copy it to the browser to open it.
I need to strip the extra dots.

Suggested change

	logger.info(f"Creating {fname}...")
	logger.info(f"Creating {fname}")

[tejlmand]

perhaps move it a level down, such as:

Suggested change

	--output ${CMAKE_BINARY_DIR}/report
	--output ${CMAKE_BINARY_DIR}/html/report

[grahamroff-dev]

@pdgendt ,

I have to say, this looks impressive. The report output is nice, and provides a lot of info in an easy accessible way.

Thanks! There is a lot of great tooling and build information available, and making it easy to generate and use without having to know about each on individually is the goal.

I do have a question regarding maintainership: Are you/Qualcomm going to oversee this? Should we create an area/subarea in the MAINTAINERS.yml file for this?

We (Qualcomm) have begun making extensive use of Zephyr internally (working on an open-source and upstreaming plan) and these optimization tools are a critical part of our work. So yes, absolutely willing to oversee and maintain it, with whatever mechanism makes the most sense.