dynamically coupling documentation to unit testing
Hi, Been thinking. What I've come to like in documentation is a section at the top of a class or functor that includes some real examples code in the 'doxygen' section that shows how to use. My question is how difficult would it be to make unit testing pick these up and actually run them? If written with unit equalities this would gain by making me and developers write good header examples at the same time as making unit tests and continuously validate documentation isn't getting out-of-date as well. And have the output from the tests be compatible with jenkins or other build system's reporting? How feasible to do this? Is this a possibility to add to boost? Or should it be an llvm based project?
Hi Roger
Op 25 jun. 2014 om 16:23 heeft Roger Martin
Hi,
Been thinking.
What I've come to like in documentation is a section at the top of a class or functor that includes some real examples code in the 'doxygen' section that shows how to use.
My question is how difficult would it be to make unit testing pick these up and actually run them? If written with unit equalities this would gain by making me and developers write good header examples at the same time as making unit tests and continuously validate documentation isn't getting out-of-date as well. And have the output from the tests be compatible with jenkins or other build system's reporting?
Boost.Geometry does this ( or quite similar to what you describe) by having small code samples for (ideally) each algorithm. They are in the doc/src/examples folder. It is not in the form of unit tests because those exhaustingly tests many cases, and the samples are minimal, and have QuickBook markup inside. But it has a Jamfile, is compilable, and automatically included in the docs by QuickBook in combination with Doxygen, in the system we use (we generate quickbook from doxygen xml). Indeed, what you say, minimal and up to date correct examples are essential. We might consider move them to the unit test folder to have them in the regression matrix. We have already many things there.... so this has not been done up to now, but this is just a small difference and trivial to change. Regards, Barend
Barend Gehrels wrote:
Op 25 jun. 2014 om 16:23 heeft Roger Martin
het volgende geschreven: Been thinking.
What I've come to like in documentation is a section at the top of a class or functor that includes some real examples code in the 'doxygen' section that shows how to use.
My question is how difficult would it be to make unit testing pick these up and actually run them? If written with unit equalities this would gain by making me and developers write good header examples at the same time as making unit tests and continuously validate documentation isn't getting out-of-date as well. And have the output from the tests be compatible with jenkins or other build system's reporting?
Boost.Geometry does this ( or quite similar to what you describe) by having small code samples for (ideally) each algorithm. They are in the doc/src/examples folder. It is not in the form of unit tests because those exhaustingly tests many cases, and the samples are minimal, and have QuickBook markup inside. But it has a Jamfile, is compilable, and automatically included in the docs by QuickBook in combination with Doxygen, in the system we use (we generate quickbook from doxygen xml).
Indeed, what you say, minimal and up to date correct examples are essential.
We might consider move them to the unit test folder to have them in the regression matrix. We have already many things there.... so this has not been done up to now, but this is just a small difference and trivial to change.
Yes, we could add test directory in geometry/doc directory and treat docs as another sublib, like extensions or index. Put a Jamfile there running test suites doc/src/examples and doc/index/src/examples. Then it should be displayed as a sublibrary of Geometry in the matrix as geometry/doc. Btw, the way how the tests for Chrono are displayed is unique. In the BOOST_ROOT/status/Jamfile only chrono/test can be found but the tests are displayed as 2 sublibs - chrono and chrono/example. It's because in the chrono/test/Jamfile, files from the parent directory are included (../example/*.cpp) and bjam handles them somehow, properly in this case. Is this intentionally supported? I don't know. Nevertheless I wonder, what if some more complicated path or several paths were used this way e.g.: ../doc/src/examples/*.cpp ../doc/index/src/examples/*.cpp Or the whole test suites: ../doc/src/examples ../doc/index/src/examples Regards, Adam
Thank all of you for pointing to examples. I'll look and learn On 06/25/2014 12:11 PM, Adam Wulkiewicz wrote:
Barend Gehrels wrote:
Op 25 jun. 2014 om 16:23 heeft Roger Martin
het volgende geschreven: Been thinking.
What I've come to like in documentation is a section at the top of a class or functor that includes some real examples code in the 'doxygen' section that shows how to use.
My question is how difficult would it be to make unit testing pick these up and actually run them? If written with unit equalities this would gain by making me and developers write good header examples at the same time as making unit tests and continuously validate documentation isn't getting out-of-date as well. And have the output from the tests be compatible with jenkins or other build system's reporting?
Boost.Geometry does this ( or quite similar to what you describe) by having small code samples for (ideally) each algorithm. They are in the doc/src/examples folder. It is not in the form of unit tests because those exhaustingly tests many cases, and the samples are minimal, and have QuickBook markup inside. But it has a Jamfile, is compilable, and automatically included in the docs by QuickBook in combination with Doxygen, in the system we use (we generate quickbook from doxygen xml).
Indeed, what you say, minimal and up to date correct examples are essential.
We might consider move them to the unit test folder to have them in the regression matrix. We have already many things there.... so this has not been done up to now, but this is just a small difference and trivial to change.
Yes, we could add test directory in geometry/doc directory and treat docs as another sublib, like extensions or index. Put a Jamfile there running test suites doc/src/examples and doc/index/src/examples. Then it should be displayed as a sublibrary of Geometry in the matrix as geometry/doc.
Btw, the way how the tests for Chrono are displayed is unique. In the BOOST_ROOT/status/Jamfile only chrono/test can be found but the tests are displayed as 2 sublibs - chrono and chrono/example. It's because in the chrono/test/Jamfile, files from the parent directory are included (../example/*.cpp) and bjam handles them somehow, properly in this case. Is this intentionally supported? I don't know.
Nevertheless I wonder, what if some more complicated path or several paths were used this way e.g.: ../doc/src/examples/*.cpp ../doc/index/src/examples/*.cpp
Or the whole test suites: ../doc/src/examples ../doc/index/src/examples
Regards, Adam
_______________________________________________ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
This is not very hard to do with either Boost Build or CMake. It doesn't require any change in boost infrastructure. It just requires that the library author invest some more time in setting up his build/test. Of course the previous sentence understates the extra hassle of dealing with the above tools. But this would still be much simpler than than any other way of doing it. Robert Ramey -- View this message in context: http://boost.2283326.n4.nabble.com/dynamically-coupling-documentation-to-uni... Sent from the Boost - Dev mailing list archive at Nabble.com.
On 06/25/2014 07:23 AM, Roger Martin wrote:
Hi,
Been thinking.
What I've come to like in documentation is a section at the top of a class or functor that includes some real examples code in the 'doxygen' section that shows how to use.
If you're already using the "standard" Boost documentation toolchain (doxygen and Quickbook), you get this out of the box. http://www.boost.org/doc/libs/1_55_0/doc/html/quickbook/syntax/block.html#qu... I see others have already pointed you to working examples. \e
participants (5)
-
Adam Wulkiewicz
-
Barend Gehrels
-
Eric Niebler
-
Robert Ramey
-
Roger Martin