[BoostBook] how to automatically test code examples?
Hi everyone, I am using BoostBook to prepare the documentation for the proposed histogram library https://github.com/HDembinski/histogram I have written a good deal of the documentation already, and it compiles fine with bjam (thanks to Klemens who showed me how). It says in the BoostBook documentation http://www.boost.org/doc/libs/1_65_1/doc/html/boostbook.html "The BoostBook format allows testcases to be generated by weaving together program fragments from example programs in the documentation. This capability is integrated with testsuite generation so that example programs are normal tests in BoostBook." That's awesome, I would love to use this feature, but the documentation is not explaining how to actually do that. I found the element "testsuite" which probably does what I want. I went on to scan the Boost repository for this BoostBook element, but couldn't find an example to copy from. Could someone help me out? How do I modify my BoostBook and Jam files to generate tests out of the code examples in the documentation? Best regards, Hans
Hello again,
It says in the BoostBook documentation http://www.boost.org/doc/libs/1_65_1/doc/html/boostbook.html "The BoostBook format allows testcases to be generated by weaving together program fragments from example programs in the documentation. This capability is integrated with testsuite generation so that example programs are normal tests in BoostBook."
That's awesome, I would love to use this feature, but the documentation is not explaining how to actually do that. I found the element "testsuite" which probably does what I want. I went on to scan the Boost repository for this BoostBook element, but couldn't find an example to copy from.
Could someone help me out? How do I modify my BoostBook and Jam files to generate tests out of the code examples in the documentation?
I didn't get any reply so far… was my question stupid? Or is it simply that no one knows? Is the author of BoostBook, Douglas Gregor, not around anymore? Since I couldn't figure out how to do it with BoostBook, I wrote a Python script which is run by CMake. It parses the documentation for code snippets and writes those to the examples directory, so that the examples are compiled along with the tests. It is not perfect, but it works at least. Best regards, Hans
-----Original Message----- From: Boost [mailto:boost-bounces@lists.boost.org] On Behalf Of Hans Dembinski via Boost Sent: 19 October 2017 09:01 To: boost@lists.boost.org Cc: Hans Dembinski Subject: Re: [boost] [BoostBook] how to automatically test code examples?
Hello again,
It says in the BoostBook documentation http://www.boost.org/doc/libs/1_65_1/doc/html/boostbook.html "The BoostBook format allows testcases to be generated by weaving together program fragments from example programs in the documentation. This capability is integrated with testsuite generation so that example programs are normal tests in BoostBook."
That's awesome, I would love to use this feature, but the documentation is not explaining how to actually do that. I found the element "testsuite" which probably does what I want. I went on to scan the Boost repository for this BoostBook element, but couldn't find an example to copy from.
Could someone help me out? How do I modify my BoostBook and Jam files to generate tests out of the code examples in the documentation?
I didn't get any reply so far… was my question stupid? Or is it simply that no one knows? Is the author of BoostBook, Douglas Gregor, not around anymore?
Sadly not - moved to higher things some years ago.
Since I couldn't figure out how to do it with BoostBook, I wrote a Python script which is run by CMake. It parses the documentation for code snippets and writes those to the examples directory, so that the examples are compiled along with the tests. It is not perfect, but it works at least.
I think that the reason is that almost everyone else is using Quickbook to generate the Boostbook, so that it is entirely transparent to them. Snippets of examples and tests are sucked in by lines of Quickbook, but the docs, examples and tests, and the jamfiles to run them otherwise entirely separate. Sorry. Paul --- Paul A. Bristow Prizet Farmhouse Kendal UK LA8 8AB +44 (0) 1539 561830
On 19/10/2017 09:01, Hans Dembinski via Boost wrote:
Hello again,
It says in the BoostBook documentation http://www.boost.org/doc/libs/1_65_1/doc/html/boostbook.html "The BoostBook format allows testcases to be generated by weaving together program fragments from example programs in the documentation. This capability is integrated with testsuite generation so that example programs are normal tests in BoostBook."
That's awesome, I would love to use this feature, but the documentation is not explaining how to actually do that. I found the element "testsuite" which probably does what I want. I went on to scan the Boost repository for this BoostBook element, but couldn't find an example to copy from.
Could someone help me out? How do I modify my BoostBook and Jam files to generate tests out of the code examples in the documentation? I didn't get any reply so far… was my question stupid? Or is it simply that no one knows? Is the author of BoostBook, Douglas Gregor, not around anymore?
Since I couldn't figure out how to do it with BoostBook, I wrote a Python script which is run by CMake. It parses the documentation for code snippets and writes those to the examples directory, so that the examples are compiled along with the tests. It is not perfect, but it works at least.
Most of us are using quickbook for this, so for example this: http://www.boost.org/doc/libs/1_55_0/libs/math/doc/html/math_toolkit/bessel/... Is generated from: https://github.com/boostorg/math/blob/develop/example/bessel_errors_example.... HTH, John. --- This email has been checked for viruses by AVG. http://www.avg.com
Dear John,
On 19. Oct 2017, at 11:22, John Maddock via Boost
wrote: Most of us are using quickbook for this, so for example this: http://www.boost.org/doc/libs/1_55_0/libs/math/doc/html/math_toolkit/bessel/... http://www.boost.org/doc/libs/1_55_0/libs/math/doc/html/math_toolkit/bessel/...
Is generated from: https://github.com/boostorg/math/blob/develop/example/bessel_errors_example.... https://github.com/boostorg/math/blob/develop/example/bessel_errors_example....
I realize now that I made mistake out of a misunderstanding. I am also using quickbook. I thought that boostbook was an extension of quickbook, but that's wrong. Thank you for the pointer to the math library, it helps to see how it is done. Best regards, Hans
I realize now that I made mistake out of a misunderstanding. I am also using quickbook. I thought that boostbook was an extension of quickbook, but that's wrong. Thank you for the pointer to the math library, it helps to see how it is done.
Ah, in that case it's trivial: you write the example code as code, add markup to it, and import into quickbook. If you start reading here: http://www.boost.org/doc/libs/1_65_1/doc/html/quickbook/syntax/block.html#qu... it explains the process. HTH, John. --- This email has been checked for viruses by AVG. http://www.avg.com
On 19. Oct 2017, at 13:30, John Maddock via Boost
wrote: I realize now that I made mistake out of a misunderstanding. I am also using quickbook. I thought that boostbook was an extension of quickbook, but that's wrong. Thank you for the pointer to the math library, it helps to see how it is done.
Ah, in that case it's trivial: you write the example code as code, add markup to it, and import into quickbook. If you start reading here: http://www.boost.org/doc/libs/1_65_1/doc/html/quickbook/syntax/block.html#qu... it explains the process.
Awesome, thank you again! I will check that out. The Quickbook documentation is pretty good. Best regards, Hans
On 19 October 2017 at 09:01, Hans Dembinski via Boost
Hello again,
It says in the BoostBook documentation http://www.boost.org/doc/libs/1_65_1/doc/html/boostbook.html "The BoostBook format allows testcases to be generated by weaving together program fragments from example programs in the documentation. This capability is integrated with testsuite generation so that example programs are normal tests in BoostBook."
That's awesome, I would love to use this feature, but the documentation is not explaining how to actually do that. I found the element "testsuite" which probably does what I want. I went on to scan the Boost repository for this BoostBook element, but couldn't find an example to copy from.
Could someone help me out? How do I modify my BoostBook and Jam files to generate tests out of the code examples in the documentation?
I didn't get any reply so far… was my question stupid? Or is it simply that no one knows? Is the author of BoostBook, Douglas Gregor, not around anymore?
Sorry, I was looking into this but a power cut got in the way. It looks like the xsl script at tools/boostbook/xsl/testing/Jamfile.xsl will generate a jamfile and test cases from tests marked up in xml, and you can find the xml schema details in the boostbook reference documentation. There doesn't seem to be any support for running this from boost build though.
Dear Daniel,
On 19. Oct 2017, at 11:43, Daniel James via Boost
wrote: I didn't get any reply so far… was my question stupid? Or is it simply that no one knows? Is the author of BoostBook, Douglas Gregor, not around anymore?
Sorry, I was looking into this but a power cut got in the way. It looks like the xsl script at tools/boostbook/xsl/testing/Jamfile.xsl will generate a jamfile and test cases from tests marked up in xml, and you can find the xml schema details in the boostbook reference documentation. There doesn't seem to be any support for running this from boost build though.
thank you for looking into this :) I am still confused how to use this, though. I guess I will stick to my poor-mans cmake/python-based solution for now. Best regards, Hans
mOn 19 October 2017 at 11:03, Hans Dembinski via Boost
Dear Daniel,
On 19. Oct 2017, at 11:43, Daniel James via Boost
wrote: I didn't get any reply so far… was my question stupid? Or is it simply that no one knows? Is the author of BoostBook, Douglas Gregor, not around anymore?
Sorry, I was looking into this but a power cut got in the way. It looks like the xsl script at tools/boostbook/xsl/testing/Jamfile.xsl will generate a jamfile and test cases from tests marked up in xml, and you can find the xml schema details in the boostbook reference documentation. There doesn't seem to be any support for running this from boost build though.
thank you for looking into this :) I am still confused how to use this, though. I guess I will stick to my poor-mans cmake/python-based solution for now.
For what it's worth, I tried creating an example that uses it. It's at: https://github.com/danieljames/boostbook-test-example If anyone actually wants to try it out, they'll need to clone it into their boost tree. But I included all the generated files, so can just look at them. The generated test doesn't actually work as it was written for boost build v1, so boostbook would need to be updated. It doesn't seem that useful to me, writing tests in xml would be a pain and less flexible than doing it normally, and the generated documentation doesn't seem useful - although what I wrote was pretty minimal, maybe with a bit more information it would be better? The boostbook file containing the test is at: https://github.com/danieljames/boostbook-test-example/blob/master/doc/tests.... The script to generate the test is at: https://github.com/danieljames/boostbook-test-example/blob/master/doc/genera... The generated test code is at: https://github.com/danieljames/boostbook-test-example/tree/master/test
On 10/19/17 3:03 AM, Hans Dembinski via Boost wrote:
Dear Daniel,
On 19. Oct 2017, at 11:43, Daniel James via Boost
wrote: I didn't get any reply so far… was my question stupid? Or is it simply that no one knows? Is the author of BoostBook, Douglas Gregor, not around anymore?
Sorry, I was looking into this but a power cut got in the way. It looks like the xsl script at tools/boostbook/xsl/testing/Jamfile.xsl will generate a jamfile and test cases from tests marked up in xml, and you can find the xml schema details in the boostbook reference documentation. There doesn't seem to be any support for running this from boost build though.
thank you for looking into this :) I am still confused how to use this, though. I guess I will stick to my poor-mans cmake/python-based solution for now.
Here's the way I handle this: I make up my examples in the "example" directory. This includes a Jamfile to build and run the examples. I create the BoostBook xml using and XMLmind - an xml editor. Within that environment I just use an "Include" directive to include the source code of the example in the correct place - easy as pie. If you download the slides associated with my recent CPPCon presentation, you can see how it's done. https://www.youtube.com/watch?v=YxmdCxX9dMk you can download the slides and example here: http://www.rrsd.com Robert Ramey
Dear Robert,
On 19. Oct 2017, at 16:24, Robert Ramey via Boost
wrote: I make up my examples in the "example" directory. This includes a Jamfile to build and run the examples.
I create the BoostBook xml using and XMLmind - an xml editor. Within that environment I just use an "Include" directive to include the source code of the example in the correct place - easy as pie. If you download the slides associated with my recent CPPCon presentation, you can see how it's done.
https://www.youtube.com/watch?v=YxmdCxX9dMk https://www.youtube.com/watch?v=YxmdCxX9dMk
you can download the slides and example here:
thank you for these pointers. :) Sorry again, I mixed up BoostBook with Quickbook. Since I already wrote 90 % of the documentation in QuickBook, so I am going to stick to that for now. I watched your talk on Youtube a few days ago and I think it was a nice talk. I think we need more talks about code documentation. I believe that writing good documentation is not solvable by a tool, although it certainly helps if the tool is easy to use and not in the way. So far, I liked the Markdown variants best of all markup languages. Since Asciidoc is similar, I am glad that it was discussed on the list recently. For me the conclusion is this: the recipe for good documentation is just like the recipe for good code. It requires reviews and iteration. Best regards, Hans
On 10/19/17 8:40 AM, Hans Dembinski via Boost wrote:
thank you for these pointers. :) Sorry again, I mixed up BoostBook with Quickbook. Since I already wrote 90 % of the documentation in QuickBook, so I am going to stick to that for now.
I don't think there's a conflict. Quickbook is just a way of producing boostbook. I'm thinking that you can just include code file from quickbook just as you can from another boostbook editor. I don't know this for a fact though. My real point is that you can let the documentation depend on your coded examples so that they are always in sync and that it is generally very easy to do.
I believe that writing good documentation is not solvable by a tool, +1 although it certainly helps if the tool is easy to use and not in the way. +1 So far, I liked the Markdown variants best of all markup languages. Since Asciidoc is similar, I am glad that it was discussed on the list recently.
I see the appeal of Quickbook and other markdowns. But all such attempts suffer from all attempts to make a "new" language. It's much more work than it first appears. So if it's helpful, the demands for maintenance and extension eventually overwhelm the original design, which then generates the next attempt. It's a never ending quest.
For me the conclusion is this: the recipe for good documentation is just like the recipe for good code. It requires reviews and iteration.+1
Best regards, Hans
_______________________________________________ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
On 10/19/2017 11:49 AM, Robert Ramey via Boost wrote:
On 10/19/17 8:40 AM, Hans Dembinski via Boost wrote:
thank you for these pointers. :) Sorry again, I mixed up BoostBook with Quickbook. Since I already wrote 90 % of the documentation in QuickBook, so I am going to stick to that for now.
I don't think there's a conflict. Quickbook is just a way of producing boostbook. I'm thinking that you can just include code file from quickbook just as you can from another boostbook editor. I don't know this for a fact though. My real point is that you can let the documentation depend on your coded examples so that they are always in sync and that it is generally very easy to do.
I believe that writing good documentation is not solvable by a tool, +1 although it certainly helps if the tool is easy to use and not in the way. +1 So far, I liked the Markdown variants best of all markup languages. Since Asciidoc is similar, I am glad that it was discussed on the list recently.
I see the appeal of Quickbook and other markdowns. But all such attempts suffer from all attempts to make a "new" language. It's much more work than it first appears. So if it's helpful, the demands for maintenance and extension eventually overwhelm the original design, which then generates the next attempt. It's a never ending quest.
XML may be decent as a way of creating and passing around data in a structured way, but having to write documentation in XML is a hopeless task AFAICS. Quickbook is hugely superior and is a no-brainer to use. If others prefer Asciidoc that's fine also. The point is that nobody need write docs in XML if they do not want to and I do not think that most people want to. I certainly don't.
For me the conclusion is this: the recipe for good documentation is just like the recipe for good code. It requires reviews and iteration.+1
Best regards, Hans
On 10/19/17 11:09 AM, Edward Diener via Boost wrote:
XML may be decent as a way of creating and passing around data in a structured way, but having to write documentation in XML is a hopeless task AFAICS.
Agreed
Quickbook is hugely superior and is a no-brainer to use. If others prefer Asciidoc that's fine also. The point is that nobody need write docs in XML if they do not want to and I do not think that most people want to. I certainly don't.
Agreed. Which is why a looked for a superior alternative editing boost book (XML) files. I found it in XMLmind. Robert Ramey
participants (6)
-
Daniel James
-
Edward Diener
-
Hans Dembinski
-
John Maddock
-
Paul A. Bristow
-
Robert Ramey