On 6/5/2017 11:34 PM, Vinícius dos Santos Oliveira via Boost wrote:

2017-06-06 0:00 GMT-03:00 Peter Dimov via Boost :

...

This is how the generated HTML documentation looks, using the default stylesheet, without any customizations on my part:

https://rawgit.com/boostorg/assert/feature/asciidoc/doc/html/assert.html

I've converted all of Boost.Http documentation to asciidoc. I don't intend to use QuickBook/BoostBook again. Too much NIH syndrome and too much inconvenient.

Using quickbook has always appeared easy to me. Generating html and pdf doc from quickbook source involves a number of steps that need to be correctly setup via a jamfile, and I think this is what always appears hard to others. I am certainly not against other methods of writing Boost documentation and will look at asciidoc. But I did want to point out that I do not think the issue with quickbook is really the quickbook syntax itself. Others of course may well disagree with this assessment.

I use the following options to generate the HTML:

asciidoctor -a toc=left -a toclevels=3 -a sectnums -a sectnumlevels=4 http.adoc

And the following to generate the ePUB:

asciidoctor -b docbook5 http.adoc pandoc -f docbook http.xml -o http.epub

I was considering paying someone to design a Boost-like theme and the opened the Boost.Asio documentation. I changed my mind. There is no reason somebody would like to mirror this theme.

It's infinitely easier to write documentation using asciidoc, and this is a plus also to receive contributions. Nobody outside Boost know how to get boostbook up and ready to generate HTML, PDF and ePUB.

Also, asciidoc was designed to make syntactic elements directly mirror docbook semantics. So you have great Docbook output with a syntax that is as pleasant to use as MarkDown.

Also, I've found many different ways to generate PDF using asciidoc. All of them are pretty different when you compare the output of each process. Just noting in case you didn't like this one generated PDF.

Have fun playing with asciidoc.

2017-06-06 12:03 GMT-03:00 Lorenzo Caminiti via Boost

:

Looks nice. I am not familiar with asciidoc...

* Does ascii interface with Doxygen? For example, can you embed in the asciidoc output a reference section generated by Doxygen and have hyperlinks in between the ascidoc and Doxygen sections? (Similar to Quickbook [xinclude ...].)

asciidoc was designed to support DocBook semantics. So, by converting the file to DocBook XML, you shouldn't loss any information from the documentation. If you can somehow interface DocBook XML and Doxygen, this should be doable. You can use a passthrough block in asciidoc syntax to directly write in DocBook XML if needed: http://asciidoctor.org/docs/user-manual/#pass-blocks * Does asciidoc allow to show code snips marked in an external source

file? For example, to show example code snips in Quickbook using [import <file-name>] and then [abc], where //[abc ... //] are used as markers in file-name.

- http://asciidoctor.org/docs/asciidoc-syntax-quick-reference/#source-code - http://asciidoctor.org/docs/user-manual/#listing-blocks I take a long time to answer, but I'm sure you can get quick answers directly in the asciidoc spaces. -- Vinícius dos Santos Oliveira https://vinipsmaker.github.io/

On Tue, Jun 6, 2017 at 11:03 AM, Lorenzo Caminiti via Boost wrote:

* Does asciidoc allow to show code snips marked in an external source file? For example, to show example code snips in Quickbook using [import <file-name>] and then [abc], where //[abc ... //] are used as markers in file-name.

This is the QuickBook feature I would miss. I've used that heavily to ensure that the code embedded in the documentation is code that actually builds and runs, without having to remember to (re-)paste updates into the document source.

...

* Does asciidoc allow to show code snips marked in an external source file? For example, to show example code snips in Quickbook using [import <file-name>] and then [abc], where //[abc ... //] are used as markers in file-name.

This is the QuickBook feature I would miss. I've used that heavily to ensure that the code embedded in the documentation is code that actually builds and runs, without having to remember to (re-)paste updates into the document source.

I too have been playing with radically different methods of generating documentation for Outcome v2, seeing as people panned the v1 Outcome documentation. This is bleeding edge stuff i.e. it currently segfaults a lot, but it is getting there. It uses Hugo to compile the Markdown source files into HTML via a Travis per-commit job. Standardese may - once Jonathan fixes the bugs - generate the reference API docs in Markdown from doxygen marked up C++ source files using libclang. If we can get it working, it'll be a whole new age of "easy" for writing docs for complex C++ codebases, and moreover one which finally can parse the worst possible C++ metaprogramming without blinking because anything clang can parse can be documented. You literally run just two build tools, both easily executed from Travis, and push the changes to gh-pages on Github. Changing documentation is as easy as modifying the Markdown source or the doxygen format comment section in the relevant header, git commit and git push. Hugo generates the syntax highlighting, the wiki linking, the search index and everything else. Oh, and it looks pretty, even on a mobile phone. You can swipe the tutorial left and right for example. You can see our joint efforts so far for Outcome v2 at https://ned14.github.io/outcome/. You can thank Andrzej for the tutorial written so far, it's all him. Niall -- ned Productions Limited Consulting http://www.nedproductions.biz/ http://ie.linkedin.com/in/nialldouglas/

On 07.10.2017 05:12, Steven Watanabe via Boost wrote:

AMDG

...

and moreover one which finally can parse the worst possible C++ metaprogramming without blinking because anything clang can parse can be documented. Just because the parser can handle it, doesn't mean that it's a good idea. For example, in documentation, I always hide complex SFINAE, and instead provide an English description of the conditions. In general, I find the more complex the

On 10/06/2017 03:38 PM, Niall Douglas via Boost wrote: metaprogramming is, the more tweaking is required to turn the doxygen output into something sane. This is rarely because doxygen can't parse the code, but is rather because the raw definitions in the source are not very human-friendly.

standardese is designed to handle such things. For example, if you have: ``` template > void foo(const T& obj); ``` You can write the following documentation comment: ``` /// \notes Thie function does not participate in overload resolution unless T is … /// \param 1 /// \exlude ``` And standardese will completely hide the SFINAE part away.

On 07/10/2017 17:25, Robert Ramey via Boost wrote:

On 10/6/17 8:12 PM, Steven Watanabe via Boost wrote:

...

...

If we can get it working, it'll be a whole new age of "easy" for writing docs for complex C++ codebases,

   Er, I've always considered the hard part of documentation to be the organization and content rather than specific tools.

So, so, so, true.  I huge problem these days is that the attractive presentation of documentation gives the impression that documentation is well done.  It's very, very misleading.  It's a plague.

Indeed, but there's another issue here I would like to raise: * How do you know when a library is well designed? The answer for me at least in part is: * Can I, starting with a blank sheet of paper, easily explain what this does? I've lost track of the number of times I've redesigned something because it failed that test. The problem I see then, is that tools that extract documentation direct from code have the effect of disengaging the authors brain from the task of explaining what their code actually does.  Well it does for me anyway ;) I've also come to the conclusion that standardese does not make good documentation - despite having authored more than my fair share. And now I'll get back to lurking again.... John. --- This email has been checked for viruses by AVG. http://www.avg.com

* How do you know when a library is well designed?

The answer for me at least in part is:

* Can I, starting with a blank sheet of paper, easily explain what this does?

I've lost track of the number of times I've redesigned something because it failed that test.

There is currently a mini-repeat of the Outcome v1 review happening on std-proposals. Many of the same points being made there by the uninitiated expert as was made here back in May. Same progressions of understanding. Same arrival at the same conclusions. It's encouraging. But also slightly surprising, the simpler the concept, the harder it seems to become for people to grok it in a strange way. But it also contradicts what you just said. You can very easily explain a library e.g. ASIO. It's a networking library. Done. But does that have much relation to its good or bad design? Same goes for Expected|Outcome. It lets functions return a T|E, like std::variant. But is the Expected or is the Outcome design good or bad? What are the merits and disadvantages of each design decision? How do you weigh them?

The problem I see then, is that tools that extract documentation direct from code have the effect of disengaging the authors brain from the task of explaining what their code actually does.  Well it does for me anyway ;)

I've also come to the conclusion that standardese does not make good documentation - despite having authored more than my fair share.

Standardese as in the word soup they put in the C++ standard is not documentation. I tried my hand at writing some of it for the operator try proposal, and immediately got nit picked on lots of bits in it. I'm no language lawyer. I care about *themes*, shapes, forms, fit like any library not compiler developer. Nit picky detail for me is just punctuation, to be added or removed as needed so long as it doesn't break ABI. Which is why I'm not on WG21! Nor would be much use there I should think. But Standardese the tool for generating reference API docs is a large improvement on doxygen, and once it matures quite a bit ought to be very useful for Boost type C++. Be appalled, after all, at how useless the doxygen "extract all" dump of Outcome v2 is: https://ned14.github.io/outcome/doxygen/namespaceoutcome__v2__xxx.html. Far too much implementation detail, it's useless. But Standardese the tool once we get it working ought to chop that very nicely down into the "just right" balance between detail and need-to-know-this. And being pretty in presentation and disability friendly doesn't hurt. Niall -- ned Productions Limited Consulting http://www.nedproductions.biz/ http://ie.linkedin.com/in/nialldouglas/

Would this be helped by characterizing Expected/Outcome as something like ... "to be used as a resultant type by an operation which might fail" or some such?  This would make all the arguments about what happens on assignment out of scope.

Thanks to your suggestion Robert to make Outcome a simple struct, assignment is now uncontroversial.

  Of course now people would say the

scope is too narrow.  etc.....  The "reviews" of these ideas tend to try to generalize them - generally a good thing - but not always.  Some times the attempt to reconcile all the requested features results in a watering down of the original good idea.

The v2 object is very considerably more simple than the v1 object. You may remember I converted over an existing codebase from v1 to v2, and for the most part it went okay. Expected is also much simpler than it was last May. Vicente took many of the same simplifying steps to Expected as I did to Outcome from the review.

When a simple idea like Outcome/Expected generates documentation of more than 4? pages, it's time to step back and ask oneself whether we've lost our and turned our original elegant idea into a monstrosity.

I know what you're saying. But when very senior members of WG21 seem to also need more than twenty pages of hand holding tutorial for a class which carries T|E and does nothing else, I think it's more a case of that the simpler and more fundamental the object, the vastly more documentation explaining it becomes needed. As an analogy, you may remember the "variant wars" which went on for nearly forever. A very simple, very well understood, object often generates the strongest opinions and people absolutely convinced that the other is wrong. Proof required to convince people thus rises accordingly. Niall -- ned Productions Limited Consulting http://www.nedproductions.biz/ http://ie.linkedin.com/in/nialldouglas/

On 10/7/17 10:25 AM, John Maddock via Boost wrote:

Indeed, but there's another issue here I would like to raise:

* How do you know when a library is well designed?

The answer for me at least in part is:

* Can I, starting with a blank sheet of paper, easily explain what this does?

I've lost track of the number of times I've redesigned something because it failed that test.

WOW - Great minds think alike! I'm hoping you can find time to skip through the video of my presentation a CPPCon 2017 - if only so you can honestly endorse it. The short versions is a) normal process code, test then write documentation. Doesn't work. b) recommended process: Write documentation in parallel with design and coding. step 1: write one sentence description of what the library is meant to do. Step 2: make a simple illustrative example of how the library is to be used. If required make more examples. step 3: make the library to make the examples work. Step 4: fix library and examples so they make sense. Step 5: add notes. step 6: Fill in reference documentation using standard format - e.g. SGI format. c) This what I do and I think its extremely helpful. The main benefit is that I find design mistakes sooner so I have to do less rework. Second benefit is that final product is better and has list "added in" quirky features.

The problem I see then, is that tools that extract documentation direct from code have the effect of disengaging the authors brain from the task of explaining what their code actually does.  Well it does for me anyway ;)

100% agree. I do agree that these tools might be helpful - but only up to a point.

I've also come to the conclusion that standardese does not make good documentation - despite having authored more than my fair share.

LOL - at least you've atoned for your sins. But really the standards have a different problem. Not so much as to help users, but more intended to help implementors. I would argue it's whole different problem.

And now I'll get back to lurking again.... John.

Right. What do we have to do to get you to make a conference appearance - if only to prove that you're an actual person rather than some robot, high school student, prison inmate with a life sentence or some thing else. Robert Ramey

AMDG On 10/07/2017 08:41 PM, Vinnie Falco via Boost wrote:

And what better place to put that documentation than in a specially formatted comment immediately preceding the function declaration? Here's an example from some code I am in the middle of writing:

/** Return `true` if the known scheme is a special scheme

The list of special schemes is as follows: ftp, file, gopher, http, https, ws, wss.

@param s The known-scheme constant to check @return `true` if the scheme is special */ bool is_special(scheme s);

I find this rather lacking. Except for the list, it's mostly an English restatement of the declaration. Saying the same thing three times is not really more useful than saying it once. - What is a "known-scheme?" - Why are these schemes special? What does it mean for a scheme to be special? why should I care whether it's special? In Christ, Steven Watanabe

On Sat, Oct 7, 2017 at 7:58 PM, Steven Watanabe via Boost wrote:

- Why are these schemes special? What does it mean for a scheme to be special?

I know you'll probably have trouble sleeping with these unanswered questions rolling around in the ole' noggin so this is what I know: https://url.spec.whatwg.org/#special-scheme Thanks

On 10/7/17 7:41 PM, Vinnie Falco via Boost wrote:

On Sat, Oct 7, 2017 at 7:12 PM, Robert Ramey via Boost wrote:

...

...

* Can I, starting with a blank sheet of paper, easily explain what this does?

I've lost track of the number of times I've redesigned something because it failed that test.

WOW - Great minds think alike!

I am also in agreement. I find that just as I am writing the declaration for a class or function, and I have a vision fixed in my mind of what the thing should do and its preconditions, postconditions, inputs, and outputs are, that it is the best time to write documentation.

And what better place to put that documentation than in a specially formatted comment immediately preceding the function declaration? Here's an example from some code I am in the middle of writing:

/** Return `true` if the known scheme is a special scheme

The list of special schemes is as follows: ftp, file, gopher, http, https, ws, wss.

@param s The known-scheme constant to check @return `true` if the scheme is special */ bool is_special(scheme s);

...

...

...tools that extract documentation direct from code have the effect of...explaining what their code actually does. ... 100% agree. I do agree that these tools might be helpful...

Much as I hate to disagree with someone who's agreeing with me I might note that I don't recommend documenting functions beyond simple statement of what it does except in very special cases. That is I personally don't recommend a page in the documentation dedicated to a member function. I also make a big distinction between concepts (type requirements), types (classes/struct), functions, and metafunctions. Robert Ramey

On 10/7/2017 12:25 PM, Robert Ramey via Boost wrote:

On 10/6/17 8:12 PM, Steven Watanabe via Boost wrote:

...

...

If we can get it working, it'll be a whole new age of "easy" for writing docs for complex C++ codebases,

   Er, I've always considered the hard part of documentation to be the organization and content rather than specific tools.

So, so, so, true.  I huge problem these days is that the attractive presentation of documentation gives the impression that documentation is well done.  It's very, very misleading.  It's a plague.  I have a lot to say about this very phenomenon in my recent presentation at cppcon 2017. Hopefully the video will be available shortly.  I wasn't totally happy with my presentation, testimony to how difficult is to capture the problem and address it.

There have been a few recent boost library submissions which have suffered in this area to the extent I believe they may have turned a possible acceptance to a rejection.  In one case I believe that a previously rejected library was accepted and a large part of that change of assessment was due to much improved documentation.

So, contrary to what one might think, I do believe we are making some progress in this area.

The major problem with library docs is always that programmers become too lazy to explain what the various things are that library actually does and what the constructs are to achieve each of those various things. Instead, for too many programmers, library documentation is 99% examples and a reference, the latter being nothing more than a listing of constructs which can be used, with no explanation of when or why one might want to use them. That's it for an awful lot of developers ! And as long as programmers in general consider that as adequate documentation, library developers will continue to follow that model. Even when you point out to programmers that such documentation is inadequate for end-users who actually want to use your library, the usual response is just bafflement that the end-user doesn't "get it". I understand that programmers do not want to write documentation, but want to create really nifty and useful libraries and programming constructs. I understand that many developers are not native English speakers and find it quite onerous to write documentation in a secondary language which they can almost never know as well as their own primary language. I sympathize with both these problems. But unless docs can explain to end-users what is useful about a library and how that usefulness translates to the constructs of that library, library documentation will not be adequate for a great many intelligent programmers. I feel like an old dinosaur writing this. The current age of computer programming expects so little of documentation that I am probably already obsolete in my documentation needs. But I hope programmers will try to understand that the most brilliant library is useless if it cannot be understood or, conversely, if understanding it takes such an inordinate amount of end-user's time that he just gives up in trying to use the library. I think very little of this has to do with tools per se. I am certainly for the best tools which anyone finds easy and adequate to use. Rather this problem has to deal with the approach to documentation as a means to explain what one offers as library functionality.

...

...

and moreover one which finally can parse the worst possible C++ metaprogramming without blinking because anything clang can parse can be documented.

   Just because the parser can handle it, doesn't mean that it's a good idea.  For example, in documentation, I always hide complex SFINAE, and instead provide an English description of the conditions.  In general, I find the more complex the metaprogramming is, the more tweaking is required to turn the doxygen output into something sane.  This is rarely because doxygen can't parse the code, but is rather because the raw definitions in the source are not very human-friendly.

The question of how to document/explain certain C++ constructs such as template parameters, concepts and meta functions is an interesting one. Tools like Doxygen and others don't address this.  It's not obvious and not described by any "tutorials" on documentation.  Actually, there isn't even a concensus.  In a recent review, I complained that requirements on template parameters weren't document.  The library submitter countered that it wasn't necessary because the code contained static_asserts to enforce (otherwise undocumented) requirements.  Even the C++ standard calls for this - yet it's often skipped.  SGI documents describe documentation standards in detail - but this is almost ignored.

...

...

You can see our joint efforts so far for Outcome v2 at https://ned14.github.io/outcome/. You can thank Andrzej for the tutorial written so far, it's all him.

Andrezej is a great writer who understands all this.  I know this personally from

a) reading his blog b) going back and forth with him on the documentation structure of the safe numerics library.  Though this effort is better than most, there have been a lot of problems with this and he is one of the few that understands this stuff.  We share a common view as to what documentation should be.  So his criticisms are on point and almost always correct. He is also very meticulous which helps me a lot.

To summarize:

a) Our (boost and C++ generally) problem is not rooted in the tools, though the tools don't help much.

b) There is growing recognition that it's a serious problem both in boost and C++ generally.

c) In reviews, we're more critical of documentation than we used to be and suggestions to improve it are more constructive.

So things are slowly getting better. I'm optimistic about the future.

Robert Ramey

On 10/7/17 2:55 PM, Niall Douglas via Boost wrote:

...

a) Our (boost and C++ generally) problem is not rooted in the tools, though the tools don't help much.

Boost tooling is painful. I have to think with Boost tooling. I don't want to have to think, I want to get on with it.

Ahhh, this explains a lot. Don't do it this way.

doxygen suffers badly from this too as soon as you push it a bit. You end up writing lots of simplifying constructs for it with #ifdef. Painful. Standardese doesn't suffer from that part at least.

I'm always down on DOxygen. But that's a little unfair. If DOxygen is restricted to filling in the reference section it's OK. It can be made work in this role. The probem comes when one starts to thing that this product IS the documentation. It's only part of - the easy part. It's a paraphrase and formating of the header files. Of some value granted. But I'm afraid that many developers stop there and consider themselves done. Or they try and "extend" DOxygen through macros to handle things for which it's not a good fit. My approach described above is really helpful. It makes the documentation much less tedious to write. and shorter too. And it makes your code simpler and shorter also. Try it and get back to me. Robert Ramey

On 10/9/17 9:08 AM, Paul A. Bristow via Boost wrote:

Doxygen syntax mark-up is the key to any tool to process well. The quality of the in-code documentation is of paramount importance. No tool can compensate for what is missing here.

I think we should judge*future* libraries by the quality of this - so that future tools like this can (re-)process it to produce future documentation.

Hmmmmm - what does "this" refer to here? in each place? Robert Ramey

...

This is bleeding edge stuff i.e. it currently segfaults a lot, but it is getting there. It uses Hugo to compile the Markdown source files into HTML via a Travis per-commit job. Standardese may - once Jonathan fixes the bugs - generate the reference API docs in Markdown from doxygen marked up C++ source files using libclang.

I'm very pleased to see the use of Clang compiler.

Jonathan has fixed the segfaults, we are now working on the complex metaprogrammed constructs it gives up upon. It gets far, far further than doxygen already. Outcome v2 assembles itself via metaprogramming from slices of implementation depending on the types you feed it, so it's just about the worst kind of class to document because its inheritance diagram is completely unstable. With doxygen you have no choice but to expose such internal implementation detail. I believe with Standardese we may reach a point where the final class as the end user sees it is presented, and nobody needs to care or think about any inheritance implemented internally.

...

You can see our joint efforts so far for Outcome v2 at https://ned14.github.io/outcome/. You can thank Andrzej for the tutorial written so far, it's all him.

Looks pretty - though I don't want to read it on a phone or tablet and resent that the layout compromises size so that the font is too big on a 'proper' desktop screen.

Maybe your eyes are younger than mine? The font size is about right for my 13 inch MacBook. To be honest I've not tried it yet on a desktop monitor. If it's too big, it can be told otherwise i.e. non-linear font scaling according to DPI. Modern CSS is great.

This is a modest library - layout may not scale well? Nor do I like the sans serif font, but

it's fashionable ;-)

It doesn't feel modest. The tutorial we have in mind is very, very long. Never ending almost. And even then I suspect people will complain about lack of sufficient tutorial coverage of the available facilities. This might be a small library of only a few thousand lines, but its use cases are very many and non-obvious to the uninitiated, and thus will need additional tutorial sections. Thanks to Hugo, the entire site is skinnable and themeable. Look and feel is completely customisable and switchable as per any Boost peer review feedback. So if the peer review wants it to look exactly like BoostBook, that can be delivered.

And people are *very* sensitive to the C++ syntax coloring - it would be nice for readers to easily control

that too, as has been discussed before. I presume snippets are using the actual C++ code, and can be referenced by hyperlinks? (As using Quickbook at present).

The syntax highlighting is done by Pygments. It's not awful. Yes linking to original source should be easy to add. It's now on my todo.

I thought that a most interesting feature was the tiny search box on the Doxygen reference pages with a dropdown to separate functions from macros etc. This is the equivalent to *part* of a conventional index. As often, the hard bit is getting people to find it and use search fully.

Did the (much larger) search box in the top left not work for you? Reference APIs aren't on there yet, and thus don't show up in the search box yet. But back before Outcome blew up old Standardese, it was working well, if you typed in some words associated with what you wanted it provided a drop down shortlist of possible matches. Niall -- ned Productions Limited Consulting http://www.nedproductions.biz/ http://ie.linkedin.com/in/nialldouglas/

On 6/6/17 7:10 AM, Edward Diener via Boost wrote:

On 6/5/2017 11:34 PM, Vinícius dos Santos Oliveira via Boost wrote:

Using quickbook has always appeared easy to me. Generating html and pdf doc from quickbook source involves a number of steps that need to be correctly setup via a jamfile, and I think this is what always appears hard to others.

Right. I skip the Jamfile and use a simple shell script/make file. maybe about 10 lines. Given that asciidoc generates docbook files, it's an easy fit into boost workflow. It's just e boostbook -> docbook step that get's skipped. This step isn't all that essential anyway. Robert Ramey