Re: [boost] To modularize, or not to modularize. What is the plan?

On 5/6/19 9:40 PM, Rene Rivera via Boost wrote:

On Mon, May 6, 2019 at 8:17 PM Robert Ramey via Boost wrote:

...

On 5/6/19 4:47 PM, Rene Rivera via Boost wrote:

...

* BoostOrg would not produce a monolithic, combined, merged, etc distribution.

I don't see it as necessary for Boost to give this up. That is, I don't see the current setup as conflicting with the ability to just download the libraries he wants.

It conflicts in that the modular arrangement is not the same as the current monolithic arrangement.

...

* BoostOrg would produce collectively tested milestone modular

...

distributions.

I don't think anything has to change here.

The change would be in holding on to the belief that the single header dir and top level build is the face we should be putting forward to users.

...

Sounds lovely right? ...I'll leave the discussion of merits to responses

...

herein ;-)

I envision the construction of a tool which just goes to github and downloads a list of boost libraries.

Can we just stop trying to build more Boost specific tools?

...

For each library the download process makes a simple transform to a standalone directory for that library. Similar to what the global distribution currently looks like except for one library at at time.

You've just created an combinatorial explosion of distributions which you'll need to test. As users will complain when their particular combination doesn't work.

This would be useful right away. In vinnies 1.70.1 situation, one could

...

put this to use right away.

...

a)the user changes the name of the current beast directory to beast-1.70. b)downloads the latest from the master into the new beast directory.

I my ideal modular view Vinnie would publish a new Beast library version of 1.70.1. Along with the requirement that it's can be used with its dependencies of 1.70.0. Users would obtain that new version with their existing package management method. Hence, not that different with how you posit, but not tied to any particular Boost custom tool. Or arrangement.

...

What would it take to reach that modular goal? Why do I keep saying we've

...

been working on this for ages and ages? Briefly here's what it would take to get there (not in any particular order):

* Abandon the single header include tree. * Abandon the monolithic build infrastructure.

One would need a more "stand alone" tool for non-header only libraries.

Why? And not sure what you mean by that?

...

But I presume lots of users would just compile the *.cpp files into their app or build their own DLL. Ideally, the library package would/should contain a CMake script to do this.

Yes, they could make use of whatever build system the library provides support for. Or they could get it from an established package manager that supports their method of building.

...

* Ban relative use of inter-library dependencies.

I don't think that's possible.

Actually it seems we are almost there on that point. As I could only find one instance in the latest release of this problem. But it was a quick search only. So there may be more.

...

But I don't think that's necessary.

Really? How would you resolve a library referring to another one with a path like "../../thread" instead of "/boost/thread" (or equivalent).

...

The only thing is that a user would need to install the dependent libraries he needs.

Right, they could say, use a package manager or "manua"l downloads of what they need.

...

One could try to make a tool to do this

Yes, they are called package and dependency managers. There are many of them.

...

- but I've argued that that is a fools errand.

Okay two things.. 1. Are you saying that package manager developers are fools? 2. Ah, earlier you say we should write a custom Boost tool to create custom distributions, and now you say it's a fools errand. Did you just call yourself a fool? ;-)

...

Rather than argue that any more. I could just imagine user does the following:

a) Adds a boost header to his project. b) "installs" that header as above c) tries to build his project d) If something missing - call a) for the missing thing

at the end of that process, he has a minimal subset of boost required to support his project.

If someone has nothing else to do he could write a tool which does generates a list of dependencies for a given app as a text file. Then the user would do most of them in one shot. This would likely be an minor enhancement of BCP or similar program. But the result would be the same.

Similar tools already exist.. they are called Package & Dependency Managers. We should probably support them by making Boost easier to package by making Boost modular.

BTW - the user already incorporates non boost libraries into his project

...

using this same procedure. Ideally any dependency checking tools would work on these as well.

They do.

...

* Explicit declaration of inter-library dependencies.

I don't think this is necessary.

Sure, it's not required. There are tools to this job for you. But they aren't as precise as an explicit declaration from the author. And being explicit helps those existing tools by making their jobs as easy as reading that explicit list and avoiding all the work of scanning and guessing. Improving everyone's lives accordingly with reduced complexity.

...

* Strict normalized library layout.

I don't think this should be necessary. But I'm aware that some libraries don't follow convention regarding header layout. So either those have to change or the "downloader" tool would have to smart enough to sort those out. I don't recommend the latter option.

Wait, another tool to write? Oh, wait, no.. You mean make the same tool even smarter. And hence prone to mistakes.

...

* Remove, and ban, dependency cycles at the inter-library user consumable

...

granularity.

I don't think this is necessary. If one is following a chain of headers rather than a chain of modules - there are no cycles.

Sure, that's a true statement given your supposition. But lets face it, the reality is that everyone except Robert Ramey thinks in terms of module granularity. All the tools we use are written in those terms. And we do it at the module granularity because it is useful to think about them that way. It simplifies things like configuring the tools we use, how we advertise the libraries, how we track the libraries in source control repositories, documentation, bug tracking, and so on.

...

There's probably more items that I've forgotten above. But this should be

...

enough to converse about.

LOL - ya think?

Haha, there's only so much time in the world.. especially when you are waiting for friends to arrive for usual Monday night political gathering :-)

I think you're concept of "modularized boost" is at least broadly

...

similar to mine.

To summarize, the only things we would need:

a) we need a tool to download/transform one boost library at a time.

Those already exist. We should facilitate their existence.

Great I didn't realize that there was already a method for downloading a boost library an transforming the directory arrange from the boost develop one to the one that users use.

b) optionally, it would be nice to have a dependency listing tool. FYI

...

this is more difficult than it looks since the user doesn't all the boost libraries on his machine. Such a tool would have to trawl the boost master on github or some oneline database of headers summaries.

Those also exist. For example the Conan package manger can produce a list or graph of such dependencies in various format.

Double great - just pick one.

...

c) There would likely need to be separate directory - boost tools with a couple of things in it. Some stuff would be moved from boost root to boost/tools. So the user wouldn't need the root on his machine.

Yep. But likely would be minimal as we would lean on existing, external, tools ecosystem.

d) A good written explanation for users who want to do this.

...

We always need documentation. There can never be enough of it!

Our problem isn't that we don't have enough documentation. It's that we write tools and procedures that lack sufficient conceptual integrity to document. We only discover that when we write the documentation and by that time it's too late to re-design.

What we don't need to do is re-organize current

...

boost/development/testing etc. This is merely an alternative deployment concept. Boost developers would not be effected.

Well... I would implore that we do not support more than one deployment arrangement (multiple "channels" is perfectly okay though). But a sans-root arrangement actually makes development easier in that it facilitates individual library testing. It would be considerably easier to just clone your one library and test it against any particular existing versions of other Boost libraries.. without the juggling with git submodules.

That's really an orthogonal question. You and your team are free to evolve the current global/build/test in a way that you see fit. But in the mean time, we can just use the current one. It doesn't need to change. The current infrastructure is already working and in use for many years. It works fine for many people. I'm suggesting an additional method of deployment for special cases.

That's it. just three or 4 simple things. And no disruption of current

...

setup.

I feel so much better that you think it's simple.

LOL - it's simple until everyone starts mucking with it. Basically a case like vinnies would be handled by just being able to produce a special version of the source that constitutes vinnie's library. If we can't do that, that what would be the point of trying to make something more ambitious?