So.... what are we going to do about modules?

John Maddock

5 Apr 2022 5 Apr '22

11:58 a.m.

The title says it all really, but in more detail: 1) Is there any existing best practice we should follow yet? 2) Can we share code content effectively between headers and modules? And by modules I mean "proper" ones: we should be able to write "import boost.whatever;" rather than importing a header. 3) What directory structure should we all be using for module definition files? 4) Do we have Boost.Build support yet? 5) What file extensions should we be using? 6) I presume we would all use the boost.libraryname and boost.libraryname.sub_feature naming convention with boost.libraryname pulling in the whole library? 7) Does anyone have a handle on the current state of std library modularization? 8) Is there demand for this? And no doubt lots of other things I haven't thought of. Curiously yours, John.

Show replies by date

René Ferdinand Rivera Morell

5 Apr 5 Apr

12:46 p.m.

On Tue, Apr 5, 2022 at 6:58 AM John Maddock via Boost wrote:

...

The title says it all really, but in more detail:

1) Is there any existing best practice we should follow yet?

No. And I mean that generally in the whole of the C++ ecosystem. 2) Can we share code content effectively between headers and modules?

...

And by modules I mean "proper" ones: we should be able to write "import boost.whatever;" rather than importing a header.

Not sure what you're asking with that.

...

3) What directory structure should we all be using for module definition files?

Might have to answer all the other ones and more regarding C++ ecosystem best practices before being able to answer this one.

...

4) Do we have Boost.Build support yet?

No. And it's not going to exist soon. As the vendor support is still "experimental".

...

5) What file extensions should we be using?

We'll have to wait for the " C++ Modules Ecosystem Technical Report" to come out https://github.com/cplusplus/modules-ecosystem-tr perhaps?

...

6) I presume we would all use the boost.libraryname and boost.libraryname.sub_feature naming convention with boost.libraryname pulling in the whole library?

Why? 7) Does anyone have a handle on the current state of std library

...

modularization?

Having a "std" and "std.compat" is up for plenary vote for C++23 < https://github.com/cplusplus/papers/issues/1115>.

...

8) Is there demand for this?

I highly doubt it. -- -- René Ferdinand Rivera Morell -- Don't Assume Anything -- No Supone Nada -- Robot Dreams - http://robot-dreams.net

Horváth V.

12:54 p.m.

On 2022. 04. 05. 14:46, René Ferdinand Rivera Morell via Boost wrote:

...

I highly doubt it.

Something I noticed is that a lot of people raise compile times as a reason they don't want to use Boost and modules supposedly help with that.

René Ferdinand Rivera Morell

1:11 p.m.

On Tue, Apr 5, 2022 at 7:54 AM Horváth V. wrote:

...

On 2022. 04. 05. 14:46, René Ferdinand Rivera Morell via Boost wrote:

...
I highly doubt it.

Something I noticed is that a lot of people raise compile times as a reason they don't want to use Boost and modules supposedly help with that.

The gains to be had depend highly on both the code structure and project structure (both of the modules and the user code). The gains are almost entirely in the re-parsing. I.e. they are not going to help much on the template instantiation. Which, IIRC, is the larger of the compile time complaints for Boost. But someone would have to do some performance analysis of a good size project that uses Boost in a compiler that supports such perf inspection to really know. -- -- René Ferdinand Rivera Morell -- Don't Assume Anything -- No Supone Nada -- Robot Dreams - http://robot-dreams.net

Niall Douglas

1:57 p.m.

On 05/04/2022 14:11, René Ferdinand Rivera Morell via Boost wrote:

...

On Tue, Apr 5, 2022 at 7:54 AM Horváth V. wrote:

...
On 2022. 04. 05. 14:46, René Ferdinand Rivera Morell via Boost wrote:

...
I highly doubt it.

Something I noticed is that a lot of people raise compile times as a reason they don't want to use Boost and modules supposedly help with that.

The gains to be had depend highly on both the code structure and project structure (both of the modules and the user code). The gains are almost entirely in the re-parsing. I.e. they are not going to help much on the template instantiation. Which, IIRC, is the larger of the compile time complaints for Boost. But someone would have to do some performance analysis of a good size project that uses Boost in a compiler that supports such perf inspection to really know.

To be specific here, what are the gains of Modules over Precompiled Headers, which Boost.Build already supports? Niall's fairly confident answer: little to none, and perhaps even a regression as ODR resolution is time expensive. A better question is whether the compartmentalisation and isolation features of Modules are worth adopting Modules? Niall's fairly confident answer: it *may* be worth it for new codebases, but it's very unlikely to be worth retrofitting an existing codebase. An awful lot of work for little gain, in my opinion, for most existing codebases. tl;dr; just turn on precompiled headers. Niall

René Ferdinand Rivera Morell

2:37 p.m.

On Tue, Apr 5, 2022 at 8:58 AM Niall Douglas via Boost < boost@lists.boost.org> wrote:

...

On 05/04/2022 14:11, René Ferdinand Rivera Morell via Boost wrote:

...
On Tue, Apr 5, 2022 at 7:54 AM Horváth V. wrote:

...
On 2022. 04. 05. 14:46, René Ferdinand Rivera Morell via Boost wrote:

...
I highly doubt it.

Something I noticed is that a lot of people raise compile times as a reason they don't want to use Boost and modules supposedly help with that.

The gains to be had depend highly on both the code structure and project structure (both of the modules and the user code). The gains are almost entirely in the re-parsing. I.e. they are not going to help much on the template instantiation. Which, IIRC, is the larger of the compile time complaints for Boost. But someone would have to do some performance analysis of a good size project that uses Boost in a compiler that supports such perf inspection to really know.

To be specific here, what are the gains of Modules over Precompiled Headers, which Boost.Build already supports?

Niall's fairly confident answer: little to none, and perhaps even a regression as ODR resolution is time expensive.

It depends :-) Assuming that you smartly organize and engineer your modules they will end up with shorter compile performance for certain compile contexts. I think that's sufficiently qualified ;-) ... The real answer is that precompiled headers need to reparse the entire TU context at any change in that TU context. While modules isolate the reparse to the "headers" specific to the modules. Hence if properly used there will be less reparsing with modules. But it's definitely not trivial or straightforward to implement that proper module structure. -- -- René Ferdinand Rivera Morell -- Don't Assume Anything -- No Supone Nada -- Robot Dreams - http://robot-dreams.net

Niall Douglas

3:24 p.m.

On 05/04/2022 15:37, René Ferdinand Rivera Morell wrote:

...

To be specific here, what are the gains of Modules over Precompiled Headers, which Boost.Build already supports?

Niall's fairly confident answer: little to none, and perhaps even a regression as ODR resolution is time expensive.

It depends :-) Assuming that you smartly organize and engineer your modules they will end up with shorter compile performance for certain compile contexts. I think that's sufficiently qualified ;-) ... The real answer is that precompiled headers need to reparse the entire TU context at any change in that TU context. While modules isolate the reparse to the "headers" specific to the modules. Hence if properly used there will be less reparsing with modules. But it's definitely not trivial or straightforward to implement that proper module structure.

You're right that Modules could and should do that optimisation. But I believe that none of the current three major implementations do do that optimisation i.e. they're no more efficient than their own precompiled header implementation, which has received many years of optimisation. Just to be clear, I'm saying that some precompiled headers implementations pre-instantiate final templates etc, and for those compilers whose precompiled headers don't do that then neither does their Modules implementation (yet). Re: ODR resolution stage, you're right that that is highly dependent on how much ODR needs resolving. However I remember talking to somebody in a WG21 meeting where they were telling me that their final binary production times rose by about 20% with Modules. They reckoned it was because in their TUs they reuse the same names for non-extern types but with different definitions, and it is O(N^2) or worse complexity for the compiler to resolve those. They apparently can't enable LTO either, because link times go through the roof. Future Modules implementations will become better if and only if enough people adopt Modules and report performance pathologies. Me personally, I suspect anybody who cares deeply about build times will probably stick with whatever techniques they currently are using, as the gain from adopting Modules over those techniques is currently not great for most codebases. I'd like to hope WG21 will propose a Modules v2 in years to come which improves the cost benefit proposition. Niall

Vinnie Falco

3:37 p.m.

Have any Boost libraries deployed C++20 modules? Thanks

John Maddock

4:56 p.m.

...

To be specific here, what are the gains of Modules over Precompiled Headers, which Boost.Build already supports?

Well they're all shiny and new of course ;) But ease of use might be one - no need to set up PCH's, just use import declarations and be done with it. Aside: for those of us of an age to remember Turbo C++, the PCH support was IMO better thought out and so stupidly fast compared to the competition of the time it was almost laughable. It probably offered a good percentage of the improvements offered by modules too, though whether it would ever have scaled to today's coding paradigms and the latest C++ features is another matter. John.

...

Niall's fairly confident answer: little to none, and perhaps even a regression as ODR resolution is time expensive.

A better question is whether the compartmentalisation and isolation features of Modules are worth adopting Modules?

Niall's fairly confident answer: it *may* be worth it for new codebases, but it's very unlikely to be worth retrofitting an existing codebase. An awful lot of work for little gain, in my opinion, for most existing codebases.

tl;dr; just turn on precompiled headers.

Niall

_______________________________________________ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost

Daniela Engert

2:56 p.m.

Am 05.04.2022 um 15:11 schrieb René Ferdinand Rivera Morell via Boost:

...

On Tue, Apr 5, 2022 at 7:54 AM Horváth V. wrote:

...
...
I highly doubt it. Something I noticed is that a lot of people raise compile times as a reason they don't want to use Boost and modules supposedly help with

On 2022. 04. 05. 14:46, René Ferdinand Rivera Morell via Boost wrote: that.

The gains to be had depend highly on both the code structure and project structure (both of the modules and the user code). The gains are almost entirely in the re-parsing. I.e. they are not going to help much on the template instantiation.

Partially true. During the compilation of the module interface unit, the following is done: 1) all parsing. That alone is a big win, in particular with Boost and its boatload of includes. 2) all name lookup that can be done in translation phase 7 is performed. In addition, all template instantiations that can be done during the compilation of that TU are done. 3) all overload resolution within the overload sets formed in 2) is done. The less unsolicited unqualified name lookup is asked for by the source, the more work can be shifted to the one-time compilation of the module interface. I've seen too many template-heavy libraries in my life that seem to be totally unaware of that. The split between visibility of names and reachability of semantic properties is all but irrelevant in the traditional header world but plays a vital role in the modules world.

...

Which, IIRC, is the larger of the compile time complaints for Boost. But someone would have to do some performance analysis of a good size project that uses Boost in a compiler that supports such perf inspection to really know.

Ciao Dani -- PGP/GPG: 2CCB 3ECB 0954 5CD3 B0DB 6AA0 BA03 56A1 2C4638C5

Daniela Engert

1:09 p.m.

Hi John, thanks for bringing up the topic of C++20 modules in the realm of Boost! Am 05.04.2022 um 13:58 schrieb John Maddock via Boost:

...

The title says it all really, but in more detail:

1) Is there any existing best practice we should follow yet?

WG21 SG15 is about to come up with a document on that. Until then, I've touched this in my Meeting 2019 talk on modules, "Modules - the beginner's guide" Boost should be pretty safe here in general.

...

2) Can we share code content effectively between headers and modules? And by modules I mean "proper" ones: we should be able to write "import boost.whatever;" rather than importing a header.

This is possible and there's more than one way to do that. Both require an additional so-called module interface unit that defines the interface of a module (at least one per library). Option 1 is to export the public names through exported using-declarations in the purview of the module. Option 2 is a kind of preprocessor gymnastics in the existing headers to declare public names as exported when compiled from within the module interface unit. I've done that in the popular {fmt} library which is a so-called 'dual-mode library' (a term coined by me to describe the capability of a library to be consumed as both a module and a traditional header)

...

3) What directory structure should we all be using for module definition files?

Module interface units are just sources that require a special invocation at compile time.

...

4) Do we have Boost.Build support yet?

None afaik.

...

5) What file extensions should we be using?

Compiler vary here (msvc: .ixx, gcc: .cpp, clang: .cppm). SG15 seems to slowly converge towards .ixx. At least this is what I see in the SG15 meetings.

...

6) I presume we would all use the boost.libraryname and boost.libraryname.sub_feature naming convention with boost.libraryname pulling in the whole library?

A perfect situation for bikeshedding...

...

7) Does anyone have a handle on the current state of std library modularization?

It is voted into the C++23 draft. Microsoft has an experimental implementation for ages, Stephan (STL) has this currently on his plate and is actively working on it. I expect this to see this Real Soon Now in msvc.

...

8) Is there demand for this?

I've tried to get a handle on a modularized Boost in the past year but its current implementation seems to be quite hostile towards modules.

...

And no doubt lots of other things I haven't thought of.

There are a few pitfalls and things to consider if you want to use existing code in the modules world. I point out some of them in my talks last year (NDC, CppCon, Meeting C++). The last one from Meeting C++ 2021, "A (Short) Tour of C++20 Modules" is the most comprehensive and polished one, slides can be found on the Meeting C++ website. I gave a company training on topic that last week because the developers there are considering switching over to modules and were asking about the way forward, considerations to be taken into account, the state of the ecosystem, and all of my experiences in our daily use of modules in our company codebase. Ciao Dani

John Maddock

4:50 p.m.

...

...
2) Can we share code content effectively between headers and modules? And by modules I mean "proper" ones: we should be able to write "import boost.whatever;" rather than importing a header.

This is possible and there's more than one way to do that. Both require an additional so-called module interface unit that defines the interface of a module (at least one per library).

Option 1 is to export the public names through exported using-declarations in the purview of the module. Option 2 is a kind of preprocessor gymnastics in the existing headers to declare public names as exported when compiled from within the module interface unit. I've done that in the popular {fmt} library which is a so-called 'dual-mode library' (a term coined by me to describe the capability of a library to be consumed as both a module and a traditional header) Excellent, so it can be done, that was exactly what I was thinking of. Do you have any feel for whether this improves performance and/or gets much use?

...
3) What directory structure should we all be using for module definition files?

Module interface units are just sources that require a special invocation at compile time. Sure, but it's helpful for folks to know where to find stuff, we could just put them under /src, but since they're completely orthogonal to regular source files I would tend to opt for a separate directory.

...
4) Do we have Boost.Build support yet?

None afaik.

...
5) What file extensions should we be using?

Compiler vary here (msvc: .ixx, gcc: .cpp, clang: .cppm). SG15 seems to slowly converge towards .ixx. At least this is what I see in the SG15 meetings.

...
6) I presume we would all use the boost.libraryname and boost.libraryname.sub_feature naming convention with boost.libraryname pulling in the whole library?

A perfect situation for bikeshedding...

Green please, they're always green ;)

...

...
7) Does anyone have a handle on the current state of std library modularization?

It is voted into the C++23 draft. Microsoft has an experimental implementation for ages, Stephan (STL) has this currently on his plate and is actively working on it. I expect this to see this Real Soon Now in msvc.

Good!

...

...
8) Is there demand for this?

I've tried to get a handle on a modularized Boost in the past year but its current implementation seems to be quite hostile towards modules.

...
And no doubt lots of other things I haven't thought of.

There are a few pitfalls and things to consider if you want to use existing code in the modules world. I point out some of them in my talks last year (NDC, CppCon, Meeting C++). The last one from Meeting C++ 2021, "A (Short) Tour of C++20 Modules" is the most comprehensive and polished one, slides can be found on the Meeting C++ website. I gave a company training on topic that last week because the developers there are considering switching over to modules and were asking about the way forward, considerations to be taken into account, the state of the ecosystem, and all of my experiences in our daily use of modules in our company codebase.

Many thanks, John.

John Maddock

5:59 p.m.

On 05/04/2022 14:09, Daniela Engert via Boost wrote:

...

Hi John,

thanks for bringing up the topic of C++20 modules in the realm of Boost!

Am 05.04.2022 um 13:58 schrieb John Maddock via Boost:

...
The title says it all really, but in more detail:

1) Is there any existing best practice we should follow yet?

WG21 SG15 is about to come up with a document on that. Until then, I've touched this in my Meeting 2019 talk on modules, "Modules - the beginner's guide"

Nice talk, the performance comparisons were pretty impressive, but no comparison to precompiled headers? Thanks, John.

Daniela Engert

6:04 p.m.

Am 05.04.2022 um 19:59 schrieb John Maddock via Boost:

...

On 05/04/2022 14:09, Daniela Engert via Boost wrote:

...
Hi John,

thanks for bringing up the topic of C++20 modules in the realm of Boost!

Am 05.04.2022 um 13:58 schrieb John Maddock via Boost:

...
The title says it all really, but in more detail:

1) Is there any existing best practice we should follow yet?

WG21 SG15 is about to come up with a document on that. Until then, I've touched this in my Meeting 2019 talk on modules, "Modules - the beginner's guide"

Nice talk, the performance comparisons were pretty impressive, but no comparison to precompiled headers?

Thanks, but no - I'm not a fan of PCHs. They don't compose. Go with header units instead if you feel like, they are PCHs on steroids. PCHs are just huuuge and don't provide the same benefits as modules do like I tried to explain in my other reply on this thread. Modules move more compiler work to the one-time compilation of the PMIU than PCHs ever can. Ciao Dani -- PGP/GPG: 2CCB 3ECB 0954 5CD3 B0DB 6AA0 BA03 56A1 2C4638C5

Boris Kolpackov

1:51 p.m.

John Maddock via Boost writes:

...

1) Is there any existing best practice we should follow yet?

AFAIC, there is no practical, real-world experience using C++20 modules in a substantial codebase. In fact, out of the big three compilers, only MSVC has usable C++20 modules implementation (or claims to have; last time I tried it, which was admittedly a while ago, it wasn't).

...

2) Can we share code content effectively between headers and modules? And by modules I mean "proper" ones: we should be able to write "import boost.whatever;" rather than importing a header.

As an experiment, I've tried to do this using the preprocessor in a smallish library and it degenerated into an unmaintainable mess faster than I expected. IMO, the two viable options are: 1. Drop headers and focus on modules only. 2. Find an automated way (likely by the build system) to translate (sensibly written) module interfaces to headers. I have some ideas on this in the context of build2. Note that I don't consider the "just re-export the header unit" approach "modularization" since you don't really use any of modules mechanisms.

...

8) Is there demand for this?

I think there will be if/when C++20 modules are well supported by the compilers. However, modularizing Boost will be a big (and largely thankless) undertaking. Perhaps the more realistic first step would be to make sure public Boost headers are importable.

Peter Dimov

5:34 p.m.

My ambitions are modest. All I want is to be able to do this: #if BOOST_MODULES_SUPPORTED import <cstddef>; import ; import <exception>; import ; import <utility>; import <functional>; // std::hash import <cstdint>; import <iosfwd>; #else #include <cstddef> #include #include <exception> #include #include <utility> #include <functional> // std::hash #include <cstdint> #include <iosfwd> #endif

Niall Douglas

5:47 p.m.

On 05/04/2022 18:34, Peter Dimov via Boost wrote:

...

My ambitions are modest. All I want is to be able to do this:

#if BOOST_MODULES_SUPPORTED

import <cstddef>; import ; import <exception>; import ; import <utility>; import <functional>; // std::hash import <cstdint>; import <iosfwd>;

#else

#include <cstddef> #include #include <exception> #include #include <utility> #include <functional> // std::hash #include <cstdint> #include <iosfwd>

#endif

The compiler will automatically as-if do the above for you if Modules are enabled for standard library headers. It's clever enough to interpret an include as an import. So no need. Niall

Daniela Engert

5:49 p.m.

Am 05.04.2022 um 19:47 schrieb Niall Douglas via Boost:

...

On 05/04/2022 18:34, Peter Dimov via Boost wrote:

...
My ambitions are modest. All I want is to be able to do this:

#if BOOST_MODULES_SUPPORTED

import <cstddef>; import ; import <exception>; import ; import <utility>; import <functional>; // std::hash import <cstdint>; import <iosfwd>;

#else

#include <cstddef> #include #include <exception> #include #include <utility> #include <functional> // std::hash #include <cstdint> #include <iosfwd>

#endif The compiler will automatically as-if do the above for you if Modules are enabled for standard library headers. It's clever enough to interpret an include as an import. So no need.

Right, but this part of the standard is optional. MSVC has it but not the others, right? Ciao Dani -- PGP/GPG: 2CCB 3ECB 0954 5CD3 B0DB 6AA0 BA03 56A1 2C4638C5

Peter Dimov

6:31 p.m.

Daniela Engert wrote:

...

Right, but this part of the standard is optional. MSVC has it but not the others, right?

I can find /translateInclude in the documentation but according to the description it doesn't automatically translate stdlib includes.

William Linkmeyer

6:42 p.m.

...

...
Neil Douglas wrote: It's clever enough to interpret an include as an import. So no need.

I know this is the case with clang, but what about gcc and msvc?

...

...
Peter Dimov wrote: My ambitions are modest. All I want is to be able to do this: #if BOOST_MODULES_SUPPORTED

import <cstddef>; import ; import <exception>; import ; import <utility>; import <functional>; // std::hash import <cstdint>; import <iosfwd>;

#else

#include <cstddef> #include #include <exception> #include #include <utility> #include <functional> // std::hash #include <cstdint> #include <iosfwd>

#endif

You may also want to declare the above as a module itself so you can import it elsewhere :)

...

...
John Maddock wrote:

...
Is there any existing best practice we should follow yet?

Unfortunately, not yet. And I’m not sure that there will be — at least not until some package management support makes its way into the standard. WL

...

On Apr 5, 2022, at 2:31 PM, Peter Dimov via Boost wrote:

Daniela Engert wrote:

...
Right, but this part of the standard is optional. MSVC has it but not the others, right?

I can find /translateInclude in the documentation but according to the description it doesn't automatically translate stdlib includes.

_______________________________________________ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost

Daniela Engert

6:45 p.m.

Am 05.04.2022 um 20:31 schrieb Peter Dimov via Boost:

...

Daniela Engert wrote:

...
Right, but this part of the standard is optional. MSVC has it but not the others, right? I can find /translateInclude in the documentation but according to the description it doesn't automatically translate stdlib includes.

...

From their Github I have gleaned the info that there is a file in there

I've been using this for my own take on a modularized standard library and Gaby is not a fan of header units at all. So this probably doesn't work right now. With Stephan finishing his work on the modularized standard library I think this will definitely work right out of the box. that tells the compiler which includes can be implicitly translated into imports. For everything else, users can take advantage of include-translation of their own modularized headers using this compiler commandline option. I've been doing that this some Boost stuff, and others, too. Ciao Dani -- PGP/GPG: 2CCB 3ECB 0954 5CD3 B0DB 6AA0 BA03 56A1 2C4638C5

Stephan T. Lavavej

10:40 p.m.

[John Maddock]

...

7) Does anyone have a handle on the current state of std library modularization?

[Daniela Engert]

...

It is voted into the C++23 draft.

Slight correction: P2465R3 has been approved by LEWG, LWG, and CWG, and is ready for the next plenary, but it has not yet been voted into C++23.

...

Stephan (STL) has this currently on his plate and is actively working on it. I expect this to see this Real Soon Now in msvc.

Yep! :-) I am auditing every line of our headers (~147k lines) and am currently 66.3% done with that. The majority of the work is minimally invasive (adding an internal macro that expands to `export` or nothing for everything that should be part of the module*) and semi-mechanical. However, we're finding and fixing compiler bugs along the way, and we still have some questions to resolve around how to deal with our separately compiled components (we are especially constrained due to bincompat; I expect Boost to have an easier time as it can break bincompat at will), exactly how our .ixx files should ship and be built, etc. * As I understand it, there is a less invasive but more repetitive alternative: adding declarations of everything that needs to be exported. This would be fairly easy for things that mostly consist of classes or even class templates, but when there are lots of functions and function templates, it would involve a fair amount of duplication and having to keep things in sync. Regarding C++20 Standard Library Header Units, they should work (and we have active test coverage for this), although you may need build system work (as I understand it, MSBuild is ready, CMake/Ninja is not yet ready), and you may encounter compiler bugs as you use the library more intensively (as this is very new - our tests validate the basic scenario, but the total amount of usage is still small). Getting the STL to be compatible with header units didn't require auditing every line of the headers, but it did involve writing test coverage to exercise every header, and reporting many many bugs found along the way. Now that they're resolved, Boost should have an easier time, although I would still expect some compiler issues to be found (as Boost is both larger than MSVC's STL, and has different patterns of code). You may also need to refactor certain patterns that are incompatible with header units (notably any expectation that source files can define macros to affect the behavior of headers). There are two ways to build Standard Library Header Units - building each header independently is easier, but results in a lot of duplication. You can instead build them in a deduplicated manner, by asking the compiler to report (via JSON) how X includes Y, and then topologically sorting the graph. I have Python code in our test harness that does this (which could be used as a reference for a real build system - see https://github.com/microsoft/STL/blob/52505b9d3249c50b33eb22152a1f18f208ea29... ). The topo sort results in the smallest (i.e. most efficient) IFC files that you can get, short of the proposed `import std;` that I'm currently working on. The tracking issue (originally for Standard Library Header Units, now also for Standard Library Modules) is https://github.com/microsoft/STL/issues/1694 . It primarily records the compiler bugs we've found and fixed along the way.

...

From their Github I have gleaned the info that there is a file in there that tells the compiler which includes can be implicitly translated into imports.

Yes, this is header-units.json (see https://github.com/microsoft/STL/blob/52505b9d3249c50b33eb22152a1f18f208ea29... ). This was designed so that libraries like Boost can also opt-in by adding such files. Thanks, STL

Niall Douglas

7:27 p.m.

On 05/04/2022 18:49, Daniela Engert via Boost wrote:

...

Am 05.04.2022 um 19:47 schrieb Niall Douglas via Boost:

...
On 05/04/2022 18:34, Peter Dimov via Boost wrote:

...
My ambitions are modest. All I want is to be able to do this:

#if BOOST_MODULES_SUPPORTED

import <cstddef>; import ; import <exception>; import ; import <utility>; import <functional>; // std::hash import <cstdint>; import <iosfwd>;

#else

#include <cstddef> #include #include <exception> #include #include <utility> #include <functional> // std::hash #include <cstdint> #include <iosfwd>

#endif The compiler will automatically as-if do the above for you if Modules are enabled for standard library headers. It's clever enough to interpret an include as an import. So no need.

Right, but this part of the standard is optional. MSVC has it but not the others, right?

clang has it too. GCC will have it soon, from what I've heard, once their Modules implementation is more mature. That's all three major compilers. Me personally, I'd consider this not a thing to worry about. Re: auto conversion of non-std includes to module imports, Outcome does it by simply having the include do the import if it detects modules enabled. Then no end user source code needs to change. #include is always the right thing to document and to do. If Boost ever modularises, I'd suggest the exact same technique, and then nobody needs to upgrade their Boost including code. Niall

Peter Dimov

10:56 p.m.

Niall Douglas wrote:

...

...
...
The compiler will automatically as-if do the above for you if Modules are enabled for standard library headers. It's clever enough to interpret an include as an import. So no need.

Right, but this part of the standard is optional. MSVC has it but not the others, right?

clang has it too.

Clang doesn't even seem to support `import <iostream>;` yet.

Niall Douglas

11:15 p.m.

On 05/04/2022 23:56, Peter Dimov via Boost wrote:

...

Niall Douglas wrote:

...
...
...
The compiler will automatically as-if do the above for you if Modules are enabled for standard library headers. It's clever enough to interpret an include as an import. So no need.

Right, but this part of the standard is optional. MSVC has it but not the others, right?

clang has it too.

Clang doesn't even seem to support `import <iostream>;` yet.

It's been a while since I last tried it, but clang has a structure of module maps and they indicate what is what. If libc++ aren't building <iostream> as a module, I assume they have good reason for that. Any automatic reinterpretation of #include into import has nothing to do with how good - or bad - the actual module implementation is, or what is supported and is not supported. To be honest, I think there's a few more years in it before considering Modules as usable across all three compilers. I definitely think we ought to await cmake build support as a sign that they are usable. Coroutines is similarly a while away yet from production stable across all three compilers. I still see unpleasant surprises from time to time. Niall

Daniela Engert

6 Apr 6 Apr

7:35 a.m.

Am 06.04.2022 um 01:15 schrieb Niall Douglas via Boost:

...

On 05/04/2022 23:56, Peter Dimov via Boost wrote:

...
Niall Douglas wrote:

...
...
...
The compiler will automatically as-if do the above for you if Modules are enabled for standard library headers. It's clever enough to interpret an include as an import. So no need.

Right, but this part of the standard is optional. MSVC has it but not the others, right? clang has it too. Clang doesn't even seem to support `import <iostream>;` yet. It's been a while since I last tried it, but clang has a structure of module maps and they indicate what is what.

If libc++ aren't building <iostream> as a module, I assume they have good reason for that. Any automatic reinterpretation of #include into import has nothing to do with how good - or bad - the actual module implementation is, or what is supported and is not supported.

If clang doesn't treat *all the standard C++ library headers* as importable and actually supports compiling them as header units, then they're simply not standards compliant in this regard. Supporting automatic include translation for them is the optional cherry on top of compliance. Supporting other headers as importable and provide include translation for them is encouraging, though. Ciao Dani

Peter Dimov

5 Apr 5 Apr

6:29 p.m.

Niall Douglas wrote:

...

The compiler will automatically as-if do the above for you if Modules are enabled for standard library headers. It's clever enough to interpret an include as an import. So no need.

I don't think it will in practice, although the feature exists in theory.

Robert Ramey

6 Apr 6 Apr

2:41 p.m.

On 4/5/22 4:58 AM, John Maddock via Boost wrote:

...

What are we going to do about modules?

The most cursory examination of this idea leads me to suspect that this is the worst idea ever and will make the life anyone who gets near it a living hell. Seems like a huge idea to solve a non-problem. So I think we should hang back until someone demonstrates this thing has any value to anyone.

...

7) Does anyone have a handle on the current state of std library modularization?

And this is a huge issue for me. We've made huge efforts and progress toward making boost "more modular". BUT a) Boost is still distributed as a monolith. So as boost gets bigger, one has to distribute bigger and bigger packages even when using boost in a small component. b) Things depend on one's linking model, static, dynamic or mixed. And this gets mixed up with C++ with questions of ODR, multiple symbol definitions, sequence of static instance initializations etc. Ideally, I'd like to write a small app, include headers (or modules) which include components I want to use, and just compile and run. I don't see that we're anywhere near being able to do this. I'm coming to think that the goal as described here is not the right goal. The C++ standard actively avoids touching on these questions - maybe for good reason - but leaves us in limbo. I'm actually thinking that C++ has reached it's peak. It's solved the question of language syntax. Maybe the next language will start with a concept of deployment separate from syntax from the start. We'll see. Robert Ramey

965

Age (days ago)

966

Last active (days ago)

List overview

Download

27 comments

11 participants

participants (11)

Boris Kolpackov
Daniela Engert
Horváth V.
John Maddock
Niall Douglas
Peter Dimov
René Ferdinand Rivera Morell
Robert Ramey
Stephan T. Lavavej
Vinnie Falco
William Linkmeyer