[config] 'requres' doesn't take differing -std into account

newer
[test] Where we are with the new...

Peter Dimov

21 Jun 2015 21 Jun '15

2:19 p.m.

I have, in my user-config.jam, using gcc ; feature.subfeature toolset gcc : std : cxx0x cxx11 cxx1y cxx14 : optional composite propagated ; feature.compose toolset-gcc:stdcxx0x : <cxxflags>-std=c++0x ; feature.compose toolset-gcc:stdcxx11 : <cxxflags>-std=c++11 ; feature.compose toolset-gcc:stdcxx1y : <cxxflags>-std=c++1y ; feature.compose toolset-gcc:stdcxx14 : <cxxflags>-std=c++14 ; using clang : : : <cxxflags>-Wno-variadic-macros <cxxflags>-Wno-c99-extensions ; feature.subfeature toolset clang : std : cxx0x cxx11 cxx1y cxx14 : optional composite propagated ; feature.compose toolset-clang:stdcxx0x : <cxxflags>-std=c++0x ; feature.compose toolset-clang:stdcxx11 : <cxxflags>-std=c++11 ; feature.compose toolset-clang:stdcxx1y : <cxxflags>-std=c++1y ; feature.compose toolset-clang:stdcxx14 : <cxxflags>-std=c++14 ; This allows me to use toolset=clang (without -std) and toolset=clang-cxx11 (with -std=c++11). The 'requires' rule however caches the results of the configuration check when I run with toolset=clang, and then when I rerun with toolset=clang-cxx11, it uses the cached values, which is wrong. I use something like run test.cpp : : : [ requires cxx11_variadic_templates cxx11_template_aliases cxx11_hdr_type_traits cxx11_hdr_tuple ] ;

Show replies by date

John Maddock

21 Jun 21 Jun

4:20 p.m.

New subject: [config] 'requres' doesn't take differing -std into account

On 21/06/2015 15:19, Peter Dimov wrote:

...

I have, in my user-config.jam,

using gcc ;

feature.subfeature toolset gcc : std : cxx0x cxx11 cxx1y cxx14 : optional composite propagated ;

feature.compose toolset-gcc:stdcxx0x : <cxxflags>-std=c++0x ; feature.compose toolset-gcc:stdcxx11 : <cxxflags>-std=c++11 ; feature.compose toolset-gcc:stdcxx1y : <cxxflags>-std=c++1y ; feature.compose toolset-gcc:stdcxx14 : <cxxflags>-std=c++14 ;

using clang : : : <cxxflags>-Wno-variadic-macros <cxxflags>-Wno-c99-extensions ;

feature.subfeature toolset clang : std : cxx0x cxx11 cxx1y cxx14 : optional composite propagated ;

feature.compose toolset-clang:stdcxx0x : <cxxflags>-std=c++0x ; feature.compose toolset-clang:stdcxx11 : <cxxflags>-std=c++11 ; feature.compose toolset-clang:stdcxx1y : <cxxflags>-std=c++1y ; feature.compose toolset-clang:stdcxx14 : <cxxflags>-std=c++14 ;

This allows me to use toolset=clang (without -std) and toolset=clang-cxx11 (with -std=c++11).

The 'requires' rule however caches the results of the configuration check when I run with toolset=clang, and then when I rerun with toolset=clang-cxx11, it uses the cached values, which is wrong.

I use something like

run test.cpp : : : [ requires cxx11_variadic_templates cxx11_template_aliases cxx11_hdr_type_traits cxx11_hdr_tuple ] ;

I don't know how to address that - the requires rule just uses Boost.Build's check_target_builds rule, so the question is why that doesn't mangle itself on your sub-features? John.

Peter Dimov

4:35 p.m.

New subject: [config] 'requires' doesn't take differing -std into account

John Maddock wrote:

...

I don't know how to address that - the requires rule just uses Boost.Build's check_target_builds rule, so the question is why that doesn't mangle itself on your sub-features?

Beats me. Boost.Build is still a mystery for me after all these years. The subfeatures do appear in the directory name: **passed** ..\..\..\bin.v2\libs\mp11\test\mp_assign.test\clang-linux-cxx11-3.5.1\debug\mp_assign.test only the caching is affected.

Peter Dimov

4:49 p.m.

New subject: [config] 'requires' doesn't take differing -std intoaccount

...

John Maddock wrote:

...
I don't know how to address that - the requires rule just uses Boost.Build's check_target_builds rule, so the question is why that doesn't mangle itself on your sub-features?

Beats me.

It's probably this bit in configure.jam: rule builds ( metatarget-reference : properties * : what ? : retry ? ) { # FIXME: This should not be hardcoded. Other checks might want to consider a # different set of features as relevant. local toolset = [ property.select <toolset> : $(properties) ] ; local toolset-version-property = "" ; local relevant = [ property.select <target-os> <toolset> $(toolset-version-property) <address-model> <architecture> : $(properties) ] ; Looking at project-cache.jam, I see that the "symlinks supported" check encodes all the features. Perhaps check-target-builds should just do the same?

John Maddock

5:26 p.m.

New subject: [config] 'requires' doesn't take differing -std intoaccount

On 21/06/2015 17:49, Peter Dimov wrote:

...

...
John Maddock wrote:

...
I don't know how to address that - the requires rule just uses > Boost.Build's check_target_builds rule, so the question is why that > doesn't mangle itself on your sub-features?

Beats me.

It's probably this bit in configure.jam:

rule builds ( metatarget-reference : properties * : what ? : retry ? ) { # FIXME: This should not be hardcoded. Other checks might want to consider a # different set of features as relevant. local toolset = [ property.select <toolset> : $(properties) ] ; local toolset-version-property = "" ; local relevant = [ property.select <target-os> <toolset> $(toolset-version-property) <address-model> <architecture> : $(properties) ] ;

Looking at project-cache.jam, I see that the "symlinks supported" check encodes all the features. Perhaps check-target-builds should just do the same?

+1, but we need a Boost.Build expert to check this, John.

Niall Douglas

22 Jun 22 Jun

1:09 a.m.

New subject: [config] 'requres' doesn't take differing -std into account

On 21 Jun 2015 at 17:19, Peter Dimov wrote:

...

run test.cpp : : : [ requires cxx11_variadic_templates cxx11_template_aliases cxx11_hdr_type_traits cxx11_hdr_tuple ] ;

Firstly, many thanks for this tip. I had no idea that Boost.Build could be told that a library requires certain C++ features. AFIO now uses that facility, and that let me clean out some cruft. Thank you. I do wish to reiterate my desire for a std=98|03|11|14|1z parameter for ./b2. I think users get confused - as you can see on stackoverflow - on how to build Boost for C++ 11 or 14 or 1z. A simple ./b2 toolset=clang std=14 ... I think is clear and obvious and memorable. Niall -- ned Productions Limited Consulting http://www.nedproductions.biz/ http://ie.linkedin.com/in/nialldouglas/

Robert Ramey

2:46 p.m.

New subject: [config] 'requres' doesn't take differing -std into account

On 6/21/15 6:09 PM, Niall Douglas wrote:

...

I do wish to reiterate my desire for a std=98|03|11|14|1z parameter for ./b2. I think users get confused - as you can see on stackoverflow - on how to build Boost for C++ 11 or 14 or 1z. A simple

./b2 toolset=clang std=14

... I think is clear and obvious and memorable.

Hmmm currently I use b2 toolset=clang cxxflags=-std=c++03 is that so bad?

...

Niall

_______________________________________________ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost

John Maddock

3:56 p.m.

New subject: [config] 'requres' doesn't take differing -std into account

...

Hmmm currently I use b2 toolset=clang cxxflags=-std=c++03 is that so bad?

It depends - on a clean run it's fine, but object files built with some other -std= option won't get rebuilt, and likewise cached config options may be out of date. John. PS in my user-config.jam I have: using gcc : 4.9.2 : "C:/Progra~1/mingw-w64/x86_64-4.9.2-win32-seh-rt_v4-rev2/mingw64/bin/g++.exe" ; using gcc : 4.9.2s11 : "C:/Progra~1/mingw-w64/x86_64-4.9.2-win32-seh-rt_v4-rev2/mingw64/bin/g++.exe" : <cxxflags>-std=gnu++11 ; using gcc : 4.9.2s14 : "C:/Progra~1/mingw-w64/x86_64-4.9.2-win32-seh-rt_v4-rev2/mingw64/bin/g++.exe" : <cxxflags>-std=gnu++14 ; which works for me.

Niall Douglas

11:50 p.m.

New subject: [config] 'requres' doesn't take differing -std into account

On 22 Jun 2015 at 7:46, Robert Ramey wrote:

...

...
I do wish to reiterate my desire for a std=98|03|11|14|1z parameter for ./b2. I think users get confused - as you can see on stackoverflow - on how to build Boost for C++ 11 or 14 or 1z. A simple

./b2 toolset=clang std=14

... I think is clear and obvious and memorable.

Hmmm currently I use b2 toolset=clang cxxflags=-std=c++03 is that so bad?

1. That command wouldn't work on MSVC. In documentation you have to keep saying "If using MSVC use the b2 command, else use this other b2 command". This is confusing for users. 2. You need std=XX put into the path generated inside bin.v2 to disambiguate binaries built using different std. Using cxxflags lets differing even incompatible binaries mix. This can be confusing. Even I have seen weird link errors at times, and realised only after some head scratching it's because I'm using a non-C++ 11 Boost and I accidentally typed the MSVC b2 command form on POSIX without thinking. What I would really prefer is a failure to find a library at all when I use mismatched std=XX config. Niall -- ned Productions Limited Consulting http://www.nedproductions.biz/ http://ie.linkedin.com/in/nialldouglas/

Edward Diener

1:11 a.m.

New subject: [config] 'requres' doesn't take differing -std into account

On 6/21/2015 10:19 AM, Peter Dimov wrote:

...

I have, in my user-config.jam,

using gcc ;

feature.subfeature toolset gcc : std : cxx0x cxx11 cxx1y cxx14 : optional composite propagated ;

feature.compose toolset-gcc:stdcxx0x : <cxxflags>-std=c++0x ; feature.compose toolset-gcc:stdcxx11 : <cxxflags>-std=c++11 ; feature.compose toolset-gcc:stdcxx1y : <cxxflags>-std=c++1y ; feature.compose toolset-gcc:stdcxx14 : <cxxflags>-std=c++14 ;

using clang : : : <cxxflags>-Wno-variadic-macros <cxxflags>-Wno-c99-extensions ;

feature.subfeature toolset clang : std : cxx0x cxx11 cxx1y cxx14 : optional composite propagated ;

feature.compose toolset-clang:stdcxx0x : <cxxflags>-std=c++0x ; feature.compose toolset-clang:stdcxx11 : <cxxflags>-std=c++11 ; feature.compose toolset-clang:stdcxx1y : <cxxflags>-std=c++1y ; feature.compose toolset-clang:stdcxx14 : <cxxflags>-std=c++14 ;

This allows me to use toolset=clang (without -std) and toolset=clang-cxx11 (with -std=c++11).

The 'requires' rule however caches the results of the configuration check when I run with toolset=clang, and then when I rerun with toolset=clang-cxx11, it uses the cached values, which is wrong.

I believe the same problem is causing predef-check problems in the predef library because I have seen local errors using predef-check which also says that a previous calculation on another toolset has been cached and is being reused, even when the toolset changes. Somehow the incorrect cached value is being retained by Boost Build even when the toolset value changes and a new calculation needs to be made. But like many other people I don't know nearly enough about Boost Build to figure out what it is doing.

...

I use something like

run test.cpp : : : [ requires cxx11_variadic_templates cxx11_template_aliases cxx11_hdr_type_traits cxx11_hdr_tuple ] ;

Vladimir Prus

11:25 a.m.

New subject: [config] 'requres' doesn't take differing -std into account

On 6/21/2015 5:19 PM, Peter Dimov wrote:

...

I have, in my user-config.jam,

using gcc ;

feature.subfeature toolset gcc : std : cxx0x cxx11 cxx1y cxx14 : optional composite propagated ;

feature.compose toolset-gcc:stdcxx0x : <cxxflags>-std=c++0x ; feature.compose toolset-gcc:stdcxx11 : <cxxflags>-std=c++11 ; feature.compose toolset-gcc:stdcxx1y : <cxxflags>-std=c++1y ; feature.compose toolset-gcc:stdcxx14 : <cxxflags>-std=c++14 ;

using clang : : : <cxxflags>-Wno-variadic-macros <cxxflags>-Wno-c99-extensions ;

feature.subfeature toolset clang : std : cxx0x cxx11 cxx1y cxx14 : optional composite propagated ;

feature.compose toolset-clang:stdcxx0x : <cxxflags>-std=c++0x ; feature.compose toolset-clang:stdcxx11 : <cxxflags>-std=c++11 ; feature.compose toolset-clang:stdcxx1y : <cxxflags>-std=c++1y ; feature.compose toolset-clang:stdcxx14 : <cxxflags>-std=c++14 ;

This allows me to use toolset=clang (without -std) and toolset=clang-cxx11 (with -std=c++11).

The 'requires' rule however caches the results of the configuration check when I run with toolset=clang, and then when I rerun with toolset=clang-cxx11, it uses the cached values, which is wrong.

The problem is that running configuration checks for all unique set of properties is quite wasteful. Say, the 'link' property usually has no effect on architecture and address-model, and if we re-run architecture/address-model checks for both static and shared variants, the result will be correct, but we'll do a lot of unnecessary checks and produce lots of unnecessary output. It does not seem possible to avoid having some 'key' properties used for configuration checks? - Volodya

Peter Dimov

11:41 a.m.

New subject: [config] 'requres' doesn't take differing -std intoaccount

Vladimir Prus wrote:

...

The problem is that running configuration checks for all unique set of properties is quite wasteful.

Maybe. It works though. :-)

...

Say, the 'link' property usually has no effect on architecture and address-model,

How could you know? The only safe choice is to encode everything. But if we're to continue to guess, I'd say that, at minimum, cxxflags can certainly affect the architecture and address-model and everything else, and are currently not encoded.

Vladimir Prus

12:08 p.m.

New subject: [config] 'requres' doesn't take differing -std intoaccount

On 6/22/2015 2:41 PM, Peter Dimov wrote:

...

Vladimir Prus wrote:

...
The problem is that running configuration checks for all unique set of properties is quite wasteful.

Maybe. It works though. :-)

...
Say, the 'link' property usually has no effect on architecture and address-model,

How could you know?

By examining all toolsets that support the 'link' property?

...

The only safe choice is to encode everything. But if we're to continue to guess, I'd say that, at minimum, cxxflags can certainly affect the architecture and address-model and everything else, and are currently not encoded.

Yes, so if you do: b2 toolset=gcc and then b2 toolset=gcc cxxflags="-m32" then you'd get wrong cached results of configuration check for address-model. It is possible to solve, in general, but it requires either creating directory paths using a hash of property set (which is ugly), or storing a hash of property set is a separate data store. I believe SCons is the only one that solves this (using the second approach). Maybe it's because it's reasonable to assume that if people adding options such as "-m32" to cxxflags know what they are doing? - Volodya

Peter Dimov

12:27 p.m.

New subject: [config] 'requres' doesn't take differing -stdintoaccount

Vladimir Prus wrote:

...

On 6/22/2015 2:41 PM, Peter Dimov wrote:

...
Vladimir Prus wrote:

...
Say, the 'link' property usually has no effect on architecture and address-model,

How could you know?

By examining all toolsets that support the 'link' property?

My point was that, in general, you can't know whether a property affects the address model. Maybe you can know that about "link" and other known properties, but that's not what the code does today - it doesn't eliminate known properties, it eliminates all unknown properties.

...

It is possible to solve, in general, but it requires either creating directory paths using a hash of property set (which is ugly), or storing a hash of property set is a separate data store.

When I look at project-cache.jam, I see this: set "hardlinks supported-<abi>ms-<address-model>32-<architecture>x86-<asynch-exceptions>off-<binary-format>pe-<cxxflags>-std=c++14-<debug-store>object-<debug-symbols>on-<deduced-address-model>32-<deduced-architecture>x86-no-xsltproc-<embed-manifest>on-<exception-handling>on-<extern-c-nothrow>off-<format>html-<hardcode-dll-paths>true-<host-os>windows-<inlining>off-<install-dependencies>off-<link>shared-<location>../../..-<log-api>generic-<midl-robust>yes-<midl-stubless-proxy>yes-<optimization>off-<os>NT-<pch>on-<preserve-test-targets>on-<profiling>off-<python-debugging>off-<python>3.3-<rtti>on-<runtime-debugging>on-<runtime-link>shared-<stdlib>native-<strip>off-<suppress-import-lib>false-<symlink-location>project-relative-<target-os>windows-on-<threadapi>win32-<threading>single-toolset-gcc:stdcxx14-toolset-gcc:version4.9.2-<toolset>gcc-<user-interface>console-<variant>debug-<warnings-as-errors>off-<warnings>on-<windows-api>deskto p" : "true" ; That is, the hardlink detection does encode cxxflags. But set "Boost.Config Feature Check: cxx11_hdr_tuple-<target-os>windows-toolset-clang:version3.5.1-<toolset>clang" : "false" ; the detection for cxx11_hdr_tuple does not.

...

Maybe it's because it's reasonable to assume that if people adding options such as "-m32" to cxxflags know what they are doing?

You could say that. But what then would be your recommendation? How can I have a working setup, in which I can issue b2 toolset=gcc,gcc-cxx11,gcc-cxx14,clang,clang-cxx11,clang-cxx14,msvc-8.0,msvc-12.0 and have it properly detect that gcc by default doesn't support C++11, but does in -std=c++11 mode?

Vladimir Prus

6:22 p.m.

New subject: [config] 'requres' doesn't take differing -stdintoaccount

On 6/22/2015 3:27 PM, Peter Dimov wrote:

...

Vladimir Prus wrote:

...
On 6/22/2015 2:41 PM, Peter Dimov wrote:

...
Vladimir Prus wrote:

...
Say, the 'link' property usually has no effect on architecture and >> address-model,

How could you know?

By examining all toolsets that support the 'link' property?

My point was that, in general, you can't know whether a property affects the address model. Maybe you can know that about "link" and other known properties, but that's not what the code does today - it doesn't eliminate known properties, it eliminates all unknown properties.

...
It is possible to solve, in general, but it requires either creating directory paths using a hash of property set (which is ugly), or storing a hash of property set is a separate data store.

When I look at project-cache.jam, I see this:

set "hardlinks supported-<abi>ms-<address-model>32-<architecture>x86-<asynch-exceptions>off-<binary-format>pe-<cxxflags>-std=c++14-<debug-store>object-<debug-symbols>on-<deduced-address-model>32-<deduced-architecture>x86-no-xsltproc-<embed-manifest>on-<exception-handling>on-<extern-c-nothrow>off-<format>html-<hardcode-dll-paths>true-<host-os>windows-<inlining>off-<install-dependencies>off-<link>shared-<location>../../..-<log-api>generic-<midl-robust>yes-<midl-stubless-proxy>yes-<optimization>off-<os>NT-<pch>on-<preserve-test-targets>on-<profiling>off-<python-debugging>off-<python>3.3-<rtti>on-<runtime-debugging>on-<runtime-link>shared-<stdlib>native-<strip>off-<suppress-import-lib>false-<symlink-location>project-relative-<target-os>windows-on-
readapi>win32-<threading>single-toolset-gcc:stdcxx14-toolset-gcc:version4.9.2-<toolset>gcc-<user-interface>console-<variant>debug-<warnings-as-errors>off-<warnings>on-<windows-api>deskto

p" : "true" ;

That is, the hardlink detection does encode cxxflags.

Even though it's independent of toolset version and C++ standard... in fact, looking at link.jam:can-hardlink, it might store all properties in config cache, but it will actually run the test only once, with whatever properties were first used - see the use of .can-hardlink variable.

...

But

set "Boost.Config Feature Check: cxx11_hdr_tuple-<target-os>windows-toolset-clang:version3.5.1-<toolset>clang" : "false" ;

the detection for cxx11_hdr_tuple does not.

This one has to actually compiler a source file with all relevant options, and put it somewhere, and if "relevant options" were to include cxxflags, we'd have to figure how to include cxxflags in a path or store full used path - the same problem I've mentioned in my previous email.

...

...
Maybe it's because it's reasonable to assume that if people adding options such as "-m32" to cxxflags know what they are doing?

You could say that.

But what then would be your recommendation? How can I have a working setup, in which I can issue

b2 toolset=gcc,gcc-cxx11,gcc-cxx14,clang,clang-cxx11,clang-cxx14,msvc-8.0,msvc-12.0

and have it properly detect that gcc by default doesn't support C++11, but does in -std=c++11 mode?

I think it's impossible to do right now, and the easiest way to support it is to make configure.build function take an additional list of properties that are relevant for this particular test, so that you can pass toolset-gcc:std in your configuration check. My naive attempt to remove filtering results in about dozen configure check results for "icu<64>" reported. alone reported, and similarly for other checks, making configure check reports quite useless. It's true that doing checks for all property sets unless a feature is positively known to have no effect is more theoretically pure, but then all other build systems will only do a single check once. - Volodya

Peter Dimov

8:24 p.m.

New subject: [config] 'requres' doesn't take differing-stdintoaccount

Vladimir Prus wrote:

...

...
But what then would be your recommendation? How can I have a working setup, in which I can issue

b2 toolset=gcc,gcc-cxx11,gcc-cxx14,clang,clang-cxx11,clang-cxx14,msvc-8.0,msvc-12.0

and have it properly detect that gcc by default doesn't support C++11, but does in -std=c++11 mode?

I think it's impossible to do right now, and the easiest way to support it is to make configure.build function take an additional list of properties that are relevant for this particular test, so that you can pass toolset-gcc:std in your configuration check.

Interestingly though, -cxx11 and -cxx14 do end up encoded in the directory names (or running tests with the above command wouldn't work.) So it seems that the directory logic and the caching logic are separate and the former seems more useful, at least in my specific case.

John Maddock

3:59 p.m.

New subject: [config] 'requres' doesn't take differing -std intoaccount

...

...
The only safe choice is to encode everything. But if we're to continue to guess, I'd say that, at minimum, cxxflags can certainly affect the architecture and address-model and everything else, and are currently not encoded.

Yes, so if you do:

b2 toolset=gcc

and then

b2 toolset=gcc cxxflags="-m32"

then you'd get wrong cached results of configuration check for address-model.

It is possible to solve, in general, but it requires either creating directory paths using a hash of property set (which is ugly), or storing a hash of property set is a separate data store. I believe SCons is the only one that solves this (using the second approach). Maybe it's because it's reasonable to assume that if people adding options such as "-m32" to cxxflags know what they are doing?

I think you're giving folks altogether too much credit ;) While it may not be possible to fix absolutely everything, if you know enough to place object files in different directories, then you know enough to configure them differently too. IMO Peter is correct - you should assume that everything makes a difference unless you know otherwise - not the other way around. John.

3446

Age (days ago)

3447

Last active (days ago)

List overview

Download

16 comments

6 participants

participants (6)

Edward Diener
John Maddock
Niall Douglas
Peter Dimov
Robert Ramey
Vladimir Prus