[build] [config] check_target_builds and feature.subfeature composition
The Boost library config uses the check_target_builds rule to provide build time feature testing. Other Boost libraries use config checks in their own build and test jamfiles. I have found that this works reliably when a particular toolset is used in a normal manner. However when using the feature.subfeature and feature.compose rules to extend the toolset-version designation with a subfeature, such that the toolset designation now becomes toolset-version-subfeature, the use of the check_target_builds rule does not work as expected. In particular the individual check_target_builds rule results are put into a subdirectory based on the toolset-version and not based on the toolset-version-subfeature when toolset subfeature composition is used. I also suspect that the cached value reflects setup. This causes wrong results in tests where the toolset designation uses subfeatures, and a library's jamfile is using config and/or predef checks. As an example My toolset designation looks like: using gcc : 6.3 : some_path ; feature.subfeature toolset gcc : cxxstd : c03 c11 c14 c17 : optional composite propagated ; feature.compose toolset-gcc:cxxstdc03 : <cxxflags>-std=c++03 ; feature.compose toolset-gcc:cxxstdc11 : <cxxflags>-std=c++11 ; feature.compose toolset-gcc:cxxstdc14 : <cxxflags>-std=c++14 ; feature.compose toolset-gcc:cxxstdc17 : <cxxflags>-std=c++1z ; A library's test looks like: import ../../config/checks/config : requires ; run some_test.cpp : : : [ requires cxx11_hdr_forward_list ] ; If I run some some_test with toolset=gcc-6.3-c03, the config result for checking if forward list is supported is put into a subdirectory reflecting the toolset-version while the test result will be put into a subdirectory reflecting the toolset-version-subfeature. Subsequently if I run some_test with toolset=gcc-6.3-c11 the config result is taken from the subdirectory reflect the toolset-version even though whether the forward list is supported or not depends very much on the subfeature I am designating as part of the toolset. This naturally causes incorrect results for the test. Is this a known problem ? Is it a problem with Boost Build or is it a problem with how config uses the check_target_builds rule ? I have run into this problem extensively in my testing once I started to use the feature.subfeature ability of Boost Build.
On 1/22/2017 12:58 PM, Edward Diener wrote:
The Boost library config uses the check_target_builds rule to provide build time feature testing. Other Boost libraries use config checks in their own build and test jamfiles.
I have found that this works reliably when a particular toolset is used in a normal manner. However when using the feature.subfeature and feature.compose rules to extend the toolset-version designation with a subfeature, such that the toolset designation now becomes toolset-version-subfeature, the use of the check_target_builds rule does not work as expected. In particular the individual check_target_builds rule results are put into a subdirectory based on the toolset-version and not based on the toolset-version-subfeature when toolset subfeature composition is used. I also suspect that the cached value reflects setup. This causes wrong results in tests where the toolset designation uses subfeatures, and a library's jamfile is using config and/or predef checks.
As an example
My toolset designation looks like:
using gcc : 6.3 : some_path ; feature.subfeature toolset gcc : cxxstd : c03 c11 c14 c17 : optional composite propagated ; feature.compose toolset-gcc:cxxstdc03 : <cxxflags>-std=c++03 ; feature.compose toolset-gcc:cxxstdc11 : <cxxflags>-std=c++11 ; feature.compose toolset-gcc:cxxstdc14 : <cxxflags>-std=c++14 ; feature.compose toolset-gcc:cxxstdc17 : <cxxflags>-std=c++1z ;
A library's test looks like:
import ../../config/checks/config : requires ; run some_test.cpp : : : [ requires cxx11_hdr_forward_list ] ;
If I run some some_test with toolset=gcc-6.3-c03, the config result for checking if forward list is supported is put into a subdirectory reflecting the toolset-version while the test result will be put into a subdirectory reflecting the toolset-version-subfeature. Subsequently if I run some_test with toolset=gcc-6.3-c11 the config result is taken from the subdirectory reflect the toolset-version even though whether the forward list is supported or not depends very much on the subfeature I am designating as part of the toolset. This naturally causes incorrect results for the test.
Is this a known problem ? Is it a problem with Boost Build or is it a problem with how config uses the check_target_builds rule ? I have run into this problem extensively in my testing once I started to use the feature.subfeature ability of Boost Build.
Ping ! Has no one else encountered the same problems I have encountered using the feature.subfeature facility in Boost Build ? I recall Peter Dimov saying he used that Boost Build facility, which is why I looked into it, and I am guessing others use it. To me using this facility is just broken because not only does config use the check_target_builds but predef uses it also ( my non-Boost cxxd library also uses it ). So any Boost jamfile using config or predef to determine what should be built is subject to the same problem I encountered when using the feature.subfeature facility. I am a bit surprised that this is a non-issue with everyone else on this mailing list.
Edward Diener wrote:
Has no one else encountered the same problems I have encountered using the feature.subfeature facility in Boost Build?
Yes I have, I brought it up, nothing happened. I stopped using check_target_builds.
AMDG On 01/23/2017 05:17 PM, Peter Dimov wrote:
Edward Diener wrote:
Has no one else encountered the same problems I have encountered using the feature.subfeature facility in Boost Build?
Yes I have, I brought it up, nothing happened. I stopped using check_target_builds.
The problem is that check_target_builds tries to guess which features are important, and does it badly (via a hard-coded list). In Christ, Steven Watanabe
Yes I have, I brought it up, nothing happened. I stopped using check_target_builds.
The problem is that check_target_builds tries to guess which features are important, and does it badly (via a hard-coded list).
IMO this is just plain wrong - any command line change could change the behaviour of the compile - I would rather loose the caching (which itself is problematic at times) that have it yield the wrong answer. John.
On 24.01.2017 03:30, John Maddock wrote:
Yes I have, I brought it up, nothing happened. I stopped using check_target_builds.
The problem is that check_target_builds tries to guess which features are important, and does it badly (via a hard-coded list).
IMO this is just plain wrong - any command line change could change the behaviour of the compile - I would rather loose the caching (which itself is problematic at times) that have it yield the wrong answer.
Allow me to tie this back to an earlier conversation: It seems this issue arises because b2 allows multiple build variants to be built at once, so (at least theoretically) each would need to run its own set of config checks. I understand that there are features that may take on different values within a single build, but to open that door widely and let multiple feature values be set at build time just makes the whole process so much more complex. Wouldn't there be a way to simplify this to get the complexity in check ? Alternatively, config checks need to become (intermediate) build targets by themselves, and "caching" would need to be handled the same as for any other target (such as object files). Regards, Stefan -- ...ich hab' noch einen Koffer in Berlin...
On 1/24/2017 9:13 AM, Stefan Seefeld wrote:
On 24.01.2017 03:30, John Maddock wrote:
Yes I have, I brought it up, nothing happened. I stopped using check_target_builds.
The problem is that check_target_builds tries to guess which features are important, and does it badly (via a hard-coded list).
IMO this is just plain wrong - any command line change could change the behaviour of the compile - I would rather loose the caching (which itself is problematic at times) that have it yield the wrong answer.
Allow me to tie this back to an earlier conversation:
It seems this issue arises because b2 allows multiple build variants to be built at once, so (at least theoretically) each would need to run its own set of config checks.
You don't need multiple build variants to generate the problem. Just build with a toolset for a compiler(-version) with a subfeature that sets -std=c++03, and then subsequently build with a compiler(-version) with a subfeature that sets -std=c++11. The check_target_builds uses the same cached value for both because it only distinguishes between compiler(-version) and not sub-feature.
I understand that there are features that may take on different values within a single build, but to open that door widely and let multiple feature values be set at build time just makes the whole process so much more complex. Wouldn't there be a way to simplify this to get the complexity in check ? Alternatively, config checks need to become (intermediate) build targets by themselves, and "caching" would need to be handled the same as for any other target (such as object files).
On 1/24/2017 3:30 AM, John Maddock wrote:
Yes I have, I brought it up, nothing happened. I stopped using check_target_builds.
The problem is that check_target_builds tries to guess which features are important, and does it badly (via a hard-coded list).
IMO this is just plain wrong - any command line change could change the behaviour of the compile - I would rather loose the caching (which itself is problematic at times) that have it yield the wrong answer.
I am glad somebody else besides me sees what there is a real problem. Thanks John.
On 1/23/2017 8:41 PM, Steven Watanabe wrote:
AMDG
On 01/23/2017 05:17 PM, Peter Dimov wrote:
Edward Diener wrote:
Has no one else encountered the same problems I have encountered using the feature.subfeature facility in Boost Build?
Yes I have, I brought it up, nothing happened. I stopped using check_target_builds.
The problem is that check_target_builds tries to guess which features are important, and does it badly (via a hard-coded list).
I would have thought that it was the feature.subfeature not distinguishing correctly between different builds, based on subfeaturees, that was the culprit. It is hardly possible for me not to use check_target_builds, since it is an integral part of predefs and also an integral part of the build time configuration of config; and I find both of these eminently useful. So my solution is to not use feature.subfeature and instead encode so-called subfeatures in the version field of a toolset. Since builds are automatically distinguished by the toolset-version encoding this always works properly for me.
In Christ, Steven Watanabe
On 1/23/2017 7:17 PM, Peter Dimov wrote:
Edward Diener wrote:
Has no one else encountered the same problems I have encountered using the feature.subfeature facility in Boost Build?
Yes I have, I brought it up, nothing happened. I stopped using check_target_builds.
But other libraries use config and predef's check_target_builds implementations so you will see errors trying to build those libraries using feature.subfeature.
participants (5)
-
Edward Diener -
John Maddock -
Peter Dimov -
Stefan Seefeld -
Steven Watanabe