PSA: Travis OS X bottleneck, <cxxstd> new Boost.Build feature
Travis's OS X resources for open source projects seem to be insufficient, so OS X jobs are very slow and seem to be falling further and further behind. Therefore, library authors are encouraged to keep the OS X jobs to the minimum necessary. Spawning many OS X jobs takes hours (~25 minutes waiting time per job at a quick estimate). On a not entirely unrelated note, Rene Rivera has added a new feature <cxxstd> to Boost.Build that controls the C++ standard in use. So for instance, instead of the old b2 libs/mylib/test toolset=gcc cxxflags=-std=c++11 one can now use b2 libs/mylib/test toolset=gcc cxxstd=11 In addition to being more convenient, this also allows several invocations to be combined into one: b2 libs/mylib/test toolset=clang cxxstd=03,11,14,1z which can be leveraged to cut down on the number of jobs. An example of using cxxstd in .travis.yml can be seen here: https://github.com/boostorg/system/blob/develop/.travis.yml
On Thu, Oct 26, 2017 at 12:43 PM, Peter Dimov via Boost < boost@lists.boost.org> wrote:
Travis's OS X resources for open source projects seem to be insufficient, so OS X jobs are very slow and seem to be falling further and further behind. Therefore, library authors are encouraged to keep the OS X jobs to the minimum necessary. Spawning many OS X jobs takes hours (~25 minutes waiting time per job at a quick estimate).
On a not entirely unrelated note, Rene Rivera has added a new feature <cxxstd> to Boost.Build that controls the C++ standard in use. So for instance, instead of the old
b2 libs/mylib/test toolset=gcc cxxflags=-std=c++11
one can now use
b2 libs/mylib/test toolset=gcc cxxstd=11
In addition to being more convenient, this also allows several invocations to be combined into one:
b2 libs/mylib/test toolset=clang cxxstd=03,11,14,1z
which can be leveraged to cut down on the number of jobs.
An example of using cxxstd in .travis.yml can be seen here:
In Travis if one updates gcc and clang as part of the update scripts, you end up with "clang++" and "clang++-5.0" for example, where the first one is the version that came with the distribution. Is there a bjam directive to change the compiler name, for example it would be nice if I could do: toolset=gcc,clang compiler={g++,g++-7},{clang++,clang++-5.0} Right now I muck with the user-config.jam file to force it one way or the other. Perhaps that's still the best way to do it? Thanks, Jim
James E. King, III wrote:
In Travis if one updates gcc and clang as part of the update scripts, you end up with "clang++" and "clang++-5.0" for example, where the first one is the version that came with the distribution.
Is there a bjam directive to change the compiler name, for example it would be nice if I could do:
toolset=gcc,clang compiler={g++,g++-7},{clang++,clang++-5.0}
I've been informed that toolset=gcc-7 already works and automagically chooses g++-7 as the compiler. Unfortunately, this trick doesn't work for clang (yet?), so one still need the user-config.
Le 26/10/2017 à 18:43, Peter Dimov via Boost a écrit :
Travis's OS X resources for open source projects seem to be insufficient, so OS X jobs are very slow and seem to be falling further and further behind. Therefore, library authors are encouraged to keep the OS X jobs to the minimum necessary. Spawning many OS X jobs takes hours (~25 minutes waiting time per job at a quick estimate).
Yes, I've notice it. Do you mean that the route cause are the new Boost builds? Or is it a bug in travis?
On a not entirely unrelated note, Rene Rivera has added a new feature <cxxstd> to Boost.Build that controls the C++ standard in use. So for instance, instead of the old
b2 libs/mylib/test toolset=gcc cxxflags=-std=c++11
one can now use
b2 libs/mylib/test toolset=gcc cxxstd=11
In addition to being more convenient, this also allows several invocations to be combined into one:
b2 libs/mylib/test toolset=clang cxxstd=03,11,14,1z
which can be leveraged to cut down on the number of jobs.
An example of using cxxstd in .travis.yml can be seen here:
https://github.com/boostorg/system/blob/develop/.travis.yml Thanks Peter for the information.
Best, vicente
It seems like this would be a great instance where the steering committee could allocate some of the organization's funds to directly improve the infrastructure being used by our team. I believe that we could upgrade our Travis account to a paid one for a couple hundred a month to get more build resources. They also claim on https://travis-ci.org/ that:
Testing your open source project is 10000% free Seriously. Always. We like to think of it as our way of giving back to a community that gives us so much as well.
Maybe we just need to have someone on the project contact them and get our free quota raised? Tom On Thu, Oct 26, 2017 at 11:43 AM, Peter Dimov via Boost < boost@lists.boost.org> wrote:
Travis's OS X resources for open source projects seem to be insufficient, so OS X jobs are very slow and seem to be falling further and further behind. Therefore, library authors are encouraged to keep the OS X jobs to the minimum necessary. Spawning many OS X jobs takes hours (~25 minutes waiting time per job at a quick estimate).
On a not entirely unrelated note, Rene Rivera has added a new feature <cxxstd> to Boost.Build that controls the C++ standard in use. So for instance, instead of the old
b2 libs/mylib/test toolset=gcc cxxflags=-std=c++11
one can now use
b2 libs/mylib/test toolset=gcc cxxstd=11
In addition to being more convenient, this also allows several invocations to be combined into one:
b2 libs/mylib/test toolset=clang cxxstd=03,11,14,1z
which can be leveraged to cut down on the number of jobs.
An example of using cxxstd in .travis.yml can be seen here:
https://github.com/boostorg/system/blob/develop/.travis.yml
_______________________________________________ Unsubscribe & other changes: http://lists.boost.org/mailman /listinfo.cgi/boost
Tom Kent wrote:
It seems like this would be a great instance where the steering committee could allocate some of the organization's funds to directly improve the infrastructure being used by our team. I believe that we could upgrade our Travis account to a paid one for a couple hundred a month to get more build resources.
One should keep in mind though that we currently get 5 jobs, which is only exceeded by the most expensive paid plan ($489). https://travis-ci.com/plans
Maybe we just need to have someone on the project contact them and get our free quota raised?
The problem is not in our quota. Travis has 180 macOS jobs allocated for open source projects, and those 180 jobs do not appear sufficient.
Tom Kent wrote:
It seems like this would be a great instance where the steering committee could allocate some of the organization's funds to directly improve the infrastructure being used by our team. I believe that we could upgrade our Travis account to a paid one for a couple hundred a month to get more build resources.
One should keep in mind though that we currently get 5 jobs, which is only exceeded by the most expensive paid plan ($489).
... although to clarify, I completely support the idea. If we can afford to do so, we should get a paid Travis account.
On Fri, Oct 27, 2017 at 1:22 PM, Peter Dimov via Boost < boost@lists.boost.org> wrote:
Tom Kent wrote:
It seems like this would be a great instance where the steering > committee could allocate some of the organization's funds to directly > improve the infrastructure being used by our team. I believe that we > could upgrade our Travis account to a paid one for a couple hundred a > month to get more build resources.
One should keep in mind though that we currently get 5 jobs, which is only exceeded by the most expensive paid plan ($489).
... although to clarify, I completely support the idea. If we can afford to do so, we should get a paid Travis account.
I would recommend attempting to negotiate with them on something that works for everyone. - Jim
Tom,
I’m not certain who we’d contact, but I suppose support@travis-ci.com is the place to start.
I’m willing to contact them on behalf of the project (although I think it would be better if you did this because you know more of what we need).
What can you tell me about our OS X resources/requirements/quota or resources/requirements/quota generally that I should know before contacting them? Do we have an account name/number/identifier of some type?
Jon
From: Boost Steering Committee
Testing your open source project is 10000% free
Seriously. Always. We like to think of it as our way of giving back to a community that gives us so much as well.
Maybe we just need to have someone on the project contact them and get our free quota raised?
Tom
On Thu, Oct 26, 2017 at 11:43 AM, Peter Dimov via Boost
On Fri, Oct 27, 2017 at 8:47 PM, Jon Kalb via Boost
Tom,
I’m not certain who we’d contact, but I suppose support@travis-ci.com is the place to start.
I have a contact from some Apache project work that I will ask about this and see if we can get contact info for decision makers. - Jim
On Fri, Oct 27, 2017 at 8:47 PM, Jon Kalb via Boost
Tom,
I’m not certain who we’d contact, but I suppose support@travis-ci.com is the place to start.
I’m willing to contact them on behalf of the project (although I think it would be better if you did this because you know more of what we need).
What can you tell me about our OS X resources/requirements/quota or resources/requirements/quota generally that I should know before contacting them? Do we have an account name/number/identifier of some type?
Jon
The info I got was: The normal, free package gives you five concurrent jobs (which equals five concurrent workers, so to speak). Beyond that, the next step up includes 10 extra concurrent jobs, effectively raising the limit to 15, for $500/month, paid annually and in advance. We are able to offer a 30% reduction for certified non-profits. So it sounds like for $4,200/yr the project can triple the number of build workers, and that's without any additional negotiation. If folks who have authority want to pursue this, you can email support@travis-ci.com to continue discussions. - Jim
On 26/10/2017 18:43, Peter Dimov via Boost wrote:
Travis's OS X resources for open source projects seem to be insufficient, so OS X jobs are very slow and seem to be falling further and further behind. Therefore, library authors are encouraged to keep the OS X jobs to the minimum necessary. Spawning many OS X jobs takes hours (~25 minutes waiting time per job at a quick estimate).
On a not entirely unrelated note, Rene Rivera has added a new feature <cxxstd> to Boost.Build that controls the C++ standard in use. So for instance, instead of the old
b2 libs/mylib/test toolset=gcc cxxflags=-std=c++11
one can now use
b2 libs/mylib/test toolset=gcc cxxstd=11
In addition to being more convenient, this also allows several invocations to be combined into one:
b2 libs/mylib/test toolset=clang cxxstd=03,11,14,1z
which can be leveraged to cut down on the number of jobs.
Thanks for the information Peter, and thanks Rene for the feature. I test different standard versions locally and I need to set up different toolsets for that. This simplifies things noticeably. Best, Ion
On 10/26/17 19:43, Peter Dimov via Boost wrote:
On a not entirely unrelated note, Rene Rivera has added a new feature <cxxstd> to Boost.Build that controls the C++ standard in use. So for instance, instead of the old
b2 libs/mylib/test toolset=gcc cxxflags=-std=c++11
one can now use
b2 libs/mylib/test toolset=gcc cxxstd=11
Sounds great. Does it always expand to c++XX or is there a way to specify gnu++XX?
Andrey Semashev wrote:
On 10/26/17 19:43, Peter Dimov via Boost wrote:
On a not entirely unrelated note, Rene Rivera has added a new feature <cxxstd> to Boost.Build that controls the C++ standard in use. So for instance, instead of the old
b2 libs/mylib/test toolset=gcc cxxflags=-std=c++11
one can now use
b2 libs/mylib/test toolset=gcc cxxstd=11
Sounds great. Does it always expand to c++XX or is there a way to specify gnu++XX?
gnu++XX can be requested with `cxxstd=XX cxxstd-dialect=gnu`.
participants (7)
-
Andrey Semashev -
Ion Gaztañaga -
James E. King, III -
Jon Kalb -
Peter Dimov -
Tom Kent -
Vicente J. Botet Escriba