Hi dear list, Hi Rene, I would like to know more about Travis and its status. From what I understand, a lot of efforts have been made recently for making things running per commit on Travis and have a quick feedback. * Naturally the first question is: is the status given by Travis the "official" one? so we consider the status given by Travis an official source of information, especially wrt. to its competing regression dashboard. * What are the development plans for Travis? Right now I see 3 types of builds, and I do not fully understand what they cover (platform, compiler) nor their differences. * For boost.test, I am testing each feature branch before merging to develop, by 1. cloning boost superproject on develop 2. switching only boost.test to the feature branch This is for preventing the breakages on develop that we observed in the past since several libraries are depending on boost.test. Would it be possible to have that as well on Travis or any continuous CI? That would be per library then. Would it also be possible to have like a template .travis.yml coming from the boost.superproject which can partly be overiden by a library? (if we want to build things slightly differently) * The CI I am using is able to parse the junit files generated by the test framework, and summarizes the information of each build nicely (also tracks the tests history). Is there anything similar in Travis? Right now, apart from looking at the full logs (which is hard), is there any intermediate information than red/green overall status? * I can see many project under the boostorg of Travis https://travis-ci.org/boostorg/ . How do we do that? Thanks, Raffi
On Thu, Aug 18, 2016 at 7:32 AM, Raffi Enficiaud < raffi.enficiaud@mines-paris.org> wrote:
Hi dear list, Hi Rene,
I would like to know more about Travis and its status. From what I understand, a lot of efforts have been made recently for making things running per commit on Travis and have a quick feedback.
* Naturally the first question is: is the status given by Travis the "official" one? so we consider the status given by Travis an official source of information, especially wrt. to its competing regression dashboard.
The integrated test results are still the "official" results. * What are the development plans for Travis? Right now I see 3 types of
builds, and I do not fully understand what they cover (platform, compiler) nor their differences.
My plans, which are official as it can get around here when it comes to testing, are to move as much of the "generic" testing and automation as possible to cloud CI (would include Travis, Circle, and Appveyor). I don't know what you mean by 3 types of builds though. Can you point out an example?
* For boost.test, I am testing each feature branch before merging to develop, by 1. cloning boost superproject on develop 2. switching only boost.test to the feature branch
This is for preventing the breakages on develop that we observed in the past since several libraries are depending on boost.test. Would it be possible to have that as well on Travis or any continuous CI? That would be per library then.
Yes. That is part of my plan.
Would it also be possible to have like a template .travis.yml coming from the boost.superproject which can partly be overiden by a library? (if we want to build things slightly differently)
Kinda :-) My plan is to have a generic .travis.yml that authors can use for their libraries. It would manage doing all the testing setups possible (and wanted). It would look roughly like the one I currently use for Boost Predef https://github.com/boostorg/predef/blob/develop/.travis.yml. And the same would apply for Circle and Appveyor. Note that the current implementation of those Travis scripts doesn't do the integrated testing yet. That's coming soonish. * The CI I am using is able to parse the junit files generated by the test
framework, and summarizes the information of each build nicely (also tracks the tests history). Is there anything similar in Travis?
I don't think there's anything like that for Travis.
Right now, apart from looking at the full logs (which is hard), is there any intermediate information than red/green overall status?
Not on Travis.. But eventually the goal is to have the above common scripts upload the CI results to our common test results. That's in the not so near future though as it's not a trivial task. * I can see many project under the boostorg of Travis
https://travis-ci.org/boostorg/ . How do we do that?
Just ask me and I can turn on any particular library to be included in that. Note that the Travis testing resources, with the free OO license, limit the number of concurrent jobs per organization. So all libraries in that list share a smallish quota. But this is one area where we will consider funding to increase that quota when we have better scope information. -- -- Rene Rivera -- Grafik - Don't Assume Anything -- Robot Dreams - http://robot-dreams.net -- rrivera/acm.org (msn) - grafikrobot/aim,yahoo,skype,efnet,gmail
Le 18/08/16 à 16:16, Rene Rivera a écrit :
On Thu, Aug 18, 2016 at 7:32 AM, Raffi Enficiaud < raffi.enficiaud@mines-paris.org> wrote:
Hi dear list, Hi Rene,
I would like to know more about Travis and its status. From what I understand, a lot of efforts have been made recently for making things running per commit on Travis and have a quick feedback.
* Naturally the first question is: is the status given by Travis the "official" one? so we consider the status given by Travis an official source of information, especially wrt. to its competing regression dashboard.
The integrated test results are still the "official" results.
* What are the development plans for Travis? Right now I see 3 types of
builds, and I do not fully understand what they cover (platform, compiler) nor their differences.
My plans, which are official as it can get around here when it comes to testing, are to move as much of the "generic" testing and automation as possible to cloud CI (would include Travis, Circle, and Appveyor). I don't know what you mean by 3 types of builds though. Can you point out an example?
Sorry I was not clear. Here for instance: https://travis-ci.org/boostorg/boost I can see 4 job builds, 3 green and one red, the 3 green saying: - SCRIPT=ci_boost_status (~4min) - SCRIPT=ci_boost_status EOL=LF (~32min) - SCRIPT=ci_boost_status EOL=CRLF (~32min) I do not know what those jobs are doing and in what type of environment.
* For boost.test, I am testing each feature branch before merging to develop, by 1. cloning boost superproject on develop 2. switching only boost.test to the feature branch
This is for preventing the breakages on develop that we observed in the past since several libraries are depending on boost.test. Would it be possible to have that as well on Travis or any continuous CI? That would be per library then.
Yes. That is part of my plan.
Wonderful!
Would it also be possible to have like a template .travis.yml coming from the boost.superproject which can partly be overiden by a library? (if we want to build things slightly differently)
Kinda :-) My plan is to have a generic .travis.yml that authors can use for their libraries. It would manage doing all the testing setups possible (and wanted). It would look roughly like the one I currently use for Boost Predef https://github.com/boostorg/predef/blob/develop/.travis.yml. And the same would apply for Circle and Appveyor. Note that the current implementation of those Travis scripts doesn't do the integrated testing yet. That's coming soonish.
Cool, many thanks for those efforts. Can we help?
* The CI I am using is able to parse the junit files generated by the test
framework, and summarizes the information of each build nicely (also tracks the tests history). Is there anything similar in Travis?
I don't think there's anything like that for Travis.
True, apparently they are not doing. That means that uploading the tests results to a dashboard will also be on boost superproject hands, is that right?
Right now, apart from looking at the full logs (which is hard), is there any intermediate information than red/green overall status?
Not on Travis.. But eventually the goal is to have the above common scripts upload the CI results to our common test results. That's in the not so near future though as it's not a trivial task.
I am not even sure this is technically allowed by Travis (https://blog.travis-ci.com/2012-12-18-travis-artifacts/).
* I can see many project under the boostorg of Travis
https://travis-ci.org/boostorg/ . How do we do that?
Just ask me and I can turn on any particular library to be included in that. Note that the Travis testing resources, with the free OO license, limit the number of concurrent jobs per organization. So all libraries in that list share a smallish quota. But this is one area where we will consider funding to increase that quota when we have better scope information.
Cool. Right now I have my own, but maybe in the near future (especially mine is in a private net). Thanks for all those information, Raffi
On Thu, Aug 18, 2016 at 10:57 AM, Raffi Enficiaud < raffi.enficiaud@mines-paris.org> wrote:
Le 18/08/16 à 16:16, Rene Rivera a écrit :
On Thu, Aug 18, 2016 at 7:32 AM, Raffi Enficiaud < raffi.enficiaud@mines-paris.org> wrote:
Hi dear list,
Hi Rene,
I would like to know more about Travis and its status. From what I understand, a lot of efforts have been made recently for making things running per commit on Travis and have a quick feedback.
* Naturally the first question is: is the status given by Travis the "official" one? so we consider the status given by Travis an official source of information, especially wrt. to its competing regression dashboard.
The integrated test results are still the "official" results.
* What are the development plans for Travis? Right now I see 3 types of
builds, and I do not fully understand what they cover (platform, compiler) nor their differences.
My plans, which are official as it can get around here when it comes to testing, are to move as much of the "generic" testing and automation as possible to cloud CI (would include Travis, Circle, and Appveyor). I don't know what you mean by 3 types of builds though. Can you point out an example?
Sorry I was not clear. Here for instance: https://travis-ci.org/boostorg/boost
I can see 4 job builds, 3 green and one red, the 3 green saying: - SCRIPT=ci_boost_status (~4min) - SCRIPT=ci_boost_status EOL=LF (~32min) - SCRIPT=ci_boost_status EOL=CRLF (~32min)
I do not know what those jobs are doing and in what type of environment.
That should be (and what they do): - SCRIPT=ci_boost_status Runs a "quick" status check on the full integrated Boost repo. The tests are dry runs of what the regular building and testing does. I.e. it runs b2 for the general building at the root (what an end user would do), and by in the status subdir (hence the name, and this is what ultimately the full testing does). And by dry run I mean (b2 -n). Hence it catches errors like missing tests, missing sources, bad jamfiles, etc. And generally provides a quick pass/fail status for the prospective release. - SCRIPT=ci_boost_release EOL=LF - SCRIPT=ci_boost_release EOL=CRLF Those generate the release archives in the two EOL flavors we support. Building one of those releases essentially corresponds to: build support tools, build all the docs, copy & filter the final release tree, generate compressed archive files, upload the archives to Bintray and SF. Any problems here are in the stop-the-presses category. As these are what we give users. - SCRIPT=ci_boost_library_check That's the newest build.. And it runs the library checks you now see in the general result matrix as "__boost_check_library__". Except it runs them for all libraries at once. It helps me to asses (and harass authors) the basic library requirements. Would it also be possible to have like a template .travis.yml coming from
the boost.superproject which can partly be overiden by a library? (if we want to build things slightly differently)
Kinda :-) My plan is to have a generic .travis.yml that authors can use for their libraries. It would manage doing all the testing setups possible (and wanted). It would look roughly like the one I currently use for Boost Predef https://github.com/boostorg/predef/blob/develop/.travis.yml. And the same would apply for Circle and Appveyor. Note that the current implementation of those Travis scripts doesn't do the integrated testing yet. That's coming soonish.
Cool, many thanks for those efforts. Can we help?
Welcome.. And maybe you can help. I'll think about how :-) * The CI I am using is able to parse the junit files generated by the test
framework, and summarizes the information of each build nicely (also tracks the tests history). Is there anything similar in Travis?
I don't think there's anything like that for Travis.
True, apparently they are not doing. That means that uploading the tests results to a dashboard will also be on boost superproject hands, is that right?
Yes, indeed. Right now, apart from looking at the full logs (which is hard), is there
any intermediate information than red/green overall status?
Not on Travis.. But eventually the goal is to have the above common scripts upload the CI results to our common test results. That's in the not so near future though as it's not a trivial task.
I am not even sure this is technically allowed by Travis ( https://blog.travis-ci.com/2012-12-18-travis-artifacts/).
This would happen as an entirely outside of their regular channels of uploading. It would either be something to process and upload to the test ftp location, or some form of custom REST upload. -- -- Rene Rivera -- Grafik - Don't Assume Anything -- Robot Dreams - http://robot-dreams.net -- rrivera/acm.org (msn) - grafikrobot/aim,yahoo,skype,efnet,gmail
On 18 Aug 2016 at 17:57, Raffi Enficiaud wrote:
Not on Travis.. But eventually the goal is to have the above common scripts upload the CI results to our common test results. That's in the not so near future though as it's not a trivial task.
I am not even sure this is technically allowed by Travis (https://blog.travis-ci.com/2012-12-18-travis-artifacts/).
One of the very early uses of Travis was to push stuff at your project's CDash dashboard. It's definitely allowed to do this, and it provides a lovely single stop location to see the colour coded status and warnings and errors of configure, build and unit testing for all your supported platforms, plus any prebuilt binaries for some platform. http://my.cdash.org/index.php?project=Boost.AFIO&date=2016-08-18 is an example of what I mean. Note the wee yellow box next to the build name, if that's present it's the binary distribution pushed by that build and it can be clicked upon to download the artifacts. Niall -- ned Productions Limited Consulting http://www.nedproductions.biz/ http://ie.linkedin.com/in/nialldouglas/
participants (3)
-
Niall Douglas -
Raffi Enficiaud -
Rene Rivera