On Wed, Dec 18, 2019 at 9:28 PM Edward Diener via Boost < boost@lists.boost.org> wrote:

On 12/18/2019 9:37 PM, Rene Rivera via Boost wrote:

...

On Wed, Dec 18, 2019 at 6:20 PM Edward Diener via Boost < boost@lists.boost.org> wrote:

...

When I started out in computer programming some 40 years ago you could submit a job, with some JCL, on an IBM mainframe computer via an input card reader and you were lucky if you got back the result a day later. How happy I was when I went to microcomputers and could get almost instantaneous results from any programming attempt.

Fast forward to now, some 40 years later, and it seems like nothing changes, despite the great advancement in computer technology and online access in our current age. Attempting to run CI using Appveyor literally has jobs queued pending a run for days and days. Surely there must be something better in this day and age than this ridiculously slow internet service. Is it 1979 again ?

The same solution from 40 years ago still applies. Buy your own CI servers and have then run your stuff instantly. Which unlike 40 years ago is financially plausible now. You can even hook them up to be managed by cloud CI like Azure Pipelines that supports custom clients. Or... Just use Azure Pipelines which has significantly better turnaround times than Appveyor.

A number of Boost libraries already run AppVeyor from their Github page to verify that a change is valid. I am certainly not against CI validity testing, but maybe Boost should explore better alternatives. When it takes days and days before a CI test in AppVeyor can be done while it is "pending", maybe there are better alternatives than AppVeyor which cover a very similar set of testing environments to what AppVeyor covers. I am gathering that AppVeyor is so slow because it is very popular and does not have the resources to run all its scheduled jobs within a more reasonable period of time.

Like I said (or maybe insinuated) Azure Pipelines is the better alternative. For example that's where I run the B2 tests < https://dev.azure.com/grafikrobot/B2/_build?definitionId=3&_a=summary>, and a few others. Assuming you aren't contending with the 10 free parallel slots among all of Boost, i.e. if you do your own CI setup, startup response is usually less than a minute. Total time is of course dependent on how much you do. Perhaps the one drawback is not as good default coverage for MinGW and Cygwin. But you can install anything you like also (with enough scripting voodoo). -- -- Rene Rivera -- Grafik - Don't Assume Anything -- Robot Dreams - http://robot-dreams.net

On 12/19/2019 2:40 AM, Alexander Grund via Boost wrote:

I had a similar problem (additionally) with travis: It is not as bad there but with about 32 jobs running with different compilers per PR and commit (on dev/master) the turnaround got quite low.

What I did first was to cut down on jobs on PRs: I only test latest + oldest clang and g++ now. I still run the full suite on dev/master commits (and hence after merge) to detect regressions in compilers causing failures in my library.

What can also help: Spread out your jobs: Currently there are at least 3 CI services which offer free windows testing:

- Appveyor - Travis - Github actions - (Azure pipelines? Haven't tried yet)

Especially GH actions is very fast (currently), so this offers a way to reduce the pressure on Appveyor jobs. Run what you can on GH actions and the rest on appveyor.

Except for the lack of YAML references I like their (GHA) Job descriptions much better. They allow matrices per job, multiple jobs per yaml and multiple yamls as well as using "building blocks" from your or other repos. Give it a try! :)

First it is not my Github Boost repository but rather other Boost repositories if/when I submit a PR. Second I do not think it is the amount of jobs in a test, as the test will say "pending" for a number of days without anything being run at all. It's like the CI Appveyor automatically triggered by the PR change in Github is just heavily backlogged and will not run for days while it is pending because there are other CI Appveyor tests from other Internet users it must run first on its queue. So some Github CI Appveyor test can be queued for days until it actually runs. I really, really doubt that the CI Appveyor test itself is actually running and taking days to complete. My point is that if CI Appveyor tests takes days before it is even started, when a PR is made against a Boost repository, maybe Boost should be a bit proactive and suggest to library maintainers that some other CI testing service, like the Azure mentioned by Rene, would be much faster and therefore should be used in place of Appveyor. Maybe I am being unduly impatient but programmers have come to expect that when tests of any kind can be run it shouldn't take days in modern computing to just start the actually testing procedure.

On 12/19/2019 3:27 AM, Alexander Grund via Boost wrote:

Am 19.12.19 um 09:17 schrieb Edward Diener via Boost:

...

First it is not my Github Boost repository but rather other Boost repositories if/when I submit a PR. Second I do not think it is the amount of jobs in a test, as the test will say "pending" for a number of days without anything being run at all. It's like the CI Appveyor automatically triggered by the PR change in Github is just heavily backlogged and will not run for days while it is pending because there are other CI Appveyor tests from other Internet users it must run first on its queue. So some Github CI Appveyor test can be queued for days until it actually runs. I really, really doubt that the CI Appveyor test itself is actually running and taking days to complete. That's not what I wanted to say. I *think* Appveyor shares its capacities over a user or a repo or an org. Hence if there are 3 open PRs, 1 commit merged you got 4xN jobs waiting to be executed. And only 1(?) will execute at a given time. So your startup latency is governed by the number of jobs the repo/user/org has queued up before. My suggestion was to reduce that number and hence the latency.

...

My point is that if CI Appveyor tests takes days before it is even started, when a PR is made against a Boost repository, maybe Boost should be a bit proactive and suggest to library maintainers that some other CI testing service, like the Azure mentioned by Rene, would be much faster and therefore should be used in place of Appveyor. Maybe I am being unduly impatient but programmers have come to expect that when tests of any kind can be run it shouldn't take days in modern computing to just start the actually testing procedure.

Hence my suggestion for GH actions. AFAIK not all MSVC versions supported on appveyor are supported on GHA, so my suggestion was to move as many jobs from appveyor as possible

I guess I do not understand under whose account an Appveyor job gets run for a particular Boost Github repository. I can see under Appveyor that I have projects based on Boost Github repositories, but does this mean that if someone triggers off an Appveyor job by updating a repository it is run under my personal Appveyor account, their personal Appveyor account, or some Appveyor account registered to the Boost organization in general ?

On Thu, Dec 19, 2019 at 11:28 AM Mateusz Loskot via Boost < boost@lists.boost.org> wrote:

There is no official organization account for Boost on AppVeyor.

There is.. https://ci.appveyor.com/project/boostorg/boost Although I don't think it's used at this time / any longer. -- -- Rene Rivera -- Grafik - Don't Assume Anything -- Robot Dreams - http://robot-dreams.net

On 12/19/2019 12:29 PM, Mateusz Loskot via Boost wrote:

On Thu, 19 Dec 2019 at 18:17, Edward Diener via Boost wrote:

...

On 12/19/2019 3:27 AM, Alexander Grund via Boost wrote:

...

Am 19.12.19 um 09:17 schrieb Edward Diener via Boost:

...

First it is not my Github Boost repository but rather other Boost repositories if/when I submit a PR. Second I do not think it is the amount of jobs in a test, as the test will say "pending" for a number of days without anything being run at all. It's like the CI Appveyor automatically triggered by the PR change in Github is just heavily backlogged and will not run for days while it is pending because there are other CI Appveyor tests from other Internet users it must run first on its queue. So some Github CI Appveyor test can be queued for days until it actually runs. I really, really doubt that the CI Appveyor test itself is actually running and taking days to complete. That's not what I wanted to say. I *think* Appveyor shares its capacities over a user or a repo or an org. Hence if there are 3 open PRs, 1 commit merged you got 4xN jobs waiting to be executed. And only 1(?) will execute at a given time. So your startup latency is governed by the number of jobs the repo/user/org has queued up before. My suggestion was to reduce that number and hence the latency.

...

My point is that if CI Appveyor tests takes days before it is even started, when a PR is made against a Boost repository, maybe Boost should be a bit proactive and suggest to library maintainers that some other CI testing service, like the Azure mentioned by Rene, would be much faster and therefore should be used in place of Appveyor. Maybe I am being unduly impatient but programmers have come to expect that when tests of any kind can be run it shouldn't take days in modern computing to just start the actually testing procedure.

Hence my suggestion for GH actions. AFAIK not all MSVC versions supported on appveyor are supported on GHA, so my suggestion was to move as many jobs from appveyor as possible

I guess I do not understand under whose account an Appveyor job gets run for a particular Boost Github repository. I can see under Appveyor that I have projects based on Boost Github repositories, but does this mean that if someone triggers off an Appveyor job by updating a repository it is run under my personal Appveyor account, their personal Appveyor account, or some Appveyor account registered to the Boost organization in general ?

AFAIK,all existing AppVeyor-s for all libraries under https://github.com/boostorg/* are attached to personal/maintainer's account, e.g. https://ci.appveyor.com/project/stefanseefeld/gil/ for https://github.com/boostorg/gil

How can I see on a Github Boost library's web page which personal maintainer's AppVeyor account is being used ? I have a number of AppVeyor projects for Boost Github libraries which I no longer follow or help maintain as a previous member of CMT, and I would like to have someone else be associated with those Appveyor projects.

There is no official organization account for Boost on AppVeyor.

It is possible to create one though https://www.appveyor.com/docs/team-setup/#setting-up-appveyor-account-for-gi...

Best regards,

Am 19.12.19 um 20:04 schrieb Mateusz Loskot via Boost:

...

How can I see on a Github Boost library's web page which personal maintainer's AppVeyor account is being used ? I don't know any way to request AppVeyor for "Who builds github.com/x/y?".

If a library maintainer sets up the AppVeyor, I'd expect her/him to add badge to the library's README.md, as many Boost libraries do.

Look at the most recent build, e.g. from the CI section of the latest PR or the badge. It has a link to the appveyor job which should show the "owner" AppVeyor can be added as an OAuth app, then it runs "as the user" or as a Github app, then it runs "as the org"(? not fully sure, maybe it was "as that app" which has a separate quota) Currently most use the OAuth, which is less secure but easier to setup as the GH app needs approval by boostorg

On 2019-12-19 12:49, Alexander Grund via Boost wrote:

Am 19.12.19 um 10:24 schrieb Andrey Semashev via Boost:

...

On 2019-12-19 03:20, Edward Diener via Boost wrote:

...

When I started out in computer programming some 40 years ago you could submit a job, with some JCL, on an IBM mainframe computer via an input card reader and you were lucky if you got back the result a day later. How happy I was when I went to microcomputers and could get almost instantaneous results from any programming attempt.

Fast forward to now, some 40 years later, and it seems like nothing changes, despite the great advancement in computer technology and online access in our current age. Attempting to run CI using Appveyor literally has jobs queued pending a run for days and days. Surely there must be something better in this day and age than this ridiculously slow internet service. Is it 1979 again ?

I think, a great deal of Appveyor time is wasted due to it not cancelling queued jobs on subsequent commits, like Travis does. I don't think we can practically do anything about it, other than manually cancelling the unneeded jobs.

I was like: There must be an option!

And there is: https://www.appveyor.com/docs/build-configuration/#rolling-builds

“Rolling builds” are great for very active OSS projects with lengthy queue. Whenever you do a new commit to the same branch /OR/ pull request all current queued/running builds for that branch or PR are cancelled and the new one is queued. Other words, rolling builds make sure that only the most recent commit is built.

For example, you do commit |A| to |master| branch - it’s being queued and then run. Then you do |B| commit to |master| branch while |A| is running - build of |A| will be cancelled immediately and |B| is queued. If you do another |C| commit while |B| is queued it will be cancelled and |C| queued.

(Note that “Rolling builds” can only be enabled in the settings UI of a project and not via |appveyor.yml| and that the presence of |appveyor.yml| does not disable this UI setting.)

So: Enable in UI

This must be a recent addition, it wasn't there when I configured Appveyor for my projects, and it is disabled now by default. I suppose, everyone should check that it is enabled in their projects then.