On 26/05/2018 19:08, Tom Kent via Boost wrote:
On Sat, May 26, 2018 at 10:49 AM, Ion Gaztañaga via Boost <
I've been having horrible problems getting all the submodules from github, that look similar to what you've got going. I'm not sure where the issue is, but I see it across linux and windows boxes at my house, but not when I run in a cloud data center. I'm thinking it has something to do with my connection (AT&T), but I'm not sure.
Looks like a GitHub problem, which is the worst case ;-) Maybe it thinks it's a DoS attack or similar with so many connections.
I ended up mirroring all the repos to a local machine, with a cron job to update them every 15 min) and then cloning from that for the regression runs. The script I use for that can be found here: https://github.com/teeks99/boost-build/blob/master/Regression/mirror_boostor...
Looks like I should do something similar. Many thanks for the script.
For more backgroun, I was seeing the same failed connections over https or ssh. I got the script failure when even one of the submoduels failed to update...which I guess is necessary. Now with my script almost all runs one of the repos fails to fetch, but since they are all just standalone repos, that failures doesn't cause an overall failure.
I wonder if other GitHub projects have similar problems, but perhaps not many projects have so many submodules. I'll have a look at your workaround and try to do something, or else I won't be able to run regression tests. Many thanks! Ion