Andrey Semashev wrote:
However, CI vs. test matrix is a bit off-topic in this discussion.
One incredibly important advantage of CI over the test matrix is that it runs on pull requests. A pull request passing CI is not a guarantee of anything, but it's still miles better than nothing. This is also why it's important for CI to pass on the base branch (before the PR) - so that failures can be attributed to the proposed change and not to other factors. Yes, this all requires constant maintenance, and forces one to pay attention to environments one doesn't normally use for local testing, and to changes to other libraries one doesn't "use". Such is life. This incidentally is one of the functions of the develop branch - sometimes several separate PRs pass in isolation, but fail when simultaneously applied.