Evolve CI builds

[AArnott]

I propose to apply some of the pattern from https://github.com/aarnott/library.template to this repo. This will give you Tests recognition within Azure Pipelines. Test results will be captured and represented as well as code coverage.

While I'm at it, I could remove AppVeyor and travis unless it's providing you with some value that can't be obtained from Azure Pipelines.

Thoughts?

[bording]

I propose to apply some of the pattern from https://github.com/aarnott/library.template to this repo. This will give you Tests recognition within Azure Pipelines. Test results will be captured and represented as well as code coverage.

Sounds good to me!

While I'm at it, I could remove AppVeyor and travis unless it's providing you with some value that can't be obtained from Azure Pipelines.

I can't think of any reason to keep them. I think the plan was to get rid of them eventually, so I say go for it.

[AArnott]

Awesome, because for my #1758 PR, getting Travis to work was a real time sink.

[AArnott]

Another option is to move away from all three CI systems you're currently using and use GitHub Actions instead. Is there anything about Azure Pipelines that you need that are not available in GitHub Actions, such as release pipelines?

[bording]

Is there anything about Azure Pipelines that you need that are not available in GitHub Actions, such as release pipelines?

AFAIK when we want to release something, we currently grab the artifact from CI and upload it manually.

[AArnott]

Then GitHub Actions may suit you well. I generally prefer Azure Pipelines, but GitHub Actions has the advantage of being easier to see and review logs from a GitHub PR's Checks tab.

I see we have some unstable tests already in master. The much awaited #1618 is blamed for having a couple unstable tests that in the last build only failed on AppVeyor. If we were to remove AppVeyor, we wouldn't have seen these failures on the last run, anyway. So that might "unblock" the PR. Not sure if that's a good thing or not. But if we have unstable tests, I don't think we should mitigate that risk by maintaining 3-4 CI systems for each PR.

[bording]

I agree. In the past we needed AppVeyor and Travis to get coverage on Windows, Mac, and Linux. We can get all of that from a single CI system now, so it makes sense to consolidate.

I haven't had a chance to check out GitHub Actions yet, but if we can get everything we need from it, I'd be fine moving to that.

[AArnott]

Another advantage to GitHub Actions is that forks of this repo will automatically be able to build too without setting up their own AppVeyor/Travis/Azure Pipelines accounts.

[AArnott]

Do you need a LEAKS_IDENTIFYING test run on all OS's, or just one?

[bording]

Good question. It seems unlikely that there could be OS-specific handle leaking, right?