BUILD specifying Python version to setup_miniconda causes conflicts

xref https://github.com/pymc-devs/pymc3/pull/4280/files#r532719476

If I specify
```
python-version: ${{ matrix.python-version }}
```
in `.github/workflows/pytest.yml` then I get
```
Found conflicts! Looking for incompatible packages.
```