Release automation

[TomAugspurger]

Tracking manual things here

• Date in header of whatsnew/vX.X.X.txt (use jinja?)
• Update to release.rst
• Tag & push
• doc build and upload
• website build and upload (web updates no longer part of release process)
• local Pip / conda tests
• Github Release
• conda-forge PR
• MacPython PR

[jbrockmendel]

How much of this goal is pandas-specific vs potentially more broadly useful? Here I'm mostly thinking of statsmodels piggybacking on effort put in here.

[TomAugspurger]

See https://github.com/pandas-dev/pandas-release

Some parts are pandas-specific, others could be adapted without too much effort.

There's also rever, which may be suitable for statsmodels.

[jorisvandenbossche]

@simonjayhawkins for the release date: I suppose that with git you can get the date of the release commit / tag, and then you can add that to the html context in conf.py and then inject that in the notes.

However, I think there is also some value in having it hardcoded in the docs I think, or at least for older releases, when looking at the source of whatsnew files to see the date (but maybe this is not really important, you can always see it online)

[simonjayhawkins]

Thanks @jorisvandenbossche. could be simpler to search and replace (the ??) after tagging in the release process. not really looked into this much yet.

[simonjayhawkins]

after tagging

actually would need to be before tagging as automated commit.

but that wouldn't be on master!

[simonjayhawkins]

I have started to experiment with GitHub Actions for the Release Process.

https://github.com/simonjayhawkins/pandas-release/actions?query=workflow%3A%22Manual+Release%22

In order to get workflows to trigger, appropriate permission are required.

To experiment, have forked https://github.com/pandas-dev/pandas-release

created a branch for the changes and then in the forked repo have made the new branch the default so actions can be triggered.

for now, using a manual trigger.

[jreback]

https://github.com/dask/dask-gateway/pull/339/files

looks interesting as an example of a GH here

cc @simonjayhawkins

[simonjayhawkins]

Thanks @jreback.

It may make sense to build the wheels for release in github actions from pandas as part of a single workflow, but we obviously don't want to duplicate code in the workflow.

If we wanted MacPython to still be the single source of truth, we could maybe create an action and re-use. (i've only created one action up to now see simonjayhawkins/pandas-release@f8b497a so not sure if this would the correct approach)

[TomAugspurger]

I haven't checked, but we can maybe use the https://github.com/peter-evans/repository-dispatch action to have an action here (pushed a tag) to trigger an action in MacPython/pandas-wheels.

[simonjayhawkins]

Thanks @TomAugspurger

dispatching events between repositories sort of leads to another big question. Should we keep pandas-release as a separate repo or include the release scripts in pandas?

[TomAugspurger]

I'm fine with moving `pandas-release` here.

…

On Mon, Oct 19, 2020 at 7:01 AM Simon Hawkins ***@***.***> wrote: Thanks @TomAugspurger <https://github.com/TomAugspurger> dispatching events between repositories sort of leads to another big question. Should we keep pandas-release as a separate repo or include the release scripts in pandas? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#21050 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAKAOIW5ZAZS32SIIRPTIJTSLQTC3ANCNFSM4E75VQVQ> .

[jorisvandenbossche]

There might be some value in having it as a separate repo? (just thinking out loud, not sure) For example, you can still make updates / fix something, even after a tag for a release is created, without having to deal with a "dirty" git state.

(eg if we want to have an action triggered on a release here in this repo, that action could also check out the separate repo)

[simonjayhawkins]

also thinking out loud... if the release is more automated, would there be less of an issue with deleting the tag and restarting to process? We would obviously want a point of no return, but would that be as late in the process as PyPI upload. I think it's possible to delete github releases? (Although IIUC only conda release needs the Github release)

so process could be re-ordered slightly

tag
conda and pip tests
build documentation
upload documentation
build wheels
push tag (if easy to remove tag on workflow failure could do this as very first step?)
create github release
upload to PyPI
start conda build process