PEP 512 – Migrating From To GitHub

Rationale

In 2014, it became obvious that Python’s custom development process was becoming a hindrance. As an example, for an external contributor to submit a fix for a bug that eventually was committed, the basic steps were:

1. Open an issue for the bug at bugs.python.org [5].
2. Checkout out the CPython source code from hg.python.org [1].
3. Make the fix.
4. Upload a patch.
5. Have a core developer review the patch using our fork of the Rietveld code review tool [6].
6. Download the patch to make sure it still applies cleanly.
7. Run the test suite manually.
8. Update the NEWS, ACKS, and “What’s New” document as necessary
9. Pull changes to avoid a merge race.
10. Commit the change manually.
11. If the change was for a bugfix release, merge into the in-development branch.
12. Run the test suite manually again.
13. Commit the merge.
14. Push the changes.

This is a very heavy, manual process for core developers. Even in the simple case, you could only possibly skip the code review step, as you would still need to build the documentation. This led to patches languishing on the issue tracker due to core developers not being able to work through the backlog fast enough to keep up with submissions. In turn, that led to a side-effect issue of discouraging outside contribution due to frustration from lack of attention, which is a dangerous problem for an open source project with no corporate backing as it runs counter to having a viable future for the project. While allowing patches to be uploaded to bugs.python.org [5] is potentially simple for an external contributor, it is as slow and burdensome as it gets for a core developer to work with.

Hence the decision was made in late 2014 that a move to a new development process was needed. A request for PEPs proposing new workflows was made, in the end leading to two: PEP 481 and PEP 507 proposing GitHub [2] and GitLab [7], respectively.

The year 2015 was spent off-and-on working on those proposals and trying to tease out details of what made them different from each other on the core-workflow mailing list [8]. PyCon US 2015 also showed that the community was a bit frustrated with our process due to both cognitive overhead for new contributors and how long it was taking for core developers to look at a patch (see the end of Guido van Rossum’s keynote at PyCon US 2015 [9] as an example of the frustration).

On January 1, 2016, the decision was made by Brett Cannon to move the development process to GitHub. The key reasons for choosing GitHub were [10]:

• Maintaining custom infrastructure has been a burden on volunteers (e.g., an unmaintained, custom fork of Rietveld [6] is currently being used).
• The custom workflow is very time-consuming for core developers (not enough automated tooling built to help support it).
• The custom workflow is a hindrance to external contributors (acts as a barrier of entry due to time required to ramp up on development process unique to CPython itself).
• There is no feature differentiating GitLab from GitHub beyond GitLab being open source.
• Familiarity with GitHub is far higher among core developers and external contributors than with GitLab.
• Our BDFL prefers GitHub (who would be the first person to tell you that his opinion shouldn’t matter, but the person making the decision felt it was important that the BDFL feel comfortable with the workflow of his own programming language to encourage his continued participation).

There’s even already an unofficial logo to represent the migration to GitHub [22].

The overarching goal of this migration is to improve the development process to the extent that a core developer can go from external contribution submission through all the steps leading to committing said contribution from within a browser on a tablet with WiFi using some development process (this does not inherently mean GitHub’s default workflow). The final solution will also allow an external contributor to contribute even if they chose not to use GitHub (although there is not guarantee in feature parity).