WEB: use sphinx to render the PDEPs

[jorisvandenbossche]

Currently the online version of the PDEPs (linked on https://pandas.pydata.org/about/roadmap.html) are rendered by the website building machinery.
This has currently some short comings:

• No code highlighting
• Subsection titles don't have anchors to link to
• No within-page table of contents which could be useful for PDEPs (like the right sidebar in the docs)
• Too wide text wrapping for comfortable reading (this is actually generally for the website, but those PDEP pages tend to be a lot text heavier than most other pages on the website)
• Non-standard markdown parser? (not fully sure about this, just remember that I had to fix some things like WEB: fix list formatting in PDEP-4 #50895)

This are of course all things that we can fix in the website building machinery. But, all those aspects are already handled by the sphinx theme we use for our docs. So personally, I would find that the easier option, and therefore the proposal from the title of this issue to use sphinx instead for rendering the PDEP pages.

The idea would not be to move them to the docs, but just to render the PDEPs pages (where they are now in the website) using sphinx, reusing the doc theme. And so it should certainly be possible to retain the current URLs for the existing PDEPs. For the landing page with the list of PDEPs, that could still be done either way (keep as is or also render using sphinx).

cc @datapythonista

[datapythonista]

No strong opinion about moving this to sphinx, but personally I think the website is a better place.

The reason we have a website and not only sphinx is that sphinx is versioning, and I think it's not great to have to wait until certain things are published (or have to make sure you're in the dev version of the docs. We already have many things that shouldn't be versioned and that are in sphinx, like the dev docs, or release notes.

That being said, the only advantage that I see in sphinx is the references among pages. That part needs to be managed manually in the website (not a big deal, but sphinx is good at that).

In my opnion, your points are not a big deal.

• I thought that I added syntax highlighting to the docs. I need to check if I never implemented it, or if it's not working. But it's just adding a css library I think.
• Subsections do have anchors: https://pandas.pydata.org//pdeps/0001-purpose-and-guidelines.html#pdep-guidelines Maybe you mean you don't get the link to it next to the titles. I guess this is just a css style that can be added
• A table of contents can be easily added in markdown inside the same document. The side panel navigation is probably nicer, but we can have a table of content if we want
• The page wide is just a style, that we can change for PDEP pages only if we want
• I don't think there is a standard markdown, just flavors. I don't remember if the Python markdown library implements one or let you chose.

As you say, those points are already in sphinx. Probably if you find them important enough for PDEP documents, they are worth for other pages in the website as well. And none of them seems difficult to implement, they seem mostly immediate, and I can create good first issues for them.

In my opinion there is also the advantage of using markdown for PDEPs if in the website. But maybe you consider that a disadvantage.

In any case, it doesn't make sense to me to move the PDEPs to sphinx, but if other people also think it's a better option, no problem with it. But I'm -1 in keeping the existing url if we move them. We can surely add a temporary redirect, but I'm not in favor of adding complexity to our nginx configuration.

[jorisvandenbossche]

but personally I think the website is a better place.

To be clear, I am not suggesting to move them out of the website, only do render those pages (at the current URLs) using sphinx.

(edited the top comment to clarify that, for other readers)

The reason we have a website and not only sphinx is that sphinx is versioning, and I think it's not great to have to wait until certain things are published

See above, there will still be only one (and directly published) version of the PDEP. The versioning is not inherent to sphinx, just that for our docs we want versions.

In my opinion there is also the advantage of using markdown for PDEPs if in the website. But maybe you consider that a disadvantage.

I should have made that clear in the first comment, but I am not proposing to move away from markdown (certainly not!). Sphinx can handle markdown just fine those days.

I thought that I added syntax highlighting to the docs. I need to check if I never implemented it, or if it's not working. But it's just adding a css library I think.

Yes -> #51458

But I'm -1 in keeping the existing url if we move them. We can surely add a temporary redirect, but I'm not in favor of adding complexity to our nginx configuration.

Can you explain in more detail what would be the problem with keeping the current urls?
As far as I understand, we currently build the website in a github action putting the build output in a directory, and afterwards we sync that directory with the server. All we would do here is when building the website also build the PDEPs with sphinx and put them in that same directory (at the same location as they currently are), and afterwards sync this output as we do now?

[datapythonista]

Ah, I see, I thought you were proposing moving them to the docs. You can ignore the comment about nginx, I thought you wanted web urls for things in the docs.

Using sphinx to render things in the website seems to me harder to implement than the fixes to the website (which would also be good for the rest of the website). Also, I think the final result would be quite complex to understand and maintain. And feels like it'd be awkward to have sphinx themed pages in the website, without nagivation I guess, and not sure where the link in the pandas logo in PDEPs would point to.

If you still think it's a good idea, maybe give it a try, and we can see the code and the preview on how it looks like, and we discuss more in practice.

[jorisvandenbossche]

In my opnion, your points are not a big deal.

Personally I think those are significant issues deterring reading those pages, but I agree they are not a big deal in the sense that it should be relatively easy to solve them also with the current website sources.
But for me it's mainly the question whether it is worth the effort if the sphinx theme we use already provide all those things, and would make it easier to fix those.

(although I think some of those would also be nice to fix in general for other pages on the website as well)

But yes, will give it a quick go to see if it is actually easier or turns out to be complicated.

[datapythonista]

Agree. Yes, I meant that are not a big deal to implement (it should be easy to get them fixed), and I was explaining why in the bullet points after that comment. I didn't phrase that well. I think it's a very good list of things to address or improve.

[jorisvandenbossche]

I pushed a quick test to #51467