DOC: Reorganization of the documentation

[datapythonista]

The documentation currently has almost all the pages in the same level. Making it difficult to navigate.

Having few top-level sections that group the pages, and some hierarchy would make things simpler to find. Here there is a prototype of how having few top-level sections could look like: https://pandas-dev.github.io/pandas-sphinx-theme/pr-datapythonista_base/

Those are the pages we currently have:

10min
advanced
api
basics
categorical
comparison_with_r
comparison_with_sas
comparison_with_sql
comparison_with_stata
computation
contributing_docstring
contributing
cookbook
developer
dsintro
ecosystem
enhancingperf
extending
gotchas
groupby
indexing
index
install
internals
io
merging
missing_data
options
overview
releases
reshaping
r_interface
sparse
style
text
timedeltas
timeseries
tutorials
visualization
whatsnew/

An initial proposal of organization is:

index

install

getting_started (probably making this a 3 or 4 part tutorial is a good idea)
    overview
    10min
    basics
    dsintro
    tutorials (I'd remove this page)
    comparisons
        comparison_with_r
            r_interface (I'd remove this page)
        comparison_with_sas
        comparison_with_sql
        comparison_with_stata

user_guide
    io
    indexing
    advanced
    merging
    reshaping
    text
    missing_data
    categorical
    visualization
    computation
    groupby
    timeseries
    timedeltas
    style
    options
    enhancingperf
    sparse
    gotchas
    cookbook (not sure if user_guide is the right place, but I don't see a better one)
    ecosystem (not sure if user_guide is the right place, but I don't see a better one)

api
    splitted in #24462, names of secions and possible changes to structure to be discussed 

development
    contributing (I'd split this one)
        contributing_docstring
    extending
    developer (this could probably be merged with extending)
    internals

whatsnew/
    releases -> index

@pandas-dev/pandas-core thoughts?

[mroeschke]

+1 on top level section for the different pages.

For reference, here's scikit-learn's documentation structure which I think is very concise and similar to what you're proposing https://scikit-learn.org/stable/documentation.html

[jorisvandenbossche]

This is actually a duplicate of #6000 :-)

But it's more or less the same. I will try to take a closer look tomorrow.

Quick feedback:

• With the current content of 'basics' and 'dsintro', I think they fit better in the User Guide section (they are very long and detailed, not really "starting" material IMO)

• I think some pages in the User guide also need splitting, but I assume the above is mainly about the top-level division? (which is a good first step to discuss)

[jorisvandenbossche]

Also, I know it will be annoying to keep those PRs open for a while (and keeping up to date with changes in master), so it might be preferred to merge them rather quickly.
But, I think we should discuss whether we only want to do this after 0.24 is released, given how close we are to the release (those PRs will break links, and if we still do some further changes we might continue breaking some links, and I think it would be even more annoying to break links multiple times if that is avoidable).

But maybe we should have a chat about how to move forward with the whole doc revamp (both content, theme, integration with website, etc)?

[datapythonista]

Not sure about 0.24, you'll know better. We're already changing the links of all the api/ since api.rst was splitted, so I'd say let's try to make the structure as close as what we want for the future for 0.24 already.

Reorganizing the current pages should be quite fast, there are almost PRs for everything, except few pages that we'll require some discussion. But some changes will surely happen in the future, as it'll take much longer to split pages, and fine tune the structure.

I'd say for what we discussed during the sprint, we're all happy with having few top level sections (user guide, development, whatsnew...), and also with the theme change. My opinion is that we should implement those, and then it'll be easier to discuss about the more "advanced" things like the home page, the navigation, the integration with the website... We tried twice to work on a one-shot documentation migration, and we failed, I think making the changes with small PRs is working much better.

I'm flexible on the basics and dsintro, I'd probably keep them in the getting started for now, as there is not much more than the 10min there, and may be move them later. But also happy having them in the user guide, it's just that the getting started will look quite poor, and I think users would probably appreciate some more recommended readings to start with (and those are the best we have for now).

[mbirdi]

I'd like to help with this one if I can. I'm a regular pandas user, but I have not contributed on github much and and would need some guidance.

[datapythonista]

@mbirdi can you better look at issues tagged as "good first issue". This requires discussion, and is not the best for a first time contributor.

[mbirdi]

@datapythonista Thank you! I'll look for that "first issue" tag.