DOC: Revise merging.rst with graphical examples

[sinhrks]

Closes #9301. Though further table size adjustment / plotter module refactoring is required, I'd like to ask any feedbacks for current output.

I've prepared table plotter under utils.doctools, but is there anywhere more appropriate?

cc @JanSchulz

[jreback]

@sinhrks looks gr8!

• can you add a release note (and put in the highlites section with a link to this page, to show this off)
• I think that you need to not render the result as that results in a blank line? (e.g. maybe need to change them to code-blocks?)

[jankatins]

@sinhrks Wow, this looks great!

[jorisvandenbossche]

Wonderful!
I will try to find some time next week to review this

[sinhrks]

Adjusted table size and updated rendered output.

I think that you need to not render the result as that results in a blank line?

Indeed, should not. But using code block prevent images from rendered automatically. Does anyone know the way to hide block but output plot using ipython-directive? Otherwise, my idea is output static images during doc build, and link them from this doc.

[jreback]

pls add a note in the highlites section of whatsnew 0.16.1 with a pointer to this pages.

@sinhrks lgtm merge away

[sinhrks]

Thanks for the review. Added release note in highlites.

@JanSchulz, @jorisvandenbossche Let me merge this if there is no further suggestions until this weekend.

[jorisvandenbossche]

@sinhrks I just wanted to finally take a look, but I don't see the image with the output anymore (get some message that the image contains errors).

[sinhrks]

@jorisvandenbossche Strange, I can built doc on my local. Can you attach error messages?

[jorisvandenbossche]

No problem anymore. I meant the image of the built docs you posted here in the PR, but now I see it (don't know what was going wrong yesterday)

Do you have any idea where the empty code cells are coming from? As normally, the ipython directive with the suppress option should work for that (to execute something, but not show the output).

• One comment: I would make the tables (certainly in the beginning) a bit smaller as 3 times 4x4 (the smaller, the easier to overview what happened I think
• Wishlist: color the data frames, so you can more easily see in the result from which dataframe the data is coming (but certainly not needed for this PR ..)

One thing I was thinking. Towards the future, would it be simpler/more flexible to use html instead of matplotlib figure? For that we could use df.to_html() (would also render multi-index better), and then try to find some way to insert that html output into the rst during build time, so it can just be rendered. But not sure if this would be an easier approach as the matplotlib one now.

[jankatins]

@jorisvandenbossche pandas html tables are probably fine in html docs but I wouldn't count on such constructs being able to be converted to PDF (e.g. pandas tables in markdown blow up horrible if converted with pandoc...)

That's actually one reason why it would be nice to have a df.to_markdown() or df.to_rst().

[shoyer]

I agree with Joris that this would look a little nicer if it leveraged the existing HTML renderer instead of matplotlib. But this is still a nice improvement.

[sinhrks]

@jorisvandenbossche Thanks for the comments. Made tables little smaller and updated the rendered output. I still see blank blocks even if I use suppress. A workaround we can do in v0.16.1 is to use static images.

The reason I used mpl is to layout multiple tables to understand merging operation easier. It gets better if it can be done using html (or rst).

[jreback]

@sinhrks pls merge when ready

[sinhrks]

@jorisvandenbossche OK for this?

[jorisvandenbossche]

yep, certainly for a first pass!
we should try to remove the empty blocks, but I also don't have the time today, so we can always improve later on (eg also the coloring)

[jorisvandenbossche]

@JanSchulz good point about the pdf output. But rst tables do work in latex output. So in principle we could specify in the sphinx build process that the output for this directive for latex should be different as for html.

[jorisvandenbossche]

My idea was a bit like this:

• extend the ipython directive to have a mode where the output of that node is 'injected' into the document (so the output is parsed as rst). In this way you can add normal text to the html file that depends on the ipython shell variables.
• use df.to_html() to output it, and possibly do some styling to the result (colors indicating from which dataframe it came)
• use some other output for the latex docs

But, I don't know if this is actually a better approach (easier, better result, ..) than what @sinhrks now already did