New module proposal: plotly.io

[jonmmease]

Overview

I propose that we introduce a new module, plotly.io, to house a new suite of functions related to the input and output of plotly figure objects.

Why now?

There are two immediate needs motivating this proposal.

1. It's time to integrate orca for static image export
2. For integration with the JupyterLab plotly-renderer and JupyterLab chart editor, we need a function to write out the JSON representation of a figure with the proper plotly metadata

How do things work now?

Here's in inventory of some of our current methods that can perform I/O operations:

• plotly.plotly.plot
  • Save figure to plot.ly (with optional filename) and return URL
  • Optionally open in browser
• plotly.offline.plot
  • Save html to a local file
  • or return an HTML div string
  • optionally open in browser
• plotly.image.save_as
  • Save image to a local file with specified image parameters by making a request to plot.ly
• plotly.get_figure
  • Convert plot.ly URL into a local figure object

Why add something new?

By my taste, these APIs are a bit scattered and not very discoverable by users without consulting external documentation. It's also unclear (to me at least) where additional I/O methods, like to two listed above, should be added.

There are good reasons that we ended up where we are (e.g. there was no offline mode when the plotly.plotly module was developed), but I'd like to propose we adopt a new approach moving forward.

The goal is that the options for saving/loading/converting figures should consistent and easily discoverable without the need to consult external documentation. By triggering tab-completion on the plotly.io module, users should be able to learn about all of the import/export options available to them.

Design

I propose that the initial functions in the plotly.io module have one of 4 prefixes, followed by the format name:

• read_{format}(file, ...) -> Figure: These methods all accept, as the first argument, either a string or a read-able file object. Strings are interpreted as local file paths, other objects are read from. These methods all return a Figure object.
• write_{format}(fig, file, ...) -> None: These methods all accept the Figure to be written as the first argument. The second argument is either a string or a write-able file object. Strings are interpreted as local file paths, other objects are written to. These methods raise on write failures and return None.
• from_{format}(data, ...) -> Figure: These methods all accept, as the first argument, some in-memory Python object and return a Figure object. These methods don't interact with the file system and have no side effects.
• to_{format}(fig, ...) -> Any: These methods all accept the Figure to be converted as the first argument. They return an in memory Python object corresponding to the format. These methods don't interact with the file system and have no side effects.

The ellipses (...) in the signatures above represent additional arguments/options that are specific to given format.

In addition to their presence as standalone functions in plotly.io, there would also be write_{format} and to_{format} methods on Figure objects. This way, people could use tab completion on a figure itself to learn how to save/convert it.

Examples

Here are some examples of functions that could follow these conventions

import plotly.graph_objs as go
import plotly.io as plio
fig = go.Figure(...)

Save to HTML file and embed plotly.js

plio.write_html(fig, 'somewhere.html', include_plotlyjs=True)
# or
fig.write_html('somewhere.html', include_plotlyjs=True)

Build an HTML div string for figure, and don't embed plotly.js

div_str = plio.to_html(fig, output_type='div', include_plotlyjs=False)
# or
div_str = fig.to_html(output_type='div', include_plotlyjs=False)

Save figure to a file as an svg image of a specific size (using orca)

plio.write_image(fig, 'somewhere.svg', format='svg', width=500, height=400)
# or
fig.write_image('somewhere.svg', format='svg', width=500, height=400)

Build a bytes string representation of the Figure converted to a PNG image

png_bytes = plio.to_image(fig, format='png')

Build a PIL Image object representing the figure as a static image

pil_img = plio.to_pil(fig, format='png')

Save to json file

plio.write_json(fig, 'fig.json')

Save as gzipped json file to an HDFS filesystem using PyArrow

import pyarrow as pa
fs = pa.hdfs.connect(host, port, user=user, ...)
with fs.open('/path/to/fig.json.gz', 'rb') as f:
    plio.write_json(fig, f, compression='gzip')

Convert to json string

json_str = plio.to_json(fig)

Convert figure to plot.ly url

plot_ly_url = plio.to_plot_ly(fig)

Get figure from plot.ly url

fig = plio.from_plot_ly(plot_ly_url)

Convert matplotlib/altair/holoviews/other object to plotly figure

fig = plio.from_matplotlib(mpl_fig)
fig = plio.from_holoviews(hv_obj)
fig = plio.from_altair(altair_fig)

What happens to plot?

Nothing, it works fine and there's no reason to change it. Some of these new methods could directly call the current plot methods, or we could look at refactoring the internals a bit so that both plot and io methods share most of their logic.

What about displaying plots?

I don't intend for any of the to_{format}/save_{format} functions to display the figure. So initially, the plot and iplot methods would continue to serve that purpose.

One option would be to eventually add a plotly.io.show method that is capable of displaying figures using different "backends" (similar in spirit to matplotlib). We actually already have 5 backends for displaying figures:

1. plot.ly in browser (plotly.plotly.plot)
2. plot.ly in notebook iframe (plotly.plotly.iplot)
3. offline in browser (plotly.offline.plot)
4. offline in notebook (plotly.offline.iplot)
5. display as ipywidget (plotly.graph_objs.FigureWidget)

Once we have orca integrated, we could even add a static image backend if people were interested in that.

Milestone

I propose adding the plotly.io module with support for json and orca-based image formats in version 3.2.0. Depending on how much work it turns out to be, it would be nice to also populate the module with existing html and plot_ly support as well.

Please chime in if you have opinions on this!

cc: @chriddyp @cldougl @Kully @nicolaskruchten @jackparmer @bcdunbar

[nicolaskruchten]

Big +1 for the high-level goals and the initiative. The API looks fine to me in principle as well :) I love the harmonization of all of these things.

[jackparmer]

👍 from me too. Love everything about this. Could really clear up a major source of confusion for Plotly's Python library users. Would love to see a parallel issue for documentation updates.

Once we have orca integrated, we could even add a static image backend if people were interested in that.

We should definitely be shooting for this 🌠 - the last frontier for eclipsing some of the legacy technical computing chart libraries 🌑

[jonmmease]

@jackparmer, two closely related documentation issues are:

• Version the docs with plotly.py: Python: Version the docs with plotly.py documentation#990
• Port examples to offline: Python: Port examples to offline documentation#992

It will help a lot if we can work out a way to release the docs for each (minor) version. This way the docs don't have to be backwards compatible, and we can update them to reflect the current best practices of each version.

If we make progress on the plotly.io.show front then this second issue just becomes porting to the show interface.

[cldougl]

@jonmmease

if we can work out a way to release the docs for each (minor) version. This way the docs don't have to be backwards compatible, and we can update them to reflect the current best practices of each version.

wouldn't minor versions be backwards compatible?

[jonmmease]

@cldougl
Yes, minor versions of the codebase will be backwards compatible (Code written for 3.1 will work with 3.2). But the documentation should be forwards compatible. For example , the docs for version 3.1 are valid for the code of 3.2, but the 3.2 docs won't necessarily be valid for version 3.1 of the code.

I would like to be able to update the docs with each version to reflect the current recommended way to do things, even if the current recommended way didn't exist in the last minor version.

Does that make sense?

[jonmmease]

The beginning of the plotly.io module (for image export) was merged in #1120 and will be released in 3.2.0.

So I'm going to remove the 3.2.0 milestone, but leave the issue open until all of our legacy I/O functionality is reproduced in this module.