Test the Exporting a Model from PyTorch to ONNX tutorial

[sekyondaMeta]

Test the following tutorial:
https://pytorch.org/tutorials/advanced/super_resolution_with_onnxruntime.html
Follow these steps:

1. Fork the pytorch/tutorials repo.
2. Switch to a new branch.
3. Run the tutorial as a python script:

python3 <path-to-tutorial>

4. Save the output into a .txt file. Did you see any errors? Add them to the output.
5. Run the tutorial in the Google Colab and save the outputs into a .txt file. Did you see any issues? Take screenshots of the issues.
6. Did the tutorial make sense? Can you suggest any improvements? Past all the outputs, screenshots, and suggestions for improvements in the issue and ping one of the reviewers. If your suggestion is approved, please submit a PR with the fix.

cc @xadupre @svekars @kit1980 @brycebortree

[diningeachox]

/assigntome

[sekyondaMeta]

@diningeachox Do you have an update on this? If not, we can unassign and re-assign someone

[diningeachox]

@sekyondaMeta Yes I will have an update soon (in an hour or so). Thanks for your patience.

[diningeachox]

@svekars @kit1980 @sekyondaMeta

I have run the ONNX superresolution tutorial locally on WSL and also in Google colab. There were no errors. Output .txt files have been attached for reference.
sr_onnx_tutorial_colab_output.txt
sr_onnx_tutorial_local_output.txt

Suggestions/Changes:

I have added a short code block to advanced_source/super_resolution_with_onnxruntime.py. Since ONNX models are supposed to accelerate inference speed, I have included a timing comparison between the original pytorch model and the ONNX model. I also increased the batch size to 64 so there would be a notable difference in the inference times. An example output of this is:

Inference of Pytorch model used 2.286515951156616 seconds
Inference of ONNX model used 1.4891066551208496 seconds

In addition, since this page is about super-resolution. I think it may be a good idea to place the original image (upscaled by bicubic interpolation) with the super-resolved image in the final section of the tutorial page. This way the reader can see the results of the model.

At the end of the tutorial, there's a link to ONNXruntime performance, but the linked page doesn't have a high-performance section anymore. My suggestion is to change the link given by https://github.com/microsoft/onnxruntime#high-performance to the new link https://onnxruntime.ai/docs/performance/

I have pushed these changes to my fork of the pytorch tutorials here under the docathon branch.

Please let me know if there's anything I missed!

[svekars]

Also, tagging @xadupre, @titaiwangms and @thiagocrepaldi

[titaiwangms]

@svekars @kit1980 @sekyondaMeta

I have run the ONNX superresolution tutorial locally on WSL and also in Google colab. There were no errors. Output .txt files have been attached for reference. sr_onnx_tutorial_colab_output.txt sr_onnx_tutorial_local_output.txt

Suggestions/Changes:

I have added a short code block to advanced_source/super_resolution_with_onnxruntime.py. Since ONNX models are supposed to accelerate inference speed, I have included a timing comparison between the original pytorch model and the ONNX model. I also increased the batch size to 64 so there would be a notable difference in the inference times. An example output of this is:

Inference of Pytorch model used 2.286515951156616 seconds
Inference of ONNX model used 1.4891066551208496 seconds

In addition, since this page is about super-resolution. I think it may be a good idea to place the original image (upscaled by bicubic interpolation) with the super-resolved image in the final section of the tutorial page. This way the reader can see the results of the model.

At the end of the tutorial, there's a link to ONNXruntime performance, but the linked page doesn't have a high-performance section anymore. My suggestion is to change the link given by https://github.com/microsoft/onnxruntime#high-performance to the new link https://onnxruntime.ai/docs/performance/

I have pushed these changes to my fork of the pytorch tutorials here under the docathon branch.

Please let me know if there's anything I missed!

Thanks! I think these are all making sense. @xadupre what do you think?

[xadupre]

It looks to me. Thanks for updating the link. My only question is about this line: https://github.com/diningeachox/tutorials/blob/docathon/advanced_source/super_resolution_with_onnxruntime.py#L232. I don't think to_numpy takes significant time in ort_inputs = {ort_session.get_inputs()[0].name: to_numpy(x)}. We usually take it out when we measure the time. But that's minor. Since the time will show up during the generation of the documentation, I wonder if you use a specific machine to avoid weird figures to appear.

[diningeachox]

Thank you for your feedback @xadupre, @titaiwangms. What do you mean by avoiding weird figures? Do you mean enforcing a specific output format for the time? (Like 3 decimal places for example)

[xadupre]

If the documentation is running multiple processes in parallel on the same machine, figures may be different. If the documentation is not run on the same machine everytime, the figures may be very different everytime the documentation is generated...

[diningeachox]

I have taken the to_numpy() out of the timing. Just to make sure I understand this better. I used Google colab and my local machine with WSL to run the timing scripts. Do you have a suggestion on how to make sure the timing is done on a designated machine? I'm not sure on which machines the official docs are generated. (I assume sphinx-gallery is run in a Docker container when all the .py files when a PR is pushed, would that be accurate?)

For reference here's another tutorial doc that does a timing script, however the output of the time is not captured on the HTML page: https://pytorch.org/tutorials/recipes/recipes/amp_recipe.html
So I'm thinking of just including the timing script and just saying in the doc something along the lines of you should see an improvement of up to 2x depending on batch size. Would that work? If so I would like to submit a PR.

[xadupre]

That works. Thanks for doing it.