TorchServe IPEX Blog #2 #2079

min-jean-cho · 2022-10-12T07:52:16Z

No description provided.

netlify · 2022-10-12T07:54:50Z

✅ Deploy Preview for pytorch-tutorials-preview ready!

Name	Link
🔨 Latest commit	`786417e`
🔍 Latest deploy log	https://app.netlify.com/sites/pytorch-tutorials-preview/deploys/6349ae04a66af800092343cf
😎 Deploy Preview	https://deploy-preview-2079--pytorch-tutorials-preview.netlify.app/intermediate/torchserve_with_ipex_2
📱 Preview on mobile	Toggle QR Code... Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify site settings.

min-jean-cho · 2022-10-12T08:27:39Z

@msaroufim, PR for our second joint TorchServe IPEX blog. Could you please help review, thanks!

msaroufim

Fantastic blog (as usual) - thank you @min-jean-cho

svekars

A few editorial suggestions. Also, need to fix these links:

/var/lib/jenkins/workspace/intermediate/torchserve_with_ipex.rst:2: WARNING: Duplicate explicit target name: "intel® vtune™ profiler".
 /var/lib/jenkins/workspace/intermediate/torchserve_with_ipex_2.rst:3: WARNING: Duplicate explicit target name: "config.properties"

Probably, just need to add double underscores in the link (my link <url>__)

svekars · 2022-10-12T21:32:26Z

intermediate_source/torchserve_with_ipex_2.rst

+Grokking PyTorch Intel CPU performance from first principles (Part 2)
+=====================================================================
+
+Authors: Min Jean Cho, Jing Xu, Mark Saroufim


let's make this links to Github profiles.

Thanks @svekars for the review. Have done so.

intermediate_source/torchserve_with_ipex_2.rst

svekars · 2022-10-12T22:12:31Z

intermediate_source/torchserve_with_ipex_2.rst

+
+Throughout this blog, we'll use `Top-down Microarchitecture Analysis (TMA) <https://www.intel.com/content/www/us/en/develop/documentation/vtune-cookbook/top/methodologies/top-down-microarchitecture-analysis-method.html>`_ to profile and show that the Back End Bound (Memory Bound, Core Bound) is often the primary bottleneck for under-optimized or under-tuned deep learning workloads, and we'll demonstrate optimization techniques via Intel® Extension for PyTorch* for improving Back End Bound. We'll also use `Intel® VTune™ Profiler's Instrumentation and Tracing Technology (ITT) <https://github.com/pytorch/pytorch/issues/41001>`_ to profile at finer granularity.
+
+*****************


We typically don't add a table of contents like this because it's autogenerated on the right hand side under Shortcuts. It is difficult to keep a manual TOC like this up-to-date with any future changes in the content.

I'm fine with either options, but I thought TOC would help when readers read the intro since this tutorial has lots of content. Let me know what you'd prefer !

I suggest to remove it as we don't have it in any other tutorials.

svekars · 2022-10-12T22:13:54Z

intermediate_source/torchserve_with_ipex_2.rst

+        - Intel® Extension for PyTorch* Optimizations  
+        - Intel® Extension for PyTorch* with TorchServe 
+        - Exercise
+


Where is prerequisites? In the TOC above they are listed as first section.

Do we need to have an explicit heading with "Prerequisites" ? The TOC is more of intended for a glance of content at one shot, rather than a link to the sections.

It would be good to have a prerequisites section and list what the user need to know / have installed before they can complete this tutorial.

OK. Let me try moving around some paragraphs, to heave content under each heading.

intermediate_source/torchserve_with_ipex_2.rst

svekars · 2022-10-12T22:29:05Z

Also, I've noticed a few images are too wide which creates formatting issues:

Co-authored-by: Svetlana Karslioglu <[email protected]>

min-jean-cho · 2022-10-12T23:57:31Z

Also, I've noticed a few images are too wide which creates formatting issues:

Thank you @svekars for checking on this. I've fixed the img sizes, could you check if they look fine now?

We typically don't add a table of contents like this because it's autogenerated on the right hand side under Shortcuts.

Could I have a look at this expanded (maybe a screenshot)? I want to see if they show up as I've intended.

min-jean-cho · 2022-10-13T00:07:19Z

/var/lib/jenkins/workspace/intermediate/torchserve_with_ipex.rst:2: WARNING: Duplicate explicit target name: "intel® vtune™ profiler".
/var/lib/jenkins/workspace/intermediate/torchserve_with_ipex_2.rst:3: WARNING: Duplicate explicit target name: "config.properties"
Probably, just need to add double underscores in the link (my link __)

May I know the source of this warning if it's from an internal CI? I've tried fixing to my link __ where applicable, but not sure if this will resolve that warning.

svekars · 2022-10-12T22:31:27Z

intermediate_source/torchserve_with_ipex_2.rst

+Additionally, let's profile with PyTorch Profiler. 
+
+.. figure:: /_static/img/torchserve-ipex-images-2/13.png
+   :width: 150%


can we please set this image to not more than 100% width since it causes a formatting issue? They should be clickable so the user should be able to enlarge them.

All images are already changed to less than 100% width. The comment is on outdated file, prior to this change.

svekars · 2022-10-12T22:32:11Z

intermediate_source/torchserve_with_ipex_2.rst

+Additionally, let's profile with PyTorch Profiler. 
+
+.. figure:: /_static/img/torchserve-ipex-images-2/15.png
+   :width: 150%


Can we please set the width to not more than 100%

svekars · 2022-10-12T22:33:13Z

intermediate_source/torchserve_with_ipex_2.rst

+When tuning CPU for optimal performance, it's useful to know where the bottleneck is. Most CPU cores have on-chip Performance Monitoring Units (PMUs). PMUs are dedicated pieces of logic within a CPU core that count specific hardware events as they occur on the system. Examples of these events may be Cache Misses or Branch Mispredictions. PMUs are used for Top-down Microarchitecture Analysis (TMA) to identify the bottlenecks. TMA consists of hierarchical levels as shown: 
+
+.. figure:: /_static/img/torchserve-ipex-images-2/2.png
+   :width: 130%


Please set the width to 100%

intermediate_source/torchserve_with_ipex_2.rst

svekars · 2022-10-13T03:59:44Z

intermediate_source/torchserve_with_ipex_2.rst

+
+Throughout this blog, we'll use `Top-down Microarchitecture Analysis (TMA) <https://www.intel.com/content/www/us/en/develop/documentation/vtune-cookbook/top/methodologies/top-down-microarchitecture-analysis-method.html>`_ to profile and show that the Back End Bound (Memory Bound, Core Bound) is often the primary bottleneck for under-optimized or under-tuned deep learning workloads, and we'll demonstrate optimization techniques via Intel® Extension for PyTorch* for improving Back End Bound. We'll also use `Intel® VTune™ Profiler's Instrumentation and Tracing Technology (ITT) <https://github.com/pytorch/pytorch/issues/41001>`_ to profile at finer granularity.
+
+*****************


I suggest to remove it as we don't have it in any other tutorials.

svekars · 2022-10-13T04:04:56Z

intermediate_source/torchserve_with_ipex_2.rst

+
+1. The feature has to be explicitly enabled by with *torch.autograd.profiler.emit_itt()*.
+
+*********************************************


Right, it looks like you have three headings on lines 66 - 71, meaning that TorchServe with Intel® Extension for PyTorch* and Leveraging Advanced Launcher Configuration: Memory Allocator are empty. Can we add a short overview paragraph under each of these headings?

min-jean-cho · 2022-10-14T18:41:54Z

Thanks @svekars for the review, I have updated according to your comments. The updates are:

Fixed syntax of title and headings
Added content under each heading (moved around some existing paragraphs)
Removed TOC
all img sizes are less than 100%

svekars

LGTM! Thank you for addressing the comments.

min-jean-cho · 2022-10-14T20:23:57Z

Thanks @svekars for the thorough review - grammatical errors, also I realize it's easier to read now with content under each heading. Thanks :)

Will you be merging this?

min-jean-cho · 2022-10-17T17:38:20Z

index.rst

+   :card_description: A case study on the TorchServe inference framework optimized with Intel® Extension for PyTorch (Part 2).
+   :image: _static/img/thumbnails/cropped/generic-pytorch-logo.png
+   :link: intermediate/torchserve_with_ipex_2
+   :tags: Model-Optimization,Production


Hi @svekars, I see this tutorial has been published to pytorch.org here. However, I don't see link to this tutorial in the left nav, could you pls. help check on this?

Fix is coming: #2085

min-jean-cho added 4 commits October 12, 2022 00:02

Create torchserve_with_ipex_2.rst

b5488b9

create torchserve-ipex-images-2

500dc2b

add torchserve-ipex-images-2 png

51e405b

add png

5508e1b

facebook-github-bot added the cla signed label Oct 12, 2022

min-jean-cho added 3 commits October 12, 2022 01:14

update wording

b359eb7

update wording

5df5bed

update wording

9124a50

min-jean-cho marked this pull request as ready for review October 12, 2022 08:26

svekars requested a review from msaroufim October 12, 2022 15:27

update wording

e76c82e

msaroufim approved these changes Oct 12, 2022

View reviewed changes

min-jean-cho added 2 commits October 12, 2022 10:48

add tutorial to matrix and left nav

de70676

Delete placeholder

a0b6350

svekars reviewed Oct 12, 2022

View reviewed changes

svekars requested a review from malfet October 12, 2022 22:24

min-jean-cho and others added 11 commits October 12, 2022 15:53

link github

d06933a

tutorial -> blog

d905bba

Co-authored-by: Svetlana Karslioglu <[email protected]>

tutorial -> blog

bece54b

Co-authored-by: Svetlana Karslioglu <[email protected]>

grammar fix

8ce095a

Co-authored-by: Svetlana Karslioglu <[email protected]>

grammar fix

a04fa71

Co-authored-by: Svetlana Karslioglu <[email protected]>

grammar fix

c71fee4

Co-authored-by: Svetlana Karslioglu <[email protected]>

blog -> tutorial

fe7dbf1

Co-authored-by: Svetlana Karslioglu <[email protected]>

un-tuned -> untuned, submetircs -> sub-metrics

32cf25d

blog -> tutorial, we'll -> we will

07a6e8d

with torch.autograd.profiler.emit_itt()

bfe11ae

grammar fix

aaff8cb

Co-authored-by: Svetlana Karslioglu <[email protected]>

min-jean-cho and others added 14 commits October 12, 2022 16:18

we'll -> we will

bdd965c

Co-authored-by: Svetlana Karslioglu <[email protected]>

we'll -> we will

bca2544

Co-authored-by: Svetlana Karslioglu <[email protected]>

2 -> two

15717d8

Co-authored-by: Svetlana Karslioglu <[email protected]>

we'll -> we will

7c33548

Co-authored-by: Svetlana Karslioglu <[email protected]>

e.g., -> for example, refer to -> see

c936e12

Co-authored-by: Svetlana Karslioglu <[email protected]>

etc -> and more

a276f35

we'll -> we will

129e933

Co-authored-by: Svetlana Karslioglu <[email protected]>

we'll -> we will

0661896

Co-authored-by: Svetlana Karslioglu <[email protected]>

we'll we will

22858fa

Co-authored-by: Svetlana Karslioglu <[email protected]>

un-tuned -> untuned

0abdb79

Co-authored-by: Svetlana Karslioglu <[email protected]>

take-aways -> conclusion

ee1e3d7

Co-authored-by: Svetlana Karslioglu <[email protected]>

blog -> tutorial, we've -> we have

bcc847d

Co-authored-by: Svetlana Karslioglu <[email protected]>

fix linking

cb5a935

Co-authored-by: Svetlana Karslioglu <[email protected]>

fix png sizes

ddaa4fe

my lin <url>__

df74a9f

min-jean-cho requested review from svekars and removed request for malfet October 14, 2022 07:01

svekars reviewed Oct 14, 2022

View reviewed changes

min-jean-cho added 2 commits October 14, 2022 11:24

(1) add content under each heading (2) fix heading syntax

15c6b3f

update

4e189ba

Merge branch 'master' into minjean/torchserve_with_ipex_2

a1f45ae

min-jean-cho requested a review from svekars October 14, 2022 18:42

blog -> tutorial

786417e

svekars approved these changes Oct 14, 2022

View reviewed changes

msaroufim merged commit be2eeaa into pytorch:master Oct 14, 2022

min-jean-cho commented Oct 17, 2022

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

TorchServe IPEX Blog #2 #2079

TorchServe IPEX Blog #2 #2079

min-jean-cho commented Oct 12, 2022

netlify bot commented Oct 12, 2022 •

edited

Loading

min-jean-cho commented Oct 12, 2022 •

edited

Loading

msaroufim left a comment

svekars left a comment

svekars Oct 12, 2022

min-jean-cho Oct 12, 2022

svekars Oct 12, 2022

min-jean-cho Oct 12, 2022

svekars Oct 13, 2022

svekars Oct 12, 2022

min-jean-cho Oct 12, 2022

svekars Oct 14, 2022

min-jean-cho Oct 14, 2022

svekars commented Oct 12, 2022

min-jean-cho commented Oct 12, 2022

min-jean-cho commented Oct 13, 2022

svekars Oct 12, 2022

min-jean-cho Oct 14, 2022

svekars Oct 12, 2022

svekars Oct 12, 2022

svekars Oct 13, 2022

svekars Oct 13, 2022

min-jean-cho commented Oct 14, 2022

svekars left a comment

min-jean-cho commented Oct 14, 2022

min-jean-cho Oct 17, 2022

svekars Oct 17, 2022


		Throughout this blog, we'll use `Top-down Microarchitecture Analysis (TMA) <https://www.intel.com/content/www/us/en/develop/documentation/vtune-cookbook/top/methodologies/top-down-microarchitecture-analysis-method.html>`_ to profile and show that the Back End Bound (Memory Bound, Core Bound) is often the primary bottleneck for under-optimized or under-tuned deep learning workloads, and we'll demonstrate optimization techniques via Intel® Extension for PyTorch* for improving Back End Bound. We'll also use `Intel® VTune™ Profiler's Instrumentation and Tracing Technology (ITT) <https://github.com/pytorch/pytorch/issues/41001>`_ to profile at finer granularity.

		*****************


		1. The feature has to be explicitly enabled by with torch.autograd.profiler.emit_itt().

		*********************************************

TorchServe IPEX Blog #2 #2079

TorchServe IPEX Blog #2 #2079

Conversation

min-jean-cho commented Oct 12, 2022

netlify bot commented Oct 12, 2022 • edited Loading

✅ Deploy Preview for pytorch-tutorials-preview ready!

min-jean-cho commented Oct 12, 2022 • edited Loading

msaroufim left a comment

Choose a reason for hiding this comment

svekars left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

svekars commented Oct 12, 2022

min-jean-cho commented Oct 12, 2022

min-jean-cho commented Oct 13, 2022

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

min-jean-cho commented Oct 14, 2022

svekars left a comment

Choose a reason for hiding this comment

min-jean-cho commented Oct 14, 2022

Choose a reason for hiding this comment

Choose a reason for hiding this comment

netlify bot commented Oct 12, 2022 •

edited

Loading

min-jean-cho commented Oct 12, 2022 •

edited

Loading