Skip to content

[New Sample] Intel Extension for PyTorch Getting Started #1314

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 2 commits into from
Jan 26, 2023
Merged

[New Sample] Intel Extension for PyTorch Getting Started #1314

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
2eb746f
add new ipex GSG notebook for dGPU
louie-tsai Jan 26, 2023
3fad686
Update sample.json
louie-tsai Jan 26, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
205 changes: 107 additions & 98 deletions ...cs/Getting-Started-Samples/Intel_Extension_For_PyTorch_GettingStarted/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,134 +1,143 @@
# `Intel Extension For PyTorch Hello World` Sample
Intel® Extension for PyTorch\* extends PyTorch with optimizations for extra performance boost on Intel hardware. Most of the optimizations will be included in stock PyTorch releases eventually, and the intention of the extension is to deliver up-to-date features and optimizations for PyTorch on Intel hardware, examples include AVX-512 Vector Neural Network Instructions (AVX512 VNNI) and Intel® Advanced Matrix Extensions (Intel® AMX).
# `Intel® Extension for PyTorch* Getting Started` Sample

| Optimized for | Description
|:--- |:---
| OS | Linux\* Ubuntu\* 18.04
| Hardware | Intel® Xeon® Scalable Processor family
| Software | Intel® AI Analytics Toolkit (AI Kit)
| What you will learn | How to get started with Intel® Optimization for PyTorch
| Time to complete | 15 minutes
Intel® Extension for PyTorch* extends PyTorch* with optimizations for extra performance boost on Intel hardware. Most of the optimizations will be included in stock PyTorch* releases eventually, and the intention of the extension is to deliver up-to-date features and optimizations for PyTorch* on Intel hardware, examples include AVX-512 Vector Neural Network Instructions (AVX512 VNNI) and Intel® Advanced Matrix Extensions (Intel® AMX).

## Purpose
This sample code shows how to get started with Intel Optimization for PyTorch. It implements an example neural network with one convolution layer, one normalization layer and one ReLU layer. Developers can quickly build and train a PyTorch\* neural network using a simple python code. Also, by controlling the build-in environment variable, the sample attempts to show how Intel® DNNL Primitives are called explicitly and their performance during PyTorch\* model training and inference with Intel® Extension for PyTorch\*.

Intel® Extension for PyTorch\* is available as part of Intel® AI Analytics Toolkit. For more information on the optimizations as well as performance data, see [this blog](http://software.intel.com/en-us/articles/intel-and-facebook-collaborate-to-boost-pytorch-cpu-performance).
This sample contains a Jupyter* NoteBook that guides you through the process of running a PyTorch* inference workload on both GPU and CPU by using Intel® AI Analytics Toolkit (AI Kit) and also analyze the GPU and CPU usage via Intel® oneAPI Deep Neural Network Library (oneDNN) verbose logs.

More examples can be found at [Examples](https://intel.github.io/intel-extension-for-pytorch/tutorials/examples.html)
| Area | Description
|:--- |:---
| What you will learn | How to get started with Intel® Extension for PyTorch
| Time to complete | 15 minutes

## Key implementation details
This Hello World sample code is implemented for CPU using the Python language.
## Prerequisites

Please **export the environment variable `DNNL_VERBOSE=1`** to display the deep learning primitives trace during execution.
| Optimized for | Description
|:--- |:---
| OS | Ubuntu* 22.04
| Hardware | Intel® Xeon® scalable processor family <br> Intel® Data Center GPUs
| Software | Intel® AI Analytics Toolkit (AI Kit)

### Notes
- The test dataset is inherited from `torch.utils.data.Dataset`.
- The model is inherited from `torch.nn.Module`.
## Purpose

## License
Code samples are licensed under the MIT license. See
[License.txt](https://github.com/oneapi-src/oneAPI-samples/blob/master/License.txt) for details.
This sample code demonstrates how to begin using the Intel® Extension for PyTorch*.

Third party program Licenses can be found here: [third-party-programs.txt](https://github.com/oneapi-src/oneAPI-samples/blob/master/third-party-programs.txt)
The sample implements an example neural network with one convolution layer, one normalization layer, and one ReLU layer.

You can quickly build and train a PyTorch* neural network using the simple Python code. Also, by controlling the built-in environment variable, the sample attempts to show how Intel® DNNL Primitives are called explicitly and shows the performance during PyTorch* model training and inference with Intel® Extension for PyTorch*.

## How to Build and Run
The Jupyter notebook in this sample also guides users how to change PyTorch* codes to run on Intel® Data Center GPU family and how to validate the GPU or CPU usages for PyTorch* workloads on Intel CPU or GPU.

> **Note**: If you have not already done so, set up your CLI
> environment by sourcing the `setvars` script located in
> the root of your oneAPI installation.
>
> Linux Sudo: . /opt/intel/oneapi/setvars.sh
>
> Linux User: . ~/intel/oneapi/setvars.sh
>
> Windows: C:\Program Files(x86)\Intel\oneAPI\setvars.bat
>**Note**: Intel® Extension for PyTorch* is available as part of Intel® AI Analytics Toolkit. For more information on the optimizations as well as performance data, see [*Intel and Facebook* collaborate to boost PyTorch* CPU performance*](http://software.intel.com/en-us/articles/intel-and-facebook-collaborate-to-boost-pytorch-cpu-performance).
>
>For more information on environment variables, see Use the setvars Script for [Linux or macOS](https://www.intel.com/content/www/us/en/develop/documentation/oneapi-programming-guide/top/oneapi-development-environment-setup/use-the-setvars-script-with-linux-or-macos.html), or [Windows](https://www.intel.com/content/www/us/en/develop/documentation/oneapi-programming-guide/top/oneapi-development-environment-setup/use-the-setvars-script-with-windows.html).
>Find more examples in the [*Examples*](https://intel.github.io/intel-extension-for-pytorch/cpu/latest/tutorials/examples.html) topic of the [*Intel® Extension for PyTorch* Documentation*](https://intel.github.io/intel-extension-for-pytorch).

1. Activate the conda environment:

```
conda activate pytorch
```

2. Activate conda environment Without Root Access (Optional)

By default, the Intel AI Analytics toolkit is installed in the inteloneapi
folder, which requires root privileges to manage it. If you would like to
bypass using root access to manage your conda environment, then you can clone
your desired conda environment using the following command:
## Key Implementation Details

```
conda create --name user_pytorch --clone pytorch
```
The sample uses pretrained model provided by Intel and published as part of [Intel Model Zoo](https://github.com/IntelAI/models). The example also illustrates how to utilize TensorFlow* and Intel® Math Kernel Library (Intel® MKL) runtime settings to maximize CPU performance on ResNet50 workload.

Then activate your conda environment with the following command:
- The Jupyter Notebook, `ResNet50_Inference.ipynb`, is implemented for both CPU and GPU using Intel® Extension for PyTorch*.
- The `Intel_Extension_For_PyTorch_Hello_World.py` script is implemented for CPU using the Python language.
- You must export the environment variable `DNNL_VERBOSE=1` to display the deep learning primitives trace during execution.

```
conda activate user_pytorch
```
> **Note**: The test dataset is inherited from `torch.utils.data.Dataset`, and the model is inherited from `torch.nn.Module`.

3. Navigate to the directory with the sample:
```
cd ~/oneAPI-samples/AI-and-Analytics/Getting-Started-Samples/Intel_Extension_For_PyTorch_GettingStarted
```
## Run the `Intel® Extension for PyTorch* Getting Started` Sample

4. Run the Python script
To run the program on Linux\*, Windows\* and MacOS\*, type the following command in the terminal with Python installed:
### On Linux*

```
python Intel_Extension_For_PyTorch_Hello_World.py
```
> **Note**: If you have not already done so, set up your CLI
> environment by sourcing the `setvars` script in the root of your oneAPI installation.
>
> Linux*:
> - For system wide installations: `. /opt/intel/oneapi/setvars.sh`
> - For private installations: ` . ~/intel/oneapi/setvars.sh`
> - For non-POSIX shells, like csh, use the following command: `bash -c 'source <install-dir>/setvars.sh ; exec csh'`
>
> For more information on configuring environment variables, see *[Use the setvars Script with Linux* or macOS*](https://www.intel.com/content/www/us/en/develop/documentation/oneapi-programming-guide/top/oneapi-development-environment-setup/use-the-setvars-script-with-linux-or-macos.html)*.

You will see the DNNL verbose trace after exporting the `DNNL_VERBOSE`:
#### Activate Conda

```
export DNNL_VERBOSE=1
```
1. Activate the conda environment:
```
conda activate pytorch
```

Please find more information about the mkldnn log [here](https://oneapi-src.github.io/oneDNN/dev_guide_verbose.html).
2. Activate conda environment without Root access (Optional).

By default, the AI Kit is installed in the `/opt/intel/oneapi` folder and requires root privileges to manage it.

### Example of Output
With successful execution, it will print out `[CODE_SAMPLE_COMPLETED_SUCCESSFULLY]` in the terminal.
You can choose to activate Conda environment without root access. To bypass root access to manage your Conda environment, clone and activate your desired Conda environment using the following commands similar to the following.
```
conda create --name user_pytorch --clone pytorch
```
Then activate your conda environment with the following command:
```
conda activate user_pytorch
```
#### Run the Script

### Running The Sample In DevCloud (Optional)
1. Navigate to the directory with the sample.
```
cd ~/oneAPI-samples/AI-and-Analytics/Getting-Started-Samples/Intel_Extension_For_PyTorch_GettingStarted
```
2. Run the Python script.
```
python Intel_Extension_For_PyTorch_Hello_World.py
```
You will see the DNNL verbose trace after exporting the `DNNL_VERBOSE`:
```
export DNNL_VERBOSE=1
```
>**Note**: Read more information about the mkldnn log at [https://oneapi-src.github.io/oneDNN/dev_guide_verbose.html](https://oneapi-src.github.io/oneDNN/dev_guide_verbose.html).

#### Run the Jupyter Notebook

1. Change to the sample directory.
2. Launch Jupyter Notebook.
```
jupyter notebook --ip=0.0.0.0 --port 8888 --allow-root
```
3. Follow the instructions to open the URL with the token in your browser.
4. Locate and select the Notebook.
```
ResNet50_Inference.ipynb
```
5. Change your Jupyter Notebook kernel to **PyTorch**.
6. Run every cell in the Notebook in sequence.

Please refer to [using samples in DevCloud](https://github.com/intel-ai-tce/oneAPI-samples/blob/devcloud/AI-and-Analytics/README.md#using-samples-in-intel-oneapi-devcloud) for general usage instructions.
### Troubleshooting

### Submit The Sample in Batch Mode
If you receive an error message, troubleshoot the problem using the **Diagnostics Utility for Intel® oneAPI Toolkits**. The diagnostic utility provides configuration and system checks to help find missing dependencies, permissions errors, and other issues. See the *[Diagnostics Utility for Intel® oneAPI Toolkits User Guide](https://www.intel.com/content/www/us/en/develop/documentation/diagnostic-utility-user-guide/top.html)* for more information on using the utility.

1. Navigate to the directory with the TensorFlow sample:
```
cd ~/oneAPI-samples/AI-and-Analytics/Getting-Started-Samples/Intel_Extension_For_PyTorch_GettingStarted
```
2. Submit this "Intel_Extension_For_PyTorch_GettingStarted" workload on the selected node with the run script.
```
./q ./run.sh
```
> the run.sh contains all the instructions needed to run this "Intel_Extension_For_PyTorch_Hello_World" workload
### Run the `Intel® Extension for PyTorch* Getting Started` Sample on Intel® DevCloud (Optional)

### Build and run additional samples
Several sample programs are available for you to try, many of which can be compiled and run in a similar fashion. Experiment with running the various samples on different kinds of compute nodes or adjust their source code to experiment with different workloads.
1. If you do not already have an account, request an Intel® DevCloud account at [*Create an Intel® DevCloud Account*](https://intelsoftwaresites.secure.force.com/DevCloud/oneapi).
2. On a Linux* system, open a terminal.
3. SSH into Intel® DevCloud.
```
ssh DevCloud
```
> **Note**: You can find information about configuring your Linux system and connecting to Intel DevCloud at Intel® DevCloud for oneAPI [Get Started](https://DevCloud.intel.com/oneapi/get_started).

### Troubleshooting
If an error occurs, troubleshoot the problem using the Diagnostics Utility for Intel® oneAPI Toolkits.
[Learn more](https://software.intel.com/content/www/us/en/develop/documentation/diagnostic-utility-user-guide/top.html)

4. Navigate to the directory with the sample.
```
cd ~/oneAPI-samples/AI-and-Analytics/Getting-Started-Samples/Intel_Extension_For_PyTorch_GettingStarted
```
5. Submit this `Intel_Extension_For_PyTorch_GettingStarted` workload on the selected node with the run script.
```
./q ./run.sh
```
The `run.sh` script contains all the instructions needed to run this `Intel_Extension_For_PyTorch_Hello_World.py` workload.

### Using Visual Studio Code\* (Optional)
### Example Output

You can use Visual Studio Code (VS Code) extensions to set your environment, create launch configurations,
and browse and download samples.
With successful execution, it will print out `[CODE_SAMPLE_COMPLETED_SUCCESSFULLY]` in the terminal.

The basic steps to build and run a sample using VS Code include:
- Download a sample using the extension **Code Sample Browser for Intel oneAPI Toolkits**.
- Configure the oneAPI environment with the extension **Environment Configurator for Intel oneAPI Toolkits**.
- Open a Terminal in VS Code (**Terminal>New Terminal**).
- Run the sample in the VS Code terminal using the instructions below.
- (Linux only) Debug your GPU application with GDB for Intel® oneAPI toolkits using the Generate Launch Configurations extension.
## License

To learn more about the extensions, see
[Using Visual Studio Code with Intel® oneAPI Toolkits](https://software.intel.com/content/www/us/en/develop/documentation/using-vs-code-with-intel-oneapi/top.html).
Code samples are licensed under the MIT license. See
[License.txt](https://github.com/oneapi-src/oneAPI-samples/blob/master/License.txt) for details.

After learning how to use the extensions for Intel oneAPI Toolkits, return to this readme for instructions on how to build and run a sample.
Third party program Licenses can be found here: [third-party-programs.txt](https://github.com/oneapi-src/oneAPI-samples/blob/master/third-party-programs.txt).
Loading