|
1 | | -# Tutorial : Optimize TensorFlow pre-trained model for inference |
2 | | -This tutorial will guide you how to optimize a pre-trained model for a better inference performance, and also |
3 | | -analyze the model pb files before and after the inference optimizations. |
4 | | -Both the Intel® Low Precision Optimization Tool (Intel® LPOT) and TensorFlow optimization tools are used in this tutorial, and Intel LPOT is the preferred tool for inference optimization on Intel Architectures. |
5 | | - |
6 | | -| Optimized for | Description |
7 | | -|:--- |:--- |
8 | | -| OS | Ubuntu* 18.04 |
9 | | -| Hardware | Intel® Xeon® Scalable processor family or newer |
10 | | -| Software | [Intel® AI Analytics Toolkit (AI Kit)](https://software.intel.com/content/www/us/en/develop/tools/oneapi/ai-analytics-toolkit.html) |
11 | | -| What you will learn | Optimize a pre-trained model for a better inference performance |
12 | | -| Time to complete | 30 minutes |
| 1 | +# `Optimize TensorFlow Pre-trained Model for Inference` Sample |
| 2 | + |
| 3 | +The `Optimize TensorFlow Pre-trained Model for Inference` sample demonstrates how to optimize a pre-trained model for a better inference performance, and also analyze the model pb files before and after the inference optimizations. |
| 4 | + |
| 5 | +| Area | Description |
| 6 | +|:--- |:--- |
| 7 | +| What you will learn | Optimize a pre-trained model for a better inference performance |
| 8 | +| Time to complete | 30 minutes |
| 9 | +| Category | Code Optimization |
| 10 | + |
| 11 | +## Prequisites |
| 12 | + |
| 13 | +| Optimized for | Description |
| 14 | +|:--- |:--- |
| 15 | +| OS | Ubuntu* 18.04 |
| 16 | +| Hardware | Intel® Xeon® Scalable processor family or newer |
| 17 | +| Software | [Intel® AI Analytics Toolkit (AI Kit)](https://software.intel.com/content/www/us/en/develop/tools/oneapi/ai-analytics-toolkit.html) |
| 18 | + |
| 19 | +### For Local Development Environments |
| 20 | + |
| 21 | +- **Intel® AI Analytics Toolkit (AI Kit)** |
| 22 | + |
| 23 | + You can get the AI Kit from [Intel® oneAPI Toolkits](https://www.intel.com/content/www/us/en/developer/tools/oneapi/toolkits.html#analytics-kit). <br> See [*Get Started with the Intel® AI Analytics Toolkit for Linux**](https://www.intel.com/content/www/us/en/develop/documentation/get-started-with-ai-linux) for AI Kit installation information and post-installation steps and scripts. |
| 24 | + |
| 25 | +- **Jupyter Notebook** |
| 26 | + |
| 27 | + Install using PIP: `$pip install notebook`. <br> Alternatively, see [*Installing Jupyter*](https://jupyter.org/install) for detailed installation instructions. |
| 28 | + |
| 29 | +The sample uses both the Intel® Low Precision Optimization Tool (Intel® LPOT) and TensorFlow* optimization tools, and Intel® LPOT is the preferred tool for inference optimization on Intel Architectures. |
| 30 | + |
| 31 | +### For Intel® DevCloud |
| 32 | + |
| 33 | +The necessary tools and components are already installed in the environment. You do not need to install additional components. See [Intel® DevCloud for oneAPI](https://devcloud.intel.com/oneapi/get_started/) for information. |
13 | 34 |
|
14 | 35 | ## Purpose |
15 | | -Show users the importance of inference optimization on performance, and also analyze TensorFlow ops difference in pre-trained models before/after the optimizations. |
16 | | -Those optimizations include: |
17 | | -* Converting variables to constants. |
18 | | -* Removing training-only operations like checkpoint saving. |
19 | | -* Stripping out parts of the graph that are never reached. |
20 | | -* Removing debug operations like CheckNumerics. |
21 | | -* Folding batch normalization ops into the pre-calculated weights. |
22 | | -* Fusing common operations into unified versions. |
23 | | - |
24 | | -## Key implementation details |
25 | | -This tutorial contains one Jupyter notebook and three python scripts listed below. |
| 36 | + |
| 37 | +Show users the importance of inference optimization on performance, and also analyze TensorFlow ops difference in pre-trained models before/after the optimizations. Those optimizations include: |
| 38 | + |
| 39 | +- Converting variables to constants. |
| 40 | +- Removing training-only operations like checkpoint saving. |
| 41 | +- Stripping out parts of the graph that are never reached. |
| 42 | +- Removing debug operations like CheckNumerics. |
| 43 | +- Folding batch normalization ops into the pre-calculated weights. |
| 44 | +- Fusing common operations into unified versions. |
| 45 | + |
| 46 | +## Key Implementation Details |
| 47 | + |
| 48 | +This tutorial contains one Jupyter Notebook and three python scripts listed below. |
| 49 | + |
26 | 50 | ### Jupyter Notebooks |
27 | 51 |
|
28 | | -| Notebook | Notes| |
29 | | -| ------ | ------ | |
30 | | -| tutorial_optimize_TensorFlow_pretrained_model.ipynb | Optimize a pre-trained model for a better inference performance, and also analyze the model pb files | |
| 52 | +| Notebook | Description |
| 53 | +|:--- |:--- |
| 54 | +|`tutorial_optimize_TensorFlow_pretrained_model.ipynb` | Optimize a pre-trained model for a better inference performance, and also analyze the model pb files |
31 | 55 |
|
32 | 56 | ### Python Scripts |
33 | | -| Scripts | Notes| |
34 | | -| ------ | ------ | |
35 | | -| tf_pb_utils.py | This script parses a pre-trained TensorFlow model PB file. | |
36 | | -| freeze_optimize_v2.py | This script optimizes a pre-trained TensorFlow model PB file. | |
37 | | -| profile_utils.py | This script helps on output processing of the Jupyter Notebook. | |
38 | | - |
39 | 57 |
|
40 | | -## License |
41 | | -Code samples are licensed under the MIT license. See |
42 | | -[License.txt](https://github.com/oneapi-src/oneAPI-samples/blob/master/License.txt) for details. |
| 58 | +| Script | Description |
| 59 | +|:--- |:--- |
| 60 | +|`tf_pb_utils.py` | Parses a pre-trained TensorFlow model PB file. <br> (See [TensorFlow* PB File Parser README](scripts/README.md) for more information.) |
| 61 | +|`freeze_optimize_v2.py` | Optimizes a pre-trained TensorFlow model PB file. |
| 62 | +|`profile_utils.py` | Helps output processing of the Jupyter Notebook. |
43 | 63 |
|
44 | | -Third party program Licenses can be found here: [third-party-programs.txt](https://github.com/oneapi-src/oneAPI-samples/blob/master/third-party-programs.txt) |
| 64 | +## Set Environment Variables |
45 | 65 |
|
46 | | -## Build and Run the Sample |
| 66 | +When working with the command-line interface (CLI), you should configure the oneAPI toolkits using environment variables. Set up your CLI environment by sourcing the `setvars` script every time you open a new terminal window. This practice ensures that your compiler, libraries, and tools are ready for development. |
47 | 67 |
|
48 | | -### Pre-requirement |
| 68 | +## Run the `Optimize TensorFlow Pre-trained Model for Inference` Sample |
49 | 69 |
|
50 | | -> NOTE: No action is required if users use Intel DevCloud as their environment. |
51 | | - Please refer to [Intel® DevCloud for oneAPI](https://intelsoftwaresites.secure.force.com/devcloud/oneapi) for Intel DevCloud. |
| 70 | +### On Linux* |
52 | 71 |
|
53 | | - 1. **Intel® AI Analytics Toolkit** |
54 | | - You can refer to the oneAPI [main page](https://software.intel.com/en-us/oneapi) for toolkit installation, |
55 | | - and the Toolkit [Getting Started Guide for Linux](https://software.intel.com/en-us/get-started-with-intel-oneapi-linux-get-started-with-the-intel-ai-analytics-toolkit) for post-installation steps and scripts. |
| 72 | +> **Note**: If you have not already done so, set up your CLI |
| 73 | +> environment by sourcing the `setvars` script in the root of your oneAPI installation. |
| 74 | +> |
| 75 | +> Linux*: |
| 76 | +> - For system wide installations: `. /opt/intel/oneapi/setvars.sh` |
| 77 | +> - For private installations: ` . ~/intel/oneapi/setvars.sh` |
| 78 | +> - For non-POSIX shells, like csh, use the following command: `bash -c 'source <install-dir>/setvars.sh ; exec csh'` |
| 79 | +> |
| 80 | +> For more information on configuring environment variables, see *[Use the setvars Script with Linux* or macOS*](https://www.intel.com/content/www/us/en/develop/documentation/oneapi-programming-guide/top/oneapi-development-environment-setup/use-the-setvars-script-with-linux-or-macos.html)*. |
56 | 81 |
|
57 | | - 2. **Jupyter Notebook** |
58 | | - Users can install via PIP by `$pip install notebook`. |
59 | | - Users can also refer to the [installation link](https://jupyter.org/install) for details. |
60 | 82 |
|
| 83 | +#### Open Jupyter Notebook |
61 | 84 |
|
| 85 | +1. Launch Jupyter Notebook. |
| 86 | + ``` |
| 87 | + jupyter notebook --ip=0.0.0.0 |
| 88 | + ``` |
| 89 | +2. Follow the instructions to open the URL with the token in your browser. |
| 90 | +3. Locate and select the Notebook. |
| 91 | + ``` |
| 92 | + tutorial_optimize_TensorFlow_pretrained_model.ipynb |
| 93 | + ``` |
| 94 | +4. Change your Jupyter Notebook kernel to **tensorflow** or **intel-tensorflow**. |
| 95 | +5. Run every cell in the Notebook in sequence. |
62 | 96 |
|
63 | | -### Running the Sample |
| 97 | +#### Troubleshooting |
64 | 98 |
|
65 | | -1. Launch Jupyter notebook: `$jupyter notebook --ip=0.0.0.0` |
| 99 | +If you receive an error message, troubleshoot the problem using the **Diagnostics Utility for Intel® oneAPI Toolkits**. The diagnostic utility provides configuration and system checks to help find missing dependencies, permissions errors, and other issues. See the [Diagnostics Utility for Intel® oneAPI Toolkits User Guide](https://www.intel.com/content/www/us/en/develop/documentation/diagnostic-utility-user-guide/top.html) for more information on using the utility. |
66 | 100 |
|
| 101 | +### Run the Sample on Intel® DevCloud (Optional) |
67 | 102 |
|
68 | | -2. Follow the instructions to open the URL with the token in your browser |
69 | | -3. Click the `tutorial_optimize_TensorFlow_pretrained_model.ipynb` file |
70 | | -4. Change your Jupyter notebook kernel to "tensorflow" or "intel-tensorflow" |
71 | | -5. Run through every cell of the notebook one by one |
| 103 | +1. If you do not already have an account, request an Intel® DevCloud account at [*Create an Intel® DevCloud Account*](https://intelsoftwaresites.secure.force.com/DevCloud/oneapi). |
| 104 | +2. On a Linux* system, open a terminal. |
| 105 | +3. SSH into Intel® DevCloud. |
| 106 | + ``` |
| 107 | + ssh DevCloud |
| 108 | + ``` |
| 109 | + > **Note**: You can find information about configuring your Linux system and connecting to Intel DevCloud at Intel® DevCloud for oneAPI [Get Started](https://devcloud.intel.com/oneapi/get_started). |
| 110 | + |
| 111 | +4. Follow the instructions to open the URL with the token in your browser. |
| 112 | +5. Locate and select the Notebook. |
| 113 | + ``` |
| 114 | + tutorial_optimize_TensorFlow_pretrained_model.ipynb |
| 115 | + ``` |
| 116 | +6. Change the kernel to **tensorflow** or **intel-tensorflow**. |
| 117 | +7. Run every cell in the Notebook in sequence. |
72 | 118 |
|
| 119 | +## Example Output |
73 | 120 |
|
| 121 | +Users should be able to see some diagrams for performance comparison and analysis. One example of performance comparison diagrams: |
74 | 122 |
|
75 | | -### Example of Output |
76 | | -Users should be able to see some diagrams for performance comparison and analysis. |
77 | | -One example of performance comparison diagrams: |
78 | | -<br><img src="images/perf_comparison.png" width="500" height="400"><br> |
| 123 | + |
79 | 124 |
|
80 | 125 | For performance analysis, users can also see pie charts for different Tensorflow* operations in the analyzed pre-trained model pb file. |
81 | | -One example of model pb file analysis diagrams: |
82 | | -<br><img src="images/saved_model_pie.png" width="800" height="600"><br> |
83 | | - |
84 | | -If an error occurs, troubleshoot the problem using the Diagnostics Utility for Intel® oneAPI Toolkits. |
85 | | -[Learn more](https://software.intel.com/content/www/us/en/develop/documentation/diagnostic-utility-user-guide/top.html) |
86 | 126 |
|
87 | | -### Using Visual Studio Code* (Optional) |
| 127 | +One example of model pb file analysis diagrams: |
88 | 128 |
|
89 | | -You can use Visual Studio Code (VS Code) extensions to set your environment, create launch configurations, |
90 | | -and browse and download samples. |
| 129 | + |
91 | 130 |
|
92 | | -The basic steps to build and run a sample using VS Code include: |
93 | | - - Download a sample using the extension **Code Sample Browser for Intel oneAPI Toolkits**. |
94 | | - - Configure the oneAPI environment with the extension **Environment Configurator for Intel oneAPI Toolkits**. |
95 | | - - Open a Terminal in VS Code (**Terminal>New Terminal**). |
96 | | - - Run the sample in the VS Code terminal using the instructions below. |
97 | | - - (Linux only) Debug your GPU application with GDB for Intel® oneAPI toolkits using the Generate Launch Configurations extension. |
| 131 | +## License |
98 | 132 |
|
99 | | -To learn more about the extensions, see |
100 | | -[Using Visual Studio Code with Intel® oneAPI Toolkits](https://software.intel.com/content/www/us/en/develop/documentation/using-vs-code-with-intel-oneapi/top.html). |
| 133 | +Code samples are licensed under the MIT license. See |
| 134 | +[License.txt](https://github.com/oneapi-src/oneAPI-samples/blob/master/License.txt) for details. |
101 | 135 |
|
102 | | -After learning how to use the extensions for Intel oneAPI Toolkits, return to this readme for instructions on how to build and run a sample. |
| 136 | +Third party program Licenses can be found here: [third-party-programs.txt](https://github.com/oneapi-src/oneAPI-samples/blob/master/third-party-programs.txt). |
0 commit comments