Rajeev Rao
169 lines
7.2 KiB
Markdown
169 lines
7.2 KiB
Markdown
# Algorithm Selection API usage example based off sampleMNIST in TensorRT
|
||
|
||
**Table Of Contents**
|
||
- [Description](#description)
|
||
- [How does this sample work?](#how-does-this-sample-work)
|
||
- [Preparing sample data](#preparing-sample-data)
|
||
- [Running the sample](#running-the-sample)
|
||
* [Sample `--help` options](#sample---help-options)
|
||
- [Additional resources](#additional-resources)
|
||
- [License](#license)
|
||
- [Changelog](#changelog)
|
||
- [Known issues](#known-issues)
|
||
|
||
## Description
|
||
|
||
This sample, sampleAlgorithmSelector, shows an example of how to use the algorithm selection API based on sampleMNIST.
|
||
[sampleMNIST documentation] (https://docs.nvidia.com/deeplearning/sdk/tensorrt-sample-support-guide/index.html#mnist_sample)
|
||
|
||
This sample demonstrates the usage of `IAlgorithmSelector` to deterministically build TRT engines.
|
||
It also shows the usage of `IAlgorithmSelector::selectAlgorithms` to define heuristics for selection of algorithms.
|
||
|
||
## How does this sample work?
|
||
|
||
This sample uses a Caffe model that was trained on the [MNIST dataset](https://github.com/NVIDIA/DIGITS/blob/master/docs/GettingStarted.md).
|
||
|
||
Specifically, this sample performs the following steps:
|
||
- Performs the basic setup and initialization of TensorRT using the Caffe parser
|
||
- [Imports a trained Caffe model using Caffe parser](https://docs.nvidia.com/deeplearning/sdk/tensorrt-developer-guide/index.html#import_caffe_c)
|
||
- Preprocesses the input and stores the result in a managed buffer
|
||
- [Sets up three instances of algorithm selector](#setup-the-algorithm-selectors)
|
||
- [Builds three engines using the algorithm selectors](https://docs.nvidia.com/deeplearning/sdk/tensorrt-developer-guide/index.html#build_engine_c)
|
||
- [Serializes and deserializes the engines](https://docs.nvidia.com/deeplearning/sdk/tensorrt-developer-guide/index.html#serial_model_c)
|
||
- [Uses the engines to perform inference on an input image](https://docs.nvidia.com/deeplearning/sdk/tensorrt-developer-guide/index.html#perform_inference_c)
|
||
|
||
To verify whether the engine is operating correctly, this sample picks a 28x28 image of a digit at random and runs inference on it using the engine it created. The output of the network is a probability distribution on the digit, showing which digit is likely to be that in the image.
|
||
|
||
### Setup the algorithm selectors
|
||
1. AlgorithmCacheWriter - Uses `IAlgorithmSelector::reportAlgorithms` to write TensorRT's default algorithm choices to a file "AlgorithmChoices.txt".
|
||
2. AlgorithmCacheReader - Uses `IAlgorithmSelector::selectAlgorithms` to replicate algorithm choices from the file "AlgorithmChoices.txt" and verifies the choices using `IAlgorithmSelector::reportAlgorithms`.
|
||
3. MinimumWorkspaceAlgorithmSelector - Uses `IAlgorithmSelector::selectAlgorithms` to select algorithms with minimum workspace requirements.
|
||
|
||
## Preparing sample data
|
||
|
||
1. Download the sample data from [TensorRT release tarball](https://developer.nvidia.com/nvidia-tensorrt-download#), if not already mounted under `/usr/src/tensorrt/data` (NVIDIA NGC containers) and set it to `$TRT_DATADIR`.
|
||
```bash
|
||
export TRT_DATADIR=/usr/src/tensorrt/data
|
||
pushd $TRT_DATADIR/mnist
|
||
pip3 install Pillow
|
||
python3 download_pgms.py
|
||
popd
|
||
```
|
||
|
||
## Running the sample
|
||
|
||
1. Compile the sample by following build instructions in [TensorRT README](https://github.com/NVIDIA/TensorRT/).
|
||
|
||
2. Run the sample to perform inference on the digit:
|
||
```bash
|
||
./sample_algorithm_selector [-h] [--datadir=/path/to/data/dir/] [--useDLA=N] [--fp16 or --int8]
|
||
```
|
||
|
||
For example:
|
||
```bash
|
||
./sample_algorithm_selector --datadir $TRT_DATADIR/mnist --fp16
|
||
```
|
||
|
||
This sample reads three Caffe files to build the network:
|
||
- `mnist.prototxt`
|
||
The prototxt file that contains the network design.
|
||
|
||
- `mnist.caffemodel`
|
||
The model file which contains the trained weights for the network.
|
||
|
||
- `mnist_mean.binaryproto`
|
||
The binaryproto file which contains the means.
|
||
|
||
This sample can be run in FP16 and INT8 modes as well.
|
||
|
||
**Note:** By default, the sample expects these files to be in either the `data/samples/mnist/` or `data/mnist/` directories. The list of default directories can be changed by adding one or more paths with `--datadir=/new/path/` as a command line argument.
|
||
|
||
3. Verify that the sample ran successfully. If the sample runs successfully you should see output similar to the following; ASCII rendering of the input image with digit 3:
|
||
```
|
||
&&&& RUNNING TensorRT.sample_algorithm_selector # ./sample_algorithm_selector
|
||
[I] Building and running a GPU inference engine for MNIST
|
||
[I] Input:
|
||
@@@@@@@@@@@@@@@@@@@@@@@@@@@@
|
||
@@@@@@@@@@@@@@@@@@@@@@@@@@@@
|
||
@@@@@@@@@@@@@@@@@@@@@@@@@@@@
|
||
@@@@@@@@@@@@@@@@@@@@@@@@@@@@
|
||
@@@@@@@@#-:.-=@@@@@@@@@@@@@@
|
||
@@@@@%= . *@@@@@@@@@@@@@
|
||
@@@@% .:+%%% *@@@@@@@@@@@@@
|
||
@@@@+=#@@@@@# @@@@@@@@@@@@@@
|
||
@@@@@@@@@@@% @@@@@@@@@@@@@@
|
||
@@@@@@@@@@@: *@@@@@@@@@@@@@@
|
||
@@@@@@@@@@- .@@@@@@@@@@@@@@@
|
||
@@@@@@@@@: #@@@@@@@@@@@@@@@
|
||
@@@@@@@@: +*%#@@@@@@@@@@@@
|
||
@@@@@@@% :+*@@@@@@@@
|
||
@@@@@@@@#*+--.:: +@@@@@@
|
||
@@@@@@@@@@@@@@@@#=:. +@@@@@
|
||
@@@@@@@@@@@@@@@@@@@@ .@@@@@
|
||
@@@@@@@@@@@@@@@@@@@@#. #@@@@
|
||
@@@@@@@@@@@@@@@@@@@@# @@@@@
|
||
@@@@@@@@@%@@@@@@@@@@- +@@@@@
|
||
@@@@@@@@#-@@@@@@@@*. =@@@@@@
|
||
@@@@@@@@ .+%%%%+=. =@@@@@@@
|
||
@@@@@@@@ =@@@@@@@@
|
||
@@@@@@@@*=: :--*@@@@@@@@@@
|
||
@@@@@@@@@@@@@@@@@@@@@@@@@@@@
|
||
@@@@@@@@@@@@@@@@@@@@@@@@@@@@
|
||
@@@@@@@@@@@@@@@@@@@@@@@@@@@@
|
||
@@@@@@@@@@@@@@@@@@@@@@@@@@@@
|
||
|
||
[I] Output:
|
||
0:
|
||
1:
|
||
2:
|
||
3: **********
|
||
4:
|
||
5:
|
||
6:
|
||
7:
|
||
8:
|
||
9:
|
||
|
||
&&&& PASSED TensorRT.sample_algorithm_selector # ./sample_algorithm_selector
|
||
```
|
||
|
||
This output shows that the sample ran successfully; `PASSED`.
|
||
|
||
|
||
### Sample --help options
|
||
|
||
To see the full list of available options and their descriptions, use the `-h` or `--help` command line option. For example:
|
||
```
|
||
Usage: ./sample_algorithm_selector [-h or --help] [-d or --datadir=<path to data directory>] [--useDLACore=<int>]
|
||
--help Display help information
|
||
--datadir Specify path to a data directory, overriding the default. This option can be used multiple times to add multiple directories. If no data directories are given, the default is to use (data/samples/mnist/, data/mnist/)
|
||
--useDLACore=N Specify a DLA engine for layers that support DLA. Value can range from 0 to n-1, where n is the number of DLA engines on the platform.
|
||
--int8 Run in Int8 mode.
|
||
--fp16 Run in FP16 mode.
|
||
```
|
||
|
||
# Additional resources
|
||
|
||
The following resources provide a deeper understanding about sampleAlgorithmSelector:
|
||
|
||
**MNIST**
|
||
- [MNIST dataset](https://github.com/NVIDIA/DIGITS/blob/master/docs/GettingStarted.md)
|
||
|
||
**Documentation**
|
||
- [Introduction To NVIDIA’s TensorRT Samples](https://docs.nvidia.com/deeplearning/sdk/tensorrt-sample-support-guide/index.html#samples)
|
||
- [Working With TensorRT Using The C++ API](https://docs.nvidia.com/deeplearning/sdk/tensorrt-developer-guide/index.html#c_topics)
|
||
- [NVIDIA’s TensorRT Documentation Library](https://docs.nvidia.com/deeplearning/sdk/tensorrt-archived/index.html)
|
||
|
||
# License
|
||
|
||
For terms and conditions for use, reproduction, and distribution, see the [TensorRT Software License Agreement](https://docs.nvidia.com/deeplearning/sdk/tensorrt-sla/index.html) documentation.
|
||
|
||
# Changelog
|
||
|
||
November 2019
|
||
This `README.md` file was recreated, updated and reviewed.
|
||
|
||
# Known issues
|
||
|
||
There are no known issues in this sample.
|