nvidia--tensorrt/samples/sampleMNIST

“Hello World” For TensorRT

Table Of Contents

• Description
• How does this sample work?
  • TensorRT API layers and ops
• Preparing sample data
• Running the sample
  • Sample --help options
• Additional resources
• License
• Changelog
• Known issues

Description

This sample, sampleMNIST, is a simple hello world example that performs the basic setup and initialization of TensorRT using the Caffe parser.

How does this sample work?

This sample uses a Caffe model that was trained on the MNIST dataset.

Specifically, this sample:

• Performs the basic setup and initialization of TensorRT using the Caffe parser
• Imports a trained Caffe model using Caffe parser
• Preprocesses the input and stores the result in a managed buffer
• Builds an engine
• Serializes and deserializes the engine
• Uses the engine to perform inference on an input image

To verify whether the engine is operating correctly, this sample picks a 28x28 image of a digit at random and runs inference on it using the engine it created. The output of the network is a probability distribution on the digit, showing which digit is likely that in the image.

TensorRT API layers and ops

In this sample, the following layers are used. For more information about these layers, see the TensorRT Developer Guide: Layers documentation.

Activation layer The Activation layer implements element-wise activation functions. Specifically, this sample uses the Activation layer with the type kRELU.

Convolution layer The Convolution layer computes a 2D (channel, height, and width) convolution, with or without bias.

FullyConnected layer The FullyConnected layer implements a matrix-vector product, with or without bias.

Pooling layer The Pooling layer implements pooling within a channel. Supported pooling types are maximum, average and maximum-average blend.

Scale layer The Scale layer implements a per-tensor, per-channel, or per-element affine transformation and/or exponentiation by constant values.

SoftMax layer The SoftMax layer applies the SoftMax function on the input tensor along an input dimension specified by the user.

Preparing sample data

1. Download the sample data from TensorRT release tarball, if not already mounted under /usr/src/tensorrt/data (NVIDIA NGC containers) and set it to $TRT_DATADIR.

   export TRT_DATADIR=/usr/src/tensorrt/data
   pushd $TRT_DATADIR/mnist
   pip3 install Pillow
   python3 download_pgms.py
   popd
   

Running the sample

1. Compile the sample by following build instructions in TensorRT README.

2. Run the sample to perform inference on the digit:

   ./sample_mnist [-h] [--datadir=/path/to/data/dir/] [--useDLA=N] [--fp16 or --int8]
   

   For example:

   ./sample_mnist --datadir $TRT_DATADIR/mnist --fp16
   

   This sample reads three Caffe files to build the network:

   • mnist.prototxt The prototxt file that contains the network design.

   • mnist.caffemodel The model file which contains the trained weights for the network.

   • mnist_mean.binaryproto The binaryproto file which contains the means.

   This sample can be run in FP16 and INT8 modes as well.

   Note: By default, the sample expects these files to be in either the data/samples/mnist/ or data/mnist/ directories. The list of default directories can be changed by adding one or more paths with --datadir=/new/path/ as a command line argument.

3. Verify that the sample ran successfully. If the sample runs successfully you should see output similar to the following; ASCII rendering of the input image with digit 3:

   &&&& RUNNING TensorRT.sample_mnist # ./sample_mnist
   [I] Building and running a GPU inference engine for MNIST
   [I] Input:
   @@@@@@@@@@@@@@@@@@@@@@@@@@@@
   @@@@@@@@@@@@@@@@@@@@@@@@@@@@
   @@@@@@@@@@@@@@@@@@@@@@@@@@@@
   @@@@@@@@@@@@@@@@@@@@@@@@@@@@
   @@@@@@@@#-:.-=@@@@@@@@@@@@@@
   @@@@@%=     . *@@@@@@@@@@@@@
   @@@@% .:+%%%  *@@@@@@@@@@@@@
   @@@@+=#@@@@@# @@@@@@@@@@@@@@
   @@@@@@@@@@@%  @@@@@@@@@@@@@@
   @@@@@@@@@@@: *@@@@@@@@@@@@@@
   @@@@@@@@@@- .@@@@@@@@@@@@@@@
   @@@@@@@@@:  #@@@@@@@@@@@@@@@
   @@@@@@@@:   +*%#@@@@@@@@@@@@
   @@@@@@@%         :+*@@@@@@@@
   @@@@@@@@#*+--.::     +@@@@@@
   @@@@@@@@@@@@@@@@#=:.  +@@@@@
   @@@@@@@@@@@@@@@@@@@@  .@@@@@
   @@@@@@@@@@@@@@@@@@@@#. #@@@@
   @@@@@@@@@@@@@@@@@@@@#  @@@@@
   @@@@@@@@@%@@@@@@@@@@- +@@@@@
   @@@@@@@@#-@@@@@@@@*. =@@@@@@
   @@@@@@@@ .+%%%%+=.  =@@@@@@@
   @@@@@@@@           =@@@@@@@@
   @@@@@@@@*=:   :--*@@@@@@@@@@
   @@@@@@@@@@@@@@@@@@@@@@@@@@@@
   @@@@@@@@@@@@@@@@@@@@@@@@@@@@
   @@@@@@@@@@@@@@@@@@@@@@@@@@@@
   @@@@@@@@@@@@@@@@@@@@@@@@@@@@
   
   [I] Output:
   0:
   1:
   2:
   3: **********
   4:
   5:
   6:
   7:
   8:
   9:
   
   &&&& PASSED TensorRT.sample_mnist # ./sample_mnist
   

   This output shows that the sample ran successfully; PASSED.

Sample --help options

To see the full list of available options and their descriptions, use the -h or --help command line option.

Additional resources

The following resources provide a deeper understanding about sampleMNIST:

MNIST

• MNIST dataset

Documentation

• Introduction To NVIDIA’s TensorRT Samples
• Working With TensorRT Using The C++ API
• NVIDIA’s TensorRT Documentation Library

License

For terms and conditions for use, reproduction, and distribution, see the TensorRT Software License Agreement documentation.

Changelog

February 2019 This README.md file was recreated, updated and reviewed.

Known issues

There are no known issues in this sample.