nvidia--tensorrt/samples/python/int8_caffe_mnist

INT8 Calibration In Python

Table Of Contents

• Description
• How does this sample work?
• Prerequisites
• Running the sample
  • Sample --help options
• Additional resources
• License
• Changelog
• Known issues

Description

This sample, int8_caffe_mnist, demonstrates how to create an INT8 calibrator, build and calibrate an engine for INT8 mode, and finally run inference in INT8 mode.

How does this sample work?

During calibration, the calibrator retrieves a total of 1003 batches, with 100 images each. We have simplified the process of reading and writing a calibration cache in Python, so that it is now easily possible to cache calibration data to speed up engine builds (see calibrator.py for implementation details).

During inference, the sample loads a random batch from the calibrator, then performs inference on the whole batch of 100 images.

Prerequisites

1. Install the dependencies for Python. python3 -m pip install -r requirements.txt

2. The MNIST dataset can be found under the data directory (usually /usr/src/tensorrt/data/mnist) if using the TensorRT containers. It is also bundled along with the TensorRT tarball.

   • This sample requires the training set, test set and test labels.

Running the sample

1. Run the sample to create a TensorRT inference engine, perform IN8 calibration and run inference: python3 sample.py [-d DATA_DIR]

   to run the sample with Python 3.

   Note: If the TensorRT sample data is not installed in the default location, for example /usr/src/tensorrt/data/, the data directory must be specified. For example: python sample.py -d /path/to/my/data/.

2. Verify that the sample ran successfully. If the sample runs successfully, the accuracy should be close to 99%.

   Calibrating batch 0, containing 64 images
   ...
   Calibrating batch 150, containing 64 images
   Validating batch 10
   ...
   Validating batch 310
   Total Accuracy: 99.09%
   
   

Sample --help options

To see the full list of available options and their descriptions, use the -h or --help command line option.

Additional resources

The following resources provide a deeper understanding about the model used in this sample:

Network:

• MNIST network

Dataset:

• MNIST dataset

Documentation

• Introduction To NVIDIA’s TensorRT Samples
• Working With TensorRT Using The Python API
• Enabling INT8 Inference Using Python
• NVIDIA’s TensorRT Documentation Library

License

For terms and conditions for use, reproduction, and distribution, see the TensorRT Software License Agreement documentation.

Changelog

March 2019 This README.md file was recreated, updated and reviewed.

Known issues

There are no known issues in this sample.