* Refactor to create abstract ParallelOpCombiner
* First draft of CombineParallelDense
* Begin to work on tests
* Test
* Refactor to move out more common code
* Clean up
* Fix
* Remove statics
* fix wording
* Start to add combine_parallel_op_batch
* Resolve PR comments
* Resolve PR comments
* dummy change to retrigger CI
* Change special case from bias_add to add
* Revert special case change
* Ignore units check
* dummy change to retrigger CI
* dummy change to re-trigger CI
* Improve docs
* Update docs
* Update docs
= Motivation
It's useful to expose the tvm::reinterpret functionality to Relay/TOPI users, as
this allows them to build (fused) operators leveraging the bitwise
reinterpretation of an operator. An example is approximate transcendental
functions, which can be implemented similar to:
```.py
def C(x):
return relay.expr.const(x, "float32")
def approx_exp(x):
x = relay.minimum(relay.maximum(x, C(-88.0)), C(88.0))
x = C(127.0) + x * C(1.44269504)
xf = relay.floor(x)
i = relay.cast(xf, "int32")
x = x - xf
Y = C(0.99992522) + x * (C(0.69583354) + x * (C(0.22606716) + x * C(0.078024523)))
exponent = relay.left_shift(i, relay.expr.const(23, "int32"))
exponent = relay.reinterpret(exponent, "float32")
return exponent * Y
def approx_sigmoid(x):
# <2.0e-5 absolute error over [-5, 5]
y = approx_exp(x)
return y / (y + C(1.0))
def approx_tanh(x):
# <4.0e-5 absolute error over [-5, 5]
x = x * C(2.0)
y = approx_exp(x)
return (y - C(1.0)) / (y + C(1.0))
```
See unit tests for implementations of these approximate transendentals.
* Add batch_dot and cpu schedule
* Add relay support for batch_dot
* Rename batch_dot to batch_matmul
* nits
* Add missing file
* Put batch_matmul and dense x86 schedule in separate files
* Fix pylint
* Remove unused import
* Add cuda schedule for batch_matmul
* Add test case with larger batch size
* Add batch_matmul in api doc
* Fix quantize pass rounding error
* Fix pylint and minor change
* bug fix
* Add mean and product operators
* Fix typo
* Fix lint
* fix test
* Fix gpu schedule
* Update doc
* remove mean from topi
* Add nnvm test
* Fix cuda schedule
* Remove cuda schedule
* [NNVM][TEST] Numerical gradient testing
* [NNVM][TEST] Make some tests a little faster
* Fix the failing test_top_level3
* Target exclusion for the check_function
* Try to ignore singularities
* grad_input_vars now can't contain shapes
* Don't pass unnecessary grad_input_vars to check_function
* Multiple outputs; fixes; testing of check_function
* Use numerical_grads_params to pass parameters to numgrad checker
* Fail when no action is requested excplicitly
* Pass additional params to functions
* Silence the linter issue
* Simplified numgrad checking
* Improved docs for check_function
* Fixed the error message when no dtype is provided
* Several fixes
* Tests with shape/dtype inference for inputs
* Don't check dense's grads on cuda
* Raise an error if output dtypes haven't been inferred
* Moved shape/dtype inference into a separate function; use float32 as fallback
* Remove redundant dtype=float32
* Fix multiple outputs
* Use check_function in the rest of the test_top_level1
Jon Soifer