pr/16-dsp-runtime-optimizer-java
50 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
|
|
0f94605d5f |
ADR/publishing prep for renamespacing and release updates (#10212)
* ADR for renamespacing and release updates * ADR for renamespacing and release updates * use new secrets for publishing * use new secrets for publishing * remove old distribution management * remove try catch for cuda build * update string functionality for cuda * update string functionality for cuda * update string functionality for cuda * update string functionality for cuda * update string functionality for cuda * update string functionality for cuda * remove old distribution management * remove more ossrh references * update environment variables * update lombok version * clean up test dependencies errant linker flags * get rid of dup flags |
||
|
|
722592d8a9 |
Update for publishing to snapshots (#10211)
* fix extra quote inserted by intellij * Fix path * fix executable permissions usage * improve directory resolution and restore some files from the previous PR * remove old unused dep versions Update to new march 2024 central release plugin * update central publishing version * temp disable mirror * fix hallucinated version * update maven invoker version for jetspeed * remove old build properties add better directory resolution for copy flatc * fix helpers sources for cpu (api updates) perform maven antrun upgrade * more fixes for onednn helpers * update more antrun usage * fix pooling param usage * remove findbugs on the 2 files it was used on * debug odd build path issues * try again * remove bad ls * try again * another attempt * Add troubleshooting logs again * Remove specific gcc versions * update build * cd to original directory instead * Remove enforcer check * remove debug steps * remove extra steps * update linux, android and cuda versions * update arm usage defaults * remove 32 bit builds * alternative path * alternative path get rid of old tools * get rid of over simplified bootstrap libnd4j download usage * remove extra cmake * remove manual path setting * arm compilation fix * debug armcompute * update android-x86_64 openblas * upgrade gcc * minor armcompute const changes * update openblas version * fix armcompute paths * remove ls * update cuda versions and envrionment variables * fix syntax errors * fix syntax errors * fix syntax errors * fix syntax errors * another rev * update msys command * rev * fix classifier * rev android-x86_64 * rev android-x86_64 * update cuda env passing * fix linux-arm64 syntax error * cuda rev * cuda rev * update arm else if branches for matching * remove external PS script? * remove external PS script? * remove external PS script? * improve architecture detection * update list operation conv2d armcompute * add flatbuffer generated code * remove old maven auth * remove old legacny average and accumulate * remove cache due to 422 error * ensure armcompute is optional * ensure armcompute is optional * disk space clean up on all workflows * deal with windows service issue * ensure nvcc is on path * Add arm64 protoc Update cuda paths * convert to use numeric types only mitigating lld errors found by clang * Add arm64 protoc Update cuda paths * fix syntax error * fix overriding properties causing libnd4j not to be built with cuda * update windows version * try to update cl.exe paths * update debugging for nvcc * update cuda flags to work with linux/windows * remove extra cuda install * fix duplicate sources * more unsupported compiler changes * remove unneeded source and javadoc * better cudnn detection * update where we put unsupported compiler * Collapse cmake logic * Collapse cmake logic * clean up consolidated file * flatbuffers fix * fix hallucinated paths * generate flatbuffers by default * fix hallucinated paths * ensure imports present * fix flatc target order * fix elif syntax * more rearrange * fix git tag * update template to allow proper configuration generation * remove guard * change quotes * change quotes * test default * fix missing functions * rearrange dependencies * set the engine * reintegrate some old cuda logic * Add back in function defs * Add back in function defs * Add back in function defs * Add back in function defs * Add back in function defs * Add back in function defs * Add back in function defs * Add back in function defs * Add back in function defs * Add back in function defs * Add back in function defs * Add back in function defs * Add back in function defs * Add back in function defs * Add back in function defs * Add back in function defs * Add back in function defs * Add back in function defs * Add back in function defs * Add back in function defs * Add back in function defs * Add back in function defs * Add back in function defs * Add back in function defs * Add back in function defs * Add back in function defs * Add back in function defs * Add back in function defs * Add back in function defs * Add back in function defs * Add back in function defs * Add back in function defs * Add back in function defs * Add back in function defs * Add back in function defs * Add back in function defs * Add back in function defs * Add back in function defs * Add back in function defs * Add back in function defs * update api to use proper cuda versions * update api to use proper cuda versions * update api to use proper cuda versions * Add back in function defs * delete old paths * aDd back include_ops.h generation * delete old paths * reinroduce old comand * delete old paths * delete old paths * better tar unpacking * fix for suffixes * Add missing cpu sources * fix duplicate profile * change mac image * change mac image * get rid of old gpg key step * get rid of javadoc on mac * remove unneeded deps for mac * update arm compute path * remove old command checks * remove gpg key step * clang specific type erro * clang specific type erro * add another missing cpu path * add another missing cpu path * fix helpers sources ordering * fix helpers sources ordering * Add back createFromDescriptor * fix helpers sources ordering * fix helpers sources ordering * fix helpers sources ordering * fix helpers sources ordering * Add back createFromDescriptor * add debug for onednn * add debug for onednn * SFINAE for type aliases * SFINAE for type aliases * SFINAE for type aliases * SFINAE for type aliases * ensure we tell compiler we're ok with certain apple version minimums * ensure we tell compiler we're ok with certain apple version minimums * ensure we tell compiler we're ok with certain apple version minimums * ensure we tell compiler we're ok with certain apple version minimums * fix imports * update linker path * fix lock type usage * change order of sources * share mutex types * share mutex types * share mutex types * update linker path * share mutex types * share mutex types * share mutex types * decrease type pairs for sort * decrease type pairs for sort * decrease type pairs for sort * decrease type pairs for sort * update the special methods to use combinations * update the special methods to use combinations * update the special methods to use combinations * update the special methods to use combinations * update the special methods to use combinations * update the special methods to use combinations * update the special methods to use combinations * update the special methods to use combinations * update the special methods to use combinations * update the special methods to use combinations * update the special methods to use combinations * standardize output paths * fix template paths * refactor compiler flags * update onednn to use similar approach to armcompute * refactor compiler flags * fix paths * fix paths * fix paths * fix paths * fix paths * fix paths * fix paths * fix paths * change target expand types * change target expand types * add new pairwise types * add new pairwise types * add new pairwise types * update linker paths * add new pairwise types * add new pairwise types * add new pairwise types * fix strings with transform * fix strings with transform * update linker paths * update default values for libnd4j.outputPath * update default values for libnd4j.outputPath * update default values for libnd4j.outputPath * update pom.xml namespaces * Update .github/workflows/build-deploy-linux-cuda-12.6.yml Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Update .github/workflows/build-deploy-linux-x86_64.yml Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> --------- Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> |
||
|
|
2f1d459933 |
Updates java cpp versions to 1.5.11 (cuda, openblas etc (#10188)
* javacpp version upgrades add new cuda versions address a few leftover api changes for nativeops address new python api change * remove commented code |
||
|
|
9ed39ba7b8 |
Ag 10120 followup (#10135)
* overhaul cuda to work with new changes update nativeops on cpu side * follow up changes to: https://github.com/deeplearning4j/deeplearning4j/pull/10120 |
||
|
|
32ef037979 |
Misc spacing cleanup/minor preset logic condensing. (#10124)
* Misc cleanup: spacing condense common preset logic in to opexclusion utils. Migrate most calls to native ops to Nd4j.getNativeOps(). * Add op exclusion utils * remove commented code add extern to different headers add more nd4j.getnativeops changes * remove commented code remove sd experimental remove old validation for multi type functions * fix typos |
||
|
|
7051233bb4 |
Clean up pom.xml across modules: logging (#10075)
Remove old test in deeplearning4j-common-tests Fix shape of created arrays in iris utils |
||
|
|
b57991671d |
Fix jdk 8 target + failing tests (#9818)
* Fix jdk 8 target * Fix kotlin vversion in tests * Fix epsilon op type Remove unmaintained lint plugin (causes builds to fail on jdk 11 due to java 9 issues) Disables long running tests * Test cleanups Fix up ArrayCacheMemoryMgr: Handles default values with respect to the static context now including the tests. Delete old misc tests that aren't consistent or need to be rewritten. Delete old misc commented code Fix up tests that had failures due to the old 2d vector assumption. These failures were related to bias shape assertions. |
||
|
|
01f340ea13 |
Misc 2d grad fixes, Spark test fixes test refactoring (#9735)
* Fix up create_view tests, assign operation for views * Misc shape related fixes for gradients and tests related to old 2d assumption with vectors * Ignore missing op * Fix updater get with 1d standardization * Fix up another 1d test case * Update TestRegressionTest080.java * Update SlicingTests.java * Fix up booleanindexing backtracklinesearch test cases * Fix up test assertion * Remove old tensor array logic for tensorarray remove, fix test * Address nits, remove noninplacevalidation listener * Remove tests with old assumptions, migrate arrowserde test to platform-tests * Add testresources profile to nd4j-presets-common * More test collapsing * Collapse tests * Remove old profiles * Delete old aurora backend, misc pom clean up * Delete old test resources profiles * Update datavec-api test locations * More test migrations * More tests * Migrate tests for spark * Collapse spark parameter server tests * Remove old tests * Collapse omnihub * Rename tests for compilation errors * Graph package moving * Add missing dep * nlp test adjustments * Update names in platform-tests datavec local modules * Add missing nlp imports * Remove hard coded test, add deepwalk import * Update pom.xml * Update pom.xml * Update pom.xml * Add mockito dep * Update TestGraphLoading.java * Update pom.xml * Remove deprecated tests, add missing dep * Clean up intellij accidental refactoring * Update AeronUdpTransportTest.java * Update EndingPreProcessorTest.java * More test updates * More test refactoring * More assertion clean up, remove deprecated test * More test clean up * Remove more deprecated tests * More test updates * More test updates * More tests * Update WeightedWalkerTest.java * Update spark versions, dependencies * Handle spark upgrade, add a set closeable false to prevent spark from closing arrays early |
||
|
|
3feca6d831 |
Python4j performance optimizations (#9688)
* WIP: adding python interpreter and new abstractions * Add tests for python update * fix up cuda build * WIP: minor clean up, add netty shading * Fix up test execution, ensure python set main thread is restored * Add caching option for python (makes objects derived available outside of GIL) * remove unused test case, extra printing |
||
|
|
8ce2d9df8f |
Java support Version updates (#9626)
* Add back CI updates * Fix android compilation error * Update TopK.java * Updates project for java 9+ * Remove old grpc server, rename backend packages, change flatbuffers to temp konduit fork for cuda compilation * More renames + remove extra generated classes * Update 0016 - Java 9+ Support.md * Add back license from flatbuffers generation * Remove solr modules, update java 9 module comments (fixed some leaving some ignored such as anything spark related) * Update pom.xml * Add cuda compute capabilities to reduce build times * Add default flags for only building certain modules for cuda on windows * Add back java 8 support * Update build-deploy-windows-cuda-11.2.yml * Update build-deploy-windows-cuda-11.4.yml * Update build-deploy-windows-cuda-11.2.yml * Update ccs, remove defaults * Update build-deploy-linux-cuda-11.4.yml * Add module info * Update IndexedTail.java * Add also make and remove deeplearning4j-cuda from build * Add also make and remove deeplearning4j-cuda from build * Update IndexedTail.java * Add missing cuda modules * Remove old unused concurrency module |
||
|
|
84427ee93e | Remove all tmp files | ||
|
|
5826a44294 | Update build versions, cuda versions | ||
|
|
2b4d930c35 | Update NumpyArray.java | ||
|
|
10f2808e0c | Add more flexible handling of creating numpy arrays | ||
|
|
30887f9374 | Update naming suggestions | ||
|
|
892b6e5e26 | Refactor constants and system properties usage in to 1 class | ||
|
|
9469401fa0 | Allow disabling of python thread management | ||
|
|
1727ab2d76 | Change python executioner to use add path rather than set path, add more configurability for calling py init. | ||
|
|
7f2ee5e158 | Add init calls to each test and ensure that forkCount is at least 1 to prevent numpy.import_array() from being called twice | ||
|
|
f1dda7b776 | Fix https://github.com/eclipse/deeplearning4j/issues/9347 | ||
|
|
7f6acc1d85 | Remove duplicate declarations for cuda single threading | ||
|
|
cc9f62fbd3 | Update pom.xmls to add dynamic cuda versions to tests | ||
|
|
42926dc7a5 | Pom changes for junit 5 upgrade | ||
|
|
5e8951cd8e | Add tags for junit 5 | ||
|
|
7c7f9db097 | Fix classpathresource for python script | ||
|
|
b2187a4c36 | Disable more tests due to permissions (to investigate in separate PR) | ||
|
|
79191c5b6c | Add profiles to python4j submodules | ||
|
|
94b14a9c74 | Update maven profiles in python4j allowing tests to run | ||
|
|
224f18a586 | Update parameterized python tests, move python tests to proper package | ||
|
|
ad4f47096c | Unify nd4j test profiles, get rid of old modules, fix more parameter issues with junit 5 tests | ||
|
|
e0077c38a9 | More junit 4 removal, all tests compile. FIxed parameterized test invocation. Deleted nd4j-parameter-server-status that used play | ||
|
|
3c6014271e | Migrate parameterized tests to junit 5 | ||
|
|
82bdcc21d2 | All tests compile | ||
|
|
b1229432d6 | Fix junit artifact in backends | ||
|
|
fa1a31c877 | Upgrade dl4j to junit 5 | ||
|
|
c3f04caef4 | Add ctc loss from KonduitAI PR, add missing java bits | ||
|
|
48856b6182 | Add ignores for tests not passing for individual processing later | ||
|
|
d93976922a | Update pom.xml | ||
|
|
1eaee7f6d9 | Copyright updates, removal of extra nlp modules | ||
|
|
46dbd0b203 | Update copyrights remove attic and relocate elsewhere | ||
|
|
4766032444 | Update copyright to mention NOTICES file | ||
|
|
c715aea405 | Update LICENSE | ||
|
|
65c6a9a42e | Dev commits | ||
|
|
f9aebec79e |
Development updates (#9098)
* RL4J: Add generic update rule (#502) Signed-off-by: Alexandre Boulanger <aboulang2002@yahoo.com> * Shyrma reduce (#481) * - start working on improving of cpu legacy code for reduce ops Signed-off-by: Yurii <iuriish@yahoo.com> * - further work on improving legacy loops Signed-off-by: Yurii <iuriish@yahoo.com> * - still working on improving reduce ops Signed-off-by: Yurii <iuriish@yahoo.com> * - further work on improving reduce ops Signed-off-by: Yurii <iuriish@yahoo.com> * - testing speed run of new reduce op Signed-off-by: Yurii <iuriish@yahoo.com> * - working on improvement of default loop for reduce op Signed-off-by: Yurii <iuriish@yahoo.com> * - update signatures of stuff which calls reduce ops Signed-off-by: Yurii <iuriish@yahoo.com> * - make corrections in cuda reduce kernels Signed-off-by: Yurii <iuriish@yahoo.com> * - change loop for default case in broadcast legacy ops Signed-off-by: Yurii <iuriish@yahoo.com> * - comment some shape stuff Signed-off-by: Yurii <iuriish@yahoo.com> * - comment unnecessary prints in RNGtests Signed-off-by: Yurii <iuriish@yahoo.com> * - finish to resolve conflicts after master has been merged Signed-off-by: Yurii <iuriish@yahoo.com> * - get rid of some compilation mistakes of cuda stuff Signed-off-by: Yurii <iuriish@yahoo.com> * - minor changes Signed-off-by: Yurii <iuriish@yahoo.com> * - further search for bug causing crash on java test Signed-off-by: Yurii <iuriish@yahoo.com> * - add scalar case in reduce_ ... exec stuff Signed-off-by: Yurii <iuriish@yahoo.com> * - minor corrections in NAtiveOps.cu Signed-off-by: Yurii <iuriish@yahoo.com> * - add switch to scalar case execReduceXD functions Signed-off-by: Yurii <iuriish@yahoo.com> * - add support for vectors old shape in ConstantShapeHelper::createShapeInfoWithNoUnitiesForReduce Signed-off-by: Yurii <iuriish@yahoo.com> * - correct cuda mirrorPad Signed-off-by: Yurii <iuriish@yahoo.com> * - add support for vectors old shape in cuda createShapeInfoWithNoUnitiesForReduce Signed-off-by: Yurii <iuriish@yahoo.com> Co-authored-by: raver119 <raver119@gmail.com> * Add support for CUDA 11.0 (#492) * Add support for CUDA 11.0 * libnd4j tweaks for CUDA 11 Signed-off-by: raver119@gmail.com <raver119@gmail.com> * bindings update, again? Signed-off-by: raver119@gmail.com <raver119@gmail.com> * * Update versions of JavaCPP Presets for FFmpeg, OpenBLAS, and NumPy * update API to match CUDA 8 Signed-off-by: raver119@gmail.com <raver119@gmail.com> * * Update version of JavaCPP Presets for CPython * C++ updated for cuDNN 8.0 Signed-off-by: raver119@gmail.com <raver119@gmail.com> * one more test Signed-off-by: raver119@gmail.com <raver119@gmail.com> * one more test Signed-off-by: raver119@gmail.com <raver119@gmail.com> * one more test Signed-off-by: raver119@gmail.com <raver119@gmail.com> * 128-bit alignment for workspaces Signed-off-by: raver119@gmail.com <raver119@gmail.com> * change seed in 1 test Signed-off-by: raver119@gmail.com <raver119@gmail.com> * Fix dependecy duplication in python4j-parent pom * Fix group id for in python4j-numpy * few tests tweaked Signed-off-by: raver119@gmail.com <raver119@gmail.com> * Remove macosx-x86_64-gpu from nd4j-tests-tensorflow * few minor tweaks for IndexReduce Signed-off-by: raver119@gmail.com <raver119@gmail.com> * one test removed Signed-off-by: raver119@gmail.com <raver119@gmail.com> Co-authored-by: raver119@gmail.com <raver119@gmail.com> Co-authored-by: Serhii Shepel <9946053+sshepel@users.noreply.github.com> * RL4J: Add SyncTrainer and AgentLearnerBuilder for a few algorithms (#504) Signed-off-by: Alexandre Boulanger <aboulang2002@yahoo.com> * * Update versions of JavaCPP Presets for OpenCV, FFmpeg, and MKL Signed-off-by: Samuel Audet <samuel.audet@gmail.com> * Fix L2NormalizeVertex and eclipse#9054 (#513) * update * Fix L2NormalizeVertex Fix eclipse#9054 * RL4J: Add async training and advantage actor-critic (#507) * Added async training & Advantage Actor Critic Signed-off-by: Alexandre Boulanger <aboulang2002@yahoo.com> * Fix compiler error Signed-off-by: Samuel Audet <samuel.audet@gmail.com> * Renamed ActorCriticPolicy back to ACPolicy Signed-off-by: Alexandre Boulanger <aboulang2002@yahoo.com> Co-authored-by: Samuel Audet <samuel.audet@gmail.com> * Python GIL overhaul (#517) * Development updates (#9053) * RL4J: Add generic update rule (#502) Signed-off-by: Alexandre Boulanger <aboulang2002@yahoo.com> * Shyrma reduce (#481) * - start working on improving of cpu legacy code for reduce ops Signed-off-by: Yurii <iuriish@yahoo.com> * - further work on improving legacy loops Signed-off-by: Yurii <iuriish@yahoo.com> * - still working on improving reduce ops Signed-off-by: Yurii <iuriish@yahoo.com> * - further work on improving reduce ops Signed-off-by: Yurii <iuriish@yahoo.com> * - testing speed run of new reduce op Signed-off-by: Yurii <iuriish@yahoo.com> * - working on improvement of default loop for reduce op Signed-off-by: Yurii <iuriish@yahoo.com> * - update signatures of stuff which calls reduce ops Signed-off-by: Yurii <iuriish@yahoo.com> * - make corrections in cuda reduce kernels Signed-off-by: Yurii <iuriish@yahoo.com> * - change loop for default case in broadcast legacy ops Signed-off-by: Yurii <iuriish@yahoo.com> * - comment some shape stuff Signed-off-by: Yurii <iuriish@yahoo.com> * - comment unnecessary prints in RNGtests Signed-off-by: Yurii <iuriish@yahoo.com> * - finish to resolve conflicts after master has been merged Signed-off-by: Yurii <iuriish@yahoo.com> * - get rid of some compilation mistakes of cuda stuff Signed-off-by: Yurii <iuriish@yahoo.com> * - minor changes Signed-off-by: Yurii <iuriish@yahoo.com> * - further search for bug causing crash on java test Signed-off-by: Yurii <iuriish@yahoo.com> * - add scalar case in reduce_ ... exec stuff Signed-off-by: Yurii <iuriish@yahoo.com> * - minor corrections in NAtiveOps.cu Signed-off-by: Yurii <iuriish@yahoo.com> * - add switch to scalar case execReduceXD functions Signed-off-by: Yurii <iuriish@yahoo.com> * - add support for vectors old shape in ConstantShapeHelper::createShapeInfoWithNoUnitiesForReduce Signed-off-by: Yurii <iuriish@yahoo.com> * - correct cuda mirrorPad Signed-off-by: Yurii <iuriish@yahoo.com> * - add support for vectors old shape in cuda createShapeInfoWithNoUnitiesForReduce Signed-off-by: Yurii <iuriish@yahoo.com> Co-authored-by: raver119 <raver119@gmail.com> * Add support for CUDA 11.0 (#492) * Add support for CUDA 11.0 * libnd4j tweaks for CUDA 11 Signed-off-by: raver119@gmail.com <raver119@gmail.com> * bindings update, again? Signed-off-by: raver119@gmail.com <raver119@gmail.com> * * Update versions of JavaCPP Presets for FFmpeg, OpenBLAS, and NumPy * update API to match CUDA 8 Signed-off-by: raver119@gmail.com <raver119@gmail.com> * * Update version of JavaCPP Presets for CPython * C++ updated for cuDNN 8.0 Signed-off-by: raver119@gmail.com <raver119@gmail.com> * one more test Signed-off-by: raver119@gmail.com <raver119@gmail.com> * one more test Signed-off-by: raver119@gmail.com <raver119@gmail.com> * one more test Signed-off-by: raver119@gmail.com <raver119@gmail.com> * 128-bit alignment for workspaces Signed-off-by: raver119@gmail.com <raver119@gmail.com> * change seed in 1 test Signed-off-by: raver119@gmail.com <raver119@gmail.com> * Fix dependecy duplication in python4j-parent pom * Fix group id for in python4j-numpy * few tests tweaked Signed-off-by: raver119@gmail.com <raver119@gmail.com> * Remove macosx-x86_64-gpu from nd4j-tests-tensorflow * few minor tweaks for IndexReduce Signed-off-by: raver119@gmail.com <raver119@gmail.com> * one test removed Signed-off-by: raver119@gmail.com <raver119@gmail.com> Co-authored-by: raver119@gmail.com <raver119@gmail.com> Co-authored-by: Serhii Shepel <9946053+sshepel@users.noreply.github.com> * RL4J: Add SyncTrainer and AgentLearnerBuilder for a few algorithms (#504) Signed-off-by: Alexandre Boulanger <aboulang2002@yahoo.com> Co-authored-by: Alexandre Boulanger <44292157+aboulang2002@users.noreply.github.com> Co-authored-by: Yurii Shyrma <iuriish@yahoo.com> Co-authored-by: raver119 <raver119@gmail.com> Co-authored-by: Serhii Shepel <9946053+sshepel@users.noreply.github.com> * Removed dead code (#9057) Signed-off-by: Dariusz Zbyrad <dariusz.zbyrad@gmail.com> * performance improvement (#9055) * performance improvement Signed-off-by: Dariusz Zbyrad <dariusz.zbyrad@gmail.com> * revert some changes Signed-off-by: Dariusz Zbyrad <dariusz.zbyrad@gmail.com> * Development updates (#9064) * Update versions of JavaCPP Presets for OpenCV, FFmpeg, and MKL Signed-off-by: Samuel Audet <samuel.audet@gmail.com> * Cherry pick rl4j changes from most recent KonduitAI/deeplearning4j PR * Update cherry pick again from last master revision. Co-authored-by: Samuel Audet <samuel.audet@gmail.com> Co-authored-by: Alexandre Boulanger <44292157+aboulang2002@users.noreply.github.com> Co-authored-by: Yurii Shyrma <iuriish@yahoo.com> Co-authored-by: raver119 <raver119@gmail.com> Co-authored-by: Serhii Shepel <9946053+sshepel@users.noreply.github.com> Co-authored-by: dariuszzbyrad <dariusz.zbyrad@gmail.com> * Ag pythongiloverhaul (#518) * Development updates (#9053) * RL4J: Add generic update rule (#502) Signed-off-by: Alexandre Boulanger <aboulang2002@yahoo.com> * Shyrma reduce (#481) * - start working on improving of cpu legacy code for reduce ops Signed-off-by: Yurii <iuriish@yahoo.com> * - further work on improving legacy loops Signed-off-by: Yurii <iuriish@yahoo.com> * - still working on improving reduce ops Signed-off-by: Yurii <iuriish@yahoo.com> * - further work on improving reduce ops Signed-off-by: Yurii <iuriish@yahoo.com> * - testing speed run of new reduce op Signed-off-by: Yurii <iuriish@yahoo.com> * - working on improvement of default loop for reduce op Signed-off-by: Yurii <iuriish@yahoo.com> * - update signatures of stuff which calls reduce ops Signed-off-by: Yurii <iuriish@yahoo.com> * - make corrections in cuda reduce kernels Signed-off-by: Yurii <iuriish@yahoo.com> * - change loop for default case in broadcast legacy ops Signed-off-by: Yurii <iuriish@yahoo.com> * - comment some shape stuff Signed-off-by: Yurii <iuriish@yahoo.com> * - comment unnecessary prints in RNGtests Signed-off-by: Yurii <iuriish@yahoo.com> * - finish to resolve conflicts after master has been merged Signed-off-by: Yurii <iuriish@yahoo.com> * - get rid of some compilation mistakes of cuda stuff Signed-off-by: Yurii <iuriish@yahoo.com> * - minor changes Signed-off-by: Yurii <iuriish@yahoo.com> * - further search for bug causing crash on java test Signed-off-by: Yurii <iuriish@yahoo.com> * - add scalar case in reduce_ ... exec stuff Signed-off-by: Yurii <iuriish@yahoo.com> * - minor corrections in NAtiveOps.cu Signed-off-by: Yurii <iuriish@yahoo.com> * - add switch to scalar case execReduceXD functions Signed-off-by: Yurii <iuriish@yahoo.com> * - add support for vectors old shape in ConstantShapeHelper::createShapeInfoWithNoUnitiesForReduce Signed-off-by: Yurii <iuriish@yahoo.com> * - correct cuda mirrorPad Signed-off-by: Yurii <iuriish@yahoo.com> * - add support for vectors old shape in cuda createShapeInfoWithNoUnitiesForReduce Signed-off-by: Yurii <iuriish@yahoo.com> Co-authored-by: raver119 <raver119@gmail.com> * Add support for CUDA 11.0 (#492) * Add support for CUDA 11.0 * libnd4j tweaks for CUDA 11 Signed-off-by: raver119@gmail.com <raver119@gmail.com> * bindings update, again? Signed-off-by: raver119@gmail.com <raver119@gmail.com> * * Update versions of JavaCPP Presets for FFmpeg, OpenBLAS, and NumPy * update API to match CUDA 8 Signed-off-by: raver119@gmail.com <raver119@gmail.com> * * Update version of JavaCPP Presets for CPython * C++ updated for cuDNN 8.0 Signed-off-by: raver119@gmail.com <raver119@gmail.com> * one more test Signed-off-by: raver119@gmail.com <raver119@gmail.com> * one more test Signed-off-by: raver119@gmail.com <raver119@gmail.com> * one more test Signed-off-by: raver119@gmail.com <raver119@gmail.com> * 128-bit alignment for workspaces Signed-off-by: raver119@gmail.com <raver119@gmail.com> * change seed in 1 test Signed-off-by: raver119@gmail.com <raver119@gmail.com> * Fix dependecy duplication in python4j-parent pom * Fix group id for in python4j-numpy * few tests tweaked Signed-off-by: raver119@gmail.com <raver119@gmail.com> * Remove macosx-x86_64-gpu from nd4j-tests-tensorflow * few minor tweaks for IndexReduce Signed-off-by: raver119@gmail.com <raver119@gmail.com> * one test removed Signed-off-by: raver119@gmail.com <raver119@gmail.com> Co-authored-by: raver119@gmail.com <raver119@gmail.com> Co-authored-by: Serhii Shepel <9946053+sshepel@users.noreply.github.com> * RL4J: Add SyncTrainer and AgentLearnerBuilder for a few algorithms (#504) Signed-off-by: Alexandre Boulanger <aboulang2002@yahoo.com> Co-authored-by: Alexandre Boulanger <44292157+aboulang2002@users.noreply.github.com> Co-authored-by: Yurii Shyrma <iuriish@yahoo.com> Co-authored-by: raver119 <raver119@gmail.com> Co-authored-by: Serhii Shepel <9946053+sshepel@users.noreply.github.com> * Removed dead code (#9057) Signed-off-by: Dariusz Zbyrad <dariusz.zbyrad@gmail.com> * performance improvement (#9055) * performance improvement Signed-off-by: Dariusz Zbyrad <dariusz.zbyrad@gmail.com> * revert some changes Signed-off-by: Dariusz Zbyrad <dariusz.zbyrad@gmail.com> * Development updates (#9064) * Update versions of JavaCPP Presets for OpenCV, FFmpeg, and MKL Signed-off-by: Samuel Audet <samuel.audet@gmail.com> * Cherry pick rl4j changes from most recent KonduitAI/deeplearning4j PR * Update cherry pick again from last master revision. * Re update python4j Co-authored-by: Samuel Audet <samuel.audet@gmail.com> Co-authored-by: Alexandre Boulanger <44292157+aboulang2002@users.noreply.github.com> Co-authored-by: Yurii Shyrma <iuriish@yahoo.com> Co-authored-by: raver119 <raver119@gmail.com> Co-authored-by: Serhii Shepel <9946053+sshepel@users.noreply.github.com> Co-authored-by: dariuszzbyrad <dariusz.zbyrad@gmail.com> * Bump formatter-maven-plugin from 2.0.0 to 2.12.1 (#505) Bumps [formatter-maven-plugin](https://github.com/revelc/formatter-maven-plugin) from 2.0.0 to 2.12.1. - [Release notes](https://github.com/revelc/formatter-maven-plugin/releases) - [Changelog](https://github.com/revelc/formatter-maven-plugin/blob/formatter-maven-plugin-2.12.1/CHANGELOG.md) - [Commits](https://github.com/revelc/formatter-maven-plugin/compare/formatter-maven-plugin-2.0.0...formatter-maven-plugin-2.12.1) Signed-off-by: dependabot-preview[bot] <support@dependabot.com> Co-authored-by: dependabot-preview[bot] <27856297+dependabot-preview[bot]@users.noreply.github.com> * Ag fix9060 (#519) * Development updates (#9053) * RL4J: Add generic update rule (#502) Signed-off-by: Alexandre Boulanger <aboulang2002@yahoo.com> * Shyrma reduce (#481) * - start working on improving of cpu legacy code for reduce ops Signed-off-by: Yurii <iuriish@yahoo.com> * - further work on improving legacy loops Signed-off-by: Yurii <iuriish@yahoo.com> * - still working on improving reduce ops Signed-off-by: Yurii <iuriish@yahoo.com> * - further work on improving reduce ops Signed-off-by: Yurii <iuriish@yahoo.com> * - testing speed run of new reduce op Signed-off-by: Yurii <iuriish@yahoo.com> * - working on improvement of default loop for reduce op Signed-off-by: Yurii <iuriish@yahoo.com> * - update signatures of stuff which calls reduce ops Signed-off-by: Yurii <iuriish@yahoo.com> * - make corrections in cuda reduce kernels Signed-off-by: Yurii <iuriish@yahoo.com> * - change loop for default case in broadcast legacy ops Signed-off-by: Yurii <iuriish@yahoo.com> * - comment some shape stuff Signed-off-by: Yurii <iuriish@yahoo.com> * - comment unnecessary prints in RNGtests Signed-off-by: Yurii <iuriish@yahoo.com> * - finish to resolve conflicts after master has been merged Signed-off-by: Yurii <iuriish@yahoo.com> * - get rid of some compilation mistakes of cuda stuff Signed-off-by: Yurii <iuriish@yahoo.com> * - minor changes Signed-off-by: Yurii <iuriish@yahoo.com> * - further search for bug causing crash on java test Signed-off-by: Yurii <iuriish@yahoo.com> * - add scalar case in reduce_ ... exec stuff Signed-off-by: Yurii <iuriish@yahoo.com> * - minor corrections in NAtiveOps.cu Signed-off-by: Yurii <iuriish@yahoo.com> * - add switch to scalar case execReduceXD functions Signed-off-by: Yurii <iuriish@yahoo.com> * - add support for vectors old shape in ConstantShapeHelper::createShapeInfoWithNoUnitiesForReduce Signed-off-by: Yurii <iuriish@yahoo.com> * - correct cuda mirrorPad Signed-off-by: Yurii <iuriish@yahoo.com> * - add support for vectors old shape in cuda createShapeInfoWithNoUnitiesForReduce Signed-off-by: Yurii <iuriish@yahoo.com> Co-authored-by: raver119 <raver119@gmail.com> * Add support for CUDA 11.0 (#492) * Add support for CUDA 11.0 * libnd4j tweaks for CUDA 11 Signed-off-by: raver119@gmail.com <raver119@gmail.com> * bindings update, again? Signed-off-by: raver119@gmail.com <raver119@gmail.com> * * Update versions of JavaCPP Presets for FFmpeg, OpenBLAS, and NumPy * update API to match CUDA 8 Signed-off-by: raver119@gmail.com <raver119@gmail.com> * * Update version of JavaCPP Presets for CPython * C++ updated for cuDNN 8.0 Signed-off-by: raver119@gmail.com <raver119@gmail.com> * one more test Signed-off-by: raver119@gmail.com <raver119@gmail.com> * one more test Signed-off-by: raver119@gmail.com <raver119@gmail.com> * one more test Signed-off-by: raver119@gmail.com <raver119@gmail.com> * 128-bit alignment for workspaces Signed-off-by: raver119@gmail.com <raver119@gmail.com> * change seed in 1 test Signed-off-by: raver119@gmail.com <raver119@gmail.com> * Fix dependecy duplication in python4j-parent pom * Fix group id for in python4j-numpy * few tests tweaked Signed-off-by: raver119@gmail.com <raver119@gmail.com> * Remove macosx-x86_64-gpu from nd4j-tests-tensorflow * few minor tweaks for IndexReduce Signed-off-by: raver119@gmail.com <raver119@gmail.com> * one test removed Signed-off-by: raver119@gmail.com <raver119@gmail.com> Co-authored-by: raver119@gmail.com <raver119@gmail.com> Co-authored-by: Serhii Shepel <9946053+sshepel@users.noreply.github.com> * RL4J: Add SyncTrainer and AgentLearnerBuilder for a few algorithms (#504) Signed-off-by: Alexandre Boulanger <aboulang2002@yahoo.com> Co-authored-by: Alexandre Boulanger <44292157+aboulang2002@users.noreply.github.com> Co-authored-by: Yurii Shyrma <iuriish@yahoo.com> Co-authored-by: raver119 <raver119@gmail.com> Co-authored-by: Serhii Shepel <9946053+sshepel@users.noreply.github.com> * Removed dead code (#9057) Signed-off-by: Dariusz Zbyrad <dariusz.zbyrad@gmail.com> * performance improvement (#9055) * performance improvement Signed-off-by: Dariusz Zbyrad <dariusz.zbyrad@gmail.com> * revert some changes Signed-off-by: Dariusz Zbyrad <dariusz.zbyrad@gmail.com> * Development updates (#9064) * Update versions of JavaCPP Presets for OpenCV, FFmpeg, and MKL Signed-off-by: Samuel Audet <samuel.audet@gmail.com> * Added support for the archunit (#9062) * Added support for the archunit Signed-off-by: Dariusz Zbyrad <dariusz.zbyrad@gmail.com> * Updated pom files Signed-off-by: Dariusz Zbyrad <dariusz.zbyrad@gmail.com> * Datavec code cleaup (#9071) * removed unnecessary semicolons Signed-off-by: Dariusz Zbyrad <dariusz.zbyrad@gmail.com> * Use standard charset object Signed-off-by: Dariusz Zbyrad <dariusz.zbyrad@gmail.com> * Removed unused imports Signed-off-by: Dariusz Zbyrad <dariusz.zbyrad@gmail.com> * WIP: Fix Conv1d causal case * Add inital tests * Update Conv1d tests to be a bit more robust * Remove redundant test * Reset from master * Remove cuda definition (left over) * Update rl4j again * Update pom.xml Co-authored-by: Samuel Audet <samuel.audet@gmail.com> Co-authored-by: Alexandre Boulanger <44292157+aboulang2002@users.noreply.github.com> Co-authored-by: Yurii Shyrma <iuriish@yahoo.com> Co-authored-by: raver119 <raver119@gmail.com> Co-authored-by: Serhii Shepel <9946053+sshepel@users.noreply.github.com> Co-authored-by: dariuszzbyrad <dariusz.zbyrad@gmail.com> * Fixes 9061 (#521) * Get rid of edge case in validation * Added support for the archunit (#9062) * Added support for the archunit Signed-off-by: Dariusz Zbyrad <dariusz.zbyrad@gmail.com> * Updated pom files Signed-off-by: Dariusz Zbyrad <dariusz.zbyrad@gmail.com> * Using embedded copying of an array instead of manual (#9073) Signed-off-by: Dariusz Zbyrad <dariusz.zbyrad@gmail.com> * Datavec bulk operation (#9075) * Bulk operation can be used instead of iteration inspection Signed-off-by: Dariusz Zbyrad <dariusz.zbyrad@gmail.com> * Redundant 'Collection.addAll()' call inspection Signed-off-by: Dariusz Zbyrad <dariusz.zbyrad@gmail.com> * Removed infinitely loop (#9076) Signed-off-by: Dariusz Zbyrad <dariusz.zbyrad@gmail.com> Co-authored-by: dariuszzbyrad <dariusz.zbyrad@gmail.com> * Revert "Merge eclipse changes" (#526) * Revert rl4j to |
||
|
|
d57b55ccf3 |
Unused imports (#9092)
* Marked some classes as static Signed-off-by: Dariusz Zbyrad <dariusz.zbyrad@gmail.com> * Removed unused imports Signed-off-by: Dariusz Zbyrad <dariusz.zbyrad@gmail.com> * Revert "Marked some classes as static" This reverts commit 50f7c34a Signed-off-by: Dariusz Zbyrad <dariusz.zbyrad@gmail.com> |
||
|
|
029b84e2b7 |
Development updates (#9053)
* RL4J: Add generic update rule (#502) Signed-off-by: Alexandre Boulanger <aboulang2002@yahoo.com> * Shyrma reduce (#481) * - start working on improving of cpu legacy code for reduce ops Signed-off-by: Yurii <iuriish@yahoo.com> * - further work on improving legacy loops Signed-off-by: Yurii <iuriish@yahoo.com> * - still working on improving reduce ops Signed-off-by: Yurii <iuriish@yahoo.com> * - further work on improving reduce ops Signed-off-by: Yurii <iuriish@yahoo.com> * - testing speed run of new reduce op Signed-off-by: Yurii <iuriish@yahoo.com> * - working on improvement of default loop for reduce op Signed-off-by: Yurii <iuriish@yahoo.com> * - update signatures of stuff which calls reduce ops Signed-off-by: Yurii <iuriish@yahoo.com> * - make corrections in cuda reduce kernels Signed-off-by: Yurii <iuriish@yahoo.com> * - change loop for default case in broadcast legacy ops Signed-off-by: Yurii <iuriish@yahoo.com> * - comment some shape stuff Signed-off-by: Yurii <iuriish@yahoo.com> * - comment unnecessary prints in RNGtests Signed-off-by: Yurii <iuriish@yahoo.com> * - finish to resolve conflicts after master has been merged Signed-off-by: Yurii <iuriish@yahoo.com> * - get rid of some compilation mistakes of cuda stuff Signed-off-by: Yurii <iuriish@yahoo.com> * - minor changes Signed-off-by: Yurii <iuriish@yahoo.com> * - further search for bug causing crash on java test Signed-off-by: Yurii <iuriish@yahoo.com> * - add scalar case in reduce_ ... exec stuff Signed-off-by: Yurii <iuriish@yahoo.com> * - minor corrections in NAtiveOps.cu Signed-off-by: Yurii <iuriish@yahoo.com> * - add switch to scalar case execReduceXD functions Signed-off-by: Yurii <iuriish@yahoo.com> * - add support for vectors old shape in ConstantShapeHelper::createShapeInfoWithNoUnitiesForReduce Signed-off-by: Yurii <iuriish@yahoo.com> * - correct cuda mirrorPad Signed-off-by: Yurii <iuriish@yahoo.com> * - add support for vectors old shape in cuda createShapeInfoWithNoUnitiesForReduce Signed-off-by: Yurii <iuriish@yahoo.com> Co-authored-by: raver119 <raver119@gmail.com> * Add support for CUDA 11.0 (#492) * Add support for CUDA 11.0 * libnd4j tweaks for CUDA 11 Signed-off-by: raver119@gmail.com <raver119@gmail.com> * bindings update, again? Signed-off-by: raver119@gmail.com <raver119@gmail.com> * * Update versions of JavaCPP Presets for FFmpeg, OpenBLAS, and NumPy * update API to match CUDA 8 Signed-off-by: raver119@gmail.com <raver119@gmail.com> * * Update version of JavaCPP Presets for CPython * C++ updated for cuDNN 8.0 Signed-off-by: raver119@gmail.com <raver119@gmail.com> * one more test Signed-off-by: raver119@gmail.com <raver119@gmail.com> * one more test Signed-off-by: raver119@gmail.com <raver119@gmail.com> * one more test Signed-off-by: raver119@gmail.com <raver119@gmail.com> * 128-bit alignment for workspaces Signed-off-by: raver119@gmail.com <raver119@gmail.com> * change seed in 1 test Signed-off-by: raver119@gmail.com <raver119@gmail.com> * Fix dependecy duplication in python4j-parent pom * Fix group id for in python4j-numpy * few tests tweaked Signed-off-by: raver119@gmail.com <raver119@gmail.com> * Remove macosx-x86_64-gpu from nd4j-tests-tensorflow * few minor tweaks for IndexReduce Signed-off-by: raver119@gmail.com <raver119@gmail.com> * one test removed Signed-off-by: raver119@gmail.com <raver119@gmail.com> Co-authored-by: raver119@gmail.com <raver119@gmail.com> Co-authored-by: Serhii Shepel <9946053+sshepel@users.noreply.github.com> * RL4J: Add SyncTrainer and AgentLearnerBuilder for a few algorithms (#504) Signed-off-by: Alexandre Boulanger <aboulang2002@yahoo.com> Co-authored-by: Alexandre Boulanger <44292157+aboulang2002@users.noreply.github.com> Co-authored-by: Yurii Shyrma <iuriish@yahoo.com> Co-authored-by: raver119 <raver119@gmail.com> Co-authored-by: Serhii Shepel <9946053+sshepel@users.noreply.github.com> |
||
|
|
99b85c5006 |
Python4j: bytes conversion fix + test (#497)
* bytes fix+test * bytes fix+test |
||
|
|
e9c13ca9f4 |
Python4j change namespace (org.eclipse->org.nd4j) (#493)
* org.eclipse->org.nd4j * Fix parent groupid Signed-off-by: Alex Black <blacka101@gmail.com> * memview->bytes * del test Co-authored-by: Alex Black <blacka101@gmail.com> |
||
|
|
9ca679e080 |
python4j-numpy (#475)
* 'initial' * 'impl' * tests * <T> * more tests * scalar fixes * lazy setup jobs * more tests * multithreading wip * multithreading fix * bytebuffer working * nits * inplace exec fixes * attempt linux cpu fix * rollback * list fixes * disable gc * log * bump jcpp + fixes * #8985 GradientSharingTrainingTest ignore for logged issue Signed-off-by: Alex Black <blacka101@gmail.com> * memview fixes * fix? Co-authored-by: Alex Black <blacka101@gmail.com> |
||
|
|
bde0a4ec98 |
Python4j (#422)
* types * pom fix * basic exec + tests * safe exec * exec fixes + tests * prim tests * lists and dicts * collections tests * list test * api * exec and return all vars * context manager + fixes * leak fixes * jobs tests * gc basic working * more gc fixed * copyright headers * try-catch-finally * gc fixes * validate var name (startswith _collapsed..) * try block refac * pythonexecutioner nits * hashset->set * call() gc fix * gc fixes * type check fix * types fixes * refacs * rem numpyarray * threadsafety check * private->public * threadsafe checks * pythonGC test * threading fixes + tests * threading tests+ * threading test fixes * make PythonException unchecked * nits * docstrings * path fixes |