* Replace wildcard lombok imports with single class imports
This helps improve compatibility with delombok
* Removed unused lombok imports
Delombok doesn't remove unused imports
* ADR for renamespacing and release updates
* ADR for renamespacing and release updates
* use new secrets for publishing
* use new secrets for publishing
* remove old distribution management
* remove try catch for cuda build
* update string functionality for cuda
* update string functionality for cuda
* update string functionality for cuda
* update string functionality for cuda
* update string functionality for cuda
* update string functionality for cuda
* remove old distribution management
* remove more ossrh references
* update environment variables
* update lombok version
* clean up test dependencies errant linker flags
* get rid of dup flags
* fix extra quote inserted by intellij
* Fix path
* fix executable permissions usage
* improve directory resolution and restore some files from the previous PR
* remove old unused dep versions
Update to new march 2024 central release plugin
* update central publishing version
* temp disable mirror
* fix hallucinated version
* update maven invoker version for jetspeed
* remove old build properties
add better directory resolution for copy flatc
* fix helpers sources for cpu (api updates)
perform maven antrun upgrade
* more fixes for onednn helpers
* update more antrun usage
* fix pooling param usage
* remove findbugs on the 2 files it was used on
* debug odd build path issues
* try again
* remove bad ls
* try again
* another attempt
* Add troubleshooting logs again
* Remove specific gcc versions
* update build
* cd to original directory instead
* Remove enforcer check
* remove debug steps
* remove extra steps
* update linux, android and cuda versions
* update arm usage defaults
* remove 32 bit builds
* alternative path
* alternative path get rid of old tools
* get rid of over simplified bootstrap libnd4j download usage
* remove extra cmake
* remove manual path setting
* arm compilation fix
* debug armcompute
* update android-x86_64 openblas
* upgrade gcc
* minor armcompute const changes
* update openblas version
* fix armcompute paths
* remove ls
* update cuda versions and envrionment variables
* fix syntax errors
* fix syntax errors
* fix syntax errors
* fix syntax errors
* another rev
* update msys command
* rev
* fix classifier
* rev android-x86_64
* rev android-x86_64
* update cuda env passing
* fix linux-arm64 syntax error
* cuda rev
* cuda rev
* update arm else if branches for matching
* remove external PS script?
* remove external PS script?
* remove external PS script?
* improve architecture detection
* update list operation conv2d armcompute
* add flatbuffer generated code
* remove old maven auth
* remove old legacny average and accumulate
* remove cache due to 422 error
* ensure armcompute is optional
* ensure armcompute is optional
* disk space clean up on all workflows
* deal with windows service issue
* ensure nvcc is on path
* Add arm64 protoc
Update cuda paths
* convert to use numeric types only mitigating lld errors found by clang
* Add arm64 protoc
Update cuda paths
* fix syntax error
* fix overriding properties causing libnd4j not to be built with cuda
* update windows version
* try to update cl.exe paths
* update debugging for nvcc
* update cuda flags to work with linux/windows
* remove extra cuda install
* fix duplicate sources
* more unsupported compiler changes
* remove unneeded source and javadoc
* better cudnn detection
* update where we put unsupported compiler
* Collapse cmake logic
* Collapse cmake logic
* clean up consolidated file
* flatbuffers fix
* fix hallucinated paths
* generate flatbuffers by default
* fix hallucinated paths
* ensure imports present
* fix flatc target order
* fix elif syntax
* more rearrange
* fix git tag
* update template to allow proper configuration generation
* remove guard
* change quotes
* change quotes
* test default
* fix missing functions
* rearrange dependencies
* set the engine
* reintegrate some old cuda logic
* Add back in function defs
* Add back in function defs
* Add back in function defs
* Add back in function defs
* Add back in function defs
* Add back in function defs
* Add back in function defs
* Add back in function defs
* Add back in function defs
* Add back in function defs
* Add back in function defs
* Add back in function defs
* Add back in function defs
* Add back in function defs
* Add back in function defs
* Add back in function defs
* Add back in function defs
* Add back in function defs
* Add back in function defs
* Add back in function defs
* Add back in function defs
* Add back in function defs
* Add back in function defs
* Add back in function defs
* Add back in function defs
* Add back in function defs
* Add back in function defs
* Add back in function defs
* Add back in function defs
* Add back in function defs
* Add back in function defs
* Add back in function defs
* Add back in function defs
* Add back in function defs
* Add back in function defs
* Add back in function defs
* Add back in function defs
* Add back in function defs
* Add back in function defs
* Add back in function defs
* update api to use proper cuda versions
* update api to use proper cuda versions
* update api to use proper cuda versions
* Add back in function defs
* delete old paths
* aDd back include_ops.h generation
* delete old paths
* reinroduce old comand
* delete old paths
* delete old paths
* better tar unpacking
* fix for suffixes
* Add missing cpu sources
* fix duplicate profile
* change mac image
* change mac image
* get rid of old gpg key step
* get rid of javadoc on mac
* remove unneeded deps for mac
* update arm compute path
* remove old command checks
* remove gpg key step
* clang specific type erro
* clang specific type erro
* add another missing cpu path
* add another missing cpu path
* fix helpers sources ordering
* fix helpers sources ordering
* Add back createFromDescriptor
* fix helpers sources ordering
* fix helpers sources ordering
* fix helpers sources ordering
* fix helpers sources ordering
* Add back createFromDescriptor
* add debug for onednn
* add debug for onednn
* SFINAE for type aliases
* SFINAE for type aliases
* SFINAE for type aliases
* SFINAE for type aliases
* ensure we tell compiler we're ok with certain apple version minimums
* ensure we tell compiler we're ok with certain apple version minimums
* ensure we tell compiler we're ok with certain apple version minimums
* ensure we tell compiler we're ok with certain apple version minimums
* fix imports
* update linker path
* fix lock type usage
* change order of sources
* share mutex types
* share mutex types
* share mutex types
* update linker path
* share mutex types
* share mutex types
* share mutex types
* decrease type pairs for sort
* decrease type pairs for sort
* decrease type pairs for sort
* decrease type pairs for sort
* update the special methods to use combinations
* update the special methods to use combinations
* update the special methods to use combinations
* update the special methods to use combinations
* update the special methods to use combinations
* update the special methods to use combinations
* update the special methods to use combinations
* update the special methods to use combinations
* update the special methods to use combinations
* update the special methods to use combinations
* update the special methods to use combinations
* standardize output paths
* fix template paths
* refactor compiler flags
* update onednn to use similar approach to armcompute
* refactor compiler flags
* fix paths
* fix paths
* fix paths
* fix paths
* fix paths
* fix paths
* fix paths
* fix paths
* change target expand types
* change target expand types
* add new pairwise types
* add new pairwise types
* add new pairwise types
* update linker paths
* add new pairwise types
* add new pairwise types
* add new pairwise types
* fix strings with transform
* fix strings with transform
* update linker paths
* update default values for libnd4j.outputPath
* update default values for libnd4j.outputPath
* update default values for libnd4j.outputPath
* update pom.xml namespaces
* Update .github/workflows/build-deploy-linux-cuda-12.6.yml
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* Update .github/workflows/build-deploy-linux-x86_64.yml
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
---------
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* add new smoke tests
fix scalar index when using the indexing api
fix usage of index2coords by migrating shape information
to shape::shapeOf
fix usage of coords2index by migrating accidental shapeOf invocations to strides
* Fix reduce3 buffer() calls in op execution
* fix test that didn't account for overlapping views
* remove print statements
fix scalar index when using the indexing api
fix usage of index2coords by migrating shape information
to shape::shapeOf
fix usage of coords2index by migrating accidental shapeOf invocations to strides
* Fix up create_view tests, assign operation for views
* Misc shape related fixes for gradients and tests related to old 2d assumption with vectors
* Ignore missing op
* Fix updater get with 1d standardization
* Fix up another 1d test case
* Update TestRegressionTest080.java
* Update SlicingTests.java
* Fix up booleanindexing backtracklinesearch test cases
* Fix up test assertion
* Remove old tensor array logic for tensorarray remove, fix test
* Address nits, remove noninplacevalidation listener
* Remove tests with old assumptions, migrate arrowserde test to platform-tests
* Add testresources profile to nd4j-presets-common
* More test collapsing
* Collapse tests
* Remove old profiles
* Delete old aurora backend, misc pom clean up
* Delete old test resources profiles
* Update datavec-api test locations
* More test migrations
* More tests
* Migrate tests for spark
* Collapse spark parameter server tests
* Remove old tests
* Collapse omnihub
* Rename tests for compilation errors
* Graph package moving
* Add missing dep
* nlp test adjustments
* Update names in platform-tests datavec local modules
* Add missing nlp imports
* Remove hard coded test, add deepwalk import
* Update pom.xml
* Update pom.xml
* Update pom.xml
* Add mockito dep
* Update TestGraphLoading.java
* Update pom.xml
* Remove deprecated tests, add missing dep
* Clean up intellij accidental refactoring
* Update AeronUdpTransportTest.java
* Update EndingPreProcessorTest.java
* More test updates
* More test refactoring
* More assertion clean up, remove deprecated test
* More test clean up
* Remove more deprecated tests
* More test updates
* More test updates
* More tests
* Update WeightedWalkerTest.java
* Update spark versions, dependencies
* Handle spark upgrade, add a set closeable false to prevent spark from closing arrays early
* Add back CI updates
* Fix android compilation error
* Update TopK.java
* Updates project for java 9+
* Remove old grpc server, rename backend packages, change flatbuffers to temp konduit fork for cuda compilation
* More renames + remove extra generated classes
* Update 0016 - Java 9+ Support.md
* Add back license from flatbuffers generation
* Remove solr modules, update java 9 module comments (fixed some leaving some ignored such as anything spark related)
* Update pom.xml
* Add cuda compute capabilities to reduce build times
* Add default flags for only building certain modules for cuda on windows
* Add back java 8 support
* Update build-deploy-windows-cuda-11.2.yml
* Update build-deploy-windows-cuda-11.4.yml
* Update build-deploy-windows-cuda-11.2.yml
* Update ccs, remove defaults
* Update build-deploy-linux-cuda-11.4.yml
* Add module info
* Update IndexedTail.java
* Add also make and remove deeplearning4j-cuda from build
* Add also make and remove deeplearning4j-cuda from build
* Update IndexedTail.java
* Add missing cuda modules
* Remove old unused concurrency module
* Refactor nd4j-common: org.nd4j.* -> org.nd4j.common.*
Signed-off-by: Alex Black <blacka101@gmail.com>
* Fix CUDA (missed nd4j-common package refactoring changes)
Signed-off-by: Alex Black <blacka101@gmail.com>
* nd4j-kryo: org.nd4j -> org.nd4j.kryo
Signed-off-by: Alex Black <blacka101@gmail.com>
* Fix nd4j-common for deeplearning4j-cuda
Signed-off-by: Alex Black <blacka101@gmail.com>
* nd4j-grppc-client: org.nd4j.graph -> org.nd4j.remote.grpc
Signed-off-by: Alex Black <blacka101@gmail.com>
* deeplearning4j-common: org.deeplearning4.* -> org.deeplearning4j.common.*
Signed-off-by: Alex Black <blacka101@gmail.com>
* deeplearning4j-core: org.deeplearning4j.* -> org.deeplearning.core.*
Signed-off-by: Alex Black <blacka101@gmail.com>
* deeplearning4j-cuda: org.deeplearning4j.nn.layers.* -> org.deeplearning4j.cuda.*
Signed-off-by: Alex Black <blacka101@gmail.com>
* Import fixes
Signed-off-by: Alex Black <blacka101@gmail.com>
* deeplearning4j-nlp-*: org.deeplearning4.text.* -> org.deeplearning4j.nlp.(language).*
Signed-off-by: Alex Black <blacka101@gmail.com>
* deeplearning4j-ui-model: org.deeplearning4j.ui -> org.deeplearning4j.ui.model
Signed-off-by: Alex Black <blacka101@gmail.com>
* datavec-spark-inference-{server/model/client}: org.datavec.spark.transform -> org.datavec.spark.inference.{server/model/client}
Signed-off-by: Alex Black <blacka101@gmail.com>
* datavec-jdbc: org.datavec.api -> org.datavec.jdbc
Signed-off-by: Alex Black <blacka101@gmail.com>
* Delete org.deeplearning4j.datasets.iterator.impl.MultiDataSetIteratorAdapter in favor of (essentially identical) org.nd4j.linalg.dataset.adapter.MultiDataSetIteratorAdapter
Signed-off-by: Alex Black <blacka101@gmail.com>
* ND4S fixes
Signed-off-by: Alex Black <blacka101@gmail.com>
* Fixes
Signed-off-by: Alex Black <blacka101@gmail.com>
* nd4j-common-tests: org.nd4j.* -> org.nd4j.common.tests
Signed-off-by: Alex Black <blacka101@gmail.com>
* Trigger CI
Signed-off-by: Alex Black <blacka101@gmail.com>
* Fixes
Signed-off-by: Alex Black <blacka101@gmail.com>
* #8878 Ignore CUDA tests on modules with 'nd4j-native under cuda' issue
Signed-off-by: Alex Black <blacka101@gmail.com>
* Fix bad imports in tests
Signed-off-by: Alex Black <blacka101@gmail.com>
* Add ignore on test (already failing) due to #8882
Signed-off-by: Alex Black <blacka101@gmail.com>
* Import fixes
Signed-off-by: Alex Black <blacka101@gmail.com>
* Additional import fixes
Signed-off-by: Alex Black <blacka101@gmail.com>
* Move locations initialization for file input splits to initialize method
* Little optimization for schema detection regular expression
Signed-off-by: Paul Dubs <paul.dubs@gmail.com>
* Fixing issues from Sonar report
* Proper logger of exceptions
* Coding style fixes
* Use dup parameter
* Cleanup, minor issues
* Cuda compilation fixed and some minor fixes
* Input format extended
* Deleted redundant code
* Added weights format to conv2d config
* Refactoring
* dl4j base test functionality
* Different tests base class per module
* Check base class for dl4j-graph subproject tests
* Check if test classes extend BaseDL4JTest
* Use nd4j-common-tests as transient dependency
* Enums and tests added
* Added codegenerated methods
* Use namespace methods
* Replace DifferentialFunctionFactory with codegenerated classes
* Fixed linspace
* Namespaces regenerated
* Namespaces used instead of factory
* Regenerated base classes
* Input format extended
* Added weights format to conv2d config
* Refactoring
* dl4j base test functionality
* Different tests base class per module
* Check base class for dl4j-graph subproject tests
* Check if test classes extend BaseDL4JTest
* Use nd4j-common-tests as transient dependency
* Enums and tests added
* Added codegenerated methods
* Use namespace methods
* Replace DifferentialFunctionFactory with codegenerated classes
* Fixed linspace
* Namespaces regenerated
* Regenerated base classes
* Regenerated namespaces
* Generate nd4j namespaces
* INDArrays accepting constructors
* Generated some ops
* Some fixes
* SameDiff ops regenerated
* Regenerated nd4j ops
* externalErrors moved
* Compilation fixes
* SquaredDifference - strict number of args
* Deprecated code cleanup. Proper base class for tests.
* Extend test classes with BaseND4JTest
* Extend test classes with BaseDL4JTest
* Legacy code
* DL4J cleanup
* Exclude test utils from base class check
* Tests fixed
* Arbiter tests fix
* Test dependency scope fix + pom.xml formatting
Signed-off-by: Alex Black <blacka101@gmail.com>
* Significant number of fixes
Signed-off-by: Alex Black <blacka101@gmail.com>
* Another round of fixes
Signed-off-by: Alex Black <blacka101@gmail.com>
* Another round of fixes
Signed-off-by: Alex Black <blacka101@gmail.com>
* Few additional fixes
Signed-off-by: Alex Black <blacka101@gmail.com>
* DataVec missing test scope dependencies
Signed-off-by: Alex Black <blacka101@gmail.com>
Co-authored-by: Alex Black <blacka101@gmail.com>
* Starting to switch configs of RL algorithms to use more fluent builder patterns. Many parameter choices in different algorithms default to SOTA and only be changed in specific cases
Signed-off-by: Bam4d <chris.bam4d@gmail.com>
* remove personal gpu-build file
Signed-off-by: Bam4d <chrisbam4d@gmail.com>
* refactored out configurations so they are heirarchical and re-usable, this is a step towards having a plug-and-play framework for different algorithms
* backwardly compatible configurations
* adding documentation to new configuration classes
Signed-off-by: Bam4d <chrisbam4d@gmail.com>
* private access modifiers are better suited here
Signed-off-by: Bam4d <chrisbam4d@gmail.com>
* RL4j does not compile without java 8 due to previous updates
fixing null pointers when listener arrays are empty
Signed-off-by: Bam4d <chrisbam4d@gmail.com>
* fixing copyright headers
Signed-off-by: Bam4d <chrisbam4d@gmail.com>
* uncomment logging line
Signed-off-by: Bam4d <chrisbam4d@gmail.com>
* fixing default value for learningUpdateFrequency
fixing test failure due to #352
Signed-off-by: Bam4d <chrisbam4d@gmail.com>
Co-authored-by: Bam4d <chris.bam4d@gmail.com>
* Adding more datatypes support in datavec-python
* Using numpy C API for creating numpy arrays
* Adding parameterized tests
* Adding support for BFLOAT16 (by converting it to FLOAT)
* Cleanup
* Using casting instead of creating an array
* Giving out a warning while casting array from BFLOAT16 to FLOAT
* Add syncToPrimary and syncToSpecial methods to BaseDataBuffer
Signed-off-by: Alex Black <blacka101@gmail.com>
* Python exec: sync to host before passing pointers
Signed-off-by: Alex Black <blacka101@gmail.com>
* Added copyright header
* use np api (#267)
* python exec / numpy - check object type before cast (#268)
* use np api
* verify object before cast
* fix cong
* cuda fix
* inplace test + tiny fix
* more test
* fix double alloc
* rem tags
* fix cuda check
* Fix implicit CUDA dependency in datavec-python tests; remove new method, add test
Signed-off-by: Alex Black <blacka101@gmail.com>
Co-authored-by: Alex Black <blacka101@gmail.com>
Co-authored-by: Fariz Rahman <farizrahman4u@gmail.com>
* wrapper
* builtins
* context mgr
* direct place
* ret all var
* call fix
* fix ndarray serde
* jobs
* try-with gil management
* cleanup
* exec tests passing
* list tests
* transforms test passing
* all pass
* headers
* dict fixes+test
* python path
* bool isinstance
* job tests
* nits
* transform fix+test
* transform tests
* leak fixes
* more mem leak fixes
* more fixes
* nits for adam
* PythonJob lombok builder
* checked exceptions
* more nits
* small leak fix
* more nits
* pythonexceptions
* fix jvm crash when bad python code
* Exception->PythonException
* Add support for boolean types in arrow records and ability to cast from float, double to int for TypeConversion (#178)
* nits for alex
* update tests
* fix test
* all pass
* refacc
* rem old code
* dtypes
* bytes working+exception pass through+cleanup (#209)
* more bytes tests
* header
* rem dummy test
* rem bad import
* alex nits + refacc
* Small error fixes (wrong type in msg) + minor formatting
Signed-off-by: AlexDBlack <blacka101@gmail.com>
* use actual python type names (dictionary->dict, boolean->bool)
Co-authored-by: Shams Ul Azeem <shamsazeem20@gmail.com>
Co-authored-by: Alex Black <blacka101@gmail.com>
DRayX