14 Commits

Author SHA1 Message Date
Adam Gibson 722592d8a9 Update for publishing to snapshots (#10211)
* fix extra quote inserted by intellij

* Fix path

* fix executable permissions usage

* improve directory resolution and restore some files from the previous PR

* remove old unused dep versions
Update to new march 2024 central release plugin

* update central publishing version

* temp disable mirror

* fix hallucinated version

* update maven invoker version for jetspeed

* remove old build properties
add better directory resolution for copy flatc

* fix helpers sources for cpu (api updates)
perform maven antrun upgrade

* more fixes for onednn helpers

* update more antrun usage

* fix pooling param usage

* remove findbugs on the 2 files it was used on

* debug odd build path issues

* try again

* remove bad ls

* try again

* another attempt

* Add troubleshooting logs again

* Remove specific gcc versions

* update build

* cd to original directory instead

* Remove enforcer check

* remove debug steps

* remove extra steps

* update linux, android and cuda versions

* update arm usage defaults

* remove 32 bit builds

* alternative path

* alternative path get rid of old tools

* get rid of over simplified bootstrap libnd4j download usage

* remove extra cmake

* remove manual path setting

* arm compilation fix

* debug armcompute

* update android-x86_64 openblas

* upgrade gcc

* minor armcompute const changes

* update openblas version

* fix armcompute paths

* remove ls

* update cuda versions and envrionment variables

* fix syntax errors

* fix syntax errors

* fix syntax errors

* fix syntax errors

* another rev

* update msys command

* rev

* fix classifier

* rev android-x86_64

* rev android-x86_64

* update cuda env passing

* fix linux-arm64 syntax error

* cuda rev

* cuda rev

* update arm else if branches for matching

* remove external PS script?

* remove external PS script?

* remove external PS script?

* improve architecture detection

* update list operation conv2d armcompute

* add flatbuffer generated code

* remove old maven auth

* remove old legacny average and accumulate

* remove cache due to 422 error

* ensure armcompute is optional

* ensure armcompute is optional

* disk space clean up on all workflows

* deal with windows service issue

* ensure nvcc is on path

* Add arm64 protoc
Update cuda paths

* convert to use numeric types only mitigating lld errors found by clang

* Add arm64 protoc
Update cuda paths

* fix syntax error

* fix overriding properties causing libnd4j not to be built with cuda

* update windows version

* try to update cl.exe paths

* update debugging for nvcc

* update cuda flags to work with linux/windows

* remove extra cuda install

* fix duplicate sources

* more unsupported compiler changes

* remove unneeded source and javadoc

* better cudnn detection

* update where we put unsupported compiler

* Collapse cmake logic

* Collapse cmake logic

* clean up consolidated file

* flatbuffers fix

* fix hallucinated paths

* generate flatbuffers by default

* fix hallucinated paths

* ensure imports present

* fix flatc target order

* fix elif syntax

* more rearrange

* fix git tag

* update template to allow proper configuration generation

* remove guard

* change quotes

* change quotes

* test default

* fix missing functions

* rearrange dependencies

* set the engine

* reintegrate some old cuda logic

* Add back in function defs

* Add back in function defs

* Add back in function defs

* Add back in function defs

* Add back in function defs

* Add back in function defs

* Add back in function defs

* Add back in function defs

* Add back in function defs

* Add back in function defs

* Add back in function defs

* Add back in function defs

* Add back in function defs

* Add back in function defs

* Add back in function defs

* Add back in function defs

* Add back in function defs

* Add back in function defs

* Add back in function defs

* Add back in function defs

* Add back in function defs

* Add back in function defs

* Add back in function defs

* Add back in function defs

* Add back in function defs

* Add back in function defs

* Add back in function defs

* Add back in function defs

* Add back in function defs

* Add back in function defs

* Add back in function defs

* Add back in function defs

* Add back in function defs

* Add back in function defs

* Add back in function defs

* Add back in function defs

* Add back in function defs

* Add back in function defs

* Add back in function defs

* Add back in function defs

* update api to use proper cuda versions

* update api to use proper cuda versions

* update api to use proper cuda versions

* Add back in function defs

* delete old paths

* aDd back include_ops.h generation

* delete old paths

* reinroduce old comand

* delete old paths

* delete old paths

* better tar unpacking

* fix for suffixes

* Add missing cpu sources

* fix duplicate profile

* change mac image

* change mac image

* get rid of old gpg key step

* get rid of javadoc on mac

* remove unneeded deps for mac

* update arm compute path

* remove old command checks

* remove gpg key step

* clang specific type erro

* clang specific type erro

* add another missing cpu path

* add another missing cpu path

* fix helpers sources ordering

* fix helpers sources ordering

* Add back createFromDescriptor

* fix helpers sources ordering

* fix helpers sources ordering

* fix helpers sources ordering

* fix helpers sources ordering

* Add back createFromDescriptor

* add debug for onednn

* add debug for onednn

* SFINAE for type aliases

* SFINAE for type aliases

* SFINAE for type aliases

* SFINAE for type aliases

* ensure we tell compiler we're ok with certain apple version minimums

* ensure we tell compiler we're ok with certain apple version minimums

* ensure we tell compiler we're ok with certain apple version minimums

* ensure we tell compiler we're ok with certain apple version minimums

* fix imports

* update linker path

* fix lock type usage

* change order of sources

* share mutex types

* share mutex types

* share mutex types

* update linker path

* share mutex types

* share mutex types

* share mutex types

* decrease type pairs for sort

* decrease type pairs for sort

* decrease type pairs for sort

* decrease type pairs for sort

* update the special methods to use combinations

* update the special methods to use combinations

* update the special methods to use combinations

* update the special methods to use combinations

* update the special methods to use combinations

* update the special methods to use combinations

* update the special methods to use combinations

* update the special methods to use combinations

* update the special methods to use combinations

* update the special methods to use combinations

* update the special methods to use combinations

* standardize output paths

* fix template paths

* refactor compiler flags

* update onednn to use similar approach to armcompute

* refactor compiler flags

* fix paths

* fix paths

* fix paths

* fix paths

* fix paths

* fix paths

* fix paths

* fix paths

* change target expand types

* change target expand types

* add new pairwise types

* add new pairwise types

* add new pairwise types

* update linker paths

* add new pairwise types

* add new pairwise types

* add new pairwise types

* fix strings with transform

* fix strings with transform

* update linker paths

* update default values for libnd4j.outputPath

* update default values for libnd4j.outputPath

* update default values for libnd4j.outputPath

* update pom.xml namespaces

* Update .github/workflows/build-deploy-linux-cuda-12.6.yml

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Update .github/workflows/build-deploy-linux-x86_64.yml

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

---------

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
2025-05-08 04:52:38 +09:00
Adam Gibson 01f340ea13 Misc 2d grad fixes, Spark test fixes test refactoring (#9735)
* Fix up create_view tests, assign operation for views

* Misc shape related fixes for gradients and tests related to old 2d assumption with vectors

* Ignore missing op

* Fix updater get with 1d standardization

* Fix up another 1d test case

* Update TestRegressionTest080.java

* Update SlicingTests.java

* Fix up booleanindexing backtracklinesearch test cases

* Fix up test assertion

* Remove old tensor array logic for tensorarray remove, fix test

* Address nits, remove noninplacevalidation listener

* Remove tests with old assumptions, migrate arrowserde test to platform-tests

* Add testresources profile to nd4j-presets-common

* More test collapsing

* Collapse tests

* Remove old profiles

* Delete old aurora backend, misc pom clean up

* Delete old test resources profiles

* Update datavec-api test locations

* More test migrations

* More tests

* Migrate tests for spark

* Collapse spark parameter server tests

* Remove old tests

* Collapse omnihub

* Rename tests for compilation errors

* Graph package moving

* Add missing dep

* nlp test adjustments

* Update names in platform-tests datavec local modules

* Add missing nlp imports

* Remove hard coded test, add deepwalk import

* Update pom.xml

* Update pom.xml

* Update pom.xml

* Add mockito dep

* Update TestGraphLoading.java

* Update pom.xml

* Remove deprecated tests, add missing dep

* Clean up intellij accidental refactoring

* Update AeronUdpTransportTest.java

* Update EndingPreProcessorTest.java

* More test updates

* More test refactoring

* More assertion clean up, remove deprecated test

* More test clean up

* Remove more deprecated tests

* More test updates

* More test updates

* More tests

* Update WeightedWalkerTest.java

* Update spark versions, dependencies

* Handle spark upgrade, add a set closeable false to prevent spark from closing arrays early
2022-07-25 07:07:14 +09:00
Adam Gibson 3feca6d831 Python4j performance optimizations (#9688)
* WIP: adding python interpreter and new abstractions

* Add tests for python update

* fix up cuda build

* WIP: minor clean up, add netty shading

* Fix up test execution, ensure python set main thread is restored

* Add caching option for python (makes objects derived available outside of GIL)

* remove unused test case, extra printing
2022-05-11 16:57:35 +09:00
Adam Gibson 8ce2d9df8f Java support Version updates (#9626)
* Add back CI updates

* Fix android compilation error

* Update TopK.java

* Updates project for java 9+

* Remove old grpc server, rename backend packages, change flatbuffers to temp konduit fork for cuda compilation

* More renames + remove extra generated classes

* Update 0016 - Java 9+ Support.md

* Add back license from flatbuffers generation

* Remove solr modules, update java 9 module comments (fixed some leaving some ignored such as anything spark related)

* Update pom.xml

* Add cuda compute capabilities to reduce build times

* Add default flags for only building certain modules for cuda on windows

* Add back java 8 support

* Update build-deploy-windows-cuda-11.2.yml

* Update build-deploy-windows-cuda-11.4.yml

* Update build-deploy-windows-cuda-11.2.yml

* Update ccs, remove defaults

* Update build-deploy-linux-cuda-11.4.yml

* Add module info

* Update IndexedTail.java

* Add also make and remove deeplearning4j-cuda from build

* Add also make and remove deeplearning4j-cuda from build

* Update IndexedTail.java

* Add missing cuda modules

* Remove old unused concurrency module
2022-02-09 19:25:39 +09:00
Adam Gibson cc9f62fbd3 Update pom.xmls to add dynamic cuda versions to tests 2021-05-02 21:31:25 +09:00
agibsonccc 5e8951cd8e Add tags for junit 5 2021-03-20 19:06:24 +09:00
agibsonccc 7c7f9db097 Fix classpathresource for python script 2021-03-19 16:48:25 +09:00
agibsonccc 79191c5b6c Add profiles to python4j submodules 2021-03-18 13:37:42 +09:00
Celebrate-future d93976922a Update pom.xml 2021-02-26 11:03:36 +08:00
agibsonccc c715aea405 Update LICENSE 2021-02-01 17:47:29 +09:00
agibsonccc 65c6a9a42e Dev commits 2021-02-01 14:31:20 +09:00
Fariz Rahman e9c13ca9f4 Python4j change namespace (org.eclipse->org.nd4j) (#493)
* org.eclipse->org.nd4j

* Fix parent groupid

Signed-off-by: Alex Black <blacka101@gmail.com>

* memview->bytes

* del test

Co-authored-by: Alex Black <blacka101@gmail.com>
2020-06-25 13:05:08 +10:00
Fariz Rahman 9ca679e080 python4j-numpy (#475)
* 'initial'

* 'impl'

* tests

* <T>

* more tests

* scalar fixes

* lazy setup jobs

* more tests

* multithreading wip

* multithreading fix

* bytebuffer working

* nits

* inplace exec fixes

* attempt linux cpu fix

* rollback

* list fixes

* disable gc

* log

* bump jcpp + fixes

* #8985 GradientSharingTrainingTest ignore for logged issue

Signed-off-by: Alex Black <blacka101@gmail.com>

* memview fixes

* fix?

Co-authored-by: Alex Black <blacka101@gmail.com>
2020-06-16 11:43:10 +10:00
Fariz Rahman bde0a4ec98 Python4j (#422)
* types

* pom fix

* basic exec + tests

* safe exec

* exec fixes + tests

* prim tests

* lists and dicts

* collections tests

* list test

* api

* exec and return all vars

* context manager + fixes

* leak fixes

* jobs tests

* gc basic working

* more gc fixed

* copyright headers

* try-catch-finally

* gc fixes

* validate var name (startswith _collapsed..)

* try block refac

* pythonexecutioner nits

* hashset->set

* call() gc fix

* gc fixes

* type check fix

* types fixes

* refacs

* rem numpyarray

* threadsafety check

* private->public

* threadsafe checks

* pythonGC test

* threading fixes + tests

* threading tests+

* threading test fixes

* make PythonException unchecked

* nits

* docstrings

* path fixes
2020-05-21 11:47:12 +10:00