dl4j_coursera
Run mvn clean install to build jar with dependencies.
Full example of importing a serialized keras 2 model, instantiating a distributed DL4J model with Spark and evaluating model performance on data read from a CSV file:
$SPARK_HOME/bin/spark-submit \
--class skymind.dsx.KerasImportCSVSparkRunner \
--files iris.txt,model.h5 \
target/dl4j-keras-runner-0.1-jar-with-dependencies.jar \
-useSparkLocal true \
-batchSizePerWorker 15 \
-indexLabel 4 \
-numClasses 3 \
-modelFileName model.h5 \
-dataFileName iris.txt
For a keras 1 model, try model_keras_1.h5 instead.
In case you just have a model, but no data in CSV format to train or evaluate it with,
set -loadData false. The model runner will then just import the model into DL4J and
exit.
To train a model, use -train true and set the number of epochs with the -epochs flag.
You can upload jars to DSX with wget, for instance:
! wget target/dl4j-keras-runner-0.1-jar-with-dependencies.jar
To run this on a cluster, e.g. with DSX, you need to specify the Spark master $MASTER and remove useSparkLocal, i.e. run:
$SPARK_HOME/bin/spark-submit \
--class skymind.dsx.KerasImportCSVSparkRunner \
--files iris.txt,model.h5 \
--master $MASTER \
target/dl4j-keras-runner-0.1-jar-with-dependencies.jar \
-batchSizePerWorker 15 \
-indexLabel 4 \
-numClasses 3 \
-modelFileName model.h5 \
-dataFileName iris.txt