oozie wofklow intermittently fails on java action for scalding - hadoop

Question

I am using CDH (Cloudera Hadoop) version 5.12.0 (which uses: Hadoop 2.6.0 Oozie 4.1.0) and Scalding 2.11
I am using a shaded jar with my dependencies built in.
I can run all my jobs properly without any error using a hadoop jar command as such:
hadoop jar /<path>/<to>/<my>/<project>.jar com.twitter.scalding.Tool
-libjars <comma>,<separated>,<list>,<of>,<dependencies>
-D mapreduce.framework.name=yarn
-D yarn.app.mapreduce.am.staging-dir=/<path>/<to>/<staging>/<dir>
-D mapreduce.map.output.compress=True
<my>.<scalding>.<job> --hdfs --input <my>/<input> --output <output>/<dir>
I can also run an oozie workflow with pig actions or other actions without any trouble. But when I try to run an oozie workflow with java actions that call scalding like so:
<action name="myAction">
<java>
<job-tracker>${jobTracker}</job-tracker>
<name-node>${nameNode}</name-node>
<prepare>
<delete path="<output>/<dir>"/>
</prepare>
<configuration>
<property>
<name>oozie.action.sharelib.for.java</name>
<value>scalding</value>
</property>
<property>
<name>oozie.launcher.mapreduce.task.classpath.user.precedence</name>
<value>true</value>
</property>
<property>
<name>oozie.launcher.mapreduce.task.classpath.first</name>
<value>true</value>
</property>
<property>
<name>oozie.launcher.mapreduce.job.user.classpath.first</name>
<value>true</value>
</property>
<property>
<name>mapreduce.job.user.classpath.first</name>
<value>true</value>
</property>
<property>
<name>mapreduce.task.classpath.user.precedence</name>
<value>true</value>
</property>
<property>
<name>mapreduce.task.classpath.first</name>
<value>true</value>
</property>
<property>
<name>mapreduce.map.log.level</name>
<value>DEBUG</value>
</property>
<property>
<name>mapreduce.reduce.log.level</name>
<value>DEBUG</value>
</property>
<property>
<name>mapreduce.job.log4j-properties-file</name>
<value>${appLocation}/conf</value>
</property>
<property>
<name>mapreduce.task.files.preserve.failedtasks</name>
<value>true</value>
</property>
<property>
<name>mapreduce.job.jvm.numtasks</name>
<value>-1</value>
</property>
<property>
<name>oozie.mapreduce.uber.jar</name>
<value>/<path>/<to>/<my>/<class>.jar</value>
</property>
</configuration>
<main-class>com.twitter.scalding.Tool</main-class>
<java-opts>-Xms2G -Xmx3G</java-opts>
<arg><my>.<scalding>.<job></arg>
<arg>--hdfs</arg>
<arg>--input</arg>
<arg><my>/<input></arg>
<arg>--output</arg>
<arg><output>/<dir></arg>
<file>/<path>/<to>/<my>/<project>.jar</file>
</java>
<ok to="end" />
<error to="sendEmailFailure" />
I have also set:
oozie.libpath=<path>/<to>/<lib>/<directory>
oozie.use.system.libpath=true
in my properties file and all the necessary jars are within the libpath, but I get errors where the java action can't find the dependencies it needs:
Error: java.io.IOException: Split class cascading.tap.hadoop.io.MultiInputSplit not found at
org.apache.hadoop.mapred.MapTask.getSplitDetails(MapTask.java:363) at
org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:432) at
org.apache.hadoop.mapred.MapTask.run(MapTask.java:343) at
org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:164) at
java.security.AccessController.doPrivileged(Native Method) at
javax.security.auth.Subject.doAs(Subject.java:422) at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1917) at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)
Caused by: java.lang.ClassNotFoundException: Class cascading.tap.hadoop.io.MultiInputSplit not found at
org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:2108) at org.apache.hadoop.mapred.MapTask.getSplitDetails(MapTask.java:361) ... 7 more
This happens inconsistently, in that if I only put one action in the workflow, sometimes the workflow completes, sometimes it fails. If I put more than one action in the workflow, it definitely fails on some action eventually, not always the same action but always with the same error.
I believe the problem is coming from the way in which the action is getting it's dependencies and sometimes it loads them in the correct order and gets the one I want first and sometimes it doesn't and is missing the dependency it needs. But I'm new to oozie, so who knows?
I think I can increase the number of max attempts for each action taken by oozie, but I feel like that is not really a solution and is sort of akin to rolling the dice against the cluster.
I saw people post this issue a while back and rolling back to an older verison of CDH (4.1) but that's not an option for me.
Suggestions?