ignore but recall malformed data : iterate & process folder with bash script + .jar - bash

Question

There is a folder full of files- each of which contains some data that I need to convert to a single output file.
I've built a conversion script- it can run like so:
java -jar tableGenerator.jar -inputfile more-adzuna-jobs-type-9.rdf -skillNames skillNames.ttl -countries countries_europe.rdf -outputcsv out.csv
The problem is- some of the files contain characters that are regarded as invalid by my .jar file, is there a way to create a bash script to run this command simultaneously on a folder full of these files (many hundreds) and for each one that generates an error to:
ignore it, i.e. not let it halt the process
remember it- so that later it can be dealt with appropriately
It seems like this would be possible but my bash-fu is quite weak- what would be a logical way to execute this task?

Answer 1

If your Java program in fact exits with an error status then it should be fairly easy to write a bash script that processes all the files in a folder and tracks which had errors. I emphasize that the Java program must exit with an error (non-zero) status for this to be easy. For example, it should terminate execution by invoking System.exit(1).
If your program does report its success or failure to the system via its exit status, then you might do something like this:
#!/bin/bash
# The name of the directory to process is expected as the first argument.
if [ $# -lt 1 ]; then
echo usage: $0 directory
exit 1
fi
# The first argument to the script is $1
if [ -e failures.txt ]; then
rm failures.txt
fi
touch failures.txt
for f in $1/*; do
if ! java -jar /path/to/tableGenerator.jar \
-inputfile $f \
-skillNames /path/to/skillNames.ttl \
-countries /path/to/countries_europe.rdf \
-outputcsv $f.out.csv
then
echo $f >> failures.txt
fi
done
That iterates over all the files in the directory specified by the first script argument, assigning each path in turn to shell variable $f, and runs your Java program for each one, passing the path as the argument following -inputfile. In the event that the program exits with a non-zero status, the script writes the name of the failing file in file failures.txt in the script's current working directory (unrelated to the data directory designated to it) and continues.
Note that it does not run the command simultaneously on all the files, but instead iteratively. I am uncertain whether that was a key component of your request. Inasmuch as the system you run this on is unlikely to have a separate core it can dedicate to each of hundreds of instances of your program, and inasmuch as the storage medium on which the files reside probably has only one data channel, you cannot effectively run the command hundreds of times simultaneously, anyway.
If you do want to run multiple jobs in parallel then bash has ways to do that, but I recommend getting the serial script working first. If processing the files serially is not good enough then you can explore ways to achieve some parallelism. However, to the extent that Java VM startup time may present an issue with starting up hundreds of JVMs, you might be better off building multiple-file handling directly into your Java program, so that you can process all the files in the same VM.