How to use File lock in Spring batch-integration? - spring

Question

In my spring-batch-integration app, file polling invokes the batchjob for eachfile and this application could be running on multiple servers(nodes) but they all are supposed to read a common directory.Now, I wrote a custom locker which takes the lock on file so that any other instance will not be able to process the same file . code as below
public class MyFileLocker extends AbstractFileLockerFilter{
private final ConcurrentMap<File, FileLock> lockCache = new ConcurrentHashMap<File, FileLock>();
private final ConcurrentMap<File, FileChannel> ChannelCache = new ConcurrentHashMap<File, FileChannel>();
#Override
public boolean lock(File fileToLock) {
FileChannel channel;
FileLock lock;
try {
channel = new RandomAccessFile(fileToLock, "rw").getChannel();
lock = channel.tryLock();
if (lock == null || !lock.isValid()) {
System.out.println(" Problem in acquiring lock!!" + fileToLock);
return false;
}
lockCache.put(fileToLock, lock);
ChannelCache.put(fileToLock, channel);
} catch (FileNotFoundException e) {
// TODO Auto-generated catch block
e.printStackTrace();
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
return true;
}
#Override
public boolean isLockable(File file) {
return file.canWrite();
}
#Override
public void unlock(File fileToUnlock) {
FileLock lock = lockCache.get(fileToUnlock);
try {
if(lock!=null){
lock.release();
ChannelCache.get(fileToUnlock).close();
}
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
}
Now, when i invoke my Spring batch and i try to read that file using flatfileitemreader it gives me
org.springframework.batch.item.file.NonTransientFlatFileException
which i believe is coming beacuse file is locked. I did some googling and found that NIOLocker locks the file in a way that even the current thread can't read it. I found a link where it shows how to read the locked file but they are using buffer.
How can I make my file accessible to my FlatfileItemReader.
Please suggest.

Answer 1

Yes, you really can get access to the locked file content only over ByteBuffer:
FileChannel fileChannel = channelCache.get(lockedFile);
ByteBuffer byteBuffer = ByteBuffer.allocate((int) fileChannel.size());
fileChannel.read(byteBuffer);
System.out.println("Read File " + lockedFile.getName() + " with content: " + new String(byteBuffer.array()));
Oh! Yeah. You really pointed to my repo :-).
So, with locker you don't have choice unless copy/paste the file byte[] that way before FlatfileItemReader or just inject some custom BufferedReaderFactory into the same FlatfileItemReader, which converts the locked file to the appropriate BufferedReader:
new BufferedReader(new CharArrayReader(byteBuffer.asCharBuffer().array()));

Answer 2

Based on the link that you shared, looks like you could try the following.
//This is from your link (except the size variable)
FileChannel fileChannel = channelCache.get(lockedFile);
int size = (int) fileChannel.size();
ByteBuffer byteBuffer = ByteBuffer.allocate(size);
fileChannel.read(byteBuffer);
//Additional code that you could try
byte[] bArray = new byte[size];
//Write data to the byte array
byteBuffer.get(bArray);
FlatFileItemReader flatFileItemReader = new FlatFileItemReader();
flatFileItemReader.setResource(new ByteArrayResource(bArray));
//Next you can try reading from your flatFileItemReader as usual
...
Let me know if it doesn't progress your issue.

Answer 3

Solution : I am creating a temporary file with content of the locked file and processing it. Once processing is done i archive that file and remove the locked and temporary both files. Key here is to create a new file with locked file content. code for below is as follows :
File tmpFile = new File(inputFile.getAbsolutePath() + ".lck");
FileChannel fileChannel = MyFileLocker.getChannelCache().get(new File(inputFile.getAbsolutePath()));
InputStream inputStream = Channels.newInputStream(fileChannel);
ByteStreams.copy(inputStream, Files.newOutputStreamSupplier(tmpFile));
Here inputFile is my locked file and tmp file is new file with locked file content. I have also created a method in my locker class to unlock and delete
public void unlockAndDelete(File fileToUnlockandDelete) {
FileLock lock = lockCache.get(fileToUnlockandDelete);
String fileName = fileToUnlockandDelete.getName();
try {
if(lock!=null){
lock.release();
channelCache.get(fileToUnlockandDelete).close();
//remove from cache
lockCache.remove(fileToUnlockandDelete);
channelCache.remove(fileToUnlockandDelete);
boolean isFiledeleted = fileToUnlockandDelete.delete();
if(isFiledeleted){
System.out.println("File deleted successfully" + fileName);
}else{
System.out.println("File is not deleted."+fileName);
}
}
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}