Flume HDFS Sink Write error "no protocol: value" - hadoop

Question

When trying to run a flume job I am getting the error given below. I am running this on a cloudera setup.
Kafka is the source
Morphline is used as an interceptor with avro records getting created from it.
Sink is HDFS
The exact same files (morphline, avro schema etc., flume config) on a test environment. But in another environment it throws this error.
2019-07-15 14:24:17,669 WARN org.apache.flume.sink.hdfs.BucketWriter: Caught IOException writing to HDFSWriter (no protocol: value). Closing file (hdfs://8.8.8.8:8020/user/hive/warehouse/folder/folder/FlumeData.1563162656585.tmp) and rethrowing exception.
2019-07-15 14:24:17,670 INFO org.apache.flume.sink.hdfs.BucketWriter: Closing hdfs://8.8.8.8:8020/user/hive/warehouse/folder/folder/FlumeData.1563162656585.tmp
2019-07-15 14:24:17,670 ERROR org.apache.flume.sink.hdfs.HDFSEventSink: process failed
java.lang.NullPointerException
at org.apache.flume.sink.hdfs.AvroEventSerializer.flush(AvroEventSerializer.java:187)
at org.apache.flume.sink.hdfs.HDFSDataStream.close(HDFSDataStream.java:131)
at org.apache.flume.sink.hdfs.BucketWriter$3.call(BucketWriter.java:327)
at org.apache.flume.sink.hdfs.BucketWriter$3.call(BucketWriter.java:323)
at org.apache.flume.sink.hdfs.BucketWriter$9$1.run(BucketWriter.java:701)
at org.apache.flume.auth.SimpleAuthenticator.execute(SimpleAuthenticator.java:50)
at org.apache.flume.sink.hdfs.BucketWriter$9.call(BucketWriter.java:698)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
2019-07-15 14:24:17,671 ERROR org.apache.flume.SinkRunner: Unable to deliver event. Exception follows.
org.apache.flume.EventDeliveryException: java.lang.NullPointerException
at org.apache.flume.sink.hdfs.HDFSEventSink.process(HDFSEventSink.java:451)
at org.apache.flume.sink.DefaultSinkProcessor.process(DefaultSinkProcessor.java:67)
at org.apache.flume.SinkRunner$PollingRunner.run(SinkRunner.java:145)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.NullPointerException
at org.apache.flume.sink.hdfs.AvroEventSerializer.flush(AvroEventSerializer.java:187)
at org.apache.flume.sink.hdfs.HDFSDataStream.close(HDFSDataStream.java:131)
at org.apache.flume.sink.hdfs.BucketWriter$3.call(BucketWriter.java:327)
at org.apache.flume.sink.hdfs.BucketWriter$3.call(BucketWriter.java:323)
at org.apache.flume.sink.hdfs.BucketWriter$9$1.run(BucketWriter.java:701)
at org.apache.flume.auth.SimpleAuthenticator.execute(SimpleAuthenticator.java:50)
at org.apache.flume.sink.hdfs.BucketWriter$9.call(BucketWriter.java:698)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
... 1 more
I was able to locate the relevant code on flume:
https://github.com/apache/flume/blob/trunk/flume-ng-sinks/flume-hdfs-sink/src/main/java/org/apache/flume/sink/hdfs/BucketWriter.java (line:602)
// write the event
try {
sinkCounter.incrementEventDrainAttemptCount();
callWithTimeout(new CallRunner<Void>() {
#Override
public Void call() throws Exception {
writer.append(event); // could block
return null;
}
});
} catch (IOException e) {
LOG.warn("Caught IOException writing to HDFSWriter ({}). Closing file (" +
bucketPath + ") and rethrowing exception.",
e.getMessage());
close(true);
throw e;
}
Error: Caught IOException writing to HDFSWriter (no protocol: value). Closing file
I am not able to workout what the error no protocol: value means.
I am unable to find any reference to this error in any context related to Flume and HDFS.

Answer 1

Incerceptor protocol was missing from the configuration -- Added "file:/" in the flume configuration file which that fixed the issue.
Similar issue reference : https://community.cloudera.com/t5/Data-Ingestion-Integration/Flume-HDFS-sink-error-quot-unknown-protocol-hdfs-quot/td-p/19344