Mutliple config files causing duplicate message - elasticsearch

Question

I have a Logstash machine running in AWS. In Logstash I have 3 config files each having 1 input defined on them. These inputs are reading logs from following sources
From s3
From http input
From filebeat
The problem is that I am getting duplicate messages in Kibana. So for 1 message generated by Filebeat I am seeing 3 messages in Kibana. I tried to remove 1 config file and the count got reduced to 2. So I am pretty sure that this is due to these config files.
What is confusing me is that why this is happening. I have separate input defined on all 3 config files, still getting duplicate messages. These are the input section of all 3 config files.
s3 input
input {
s3 {
bucket => "elb-logs"
region => "us-east-1"
prefix => "demo/AWSLogs/792177735214/"
type => "elb-logs"
delete => true
}
}
Http input
input {
http {
type => "frontend-logs"
codec => "json"
}
}
Filebeat
input {
beats {
port => "5043"
}
}
For all 3 config files there is common output section i.e.
output {
elasticsearch { hosts => [ "10.0.0.1:9200" ] }
}

Answer 1

Logstash will concatenate the three config files together (s3 input, Http input, Filebeat) and see three output sections.
The three output sections are not related to the specific inputs - instead Logstash will send an input from any one of the three sources to all of the configured outputs. As a result your message will be output three times to the same destination.
I would create a separate, single output config file and remove the output section from your 3 input config files.