run filebeat as ansible - ansible

Question

I want to run filebeats installed using ansible.
It worked fine when I executed command in terminal.
However, when I ran filebeat with ansible, the log was outputted as shown below.
ansible filebeat run role
- name: run vote history filebeat
remote_user: irteam
shell: /home1/irteam/apps/filebeat/filebeat -e -c /home1/irteam/apps/filebeat/vote_history.yml -d publish &
ansible error log
"stderr": "2019-10-18T21:37:39.793+0900\tINFO\tinstance/beat.go:606\tHome path: [/home1/irteam/apps/filebeat-7.2.1-linux-x86_64] Config path: [/home1/irteam/apps/filebeat-7.2.1-linux-x86_64] Data path: [/home1/irteam/apps/filebeat-7.2.1-linux-x86_64/data] Logs path: [/home1/irteam/apps/filebeat-7.2.1-linux-x86_64/logs]\n2019-10-18T21:37:39.794+0900\tINFO\tinstance/beat.go:614\tBeat ID: 4e840153-d8fd-44c4-ab32-7e2d3dd34d3f\n2019-10-18T21:37:39.794+0900\tINFO\t[seccomp]\tseccomp/seccomp.go:93\tSyscall filter could not be installed because the kernel does not support seccomp\n2019-10-18T21:37:39.794+0900\tINFO\t[beat]\tinstance/beat.go:902\tBeat info\t{\"system_info\": {\"beat\": {\"path\": {\"config\": \"/home1/irteam/apps/filebeat-7.2.1-linux-x86_64\", \"data\": \"/home1/irteam/apps/filebeat-7.2.1-linux-x86_64/data\", \"home\": \"/home1/irteam/apps/filebeat-7.2.1-linux-x86_64\", \"logs\": \"/home1/irteam/apps/filebeat-7.2.1-linux-x86_64/logs\"}, \"type\": \"filebeat\", \"uuid\": \"4e840153-d8fd-44c4-ab32-7e2d3dd34d3f\"}}}\n2019-10-18T21:37:39.794+0900\tINFO\t[beat]\tinstance/beat.go:911\tBuild info\t{\"system_info\": {\"build\": {\"commit\": \"dd3f47f0fb299aa5de9c5c1468faacc1b9b3c27f\", \"libbeat\": \"7.2.1\", \"time\": \"2019-07-24T17:10:04.000Z\", \"version\": \"7.2.1\"}}}\n2019-10-18T21:37:39.794+0900\tINFO\t[beat]\tinstance/beat.go:914\tGo runtime info\t{\"system_info\": {\"go\": {\"os\":\"linux\",\"arch\":\"amd64\",\"max_procs\":4,\"version\":\"go1.12.4\"}}}\n2019-10-18T21:37:39.796+0900\tINFO\t[beat]\tinstance/beat.go:918\tHost info\t{\"system_info\": {\"host\": {\"architecture\":\"x86_64\",\"boot_time\":\"2019-09-16T14:01:30+09:00\",\"containerized\":false,\"name\":\"dev-vos-api-ncl\",\"ip\":[\"127.0.0.1/8\",\"10.113.103.6/23\"],\"kernel_version\":\"3.10.0-693.2.2.el7.x86_64\",\"mac\":[\"7e:76:cd:f1:39:c6\"],\"os\":{\"family\":\"redhat\",\"platform\":\"centos\",\"name\":\"CentOS Linux\",\"version\":\"7 (Core)\",\"major\":7,\"minor\":4,\"patch\":1708,\"codename\":\"Core\"},\"timezone\":\"KST\",\"timezone_offset_sec\":32400,\"id\":\"97b3ae2e453f442cb387546f7d3d3214\"}}}\n2019-10-18T21:37:39.796+0900\tINFO\t[beat]\tinstance/beat.go:947\tProcess info\t{\"system_info\": {\"process\": {\"capabilities\": {\"inheritable\":null,\"permitted\":null,\"effective\":null,\"bounding\":[\"chown\",\"dac_override\",\"dac_read_search\",\"fowner\",\"fsetid\",\"kill\",\"setgid\",\"setuid\",\"setpcap\",\"linux_immutable\",\"net_bind_service\",\"net_broadcast\",\"net_admin\",\"net_raw\",\"ipc_lock\",\"ipc_owner\",\"sys_module\",\"sys_rawio\",\"sys_chroot\",\"sys_ptrace\",\"sys_pacct\",\"sys_admin\",\"sys_boot\",\"sys_nice\",\"sys_resource\",\"sys_time\",\"sys_tty_config\",\"mknod\",\"lease\",\"audit_write\",\"audit_control\",\"setfcap\",\"mac_override\",\"mac_admin\",\"syslog\",\"wake_alarm\",\"block_suspend\"],\"ambient\":null}, \"cwd\": \"/home1/irteam/apps/filebeat-7.2.1-linux-x86_64\", \"exe\": \"/home1/irteam/apps/filebeat-7.2.1-linux-x86_64/filebeat\", \"name\": \"filebeat\", \"pid\": 115545, \"ppid\": 1, \"seccomp\": {\"mode\":\"disabled\"}, \"start_time\": \"2019-10-18T21:37:39.160+0900\"}}}\n2019-10-18T21:37:39.796+0900\tINFO\tinstance/beat.go:292\tSetup Beat: filebeat; Version: 7.2.1\n2019-10-18T21:37:39.797+0900\tINFO\t[publisher]\tpipeline/module.go:97\tBeat name: dev-vos-api-ncl\n2019-10-18T21:37:39.797+0900\tWARN\tbeater/filebeat.go:152\tFilebeat is unable to load the Ingest Node pipelines for the configured modules because the Elasticsearch output is not configured/enabled. If you have already loaded the Ingest Node pipelines or are using Logstash pipelines, you can ignore this warning.\n2019-10-18T21:37:39.797+0900\tINFO\tinstance/beat.go:421\tfilebeat start running.\n2019-10-18T21:37:39.797+0900\tINFO\t[monitoring]\tlog/log.go:118\tStarting metrics logging every 30s\n2019-10-18T21:37:39.797+0900\tINFO\tregistrar/registrar.go:145\tLoading registrar data from /home1/irteam/apps/filebeat-7.2.1-linux-x86_64/data/registry/filebeat/data.json\n2019-10-18T21:37:39.798+0900\tINFO\tregistrar/registrar.go:152\tStates Loaded from registrar: 2\n2019-10-18T21:37:39.798+0900\tWARN\tbeater/filebeat.go:368\tFilebeat is unable to load the Ingest Node pipelines for the configured modules because the Elasticsearch output is not configured/enabled. If you have already loaded the Ingest Node pipelines or are using Logstash pipelines, you can ignore this warning.\n2019-10-18T21:37:39.798+0900\tINFO\tcrawler/crawler.go:72\tLoading Inputs: 1\n2019-10-18T21:37:39.798+0900\tINFO\tlog/input.go:148\tConfigured paths: [/home1/irteam/logs/vos_api/vote_history*.log]\n2019-10-18T21:37:39.798+0900\tINFO\tinput/input.go:114\tStarting input of type: log; ID: 5211242161898657702 \n2019-10-18T21:37:39.798+0900\tINFO\tcrawler/crawler.go:106\tLoading and starting Inputs completed. Enabled inputs: 1",
"stderr_lines": [
"2019-10-18T21:37:39.793+0900\tINFO\tinstance/beat.go:606\tHome path: [/home1/irteam/apps/filebeat-7.2.1-linux-x86_64] Config path: [/home1/irteam/apps/filebeat-7.2.1-linux-x86_64] Data path: [/home1/irteam/apps/filebeat-7.2.1-linux-x86_64/data] Logs path: [/home1/irteam/apps/filebeat-7.2.1-linux-x86_64/logs]",
"2019-10-18T21:37:39.794+0900\tINFO\tinstance/beat.go:614\tBeat ID: 4e840153-d8fd-44c4-ab32-7e2d3dd34d3f",
"2019-10-18T21:37:39.794+0900\tINFO\t[seccomp]\tseccomp/seccomp.go:93\tSyscall filter could not be installed because the kernel does not support seccomp",
"2019-10-18T21:37:39.794+0900\tINFO\t[beat]\tinstance/beat.go:902\tBeat info\t{\"system_info\": {\"beat\": {\"path\": {\"config\": \"/home1/irteam/apps/filebeat-7.2.1-linux-x86_64\", \"data\": \"/home1/irteam/apps/filebeat-7.2.1-linux-x86_64/data\", \"home\": \"/home1/irteam/apps/filebeat-7.2.1-linux-x86_64\", \"logs\": \"/home1/irteam/apps/filebeat-7.2.1-linux-x86_64/logs\"}, \"type\": \"filebeat\", \"uuid\": \"4e840153-d8fd-44c4-ab32-7e2d3dd34d3f\"}}}",
"2019-10-18T21:37:39.794+0900\tINFO\t[beat]\tinstance/beat.go:911\tBuild info\t{\"system_info\": {\"build\": {\"commit\": \"dd3f47f0fb299aa5de9c5c1468faacc1b9b3c27f\", \"libbeat\": \"7.2.1\", \"time\": \"2019-07-24T17:10:04.000Z\", \"version\": \"7.2.1\"}}}",
"2019-10-18T21:37:39.794+0900\tINFO\t[beat]\tinstance/beat.go:914\tGo runtime info\t{\"system_info\": {\"go\": {\"os\":\"linux\",\"arch\":\"amd64\",\"max_procs\":4,\"version\":\"go1.12.4\"}}}",
"2019-10-18T21:37:39.796+0900\tINFO\t[beat]\tinstance/beat.go:918\tHost info\t{\"system_info\": {\"host\": {\"architecture\":\"x86_64\",\"boot_time\":\"2019-09-16T14:01:30+09:00\",\"containerized\":false,\"name\":\"dev-vos-api-ncl\",\"ip\":[\"127.0.0.1/8\",\"10.113.103.6/23\"],\"kernel_version\":\"3.10.0-693.2.2.el7.x86_64\",\"mac\":[\"7e:76:cd:f1:39:c6\"],\"os\":{\"family\":\"redhat\",\"platform\":\"centos\",\"name\":\"CentOS Linux\",\"version\":\"7 (Core)\",\"major\":7,\"minor\":4,\"patch\":1708,\"codename\":\"Core\"},\"timezone\":\"KST\",\"timezone_offset_sec\":32400,\"id\":\"97b3ae2e453f442cb387546f7d3d3214\"}}}",
"2019-10-18T21:37:39.796+0900\tINFO\t[beat]\tinstance/beat.go:947\tProcess info\t{\"system_info\": {\"process\": {\"capabilities\": {\"inheritable\":null,\"permitted\":null,\"effective\":null,\"bounding\":[\"chown\",\"dac_override\",\"dac_read_search\",\"fowner\",\"fsetid\",\"kill\",\"setgid\",\"setuid\",\"setpcap\",\"linux_immutable\",\"net_bind_service\",\"net_broadcast\",\"net_admin\",\"net_raw\",\"ipc_lock\",\"ipc_owner\",\"sys_module\",\"sys_rawio\",\"sys_chroot\",\"sys_ptrace\",\"sys_pacct\",\"sys_admin\",\"sys_boot\",\"sys_nice\",\"sys_resource\",\"sys_time\",\"sys_tty_config\",\"mknod\",\"lease\",\"audit_write\",\"audit_control\",\"setfcap\",\"mac_override\",\"mac_admin\",\"syslog\",\"wake_alarm\",\"block_suspend\"],\"ambient\":null}, \"cwd\": \"/home1/irteam/apps/filebeat-7.2.1-linux-x86_64\", \"exe\": \"/home1/irteam/apps/filebeat-7.2.1-linux-x86_64/filebeat\", \"name\": \"filebeat\", \"pid\": 115545, \"ppid\": 1, \"seccomp\": {\"mode\":\"disabled\"}, \"start_time\": \"2019-10-18T21:37:39.160+0900\"}}}",
"2019-10-18T21:37:39.796+0900\tINFO\tinstance/beat.go:292\tSetup Beat: filebeat; Version: 7.2.1",
"2019-10-18T21:37:39.797+0900\tINFO\t[publisher]\tpipeline/module.go:97\tBeat name: dev-vos-api-ncl",
"2019-10-18T21:37:39.797+0900\tWARN\tbeater/filebeat.go:152\tFilebeat is unable to load the Ingest Node pipelines for the configured modules because the Elasticsearch output is not configured/enabled. If you have already loaded the Ingest Node pipelines or are using Logstash pipelines, you can ignore this warning.",
"2019-10-18T21:37:39.797+0900\tINFO\tinstance/beat.go:421\tfilebeat start running.",
"2019-10-18T21:37:39.797+0900\tINFO\t[monitoring]\tlog/log.go:118\tStarting metrics logging every 30s",
"2019-10-18T21:37:39.797+0900\tINFO\tregistrar/registrar.go:145\tLoading registrar data from /home1/irteam/apps/filebeat-7.2.1-linux-x86_64/data/registry/filebeat/data.json",
"2019-10-18T21:37:39.798+0900\tINFO\tregistrar/registrar.go:152\tStates Loaded from registrar: 2",
"2019-10-18T21:37:39.798+0900\tWARN\tbeater/filebeat.go:368\tFilebeat is unable to load the Ingest Node pipelines for the configured modules because the Elasticsearch output is not configured/enabled. If you have already loaded the Ingest Node pipelines or are using Logstash pipelines, you can ignore this warning.",
"2019-10-18T21:37:39.798+0900\tINFO\tcrawler/crawler.go:72\tLoading Inputs: 1",
"2019-10-18T21:37:39.798+0900\tINFO\tlog/input.go:148\tConfigured paths: [/home1/irteam/logs/vos_api/vote_history*.log]",
"2019-10-18T21:37:39.798+0900\tINFO\tinput/input.go:114\tStarting input of type: log; ID: 5211242161898657702 ",
"2019-10-18T21:37:39.798+0900\tINFO\tcrawler/crawler.go:106\tLoading and starting Inputs completed. Enabled inputs: 1"

Answer 1

It looks like the command is not returning 0 as a return code although it succeeded (with warnings).
By default, the shell module in ansible will consider that any command with a return code other than 0 is an error.
You should check your command return codes with echo $? right after executing it in your terminal. Then you can either:
fix everything so that it returns 0 all the time (unless there is really an error)
adjust failed_when on your task to include return codes considered as success (replace X with an actual return code you want to trust):
- name: run vote history filebeat
remote_user: irteam
shell: /home1/irteam/apps/filebeat/filebeat -e -c /home1/irteam/apps/filebeat/vote_history.yml -d publish &
register: filebeat_cmd
failed_when: filebeat_cmd.rc not in [0,X]