How can I send log from fluentd to influxDB? - ruby

Question

I'm in trouble with the parser of Fluentd not working.
I don't know where the hell the problem is.
fluentd.conf
<source>
#type tail
#id in_tail_app_logs
path /tmp/log/test/*.log
pos_file /var/log/app.log.pos
tag logging.app
refresh_interval 1
read_from_head true
<parse>
#type regexp
expression /^(?<time>.+?)\t(?<logname>.+?)\t(?<log>.+?)$/
time_key time
time_format %Y-%m-%dT%H:%M:%S%:z
</parse>
</source>
<match logging.app>
#type stdout
#type copy
<store ignore_error>
#type file
path /tmp/log/influx
</store>
<store>
#type influxdb
host 20.10.222.22
port 8086
user test
password test123
use_ssl false
dbname test
measurement test_measurement
time_precision s
auto_tags true
flush_interval 10
verify_ssl false
sequence_tag _seq
</store>
</match>
/tmp/log/test/buffer.*.log
...
2023-02-06T08:09:46+00:00 kubernetes.var.log.containers.app-test-app-test-space-fb5ffca1ec-0_cf-workloads_opi-05942675f9f0dd17039304733f228e7abd6ebdfd91609baf1a7afefaeb33ced8.log {"stream":"stdout","log":"Console output from test-node-app","docker":{"container_id":"05942675f9f0dd17039304733f228e7abd6ebdfd91609baf1a7afefaeb33ced8"},"kubernetes":{"container_name":"opi","namespace_name":"cf-workloads","pod_name":"app-test-app-test-space-fb5ffca1ec-0","pod_id":"f4f2592e-ed57-4375-aff7-40c7b214abe0","host":"ap-joy-sidecar-5","labels":{"controller-revision-hash":"app-test-app-test-space-fb5ffca1ec-5bb7dc5769","cloudfoundry_org/app_guid":"8fc07280-506c-49b2-ab00-a97222fcf0a5","cloudfoundry_org/guid":"8fc07280-506c-49b2-ab00-a97222fcf0a5","cloudfoundry_org/org_guid":"fdf0a222-33a4-46fd-a7f9-7955b9ea862c","cloudfoundry_org/org_name":"system","cloudfoundry_org/process_type":"web","cloudfoundry_org/source_type":"APP","cloudfoundry_org/space_guid":"f1b70a4b-7581-4214-b128-2f1597f7789d","cloudfoundry_org/space_name":"app-test-space","cloudfoundry_org/version":"ebfde654-e73c-48da-b55b-42a37a6ba139","security_istio_io/tlsMode":"istio","service_istio_io/canonical-name":"app-test-app-test-space-fb5ffca1ec","service_istio_io/canonical-revision":"latest","statefulset_kubernetes_io/pod-name":"app-test-app-test-space-fb5ffca1ec-0"}},"app_id":"8fc07280-506c-49b2-ab00-a97222fcf0a5","instance_id":"f4f2592e-ed57-4375-aff7-40c7b214abe0","structured_data":"[tags#47450 source_type=\"APP/PROC/WEB\"]"}
...
I think \t exists between time, logname, and log.
I tried the following methods.
What I tried
1. fluentd.conf
json
<parse>
#type json
time_format %Y-%m-%dT%H:%M:%S%:z
</parse>
tsv
<parse>
#type tsv
keys time,logname,logs
time_key time
time_format %Y-%m-%dT%H:%M:%S%:z
</parse>
none
<parse>
#type none
</parse>
2. result
Log files did not exist for all.
There is no output when using stdout parser except json parser.
What I want
I want the parser to work normally and to send data to influxdb.
Please help me...