How to write fluent bit input logs to localhost syslog server - docker

Question

I'm working on collecting logs from docker containerized application. I'm able to bring the logs to stdout output plugin but when I am trying syslog output plugin then it is not writing on syslog server.
Below is the configuration file.
[SERVICE]
Parsers_File /etc/td-agent-bit/parsers.conf
[INPUT]
Name forward
[Output]
name syslog
match *
host 127.0.0.1
port 514
mode udp
syslog_format rfc5424
syslog_hostname_key hostname
syslog_appname_key appname
syslog_procid_key procid
syslog_message_key log
The logging property for container application is set to
logging:
driver: fluentd
options:
fluentd-address: localhost:24224
tag: logs
After running the fluent bit /opt/td-agent-bit/bin/td-agent-bit -c fluent.conf
[2021/09/20 08:47:16] [ warn] [engine] failed to flush chunk '8481-1632152835.361162854.flb', retry in 7 seconds: task_id=0, input=forward.0 > output=syslog.0 (out_id=0)
[2021/09/20 08:47:23] [ warn] [engine] chunk '8481-1632152835.361162854.flb' cannot be retried: task_id=0, input=forward.0 > output=syslog.0
[2021/09/20 08:47:26] [ warn] [engine] failed to flush chunk '8481-1632152845.361118393.flb', retry in 6 seconds: task_id=0, input=forward.0 > output=syslog.0 (out_id=0)
[2021/09/20 08:47:32] [ warn] [engine] chunk '8481-1632152845.361118393.flb' cannot be retried: task_id=0, input=forward.0 > output=syslog.0
[2021/09/20 08:47:36] [ warn] [engine] failed to flush chunk '8481-1632152855.361556013.flb', retry in 8 seconds: task_id=0, input=forward.0 > output=syslog.0 (out_id=0)
Can anyone tell what is going wrong here?
Even for simple cpu input plugin syslog does not work. like
/opt/td-agent-bit/bin/td-agent-bit -i cpu -o syslog
output
[2021/09/20 08:53:43] [ info] [cmetrics] version=0.2.1
[2021/09/20 08:53:43] [ info] [output:syslog:syslog.0] setup done for 127.0.0.1:514
[2021/09/20 08:53:43] [ info] [sp] stream processor started
[2021/09/20 08:53:48] [ warn] [engine] failed to flush chunk '8765-1632153224.515974981.flb', retry in 10 seconds: task_id=0, input=cpu.0 > output=syslog.0 (out_id=0)
[2021/09/20 08:53:53] [ warn] [engine] failed to flush chunk '8765-1632153228.516869744.flb', retry in 6 seconds: task_id=1, input=cpu.0 > output=syslog.0 (out_id=0)
[2021/09/20 08:53:58] [ warn] [engine] chunk '8765-1632153224.515974981.flb' cannot be retried: task_id=0, input=cpu.0 > output=syslog.0

Answer 1

i faced similar issue, happening because fluentbit pod itself is 127.0.0.1. i created another rsyslog pod and used its IP to get logs from fluentbit.
https://artifacthub.io/packages/helm/rsyslog/rsyslog