fluentd not picking logs from beginning - docker

Question

I am trying to take my docker container logs with fluentd. both application and fluentd process start through supervisord and both are in the same container but fluentd only taking half of the application logs. I need to fetch the logs from the beginning. adding fluentd conf below:-
<source>
type tail
path /var/log/*
path_key path
format none
read_from_head true
<parse>
#type grok
<grok>
pattern (?<logtm>%{MONTHDAY}-%{MONTH}-%{YEAR} %{HOUR}:%{MINUTE}:%{SECOND}) %{LOGLEVEL:loglevel} \[%{DATA:thread}] %{GREEDYDATA:message}
</grok>
<grok>
pattern %{GREEDYDATA:timestamp} %{LOGLEVEL:loglevel} %{GREEDYDATA:message}
</grok>
<grok>
pattern %{URIHOST:remote_host} - - \[%{HTTPDATE:request_time}] "%{WORD:method} %{NOTSPACE:request_page} %{GREEDYDATA:message}/%{NUMBER:http_version}\" %{NUMBER:response_code} %{NUMBER:bytes} %{INT:time_taken} %{QS:referrer} %{QS:user_agent}
</grok>
</parse>
keep_time_key true
# time_format yyyy-MM-dd HH:mm:ss.SSSZ
tag graylog2.*
</source>
<filter **>
#type record_transformer
enable_ruby
<record>
logtm ${record["logtm"]}
thread ${record["thread"]}
instance "#{Socket.gethostname}"
namespace "#{ENV.fetch('INSTANCE_PREFIX'){'default'}}"
app "#{ENV.fetch('APPZ_APP_NAME'){'wordpress'}}"
level ${if record["loglevel"] == "EMERG";record["level"] = "0" ; record["loglevel"] == "ALERT";record["level"] = "1";elsif record["loglevel"] == "CRIT" || record["loglevel"] == "SEVERE"; record["level"]= "2" ;elsif record["loglevel"] == "ERROR" ; record["level"]= "3" ;elsif record["loglevel"] == "WARN" || record["loglevel"] == "WARNING" ; record["level"]= "4" ;elsif record["loglevel"] == "NOTICE" ; record["level"]= "5";elsif record["loglevel"] == "INFO" || record["loglevel"] == nil || record["loglevel"] == 0 ; record["level"]= "6";else record["loglevel"] == "DEBUG" || record["loglevel"] == "debug"; record["level"]= "7";end}
</record>
</filter>
<match **>
type graylog
host "#{ENV.fetch('LOG_HOST'){'GL'}}"
port "#{ENV.fetch('LOG_PORT'){12201}}"
# BufferedOutput config
flush_interval 5s
num_threads 2
# ...
</match>
note:- when i change /var/log/* to /var/log/log_file_name.log its working fine.

Answer 1

Could you try with a POS file within in_tail
path /var/log/httpd-access.log
pos_file /var/log/td-agent/httpd-access.log.pos