Fluentd regex filter removes other keys - fluentd

Question

I'm getting a message into fluentd with a few keys already populated from previous stages (fluent-bit on another host). I'm trying to parse the content of the log field as follows:
# Parse app_logs
<filter filter.app.backend.app_logs>
#type parser
key_name log
<parse>
#type regexp
expression /^(?<module>[^ ]*) *(?<time>[\d ,-:]*) (?<severity>[^ ]*) *(?<file>[\w\.]*):(?<function>[\w_]*) (?<message>.*)$/
time_format %Y-%m-%d %H:%M:%S,%L
</parse>
</filter>
It works (kind of), as it extracts the fields as expected. That said, it removes all the other fields that were there before.
Example message before the filter:
filter.app.backend.app_logs: {"docker.container_name":"intranet-worker","docker.container_id":"98b7784f27f93a056c05b4c5066c06cb5e23d7eeb436a6e4a66cdf8ff045d29f","time":"2022-06-10T17:00:00.248932151Z","log":"org-worker 2022-06-10 19:00:00,248 INFO briefings.py:check_expired_registrations Checking for expired registrations\n","docker.container_image":"registry.my-org.de/org-it-infrastructure/org-fastapi-backend/backend-worker:v0-7-11","stream":"stdout","docker.container_started":"2022-06-10T14:57:27.925959889Z"}
After the filter, the message looks like this (its a slightly different one, but same stream):
filter.app.backend.app_logs: {"module":"mksp-api","severity":"DEBUG","file":"authToken.py","function":"verify_token","message":"Token is valid, checking permission"}
So only the parsed fields are kept, the rest is removed. Can I somehow use that filter to add the fields to the message, instead of replacing it?

Answer 1

Actually, this scenario is described in the documentation, its not part of the regexp documentation but of the corresponding parser filter documentation:
reserve_data
Keeps the original key-value pair in the parsed result.
Therefore, the following configuration works:
<filter filter.app.backend.app_logs>
#type parser
key_name log
reserve_data true
<parse>
#type regexp
expression /^(?<module>[^ ]*) *(?<time>[\d ,-:]*) (?<severity>[^ ]*) *(?<file>[\w\.]*):(?<function>[\w_]*) (?<message>.*)$/
time_format %Y-%m-%d %H:%M:%S,%L
</parse>
</filter>