stdout put 404 in logstash - docker

Question

I'm new to elk stack, and I'm trying to do a very basic experiment: send a message to logstash stdout with a PUT request, based on this repo: link
The logstash's port is 9600, and I use postman to send a PUT request. It returns 404
My logstash.conf is very simple.
input {
http {
}
}
output {
stdout {
}
}
As for the settings in the docker-compose file, here they are:
logstash:
build:
context: logstash/
args:
ELK_VERSION: $ELK_VERSION
volumes:
- type: bind
source: ./logstash/config/logstash.yml
target: /usr/share/logstash/config/logstash.yml
read_only: true
- type: bind
source: ./logstash/pipeline
target: /usr/share/logstash/pipeline
read_only: true
ports:
- "5044:5044"
- "5000:5000/tcp"
- "5000:5000/udp"
- "9600:9600"
environment:
LS_JAVA_OPTS: "-Xmx256m -Xms256m"
networks:
- elk
depends_on:
- elasticsearch
A GET request works, and here is the result:
{
"host": "b32085c40331",
"version": "7.10.2",
"http_address": "0.0.0.0:9600",
"id": "0079f53f-1d2e-4278-85eb-0817fa95506c",
"name": "b32085c40331",
"ephemeral_id": "d0c18df3-9a0b-48c9-abb4-9e41543ed7ac",
"status": "green",
"snapshot": false,
"pipeline": {
"workers": 4,
"batch_size": 125,
"batch_delay": 50
},
"monitoring": {
"hosts": [
"http://elasticsearch:9200"
],
"username": "elastic"
},
"build_date": "2021-01-13T02:43:06Z",
"build_sha": "7cebafee7a073fa9d58c97de074064a540d6c317",
"build_snapshot": false
}
About logstash, with docker-compose logs logstash, I get a large log, and I don't know even where to start:
logstash_1 | Using bundled JDK: /usr/share/logstash/jdk
logstash_1 | OpenJDK 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
logstash_1 | Using bundled JDK: /usr/share/logstash/jdk
logstash_1 | OpenJDK 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
logstash_1 | WARNING: An illegal reflective access operation has occurred
logstash_1 | WARNING: Illegal reflective access by org.jruby.ext.openssl.SecurityHelper (file:/tmp/jruby-1/jruby5118775578707886457jopenssl.jar) to field java.security.MessageDigest.provider
logstash_1 | WARNING: Please consider reporting this to the maintainers of org.jruby.ext.openssl.SecurityHelper
logstash_1 | WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations
logstash_1 | WARNING: All illegal access operations will be denied in a future release
logstash_1 | Sending Logstash logs to /usr/share/logstash/logs which is now configured via log4j2.properties
logstash_1 | [2021-01-29T12:00:35,199][INFO ][logstash.runner ] Starting Logstash {"logstash.version"=>"7.10.2", "jruby.version"=>"jruby 9.2.13.0 (2.5.7) 2020-08-03 9a89c94bcc OpenJDK 64-Bit Server VM 11.0.8+10 on 11.0.8+10 +indy +jit [linux-x86_64]"}
logstash_1 | [2021-01-29T12:00:35,412][INFO ][logstash.setting.writabledirectory] Creating directory {:setting=>"path.queue", :path=>"/usr/share/logstash/data/queue"}
logstash_1 | [2021-01-29T12:00:35,440][INFO ][logstash.setting.writabledirectory] Creating directory {:setting=>"path.dead_letter_queue", :path=>"/usr/share/logstash/data/dead_letter_queue"}
logstash_1 | [2021-01-29T12:00:37,687][INFO ][logstash.agent ] No persistent UUID file found. Generating new UUID {:uuid=>"0079f53f-1d2e-4278-85eb-0817fa95506c", :path=>"/usr/share/logstash/data/uuid"}
logstash_1 | [2021-01-29T12:00:38,657][WARN ][deprecation.logstash.monitoringextension.pipelineregisterhook] Internal collectors option for Logstash monitoring is deprecated and targeted for removal in the next major version.
logstash_1 | Please configure Metricbeat to monitor Logstash. Documentation can be found at:
logstash_1 | https://www.elastic.co/guide/en/logstash/current/monitoring-with-metricbeat.html
logstash_1 | [2021-01-29T12:00:42,951][WARN ][deprecation.logstash.outputs.elasticsearch] Relying on default value of `pipeline.ecs_compatibility`, which may change in a future major release of Logstash. To avoid unexpected changes when upgrading Logstash, please explicitly declare your desired ECS Compatibility mode.
logstash_1 | [2021-01-29T12:00:46,669][INFO ][logstash.licensechecker.licensereader] Elasticsearch pool URLs updated {:changes=>{:removed=>[], :added=>[http://elastic:xxxxxx#elasticsearch:9200/]}}
logstash_1 | [2021-01-29T12:00:50,290][WARN ][logstash.licensechecker.licensereader] Restored connection to ES instance {:url=>"http://elastic:xxxxxx#elasticsearch:9200/"}
logstash_1 | [2021-01-29T12:00:50,515][INFO ][logstash.licensechecker.licensereader] ES Output version determined {:es_version=>7}
logstash_1 | [2021-01-29T12:00:50,518][WARN ][logstash.licensechecker.licensereader] Detected a 6.x and above cluster: the `type` event field won't be used to determine the document _type {:es_version=>7}
logstash_1 | [2021-01-29T12:00:50,694][INFO ][logstash.monitoring.internalpipelinesource] Monitoring License OK
logstash_1 | [2021-01-29T12:00:50,695][INFO ][logstash.monitoring.internalpipelinesource] Validated license for monitoring. Enabling monitoring pipeline.
logstash_1 | [2021-01-29T12:00:53,243][INFO ][org.reflections.Reflections] Reflections took 606 ms to scan 1 urls, producing 23 keys and 47 values
logstash_1 | [2021-01-29T12:00:54,045][WARN ][deprecation.logstash.outputs.elasticsearchmonitoring] Relying on default value of `pipeline.ecs_compatibility`, which may change in a future major release of Logstash. To avoid unexpected changes when upgrading Logstash, please explicitly declare your desired ECS Compatibility mode.
logstash_1 | [2021-01-29T12:00:54,339][INFO ][logstash.outputs.elasticsearchmonitoring][.monitoring-logstash] Elasticsearch pool URLs updated {:changes=>{:removed=>[], :added=>[http://elastic:xxxxxx#elasticsearch:9200/]}}
logstash_1 | [2021-01-29T12:00:54,417][WARN ][logstash.outputs.elasticsearchmonitoring][.monitoring-logstash] Restored connection to ES instance {:url=>"http://elastic:xxxxxx#elasticsearch:9200/"}
logstash_1 | [2021-01-29T12:00:54,500][INFO ][logstash.outputs.elasticsearchmonitoring][.monitoring-logstash] ES Output version determined {:es_version=>7}
logstash_1 | [2021-01-29T12:00:54,500][WARN ][logstash.outputs.elasticsearchmonitoring][.monitoring-logstash] Detected a 6.x and above cluster: the `type` event field won't be used to determine the document _type {:es_version=>7}
logstash_1 | [2021-01-29T12:00:54,627][INFO ][logstash.outputs.elasticsearchmonitoring][.monitoring-logstash] New Elasticsearch output {:class=>"LogStash::Outputs::ElasticSearchMonitoring", :hosts=>["http://elasticsearch:9200"]}
logstash_1 | [2021-01-29T12:00:54,691][WARN ][logstash.javapipeline ][.monitoring-logstash] 'pipeline.ordered' is enabled and is likely less efficient, consider disabling if preserving event order is not necessary
logstash_1 | [2021-01-29T12:00:54,953][INFO ][logstash.javapipeline ][.monitoring-logstash] Starting pipeline {:pipeline_id=>".monitoring-logstash", "pipeline.workers"=>1, "pipeline.batch.size"=>2, "pipeline.batch.delay"=>50, "pipeline.max_inflight"=>2, "pipeline.sources"=>["monitoring pipeline"], :thread=>"#<Thread:0x37941be run>"}
logstash_1 | [2021-01-29T12:00:55,984][INFO ][logstash.javapipeline ][main] Starting pipeline {:pipeline_id=>"main", "pipeline.workers"=>4, "pipeline.batch.size"=>125, "pipeline.batch.delay"=>50, "pipeline.max_inflight"=>500, "pipeline.sources"=>["/usr/share/logstash/pipeline/logstash.conf"], :thread=>"#<Thread:0x3e7f065e run>"}
logstash_1 | [2021-01-29T12:01:00,012][INFO ][logstash.javapipeline ][.monitoring-logstash] Pipeline Java execution initialization time {"seconds"=>5.05}
logstash_1 | [2021-01-29T12:01:00,013][INFO ][logstash.javapipeline ][main] Pipeline Java execution initialization time {"seconds"=>4.03}
logstash_1 | [2021-01-29T12:01:00,142][INFO ][logstash.javapipeline ][.monitoring-logstash] Pipeline started {"pipeline.id"=>".monitoring-logstash"}
logstash_1 | [2021-01-29T12:01:01,027][INFO ][logstash.inputs.beats ][main] Starting input listener {:address=>"0.0.0.0:5044"}
logstash_1 | [2021-01-29T12:01:01,209][INFO ][logstash.javapipeline ][main] Pipeline started {"pipeline.id"=>"main"}
logstash_1 | [2021-01-29T12:01:01,245][INFO ][logstash.inputs.http ][main][2d26a22d7786b5d1d6a62684242754061f0e7699167308954d8cf88e52c80903] Starting http input listener {:address=>"0.0.0.0:8080", :ssl=>"false"}
logstash_1 | [2021-01-29T12:01:01,217][INFO ][logstash.inputs.tcp ][main][6ca97606e772405a9e65bc09f9b369d784557cb3e3fea379b981c5d16a9573f1] Starting tcp input listener {:address=>"0.0.0.0:5000", :ssl_enable=>"false"}
logstash_1 | [2021-01-29T12:01:01,306][INFO ][org.logstash.beats.Server][main][d704d487716580c50daa3a9bb4e99ad2bfa9542e31e8b0b06a9e0ea687e6f15a] Starting server on port: 5044
logstash_1 | [2021-01-29T12:01:01,340][INFO ][logstash.agent ] Pipelines running {:count=>2, :running_pipelines=>[:".monitoring-logstash", :main], :non_running_pipelines=>[]}
logstash_1 | [2021-01-29T12:01:02,200][INFO ][logstash.agent ] Successfully started Logstash API endpoint {:port=>9600}
How can this problem be fixed?

Answer 1

The port 9600 is the port for the Logstash API, for monitoring logstash, not the port for the http input.
If you want to use the http input and since you didn't specify a port in the configuration, you should use the port 8080, which is the default port for this input.
You will need to expose this port also in your docker configuration.