I am trying to up elasticsearch kibana and metricbeat through docker-compose.
This is the code.
ELK is working fine but metricbeat failed, I am getting this error, and not able to figure out how can i solve this :
MetricBeat Log:
2020-05-09T19:27:03.353Z ERROR pipeline/output.go:100 Failed to connect to backoff(elasticsearch(http://elasticsearch:9200)): Connection marked as failed because the onConnect callback failed: cannot retrieve the elasticsearch license from the /_xpack endpoint, Metricbeat requires the default distribution of Elasticsearch. Please make the endpoint accessible to Metricbeat so it can verify the license.: could not extract license information from the server response: unknown state, received: 'expired'
The license expired but i was hoping it would work without it.
Related
I am trying to run/configure a Spring Data Cloud Data Flow (SCDF) to schedule a task for a Spring Batch Job.
I am running in a minikube that connects to a local postgresql(localhost:5432). The minikube runs in a virtualbox where I assigned a vnet thru the --cidr, so minikube can connect to the local postgres.
Here is the postgresql service yaml:
https://github.com/msuzuki23/SpringCloudDataFlowDemo/blob/main/postgres-service.yaml
Here is the SCDF config yaml:
https://github.com/msuzuki23/SpringCloudDataFlowDemo/blob/main/server-config.yaml
Here is the SCDF deployment yaml:
https://github.com/msuzuki23/SpringCloudDataFlowDemo/blob/main/server-deployment.yaml
Here is the SCDF server-svc.yaml:
https://github.com/msuzuki23/SpringCloudDataFlowDemo/blob/main/server-svc.yaml
To launch the SCDF server in minikube I do the following kubectl commands:
kubectl apply -f secret.yaml
kubectl apply -f configmap.yaml
kubectl apply -f postgres-service.yaml
kubectl create -f server-roles.yaml
kubectl create -f server-rolebinding.yaml
kubectl create -f service-account.yaml
kubectl apply -f server-config.yaml
kubectl apply -f server-svc.yaml
kubectl apply -f server-deployment.yaml
I am not running Prometeus, Grafana, Kafka/Rabbitmq as I want to test and make sure I can launch the Spring Batch Job from SCDF. I did not run the skipper deployment (Spring Cloud DataFlow server runnning locally pointing to Skipper in Kubernetes) it is not necessary if just running tasks.
This is the error I am getting when trying to add an application from a docker private repo:
And this is the full error stack from the pod:
https://github.com/msuzuki23/SpringCloudDataFlowDemo/blob/main/SCDF_Log_Error
Highlights from the error stack:
2021-07-08 13:04:13.753 WARN 1 --- [-nio-80-exec-10] o.s.c.d.s.controller.AboutController : Skipper Server is not accessible
org.springframework.web.client.ResourceAccessException: I/O error on GET request for "http://localhost:7577/api/about": Connect to localhost:7577 [localhost/127.0.0.1] failed: Connection refused (Connection refused); nested exception is org.apache.http.conn.HttpHostConnectException: Connect to localhost:7577 [localhost/127.0.0.1] failed: Connection refused (Connection refused)
Postges Hibernate Error:
2021-07-08 13:05:22.142 WARN 1 --- [p-nio-80-exec-5] o.h.engine.jdbc.spi.SqlExceptionHelper : SQL Error: 0, SQLState: 42P01
2021-07-08 13:05:22.200 ERROR 1 --- [p-nio-80-exec-5] o.h.engine.jdbc.spi.SqlExceptionHelper : ERROR: relation "hibernate_sequence" does not exist
Position: 17
2021-07-08 13:05:22.214 ERROR 1 --- [p-nio-80-exec-5] o.s.c.d.s.c.RestControllerAdvice : Caught exception while handling a request
org.springframework.dao.InvalidDataAccessResourceUsageException: could not extract ResultSet; SQL [n/a]; nested exception is org.hibernate.exception.SQLGrammarException: could
The first couple of errors are from the SCDF trying to connect to the skipper, since it was not configured, that was expected.
The second error is the Postgres JDBC Hibernate. How do I solve that?
Is there a configuration I am missing when setting the SCDF to point into the local postgres?
Also, in my docker jar I have not added any annotation such as #EnableTask.
Any help is appreciated, thx! Markus.
I did a search on
Caused by: org.postgresql.util.PSQLException: ERROR: relation
"hibernate_sequence" does not exist Position: 17
And found this stackoverflow anwer:
Postgres error in batch insert : relation "hibernate_sequence" does not exist position 17
Went to the postgres and created the hibernate_sequence:
CREATE SEQUENCE my_seq_gen START 1;
Then, add application worked.
I want to use composer-rest-server to generate rest api.
1,I run composer network ping -c admin#trade-network
the result is :
```The connection to the network was successfully tested: trade-network
Business network version: 0.2.6-20180530153450
Composer runtime version: 0.19.8
participant: org.hyperledger.composer.system.NetworkAdmin#admin
identity: org.hyperledger.composer.system.Identity#8633aef10e9d998be8bec4bb4ab535eb74e3d6832cb21286b89cadf0e95863c5
Command succeeded
2,run `docker run -e COMPOSER_CARD=admin#trade-network -e COMPOSER_NAMESPACES=never --name rest -p 3000:3000 hyperledger/composer-rest-server`
while error appears:
[2018-06-15 08:18:17] PM2 log: Launching in no daemon mode
[2018-06-15 08:18:17] PM2 log: Starting execution sequence in -fork mode- for app name:composer-rest-server id:0
[2018-06-15 08:18:17] PM2 log: App name:composer-rest-server id:0 online
WARNING: NODE_APP_INSTANCE value of '0' did not match any instance config file names.
WARNING: See https://github.com/lorenwest/node-config/wiki/Strict-Mode
Discovering types from business network definition ...
Exception: Error: Error trying to ping. Error: REQUEST_TIMEOUT
Connection fails: Error: Error trying to ping. Error: REQUEST_TIMEOUT
It will be retried for the next request.
Error: Error trying to ping. Error: REQUEST_TIMEOUT
at _checkRuntimeVersions.then.catch (/home/composer/.npm-global/lib/node_modules/composer-rest-server/node_modules/composer-connector-hlfv1/lib/hlfconnection.js:790:34)
at
at process._tickDomainCallback (internal/process/next_tick.js:228:7)
[2018-06-15 08:23:21] PM2 log: App [composer-rest-server] with id [0] and pid [15], exited with code [1] via signal [SIGINT]
[2018-06-15 08:23:21] PM2 log: Starting execution sequence in -fork mode- for app name:composer-rest-server id:0
[2018-06-15 08:23:21] PM2 log: App name:composer-rest-server id:0 online
WARNING: NODE_APP_INSTANCE value of '0' did not match any instance config file names.
WARNING: See https://github.com/lorenwest/node-config/wiki/Strict-Mode
Discovering types from business network definition ...
Connection fails: Error: Error trying to ping. Error: REQUEST_TIMEOUT
It will be retried for the next request.
Exception: Error: Error trying to ping. Error: REQUEST_TIMEOUT
Error: Error trying to ping. Error: REQUEST_TIMEOUT
```
so I want to ask why?
This is a networking issue ... The URL values in your connection.json (part of the "Card") have localhost in them which works fine from the command line of your computer where Docker sets up Port forwarding into the containers of the Fabric. When you use the same card inside a rest container localhost loops back into the container and does not "see" the Fabric.
Step 6 of this tutorial deals with the problem by creating a special restadmin card, and modifying the addresses in the connection.json to 'find' the Fabric - Google Oauth2 Tutorial.
I assume you are already sharing a volume on the docker run command so that the Rest Container can find the folder with the cards. E.g. -v ~/.composer:/home/composer/.composer
I am trying to create a pod using kubernetes with the following simple command
kubectl run example --image=nginx
It runs and assigns the pod to the minion correctly but the status is always in ContainerCreating status due to the following error. I have not hosted GCR or GCloud on my machine. So not sure why its picking from there only.
1h 29m 14s {kubelet centos-minion1} Warning FailedSync Error syncing pod, skipping:
failed to "StartContainer" for "POD" with ErrImagePull: "image pull failed
for gcr.io/google_containers/pause:2.0, this may be because there are no
credentials on this request. details: (unable to ping registry endpoint
https://gcr.io/v0/\nv2 ping attempt failed with error: Get https://gcr.io/v2/:
http: error connecting to proxy http://87.254.212.120:8080: dial tcp
87.254.212.120:8080: i/o timeout\n v1 ping attempt failed with error:
Get https://gcr.io/v1/_ping: http: error connecting to proxy
http://87.254.212.120:8080: dial tcp 87.254.212.120:8080: i/o timeout)
Kubernetes is trying to create a pause container for your pod; this container is used to create the pod's network namespace. See this question and its answers for more general information on the pause container.
To your specific error: Kubernetes tries to pull the pause container's image (which would be gcr.io/google_containers/pause:2.0, according to your error message) from the Google Container Registry (gcr.io). Apparently, your Docker engine tries to connect to GCR using a HTTP proxy located at 87.254.212.120:8080, to which it apparently cannot connect (i/o timeout).
To correct this error, either make sure that you HTTP proxy server is online and does not block HTTP requests to GCR, or (if you do have public Internet access) disable the proxy connection for your Docker engine (this would typically be done using the http_proxy and https_proxy environment variables, which would have been set in /etc/sysconfig/docker or /etc/default/docker, depending on your Linux distribution).
I was trying to upload my image to Google Container Registry, but it return some error and I don't know how to troubleshooting.
$> gcloud docker -- push asia.gcr.io/dtapi-1314/web
The push refers to a repository [asia.gcr.io/dtapi-1314/web]
53ccd4e59f47: Retrying in 1 second
32ca8635750d: Retrying in 1 second
e5363ba7dd4d: Retrying in 1 second
d575d439624a: Retrying in 1 second
5c1cba20b78d: Retrying in 1 second
7198e99c156d: Waiting
6ca37046de16: Waiting
b8f2f07b3eab: Waiting
16681562a534: Waiting
92ea1d98cb79: Waiting
97ca462ad9ee: Waiting
unable to decode token response: read tcp 10.0.2.10:54718->74.125.23.82:443: read: connection reset by peer
I checked permission on my Mac.
$> gsutil acl get gs://asia.artifacts.dtapi-1314.appspot.com
It returned a list of correct permission.
I'd tested push on the cloud console, it works.
Does anyone have clue?
Thanks a lot if anyone could help. :)
Other troubleshooting
gcloud auth login
gcloud docker -- login -p $(gcloud auth print-access-token) -u _token https://asia.gcr.io
gsutil acl get gs://asia.artifacts.{%PROJECT_ID}.appspot.com
Add insecure-registry to dockerd startup command.
--insecure-registry asia.gcr.io
Might be the same cause
gcloud docker -- pull google/python
The error was
Error response from daemon: Get https://registry-1.docker.io/v2/google/python/manifests/latest: read tcp 10.0.2.15:37762->52.45.33.149:443: read: connection reset by peer
docker server log
DEBU[0499] Increasing token expiration to: 60 seconds
ERRO[0500] Error trying v2 registry: Get https://registry-1.docker.io/....../python/manifests/latest: read tcp 10.0.2.15:37762->52.45.33.149:443: read: connection reset by peer
ERRO[0500] Attempting next endpoint for pull after error: Get https://registry-1.docker.io/....../python/manifests/latest: read tcp 10.0.2.15:37762->52.45.33.149:443: read: connection reset by peer
DEBU[0500] Skipping v1 endpoint https://index.docker.io because v2 registry was detected
ERRO[0500] Handler for POST /v1.24/images/create returned error: Get https://registry-1.docker.io/....../python/manifests/latest: read tcp 10.0.2.15:37762->52.45.33.149:443: read: connection reset by peer
Environment
MacOS: 10.11.6
Docker Toolbox (on MAC)
Docker 1.12.3 (Git commit: 6b644ec, Built: Wed Oct 26 23:26:11 2016)
The root cause was stupid, but I'd like to update this for anyone who see this question. I found when I attached my computer to company's WIFI. Then It would work (Still some reset). The cable network of my company is mysterious broken to Google Container Registry. The cable network works for all other services (google/youtube/mobile services) we used but broken to Google Container Registry.
Seems like a permission issue. Try running
gcloud auth login
I remember running into a similar issue and this helped.
When I login to the IBM containers on Bluemix, I get the following error:
cf ic login
** Retrieving client certificates from IBM Containers
** Storing client certificates in /Users/triplez/.ice/certs Successfully retrieved client certificates
** Checking local docker configuration OK
** Authenticating with registry at registry.ng.bluemix.net
Could not authenticate with IBM Containers registry at registry.ng.bluemix.net
**** exit status 1
****Error response from daemon: invalid registry endpoint registry.ng.bluemix.net/v0/: unable to ping registry endpoint
registry.ng.bluemix.net/v0/
v2 ping attempt failed with error: Get registry.ng.bluemix.net/v2/: dial tcp: lookup
registry.ng.bluemix.net on 103.11.48.126:53: read udp
103.11.48.126:53: i/o timeout
v1 ping attempt failed with error: Get registry.ng.bluemix.net/v1/_ping: dial tcp: lookup registry.ng.bluemix.net on 103.11.48.126:53: read udp
103.11.48.126:53: i/o timeout. If this private registry supports only HTTP or HTTPS with an unknown CA certificate, please add
--insecure-registry registry.ng.bluemix.net to the daemon's
arguments. In the case of HTTPS, if you have access to the registry's
CA certificate, no need for the flag; simply place the CA certificate
at /etc/docker/certs.d/registry.ng.bluemix.net/ca.crt
I've already uninstalled docker and reinstalled, giving me the same error. I've also deleted ~/.cf/config.json and ~/.ice/ice-cfg.ini and ~/.ice/certs/ and tried logging in again. I still receive the same error.
I've also tested this with ice cli with the same error.
EDITED
Tried this command:
ice login -a 'https://api.ng.bluemix.net' --registry 'registry.ng.bluemix.net' --host 'https://containers-api.ng.bluemix.net/v3/containers'
Got the same error:
Authentication with container cloud service at
containers-api.ng.bluemix.net/v3/containers completed
successfully
You can issue commands now to the container service
Proceeding to authenticate with the container cloud registry at
registry.ng.bluemix.net/v3
Error response from daemon: invalid registry endpoint
registry.ng.bluemix.net/v3/v0/:
unable to ping registry endpoint registry.ng.bluemix.net/v3/v0/
v2 ping attempt failed with error: Get registry.ng.bluemix.net/v3/v2/:
dial tcp: lookup registry.ng.bluemix.net on 192.168.0.1:53: read ump
192.168.0.1:53: i/o timeout
v1 ping attempt failed with error: Get registry.ng.bluemix.net/v3/v1/_ping: dial tcp: lookup registry.ng.bluemix.net on 192.168.0.1:53: read udp 192.168.0.1:53: i/o timeout. If this private registry supports only HTTP or HTTPS with an unknown CA certificate, please add --insecure-registry registry.ng.bluemix.net to the daemon's arguments. In the case of
HTTPS, if you have access to the registry's CA certificate, no need
for the flag; simply place the CA certificate at
/etc/docker/certs.d/registry.ng.bluemix.net/ca.crt
docker is not available on your system or is not properly configured
Could not authenticate with cloud registry at registry.ng.bluemix.net/v3
You can still use IBM Containers but will not be able to run local docker containers, push, or pull images
It looks like the container cloud service host name is not correct.
Please try to run the ice login command as below:
ice login -a 'https://api.ng.bluemix.net' --registry 'registry.ng.bluemix.net' --host 'https://containers-api.ng.bluemix.net/v3/containers'
Based on your comment "boot2docker has been deprecated" I assume you are using Docker 1.8.
This version of Docker is not supported by IBM Containers on Bluemix yet.
Please see documentation in link below saying Docker 1.6 or 1.7 is required:
https://www.ng.bluemix.net/docs/containers/container_cli_ov.html
I know IBM Containers team is working to add support for Docker 1.8 as well. Please keep an eye on documentation for updates in the future.