I have been using flume for a while now, I have got agent and collector running on same machine.
Configuration
agent: exec("/usr/bin/tail -n +0 -F /path/to/file") | agentE2ESink("hostname", 35855)
collector: collectorSource(35855) | collector(10000) { collectorSink("/hdfs/path/to/sink","name") }
Facing issues in the agent node:
2012-06-04 19:13:33,625 [naive file wal consumer-27] INFO debug.InsistentOpenDecorator: open attempt 0 failed, backoff (1000ms): Failed to open thrift event sink to hostname:35855 : java.net.ConnectException: Connection refused
2012-06-04 19:13:34,625 [logicalNode hostname-19] ERROR connector.DirectDriver: Expected ACTIVE but timed out in state OPENING
2012-06-04 19:13:34,632 [naive file wal consumer-27] INFO debug.InsistentOpenDecorator: open attempt 1 failed, backoff (2000ms): Failed to open thrift event sink to hostname:35855 : java.net.ConnectException: Connection refused
2012-06-04 19:13:36,635 [naive file wal consumer-27] INFO debug.InsistentOpenDecorator: open attempt 2 failed, backoff (4000ms): Failed to open thrift event sink to hostname:35855 : java.net.ConnectException: Connection refused
and then empty ACKs will be sent continuously
2012-06-04 19:19:56,960 [Roll-TriggerThread-0] INFO endtoend.AckListener$Empty: Empty Ack Listener began 20120604-191956958+0530.881565921235084.00000026
2012-06-04 19:20:07,043 [Roll-TriggerThread-0] INFO hdfs.SeqfileEventSink: closed /tmp/flume-user1/agent/hostname/writing/20120604-191956958+0530.881565921235084.00000026
I dont understand why the connection is refused. Are there any system level changes that needs to be done ?
Note: the collector is listening to the port but agent is unable to send data through the 35855 port.
Can anyone help me with this problem.
Thanks
If you are running both the agent and the collector on the same box, you should be using localhost as the address.
agentE2ESink("localhost", 35855)
Related
Nebula Graph fails on CentOS 6.5, the error message is as follows:
# storage log
Heartbeat failed, status:RPC failure in MetaClient: N6apache6thrift9transport19TTransportExceptionE: AsyncSocketException: connect failed, type = Socket not open, errno = 111 (Connection refused): Connection refused
# meta log
Log line format: [IWEF]mmdd hh:mm:ss.uuuuuu threadid file:line] msg
E0415 22:32:38.944437 15532 AsyncServerSocket.cpp:762] failed to set SO_REUSEPORT on async server socket Protocol not available
E0415 22:32:38.945001 15510 ThriftServer.cpp:440] Got an exception while setting up the server: 92failed to bind to async server socket: [::]:0: Protocol not available
E0415 22:32:38.945057 15510 RaftexService.cpp:90] Setup the Raftex Service failed, error: 92failed to bind to async server socket: [::]:0: Protocol not available
E0415 22:32:38.949586 15463 NebulaStore.cpp:47] Start the raft service failed
E0415 22:32:38.949597 15463 MetaDaemon.cpp:88] Nebula store init failed
E0415 22:32:38.949796 15463 MetaDaemon.cpp:215] Init kv failed!
Nebula service status is as follows:
[root#redhat6 scripts]# ./nebula.service status all
[WARN] The maximum files allowed to open might be too few: 1024
[INFO] nebula-metad: Exited
[INFO] nebula-graphd: Exited
[INFO] nebula-storaged: Running as 15547, Listening on 44500
Reason for error: CentOS 6.5 system kernel version is 2.6.32, which is less than 3.9. However, SO_REUSEPORT only supports Linux 3.9 and above.
Upgrading the system to CentOS 7.5 can solve the problem by itself.
Setup: zeppelin-0.6.0-dse-5.0.0-5.0.1 + Spark 1.6.1
connection issue while running a notebook containing a single statement %spark.
Error Trace: INFO [2017-11-26 09:37:27,981] ({pool-1-thread-2}
RemoteInterpreterProcess.java[reference]:143) - Run interpreter
process [/data/zeppelin-0.6.0/bin/interpreter.sh, -d,
/data/zeppelin-0.6.0/interpreter/spark, -p, 34222, -l,
/data/zeppelin-0.6.0/local-repo/2CYEPSQ2N] INFO [2017-11-26
09:37:28,009] ({Exec Default Executor}
RemoteInterpreterProcess.java[onProcessComplete]:276) - Interpreter
process exited 0 ERROR [2017-11-26 09:37:58,054] ({Thread-15}
RemoteScheduler.java[getStatus]:255) - Can't get status information
org.apache.zeppelin.interpreter.InterpreterException:
org.apache.thrift.transport.TTransportException:
java.net.ConnectException: Connection refused (Connection refused)
How can i detect this exception when i start my application:
org.springframework.amqp.AmqpConnectException: java.net.ConnectException: Connection timed out: connect
The project is configured in the Spring way (http://projects.spring.io/spring-amqp/) and I can see the errors in the logs, but I can't detect it in my java Classes.
Consumer raised exception, processing can restart if the connection factory supports it. Exception summary: org.springframework.amqp.AmqpConnectException: java.net.ConnectException: Connection timed out: connect
Restarting Consumer: tags=[{}], channel=null, acknowledgeMode=AUTO local queue size=0
Failed to check/redeclare auto-delete queue(s).org.springframework.amqp.AmqpConnectException: java.net.ConnectException: Connection timed out: connect
at org.springframework.amqp.rabbit.support.RabbitExceptionTranslator.convertRabbitAccessException(RabbitExceptionTranslator.java:54)
at org.springframework.amqp.rabbit.connection.AbstractConnectionFactory.createBareConnection(AbstractConnectionFactory.java:217)
at org.springframework.amqp.rabbit.connection.CachingConnectionFactory.createConnection(CachingConnectionFactory.java:444)
at org.springframework.amqp.rabbit.connection.ConnectionFactoryUtils$1.createConnection(ConnectionFactoryUtils.java:80)
at org.springframework.amqp.rabbit.connection.ConnectionFactoryUtils.doGetTransactionalResourceHolder(ConnectionFactoryUtils.java:130)
at org.springframework.amqp.rabbit.connection.ConnectionFactoryUtils.getTransactionalResourceHolder(ConnectionFactoryUtils.java:67)
at org.springframework.amqp.rabbit.core.RabbitTemplate.doExecute(RabbitTemplate.java:1035)
at org.springframework.amqp.rabbit.core.RabbitTemplate.execute(RabbitTemplate.java:1028)
at org.springframework.amqp.rabbit.core.RabbitTemplate.execute(RabbitTemplate.java:1004)
at org.springframework.amqp.rabbit.core.RabbitAdmin.getQueueProperties(RabbitAdmin.java:254)
at org.springframework.amqp.rabbit.listener.SimpleMessageListenerContainer.redeclareElementsIfNecessary(SimpleMessageListenerContainer.java:947)
at org.springframework.amqp.rabbit.listener.SimpleMessageListenerContainer.access$300(SimpleMessageListenerContainer.java:82)
at org.springframework.amqp.rabbit.listener.SimpleMessageListenerContainer$AsyncMessageProcessingConsumer.run(SimpleMessageListenerContainer.java:1065)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.net.ConnectException: Connection timed out: connect
at java.net.DualStackPlainSocketImpl.connect0(Native Method)
at java.net.DualStackPlainSocketImpl.socketConnect(DualStackPlainSocketImpl.java:79)
at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)
at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)
at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)
at java.net.PlainSocketImpl.connect(PlainSocketImpl.java:172)
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
at java.net.Socket.connect(Socket.java:579)
at com.rabbitmq.client.impl.FrameHandlerFactory.create(FrameHandlerFactory.java:32)
at com.rabbitmq.client.ConnectionFactory.newConnection(ConnectionFactory.java:615)
at com.rabbitmq.client.ConnectionFactory.newConnection(ConnectionFactory.java:651)
at org.springframework.amqp.rabbit.connection.AbstractConnectionFactory.createBareConnection(AbstractConnectionFactory.java:208)
... 12 more
This can happen for example when you have no access to the AMQP port.
Thank you!
There is currently no event emitted when this error occurs (I have created a JIRA issue to add one.
In the meantime, since the the connection is shared, you could call createConnection() on the connection factory from your own code from time to time, to check the connection state.
Or, you could hook into the logging subsystem to capture the WARN log, as I described in this answer.
I am using Datastax enterprise edition and in cluster one is Hadoop/Hive .I am trying to connect to hive with datastax hive odbc connector.I am getting error like :
Connector Version: V1.0.0.1007
Running connectivity tests...
Attempting connection
Failed to establish connection
SQLSTATE: HY000[DataStax][Hardy] (22) Error from ThriftHiveClient: connect() failed: errno = 10061
TESTS COMPLETED WITH ERROR
The error 10061 means connection refused
Seems like you have not started the hive service on your Analytics node therefore nothing is listening on TCP 10000
Please login into one of your DSE Analytics node and execute:
dse hive --service hiveserver
Then try again your test from your Windows system
Source: http://www.datastax.com/documentation/datastax_enterprise/4.0/datastax_enterprise/ana/anaHivStrtSvr.html
I am using grails-2.0.4 version with GGTS ide. When i tried to install mongodb some errors occuring. I have tried in two ways.
**1)compile ":mongodb:1.0.0.GA"**
Then it is getting this error:
| Loading Grails 2.0.4
| Configuring classpath
:: problems summary ::
:::: ERRORS
Server access Error: Connection timed out: connect url=http://plugins.grails.org/grails-mongodb/tags/RELEASE_1_0_0_GA/mongodb-1.0.0.GA.pom
Server access Error: Connection timed out: connect url=http://plugins.grails.org/grails-mongodb/tags/RELEASE_1_0_0_GA/grails-mongodb-1.0.0.GA.jar
Server access Error: Connection timed out: connect url=http://repo.grails.org/grails/plugins//mongodb/1.0.0.GA/mongodb-1.0.0.GA.pom
Server access Error: Connection timed out: connect url=http://repo.grails.org/grails/plugins//mongodb/1.0.0.GA/mongodb-1.0.0.GA.jar
Server access Error: Connection timed out: connect url=http://repo.grails.org/grails/core//mongodb/1.0.0.GA/mongodb-1.0.0.GA.pom
Server access Error: Connection timed out: connect url=http://repo.grails.org/grails/core//mongodb/1.0.0.GA/mongodb-1.0.0.GA.jar
Server access Error: Connection timed out: connect url=http://svn.codehaus.org/grails/trunk/grails-plugins/grails-mongodb/tags/RELEASE_1_0_0_GA/mongodb-1.0.0.GA.pom
Server access Error: Connection timed out: connect url=http://svn.codehaus.org/grails/trunk/grails-plugins/grails-mongodb/tags/RELEASE_1_0_0_GA/grails-mongodb-1.0.0.GA.jar
Server access Error: Connection timed out: connect url=http://repo1.maven.org/maven2//mongodb/1.0.0.GA/mongodb-1.0.0.GA.pom
Server access Error: Connection timed out: connect url=http://repo1.maven.org/maven2//mongodb/1.0.0.GA/mongodb-1.0.0.GA.jar
| Error Failed to resolve dependencies (Set log level to 'warn' in BuildConfig.groovy for more information):
:mongodb:1.0.0.GA
**2) grails install-plugin C:\Documents and Settings\marao18\Desktop\gorm-mongodb-0.5.4**
Then it is getting this error: Please help me anyone.
| Loading Grails 2.0.4
| Configuring classpath.
| Environment set to development.....
| Resolving plugin C:\Documents. Please wait...
:: problems summary ::
:::: ERRORS
Server access Error: Connection timed out: connect url=http://plugins.grails.org/grails-%5CDocuments/tags/RELEASE_and/%5CDocuments-and.pom
Server access Error: Connection timed out: connect url=http://plugins.grails.org/grails-%5CDocuments/tags/RELEASE_and/grails-%5CDocuments-and.zip
Server access Error: Connection timed out: connect url=http://repo.grails.org/grails/plugins/C/%5CDocuments/and/%5CDocuments-and.pom
Server access Error: Connection timed out: connect url=http://repo.grails.org/grails/plugins/C/%5CDocuments/and/%5CDocuments-and.zip
Server access Error: Connection timed out: connect url=http://repo.grails.org/grails/core/C/%5CDocuments/and/%5CDocuments-and.pom
Server access Error: Connection timed out: connect url=http://repo.grails.org/grails/core/C/%5CDocuments/and/%5CDocuments-and.zip
Server access Error: Connection timed out: connect url=http://svn.codehaus.org/grails/trunk/grails-plugins/grails-%5CDocuments/tags/RELEASE_and/%5CDocuments-and.pom
Server access Error: Connection timed out: connect url=http://svn.codehaus.org/grails/trunk/grails-plugins/grails-%5CDocuments/tags/RELEASE_and/grails-%5CDocuments-and.zip
Server access Error: Connection timed out: connect url=http://repo1.maven.org/maven2/C/%5CDocuments/and/%5CDocuments-and.pom
Server access Error: Connection timed out: connect url=http://repo1.maven.org/maven2/C/%5CDocuments/and/%5CDocuments-and.zip
| Error resolving plugin [name:\Documents, group:C, version:and]. Plugin not found.
| Error Plugin not found for name [C:\Documents] and version [and]