ZMQ event publisher in Jenkins doesn't send a notification - jenkins

I have been trying to figure out what is wrong with my Jenkins ZMQ-event-publisher configuration for more than 23 hours and have given up. Hopefully, you may have an idea what I am doing wrong.
I've installed Jenkins with ZMQ-event-publisher plugin and under Manage Jenkins->Configure System checked Enable on all Jobs (Note TCP port to publish on is set to 8888).
Created a new job, checked Check if ZMQ events should be published for this project and clicked on Save.
I have written a Python script using pyZMQ
#!/usr/bin/env python
import zmq
port = "8888"
context = zmq.Context()
socket = context.socket(zmq.SUB)
socket.connect("tcp://localhost:%s" % port)
socket.setsockopt(zmq.SUBSCRIBE, '')
print "Jenkins... waiting..."
string = socket.recv()
print "recv =>", string
Executing the above script on the Jenkins machine + running the Jenkins job.
Unfortunately, the script doesn't receive any ZMQ message from Jenkins.
Trying to capture the ZMQ message using either tcpdump -i eth0 'port 8888' or tcpdump -i lo 'port 8888' didn't help too.
In addition to that, looking at the /var/log/jenkins/jenkins.log, I get:
Sep 25, 2014 8:54:47 PM org.jenkinsci.plugins.ZMQEventPublisher.ZMQRunnable bindSocket
INFO: Binding ZMQ PUB to port 8888
Sep 25, 2014 8:54:48 PM hudson.model.Run execute
INFO: MyJob #18 main build action completed: SUCCESS
Moreover, netstat -ntlp verifies that
Proto Recv-Q Send-Q Local Address Foreign Address State PID/Program name
tcp 0 0 :::8888 :::* LISTEN 31/java
tcp 0 0 :::57467 :::* LISTEN 31/java
tcp 0 0 :::8009 :::* LISTEN 31/java
tcp 0 0 :::59373 :::* LISTEN 31/java
tcp 0 0 :::8080 :::* LISTEN 31/java
So, what am I doing wrong?
There is no good explanation on how to configure correctly the Jenkins ZMQ plugin and looking at the plugin code doesn't reveal much.
Your help will be more than appreciated.
Thanks.
EDIT : Dave's suggestion was great, but it hasn't fixed the problem, yet.
EDIT 2: It looks like it fails because Jenkins has been running from a Docker container and I've forgotten to expose all its ports. Nevertheless, It looks like Dave's suggestions does fix the problem. Yay!!!

From the ZMQ Guide:
Note that when you use a SUB socket you must set a subscription using zmq_setsockopt() and SUBSCRIBE
I suspect that your subscriber script is not seeing events because you need to set the subscribe filter. As described on the zmq_setsockopt page, setting the filter to the empty string subscribes to all messages.
Try adding:
socket.setsockopt(zmq.SUBSCRIBE, '')

Related

MQTT5 User Properties with Mosquitto Bridge

I am running a local Mosquitto (MQTT) Broker that connects to a remote Mosquitto Broker using the build in MQTT Bridge functionality of Mosquitto. My mosquitto.conf looks like this:
# =================================================================
# Listeners
# =================================================================
listener 1883
# =================================================================
# Security
# =================================================================
allow_anonymous true
# =================================================================
# Bridges
# =================================================================
connection myConnectionName
address <<Remote Broker IP>>:1883
remote_username <<Remote Broker Username>>
remote_password <<Remote Broker Password>>
topic mytopic/# out 1 "" B2/
bridge_protocol_version mqttv50
cleansession false
bridge_attempt_unsubscribe true
upgrade_outgoing_qos true
max_queued_messages 5000
For testing I run a MqttPublisher using a C# console application which uses the MQTTnet library (version 3) and a MqttSubsbriber (also C# console application with MqttNet).
Now I want the Publisher to publish MQTT messages with User Properties (introduced by MQTT 5).
I build the message like this:
using MQTTnet;
using MQTTnet.Client;
using MQTTnet.Client.Options;
class Program
{
static void Main()
{
// Create a new MQTT client instance
var factory = new MqttFactory();
var mqttClient = factory.CreateMqttClient();
// Setup the options for the MQTT client
var options = new MqttClientOptionsBuilder()
.WithClientId("MqttPublisher")
.WithTcpServer("localhost", 1883)
.WithProtocolVersion(MQTTnet.Formatter.MqttProtocolVersion.V500)
.WithCleanSession()
.Build();
mqttClient.ConnectAsync(options).Wait();
var i = 0;
while (true)
{
Console.WriteLine("Client connected: " + mqttClient.IsConnected);
var message = new MqttApplicationMessageBuilder()
.WithTopic("mytopic/test")
.WithUserProperty("UPTest","Hi UP")
.WithPayload("Hello World: " + i)
.Build();
mqttClient.PublishAsync(message).Wait();
Console.WriteLine("Published message with payload: " + System.Text.Encoding.UTF8.GetString(message.Payload));
i++;
System.Threading.Thread.Sleep(1000);
}
mqttClient.DisconnectAsync().Wait();
}
}
With the subscriber (also with WithProtocolVersion(MQTTnet.Formatter.MqttProtocolVersion.V500) if I subscribe to the topic I get all the messages and I can read the MQTTnet.MqttApplicationMessage like shown in the following screenshot:
The messages are also published to the remote MQTT Broker due to the MQTT Bride configured. However, if I subscribe to the remote Broker with my MqttSubscriber, I am not getting the User Properties anymore:
Is there any way to configure the Mosquitto Bridge that also the user properties are send? I cant find a way and any help and comments are appreciated.
Thanks
Joshua
Using mosqutto 2.0.15 I have verified that MQTTv5 message properties are passed over the bridge.
Broker one.conf
listener 1883
allow_anonymous true
Broker two.conf
listener 1884
allow_anonymous true
connection one
address 127.0.0.1:1883
topic foo/# both 0
bridge_protocol_version mqttv50
Subscriber connected to Broker two
$ mosquitto_sub -t 'foo/#' -V mqttv5 -p 1884 -F "%t %P %p"
foo/bar foo:bar ben
Publisher connected to Broker one
$ mosquitto_pub -D PUBLISH user-property foo bar -t 'foo/bar' -m ben -p 1883
As you can see the the %P in the output format for the subscriber is outputting the user property foo with a value of bar when subscribed over the bridge.

NAT traversal requires STUN or TURN

I'm a novice setting up a server for the first time to implement WebRTC
Linux is using Centos7 and has set up KMS and Coturn.
However, there is one problem.
The client and server are not connected on the screen, so I checked the logs of kms
docker logs --follow kms
0:00:01.206579656 1 0x56191aac5010 INFO KurentoServerMethods ServerMethods.cpp:90:ServerMethods: Using above 80% of system limits will throw NOT_ENOUGH_RESOURCES exception
0:00:01.206607827 1 0x56191aac5010 INFO KurentoServerMethods ServerMethods.cpp:109:ServerMethods: System limits: unlimited threads, 32768 files
0:00:01.206902099 1 0x56191aac5010 INFO KurentoWorkerPool WorkerPool.cpp:67:WorkerPool: Worker thread pool size: 2
0:00:01.207158442 1 0x56191aac5010 INFO KurentoServerMethods ServerMethods.cpp:144:ServerMethods: RPC Request Cache is ENABLED
0:00:01.207351433 1 0x56191aac5010 INFO KurentoWebSocketTransport WebSocketTransport.cpp:187:initWebSocket: WebSocket server (ws://) listening on address '::', port 8888
0:00:01.207411744 1 0x56191aac5010 INFO KurentoWebSocketTransport WebSocketTransport.cpp:88:WebSocketTransport: Secure WebSocket server (wss://) not enabled
0:00:01.208078290 1 0x56191aac5010 INFO KurentoMediaServer main.cpp:259:main: Kurento Media Server started
0:02:29.095818552 1 0x7f5070017630 INFO KurentoWebRtcEndpointImpl WebRtcEndpointImpl.cpp:164:generateDefaultCertificates: Unable to load the RSA certificate from file. Using the default certificate.
0:02:29.284074137 1 0x7f5070017630 INFO KurentoWebRtcEndpointImpl WebRtcEndpointImpl.cpp:174:generateDefaultCertificates: Unable to load the ECDSA certificate from file. Using the default certificate.
0:02:29.290405426 1 0x7f5070017630 INFO KurentoWebRtcEndpointImpl WebRtcEndpointImpl.cpp:110:remove_not_supported_codecs_from_array:<kmswebrtcendpoint0> Removing not supported codec 'AMR/8000'
0:02:29.515589312 1 0x7f5064039e00 INFO basertpendpoint kmsbasertpendpoint.c:1132:kms_base_rtp_endpoint_start_transport_send:<kmswebrtcendpoint0> Media 'video' has REMB
0:02:29.515721223 1 0x7f5064039e00 INFO basertpendpoint kmsbasertpendpoint.c:1078:kms_base_rtp_endpoint_create_remb_manager:<kmswebrtcendpoint0> Creating REMB for session ID 0 (kmswebrtcendpoint0-sess0) and remote video SSRC 3653849939
0:02:29.515746113 1 0x7f5064039e00 INFO basertpendpoint kmsbasertpendpoint.c:1089:kms_base_rtp_endpoint_create_remb_manager:<kmswebrtcendpoint0> REMB: Set RTCP min interval to 500 ms
0:02:29.519063004 1 0x7f5064007580 WARN kmswebrtcsession kmswebrtcsession.c:823:kms_webrtc_session_set_stun_server_info:<kmswebrtcsession0> STUN server not configured! NAT traversal requires STUN or TURN
0:02:29.519107324 1 0x7f5064007580 WARN kmswebrtcsession kmswebrtcsession.c:843:kms_webrtc_session_set_relay_info:<kmswebrtcsession0> TURN relay server not configured! NAT traversal requires STUN or TURN
0:02:29.522346434 1 0x7f50700054f0 INFO KurentoWorkerPool WorkerPool.cpp:67:WorkerPool: Worker thread pool size: 2
0:02:40.930306053 1 0x7f5050001630 INFO KurentoWebRtcEndpointImpl WebRtcEndpointImpl.cpp:110:remove_not_supported_codecs_from_array:<kmswebrtcendpoint1> Removing not supported codec 'AMR/8000'
0:02:40.951376487 1 0x7f5064018b30 INFO basertpendpoint kmsbasertpendpoint.c:1132:kms_base_rtp_endpoint_start_transport_send:<kmswebrtcendpoint1> Media 'video' has REMB
0:02:40.951898082 1 0x7f5064018b30 INFO basertpendpoint kmsbasertpendpoint.c:1078:kms_base_rtp_endpoint_create_remb_manager:<kmswebrtcendpoint1> Creating REMB for session ID 0 (kmswebrtcendpoint1-sess0) and remote video SSRC 3442416509
"NAT traversal requires STUN or TURN."
I don't know how to solve this part.
This is because the STUN server results from Trickle ICE were also successful.
If you know what I need to do, I'd appreciate it if you could tell me all the actions.
And please let me know if there is anything else I need to fill out!
STUN and TURN
You dont have to have coturn if you are doing local testing. The warning is saying if you want to go outside of your network (out of your router and to the web) you will need a STUN or TURN server.
Docker
Docker doesn't open the port 8888 by itself. You may need to open that port manually. To do this, add this -p 8888:8888 when creating your container.
Or if you are using the Desktop version you can enter it into Host port under the Optional settings when you first run it.

Why does FreeRadius fail to process the accounting response from Fortigate?

I have configured a freeradius proxy (3.0.16) on Ubuntu (4.15.0-47-generic). It receives the radius accounting packets from another radius server running on Ubuntu and writes those to another radius server on running on Fortigate.
Radius Server ---> Proxy Radius Server ---> Fortigate Radius Server
I have configured copy-acct-to-home-server to include the Realm in proxy.conf
proxy.conf ( Realm definition )
home_server myFortigate {
type = acct
ipaddr = <IP address of Fortigate Interface Running Radius>
port = 1813
secret = superSecret
}
home_server_pool myFortigatePool {
type = fail-over
home_server = myFortigate
}
realm myFortigateRealm {
acct_pool = myFortigatePool
nostrip
}
copy-acct-to-home-server entry
preacct {
preprocess
update control {
Proxy-To-Realm := myFortigateRealm
}
suffix
}
After I run the freeradius -X, I also run tcpdump from a new session
tcpdump -ni eth01 port 1812 or port 1813
and get the following log
15:03:40.225570 IP RADIUS_PROXY_IP.56813 > FORTIGATE_INTERFACE_IP.1813: RADIUS, Accounting-Request (4), id: 0x31 length: 371
15:03:40.236155 IP FORTIGATE_INTERFACE_IP.1813 > RADIUS_PROXY_IP.56813: RADIUS, Accounting-Response (5), id: 0x31 length: 27
Which basically shows it is sending the account request to fortigate radius server and receiving the accounting response.
But strangely freeradius -X debug output shows a request time out for the same radius server on Fortigate and it ultimately tags the server as zombie
Starting proxy to home server FORTIGATE_INTERFACE_IP port 1813
(14) Proxying request to home server FORTIGATE_INTERFACE_IP port 1813 timeout 30.000000
Waking up in 0.3 seconds.
(14) Expecting proxy response no later than 29.667200 seconds from now
Waking up in 3.5 seconds.
and Finally it gives up
25) accounting {
(25) [ok] = ok
(25) } # accounting = ok
(25) ERROR: Failed to find live home server: Cancelling proxy
(25) WARNING: No home server selected
(25) Clearing existing &reply: attributes
(25) Found Post-Proxy-Type Fail-Accounting
(25) Post-Proxy-Type sub-section not found. Ignoring.
So the situation is the Radius proxy is sending accounting packets to Fortigate Radius server (could be seen in both freeradius and fortigate logs)
tcpdump shows that Radius proxy is receiving accounting response from the fortigate, but for some reason freeradius process doesn't recognize (or can not read) accounting response. It may be some interoperability issue or I have missed to set some flag. Requesting help from the experts to isolate and rectify the issue.

Ganglia:No nodes were viewed in ganglia web (centOS7)

I installed ganglia server and client at the same machine. But no nodes can view in the web when it finished. No matter google or baidu,no resolution about this problem appeared.I need help.
So this is my gmetad.conf:
[root#tools etc]# egrep -v "^#|^$" gmetad.conf
data_source "trainor" localhost 127.0.0.1
setuid_username "apache"
rrd_rootdir "/var/lib/ganglia/rrds"
case_sensitive_hostnames 0
here is my gmond.conf:
[root#tools etc]# egrep -v "^#|^$" gmond.conf
globals {
user = apache
}
cluster{
name = "trainor"
owner = "apache"
latlong = "unspecified"
url = "unspecified"
}
udp_recv_channel {
port = 8649
}
tcp_accept_channel {
port = 8649
}
Do you have a udp_send_channel set? In my experience (3.1.7), gmond doesn't report a node's own stats over the TCP channel (xml reporting) unless it receives them over UDP (raw stats collection).
You can use "gstat" to connect to gmond to see what it's outputting, or netcat to the TCP port:
nc node1.domain.com 8649
I found these pages the most useful:
https://github.com/ganglia/monitor-core/wiki/Ganglia-Quick-Start
http://timstaley.co.uk/posts/ganglia-setup-explained/

Why does lighttpd mod_fastcgi start a listening socket?

I read in lighttpd 1.4.19 source codes, now I got stuck at the function fcgi_spawn_connection,
if (-1 == connect(fcgi_fd, fcgi_addr, servlen)) {
...
bind(fcgi_fd, fcgi_addr, servlen)
...
listen(fcgi_fd, 1024)
}
The question is, why does mod_fastcgi create a listening socket, what's it use for? Isn't that mod_fastcgi works as client connect to fastcgi processes(etc, php-cgi)? - the php-cgi processes will listen.
Thanks.
OK, I thought i got it.
the php-cgi will not create a listen socket, but inherits from fcgi_spawn_connection, while this listen socket has been dup2 FCGI_LISTENSOCK_FILENO(usually 0).

Resources