I am writing a program that will parse and manipulate logs from Darwin Streaming Servers. Where can I get a definition of what the individual fields headers mean? Are there some kind of documentation in the DSS doc files?
For example, I have this as the log output header:
c-ip date time c-dns cs-uri-stem c-starttime x-duration c-rate c-status c-playerid c-playerversion c-playerlanguage cs(User-Agent) c-os c-osversion c-cpu filelength filesize avgbandwidth protocol transport audiocodec videocodec sc-bytes cs-bytes c-bytes s-pkts-sent c-pkts-received c-pkts-lost-client c-buffercount c-totalbuffertime c-quality s-ip s-dns s-totalclients s-cpu-util cs-uri-query c-username sc(Realm)
Please check:
QuickTime Streaming Server 2.0 Access Log Format
http://support.apple.com/kb/TA26756?viewlocale=en_US
I've managed to get some information about the log format from the net, but this is not complete. Is there a link to the complete log format?
DSS Log Format (Google Docs)
Related
Telegraf plugin:
HTTP Input Plugin
I'm trying to use telegraf to collect data from an vendor API.
test.conf file looks like this:
[[inputs.http]] urls = ["https://10.10.10.10"] method = "POST" body = '{"F_":"LOGIN","DATA":{"ID":"user","PWD":"password"}}'
When debugging i can see this is the error i get:
[inputs.http] Error in plugin: [url=https://10.10.10.10]: received status code 411 (Length Required), expected any value out of [200]
The API documentation for my vendor API states that the field "Content-Length" and "Host" are mandatory, but I can find now way to enable that in the plugin.
I have also tried using the http_response plugin, and I am able to get the JSON reply, but unfortuately i have not been able to find a good way to get the response_body_field JSON appended to the influxdb.
Does anybody know if either:
I can enable the two headers dynamicly(The POST lenght will vary)
or:
use the response_body_field from http_response and smoothly parse it to the influxdb?
Need to post a byte message to solace queue using Jmeter. I have tried in following manner might be am incorrect but tried with following:
Use JMSPublisher sampler
create jndi.properties file and put in jmeter/lib
jndi.properties
java.naming.factory.initial = com.solacesystems.jndi.SolJNDIInitialContextFactory
java.naming.provider.url = smf://<remote IP and port>
java.naming.security.principal=<username>
java.naming.security.credentials=<password>
Solace_JMS_VPN=<VPN Name>
in JMSPublisher sampler (in GUI)
Connection Factory = connectionFactory
Destination = (Queue Name )
Message Type (radio button---Byte message)
Content encoding -- RAW
in text area ---> (Byte message)
Note : I have used actual values of IP/port/username/port/queuename/bytemessage, cannot share those. Soljms jar is available in lib folder too.
getting error :
Response message: javax.naming.NamingException: JNDI lookup failed - 503: Service Unavailable [Root exception is (null) com.solacesystems.jcsmp.JCSMPErrorResponseException: 503: Service Unavailable]
Though it is working perfectly fine when did with java spring boot. There used properties files in place of JNDI.
It would be great if anyone can guide me , please do not give activeMQ JNDI am actively looking for posting on solace queue or create connection to solace appliances through Jmeter.
I don't think you should be putting your Byte message into the textarea as it accepts either plain text or an XStream object, consider providing your payload via binary file(s) instead
If you're capable of sending the message using Java code you should be able to replicate the same using:
JMeter's JSR223 Sampler with Groovy language (Java syntax will work)
Or JUnit Request sampler if you need "strict" java
Trying to run producer.py from solace-samples-amqp-qpid-proton-python with payload of python dict
Message(id=(self.sent+1), body={'sequence':(self.sent+1)})
Get following error
Reject message: 1 Remote disposition:
Condition('amqp:not-implemented', 'unsupported AMQP value type:
TOK_MAP_START')
Get similar error when trying to send integer value in body - TOK_TYPE_INT
Does solace support only Strings over AMQP?
Solace message brokers support amqp-value message sections containing values of types null, string, binary, symbol, or uuid. (https://docs.solace.com/Open-APIs-Protocols/AMQP/AMQP-Protocol-Conformance.htm#Sec3-2-8)
This is done in order to preserve maximum message inter-operability.
Any published message using a language specific semantic can only be consumed using the same semantic. I.e. if you publish with Python dict, you can only decode using Python dict, so if you are using a MQTT or REST consumer, it will not be able to decode the message.
The best option is to use a cross-language serialization library, which will make it easier for future expansions. For example, you might decide to add an REST consumer to in future, which can decode the data using the cross-language serialization library.
I want to build a small script in python which needs to fetch an url. The server is a kind of crappy though and replies pure ASCII without any headers.
When I try:
import urllib.request
response = urllib.request.urlopen(url)
print(response.read())
I obtain a http.client.BadStatusLine: 100 error because this isn't a properly formatted HTTP response.
Is there another way to fetch an url and get the raw content, without trying to parse the response?
Thanks
It's difficult to answer your direct question without a bit more information; not knowing exactly how the (web) server in question is broken.
That said, you might try using something a bit lower-level, a socket for example. Here's one way (python2.x style, and untested):
#!/usr/bin/env python
import socket
from urlparse import urlparse
def geturl(url, timeout=10, receive_buffer=4096):
parsed = urlparse(url)
try:
host, port = parsed.netloc.split(':')
except ValueError:
host, port = parsed.netloc, 80
sock = socket.create_connection((host, port), timeout)
sock.sendall('GET %s HTTP/1.0\n\n' % parsed.path)
response = [sock.recv(receive_buffer)]
while response[-1]:
response.append(sock.recv(receive_buffer))
return ''.join(response)
print geturl('http://www.example.com/') #<- the trailing / is needed if no
other path element is present
And here's a stab at a python3.2 conversion (you may not need to decode from bytes, if writing the response to a file for example):
#!/usr/bin/env python
import socket
from urllib.parse import urlparse
ENCODING = 'ascii'
def geturl(url, timeout=10, receive_buffer=4096):
parsed = urlparse(url)
try:
host, port = parsed.netloc.split(':')
except ValueError:
host, port = parsed.netloc, 80
sock = socket.create_connection((host, port), timeout)
method = 'GET %s HTTP/1.0\n\n' % parsed.path
sock.sendall(bytes(method, ENCODING))
response = [sock.recv(receive_buffer)]
while response[-1]:
response.append(sock.recv(receive_buffer))
return ''.join(r.decode(ENCODING) for r in response)
print(geturl('http://www.example.com/'))
HTH!
Edit: You may need to adjust what you put in the request, depending on the web server in question. Guanidene's excellent answer provides several resources to guide you on that path.
What you need to do in this case is send a raw HTTP request using sockets.
You would need to do a bit of low level network programming using the socket python module in this case. (Network sockets actually return you all the information sent by the server as it as, so you can accordingly interpret the response as you wish. For example, the HTTP protocol interprets the response in terms of standard HTTP headers - GET, POST, HEAD, etc. The high-level module urllib hides this header information from you and just returns you the data.)
You also need to have some basic information about HTTP headers. For your case, you just need to know about the GET HTTP request. See its definition here - http://djce.org.uk/dumprequest, see an example of it here - http://en.wikipedia.org/wiki/HTTP#Example_session. (If you wish to capture live traces of HTTP requests sent from your browser, you would need a packet sniffing software like wireshark.)
Once you know basics about socket module and HTTP headers, you can go through this example - http://coding.debuntu.org/python-socket-simple-tcp-client which tells you how to send a HTTP request over a socket to a server and read its reply back. You can also refer to this unclear question on SO.
(You can google python socket http to get more examples.)
(Tip: I am not a Java fan, but still, if you don't find enough convincing examples on this topic under python, try finding it under Java, and then accordingly translate it to python.)
urllib.urlretrieve('http://google.com/abc.jpg', 'abc.jpg')
I have a GPS tracker . It's a chinese model, with sparse documentation. It's got a built in gps and a gprs module (sim) and it's sending me my data to a particular ip address.
No idea what the format is
I using "template" OpenGTS server
Log:
[INFO_|06/20 11:47:28|TrackClientPacketHandler.sessionStarted:256] Begin TCP communication: 84.15.15.12 [Mon Jun 20 11:47:28 EE ST 2011]
[INFO_|06/20 11:47:37|TrackClientPacketHandler.parseInsertRecord_ASCII_1:565] Parsing: �32472798�?
[WARN_|06/20 11:47:37|TrackClientPacketHandler.parseInsertRecord_ASCII_1:576] Invalid number of fields
[WARN_|06/20 11:47:41|ServerSocketThread$ServerSessionThread.readPacket:2001] Timeout: 0x007F070F00FFFFFFFF00F0640000000000000000000000000000000000000000000000000000000000000000000000000000B822511D1013031E00067360215281F2000000000630079659105100000000000000000000000000000000000000000000000000000000000000000000000000B4
[WARN_|06/20 11:47:41|ServerSocketThread$ServerSessionThread.run:1527] Read timeout [# 115]
[INFO_|06/20 11:47:41|ServerSocketThread$ServerSessionThread.run:1550] End of session ...
[INFO_|06/20 11:47:41|TrackClientPacketHandler.sessionTerminated:270] End TCP communication: 84.15.15.12
�32472798�? is part of IMEI number
Please help to understand data format
It seems that this tracker uses a binary protocol and it will be too hard to implement this protocol without documentation.