send xml data via socket with message (VLI) Ruby - ruby-on-rails

Ruby novice. First time post so excuse any communication protocol inadequacies :)
This site has been a great help and a "HUGE!!!" shoutout of thanks to all.
I need to connect my rails app to an electricity providers api so I can vend electricity to my web customers. I'm needing some help to simply get an initial request sent to the API
IP: 41.204.194.188
Port: 8945
First block: What is a message variable length indicator (VLI)?
"2 bytes precede every message sent to/from BizSwitch. The 2 bytes are referred to as a variable length indicator. Bytes 1-2 indicate the number of bytes in the message (excluding the first 2 bytes). The 2 bytes represent a 16bit unsigned integer in network byte order. Note that if a compressed message is being sent, the message will have to first be compressed, in order to determine its length, before being sent."
Ignore compression.
link to api doc: https://dl.dropboxusercontent.com/u/3815995/Ipay-prepaidElecTransactionSpec.pdf
Simple Vend Request example:
<ipayMsg client="ipay" term="1" seqNum="0" time="2002-05-16 10:55:30 +0200">
<elecMsg ver="2.37">
<vendReq>
<ref>136105500001</ref>
<amt cur="ZAR">11400</amt>
<numTokens>1</numTokens>
<meter>A12C3456789</meter>
<payType>cash</payType>
</vendReq >
</elecMsg>
</ipayMsg>
Simple Vend Response example
<ipayMsg client="ipay" term="1" seqNum="0" time="2002-05-16 10:55:35 +0200">
<elecMsg ver="2.37"
<vendRes> <ref>136105500001</ref>
<res code="elec000">OK</res>
<util addr="Megawatt Park, Contact Centre tel 086-003-7566" taxRef="4740101508" distId="6004708001509">Eskom Online</util>
<stdToken units="346.34" rctNum="12345678" amt="10000" tax="1400">12345678901234567890</stdToken>
<rtlrMsg>060000 Warning: This meter is not configured for FBE.</rtlrMsg>
<customerMsg>Meter not registered for Free Basic Electricity. Please apply at your local office.</customerMsg>
</vendRes>
</elecMsg>
</ipayMsg>
I've got this far and I seem connected but how do I actually send and receive responses? I've tried googling for help but yet to find how to send the XML packet and then receive the response.
#!/usr/bin/env ruby
require 'socket'
begin
socket = TCPSocket.new('41.204.194.188', 8945)
rescue => e
puts "error: #{e}"
else
puts "connected"
end
socket.close
Would appreciate any assistance or a nudge in the right direction.
Kind regards,
Jamie

Great I figured it out. Main issue regarding communication to the socket was sending a message variable length indicator. This stackoverflow question put me on the write path "Ruby - How to represent message length as 2 binary bytes"
Step 1: determine the length of my xml message length = message.size
The first field in the header must be the message length which is defined as a 2 binary byte message length in network byte order.
Step 2: message_variable_length_indicator = [length].pack("n")
Step 3: Connect to socket streamSock = TCPSocket::new('41.204.194.188', 8945)
Step 4: streamSock.write(message_variable_length_indicator)
Step 5: streamSock.write(message)
Step 6: get a response str = streamSock.recvfrom(1000)
Now to deal with timeout but at least I'm connecting :)

I have done this in PHP may be you will get idea from below code its working fine for my request.
function sendSocketRequest($XmlString, $Socket_Request) {
if (!($sock = socket_create(AF_INET, SOCK_STREAM, 0))) {
$errorcode = socket_last_error();
$errormsg = socket_strerror($errorcode);
die("Couldn't create socket: [$errorcode] $errormsg \n");
}
if (!socket_connect($sock, $Socket_Request['HostName'], $Socket_Request['Port'])) {
$errorcode = socket_last_error();
$errormsg = socket_strerror($errorcode);
die("Could not connect: [$errorcode] $errormsg \n");
}
$status = socket_write($sock, pack_int32be(strlen($XmlString)), 4);
$status = socket_write($sock, $XmlString, strlen($XmlString));
$response = socket_read($sock, $this->_socketReadLength);
socket_close($sock);
return substr($response, 2);
}
function pack_int32be($i) {
if ($i < -2147483648 || $i > 2147483647) {
die("Out of bounds");
}
return pack('C4', ($i >> 24) & 0xFF, ($i >> 16) & 0xFF, ($i >> 8) & 0xFF, ($i >> 0) & 0xFF
);
}
$socketResponse = sendSocketRequest($yourXMLString, array('HostName'=>'<HostName>','Port'=>'<Port>');

Related

Can't Parse DHCP packets with Ryu's get_protocol(dhcp.dhcp)

I'm using the Ryu SDN controller with an Open vSwitch on mininet using OpenFlow 1.3 to parse DHCP packets. Following online examples and the Ryu resources, I've implemented a DHCP packet parser. However, it does not work as I expected it to, and I'm wondering if anyone has any insight as to why my first solution does not work?
An example of a code snippet for parsing a DHCP packet is below:
from ryu.lib.packet import dhcp
...
...
#set_ev_cls(ofp_event.EventOFPPacketIn, MAIN_DISPATCHER)
def _packet_in_handler(self, ev):
msg = ev.msg
datapath = msg.datapath
pkt = packet.Packet(msg.data)
dhcpPacket = pkt.get_protocol(dhcp.dhcp)
My code follows a similar vein:
from ryu.lib.packet import dhcp
...
...
#set_ev_cls(ofp_event.EventOFPPacketIn, MAIN_DISPATCHER)
def _packet_in_handler(self, ev):
pkt = {}
pkt['msg'] = ev.msg
pkt['dp'] = pkt['msg'].datapath
pkt['pkt'] = packet.Packet(pkt['msg'].data)
pkt['dhcp'] = pkt['pkt'].get_protocol(dhcp.dhcp)
This seems reasonable as I am following this exact sequence with other protocols like ARP, ICMP, IP, etc. Examples below.
pkt['arp'] = pkt['pkt'].get_protocol(arp.arp)
pkt['ip'] = pkt['pkt'].get_protocol(ipv4.ipv4)
pkt['icmp'] = pkt['pkt'].get_protocol(icmp.icmp)
The only problem is that the three parsers I list above actually return data, while the get_protocol for DHCP consistently returns None. I have already tested this by sending DHCP packets through my switch.
What does work is the following code snippet where I identify packet lists having more than three values. I save the value at index three and set that as my DHCP packet. In the DHCP packet, I concentrate of parsing the string at index 2. That contains the data I'm interested in.
# pkt['dhcp'] = pkt['pkt'].get_protocol(dhcp.dhcp)
# Check if pkt['pkt]] > 3 elements, if so, parse DHCP string
#Standard pkt['dhcp'] = (None, None, String)
if len(pkt['pkt']) > 3:
pkt['dhcp'] = dhcp.dhcp.parser(pkt['pkt'][3])
pkt['op'] = hex(ord(dhcp_p[2][0]))
pkt['htype'] = hex(ord(dhcp_p[2][1]))
pkt['hlen'] = hex(ord(dhcp_p[2][2]))
pkt['hops'] = hex(ord(dhcp_p[2][3]))
def parseDHCP(pkt_d,start,stop):
s_value = ''
stop += 1
for val in range(start,stop):
s_value += str(hex(ord(pkt_d[val])))
return s_value
pkt['xid'] = parseDHCP(dhcp_p[2],4,7)
pkt['secs'] = parseDHCP(dhcp_p[2],8,9)
pkt['flags'] = parseDHCP(dhcp_p[2],10,11)
pkt['ciaddr'] = parseDHCP(dhcp_p[2],12,15)
pkt['yiaddr'] = parseDHCP(dhcp_p[2],16,19)
pkt['siaddr'] = parseDHCP(dhcp_p[2],20,23)
pkt['giaddr'] = parseDHCP(dhcp_p[2],24,27)
pkt['chaddr'] = parseDHCP(dhcp_p[2],28,33)
pkt['pad'] = parseDHCP(dhcp_p[2],34,43)
A print out of these values looks like so:
0x1
0x1
0x6
0x0
0x440x30x980x11
0x00x0
0x00x0
0x00x00x00x0
0x00x00x00x0
0x00x00x00x0
0x00x00x00x0
0x7e0x1d0xcc0xe70xee0x4f
0x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00x0
The above code allows me to observe the contents of DHCP packets, but I'm really trying to figure out why I'm not achieving similar results using the pkt['pkt'].get_protocol(dhcp.dhcp) method?
Ok, I found the problem. In lines 200 – 218 of dhcp.py there is a try except statement. I pulled the cls._parser out of the try to see the error it throws and I get this:
Ryuretic_coupler: Exception occurred during handler processing. Backtrace
from offending handler [initial_event] servicing event [EventOFPPacketIn] follows.
Traceback (most recent call last):
File "/home/ubuntu/ryu/ryu/base/app_manager.py", line 290, in _event_loop
handler(ev)
File "/home/ubuntu/ryu/ryu/app/Ryuretic/Ryuretic.py", line 72, in initial_event
pkt = parsPkt.handle_pkt(ev)
File "/home/ubuntu/ryu/ryu/app/Ryuretic/Pkt_Parse13.py", line 81, in handle_pkt
dhcp_p = pkt['dhcp'] = dhcp.dhcp.parser(pkt['pkt'][3])
File "/home/ubuntu/ryu/ryu/lib/packet/dhcp.py", line 212, in parser
return cls._parser(buf)
File "/home/ubuntu/ryu/ryu/lib/packet/dhcp.py", line 192, in _parser
) = struct.unpack_from(unpack_str, buf)
error: unpack_from requires a buffer of at least 233 bytes
So, dhcp.py is not receiving the 233 bytes that it requires. Unfortunately, I am using an Open vSwitch on the VM provided by SDN hub, and 128 bytes seems to be a limit. So, I have a conflict with Ryu's dhcp.py file. My solution was to modify dhcp.py. How that was done follows.
Before modifying the code, I recommend you first update your Ryu controller. The procedures are as follows:
Step 1: If you are using a VM. Take a stapshot or clone it now.
Step 2: Update Ryu
cd ryu
git pull
If you were running an older version like I was, then the following error may occur when you next attempt to run your Ryu controller:
Traceback (most recent call last):
File "./bin/ryu-manager", line 18, in <module>
from ryu.cmd.manager import main
File "/home/ubuntu/ryu/ryu/cmd/manager.py", line 31, in <module>
from ryu.base.app_manager import AppManager
File "/home/ubuntu/ryu/ryu/base/app_manager.py", line 37, in <module>
from ryu.controller.controller import Datapath
File "/home/ubuntu/ryu/ryu/controller/controller.py", line 74, in <module>
help='Maximum number of unreplied echo requests before datapath is disconnected.')
File "/usr/local/lib/python2.7/dist-packages/oslo_config/cfg.py", line 1033, in __init__
super(IntOpt, self).__init__(name, type=types.Integer(), **kwargs)
TypeError: __init__() got an unexpected keyword argument 'min'
Step 3: Update your oslo_config file
sudo pip install oslo.config --upgrade
The TypeError from step 2 should now be resolved. (Hopefully, you cloned your VM just in case.)
Step 3: Modify Ryu's dhcp.py file
Ryu's dhcp.py file (located at /ryu/ryu/lib/packet) expects to receive a buffer of more than 235 bytes. Otherwise, it throws and error and returns nothing to the controller. Since my buffer only receives a buffer size of about 81 bytes. I modified the Ryu dhcp.py file as follows.
The error occurs because dhcp.py specifies a string format of '!BBBBIHH4s4s4s4s16s64s128s'. In #1, I created a second option. Do so, allows me to insert a few if statements to handle the packet differently if a packet arrives that is smaller than 100 bytes. Likewise, if the packet is greater than 235 bytes, then dhcp.py's handles the packet normally and returns the extra values.
class dhcp(packet_base.PacketBase):
"""DHCP (RFC 2131) header encoder/decoder class.
....deleted....
"""
_MIN_LEN = 236
_HLEN_UNPACK_STR = '!BBB'
_HLEN_UNPACK_LEN = struct.calcsize(_HLEN_UNPACK_STR)
_DHCP_UNPACK_STR = '!BIHH4s4s4s4s%ds%ds64s128s'
##################################################
#1(mod) Created second option for unpacking string
_DHCP_UNPACK_STR2 = '!BIHH4s4s4s4s%ds%ds40s'
_DHCP_PACK_STR = '!BBBBIHH4s4s4s4s16s64s128s'
#################################################
_DHCP_CHADDR_LEN = 16
_HARDWARE_TYPE_ETHERNET = 1
_class_prefixes = ['options']
_TYPE = {
'ascii': [
'ciaddr', 'yiaddr', 'siaddr', 'giaddr', 'chaddr', 'sname'
]
}
def __init__(self, op, chaddr, options, htype=_HARDWARE_TYPE_ETHERNET,
hlen=0, hops=0, xid=None, secs=0, flags=0,
ciaddr='0.0.0.0', yiaddr='0.0.0.0', siaddr='0.0.0.0',
giaddr='0.0.0.0', sname='', boot_file=b''):
super(dhcp, self).__init__()
#...Deleted No Changes made to init.
#classmethod
def _parser(cls, buf):
(op, htype, hlen) = struct.unpack_from(cls._HLEN_UNPACK_STR, buf)
buf = buf[cls._HLEN_UNPACK_LEN:]
####################################################
#2(mod) provided option for smaller packet sizes
if len(buf) < 100:
unpack_str = cls._DHCP_UNPACK_STR2 % (hlen,
(cls._DHCP_CHADDR_LEN - hlen))
else:
unpack_str = cls._DHCP_UNPACK_STR % (hlen,
(cls._DHCP_CHADDR_LEN - hlen))
#####################################################
min_len = struct.calcsize(unpack_str)
######################################################
#3(mod) provided option for smaller packets, set bootfile to b''
if min_len > 233:
(hops, xid, secs, flags, ciaddr, yiaddr, siaddr, giaddr,
chaddr, dummy, sname, boot_file
) = struct.unpack_from(unpack_str, buf)
else:
boot_file=b''
(hops, xid, secs, flags, ciaddr, yiaddr, siaddr, giaddr,
chaddr, dummy, sname, boot_file
) = struct.unpack_from(unpack_str, buf)+(boot_file,)
########################################################
length = min_len
########################################################
# (mod) provided option for smaller packet sizes, no parse_opt
if len(buf) > 233:
parse_opt = options.parser(buf[min_len:])
length += parse_opt.options_len
else:
parse_opt = None
length = min_len
#########################################################
return (cls(op, addrconv.mac.bin_to_text(chaddr), parse_opt,
htype, hlen, hops, xid, secs, flags,
addrconv.ipv4.bin_to_text(ciaddr),
addrconv.ipv4.bin_to_text(yiaddr),
addrconv.ipv4.bin_to_text(siaddr),
addrconv.ipv4.bin_to_text(giaddr),
sname.decode('ascii'), boot_file),
None, buf[length:])
The complete file will soon be on https://github.com/Ryuretic/RyureticLabs/tree/master/ryu/ryu/app/Ryuretic/Support_Files.
If you make these adjustements to the dhcp.py file, then you will gain access to the following header fields:
============== ====================
Attribute Description
============== ====================
op Message op code / message type.\
1 = BOOTREQUEST, 2 = BOOTREPLY
htype Hardware address type (e.g. '1' = 10mb ethernet).
hlen Hardware address length (e.g. '6' = 10mb ethernet).
hops Client sets to zero, optionally used by relay agent\
when booting via a relay agent.
xid Transaction ID, a random number chosen by the client,\
used by the client and serverto associate messages\
and responses between a client and a server.
secs Filled in by client, seconds elapsed since client\
began address acquisition or renewal process.
flags Flags.
ciaddr Client IP address; only filled in if client is in\
BOUND, RENEW or REBINDING state and can respond\
to ARP requests.
yiaddr 'your' (client) IP address.
siaddr IP address of next server to use in bootstrap;\
returned in DHCPOFFER, DHserver.
giaddr Relay agent IP address, used in booting via a\
relay agent.
chaddr Client hardware address.
sname(partial) Optional server host name, null terminated string.

NodeMCU timeout when using while loop

I have a Lua script that sends an email to myself via SMTP. Everything works fine when uploading to the NodeMCU and saying dofile("sendemail.lua").
-- sendmail.lua
-- The email and password from the account you want to send emails from
MY_EMAIL = "REDACTED"
EMAIL_PASSWORD = "REDACTED"
-- The SMTP server and port of your email provider.
-- If you don't know it google [my email provider] SMTP settings
SMTP_SERVER = "isp.smtp.server"
SMTP_PORT = 25
-- The account you want to send email to
mail_to = "REDACTED"
-- Your access point's SSID and password
SSID = "REDACTED"
SSID_PASSWORD = "REDACTED"
-- configure ESP as a station
wifi.setmode(wifi.STATION)
wifi.sta.config(SSID,SSID_PASSWORD)
wifi.sta.autoconnect(1)
email_subject = ""
email_body = ""
count = 0
local smtp_socket = nil -- will be used as socket to email server
-- The display() function will be used to print the SMTP server's response
function display(sck,response)
print(response)
end
-- The do_next() function is used to send the SMTP commands to the SMTP server in the required sequence.
-- I was going to use socket callbacks but the code would not run callbacks after the first 3.
function do_next()
if(count == 0)then
count = count+1
IP_ADDRESS = wifi.sta.getip()
smtp_socket:send("HELO "..IP_ADDRESS.."\r\n")
elseif(count==1) then
count = count+1
smtp_socket:send("AUTH LOGIN\r\n")
elseif(count == 2) then
count = count + 1
smtp_socket:send("REDACTED".."\r\n")
elseif(count == 3) then
count = count + 1
smtp_socket:send("REDACTED".."\r\n")
elseif(count==4) then
count = count+1
smtp_socket:send("MAIL FROM:<" .. MY_EMAIL .. ">\r\n")
elseif(count==5) then
count = count+1
smtp_socket:send("RCPT TO:<" .. mail_to ..">\r\n")
elseif(count==6) then
count = count+1
smtp_socket:send("DATA\r\n")
elseif(count==7) then
count = count+1
local message = string.gsub(
"From: \"".. MY_EMAIL .."\"<"..MY_EMAIL..">\r\n" ..
"To: \"".. mail_to .. "\"<".. mail_to..">\r\n"..
"Subject: ".. email_subject .. "\r\n\r\n" ..
email_body,"\r\n.\r\n","")
smtp_socket:send(message.."\r\n.\r\n")
elseif(count==8) then
count = count+1
tmr.stop(0)
smtp_socket:send("QUIT\r\n")
print("msg sent")
else
smtp_socket:close()
end
print(count)
end
-- The connectted() function is executed when the SMTP socket is connected to the SMTP server.
-- This function will create a timer to call the do_next function which will send the SMTP commands
-- in sequence, one by one, every 5000 seconds.
-- You can change the time to be smaller if that works for you, I used 5000ms just because.
function connected(sck)
tmr.alarm(0,5000,1,do_next)
end
-- #name send_email
-- #description Will initiated a socket connection to the SMTP server and trigger the connected() function
-- #param subject The email's subject
-- #param body The email's body
function send_email(subject,body)
count = 0
email_subject = subject
email_body = body
smtp_socket = net.createConnection(net.TCP,0)
smtp_socket:on("connection",connected)
smtp_socket:on("receive",display)
smtp_socket:connect(SMTP_PORT, SMTP_SERVER)
end
-- Send an email
send_email("ESP8266", "[[Hi, How are your IoT projects coming along? Best Wishes,ESP8266]]")
However, I want to use a loop to monitor an analog input value and only send the email when certain analog input values are detected. Therefore, I added this code at the end of the script, after the sendemail() function definition and immediately before the function sendmail('subject', 'body') is called
vp = 0
gpio.mode(vp, gpio.INPUT)
while true do
local v = adc.read(vp)
if v < 840 or v > 870 then
print(v)
break
end
tmr.wdclr()
end
sendmail('subject', 'body')
The while loop works perfectly, waiting indefinitely for input from the analog pin. Once that input is found, it breaks correctly and calls the sendmail function. However, once that function is called, NodeMCU eventually resets. Sometimes it will get as far as successfully authenticating the SMTP credentials with the server, and sometimes it will not even make the HELO before it shuts down. What could possibly be causing this? Why would the sendmail.lua script work fine then suddenly decide not to work when adding this one small while loop that appears to work perfectly fine on its own?
A little quote from the NodeMCU reference:
tmr.wdclr() Feed the system watchdog.
In general, if you ever need to use this function, you are doing it
wrong.
The event-driven model of NodeMCU means that there is no need to be
sitting in hard loops waiting for things to occur. Rather, simply use
the callbacks to get notified when somethings happens. With this
approach, there should never be a need to manually feed the system
watchdog.
Please note the second line. :)
Not sure what your problem is, but why do you use a while loop in the first place? Why not use timer events to poll your ADC regularly?
Maybe the watchdog is triggered because your feed comes to late for some reason. In the if case you don't feed it at all befor you leave the loop.
Even if it may not be the definite answer I post it as such since the comment input is too small.
First, I suggest you use the script I posted for your previous question. This one isn't handling WiFi setup correctly. You need to wait in a timer until the device got an IP before you can continue. Remember, wifi.sta.config is non-blocking. And since it uses auto-connect=true if not set explicitly it'll try to connect to the AP immediately. That's also the reason why wifi.sta.autoconnect(1) is superfluous.
I don't understand the ADC reading code you posted.
vp = 0
gpio.mode(vp, gpio.INPUT)
Seems unnecessary to me because a) you don't do anything with GPIO 0 and b) adc.read only supports 0.
Rather than using a busy loop and constantly feeding the watch dog, which is a very bad sign, I suggest you use an interval based timer. Furthermore, I guess you don't wanna break the loop the first time the condition is met and never come back? So, you need to stay in the loop and keep triggering send mail, no? Something like this maybe (untested):
tmr.alarm(0, 200, tmr.ALARM_AUTO, function()
local v = adc.read(0)
if v < 840 or v > 870 then
node.task.post(function()
send_email("ESP8266", "[[Hi, How are your IoT projects coming along? Best Wishes,ESP8266]]")
end)
end
end)

How to receive UDP data in Vala?

another Vala problem occured: I try to send and receive data via UDP. The sending works and via Wireshark I can see that the server sends the expected result. Problem is: My program doesn't get the data.
I checked and I can see that, when a socket has been created to send the UDP data, the specific port stays open, which is confirmed by Wireshark because my PC doesn't send any of those ICMP messages back to the server.
What I got so far:
try
{
SocketClient mySocket = new SocketClient();
mySocket.protocol = SocketProtocol.UDP;
mySocket.type = SocketType.DATAGRAM;
var conn = mySocket.connect (new InetSocketAddress(addr,targetPort));
conn.output_stream.write(themessage_in_a_uint8_array);
DataInputStream response = new DataInputStream (conn.input_stream);
string resp ="";
char myChar;
try
{
do
{
myChar = (char)response.read_byte();
print ("Response" + myChar.to_string());
}while(true);
}
catch(Error e)
{
print(e.message);
}
}
catch(Error e)
{print(e.message);}
What currently happens: The message is send, the string 'Response' is printed once into the console and after that it just loops.
If I check response.get_available() it returns 0.
I can check with lsof | grep used_portnumber and sure enough, the used socket stays open. What am I doing wrong?
I am not sure but this is what I suspect:
UDP is a datagram protocol (data is explicitly chopped into data). Server have sent one datagram to client. Now in BSD Sockets (and after it everywhere) if the underlaying socket have datagram type then read reads the full packet. If the buffer have insufficient length the message is truncated.
The solution is read in one byte. For example
uint8[] buffer = new uint8[1 << 16]; // Maximum UDP length - we don't loose anything
unowned string locale;
bool need_convert = GLib.get_charset (out locale);
do {
ssize_t len = response.read (buffer);
string text;
if (need_convert) {
text = GLib.convert ((string)buffer, len, locale, "UTF-8");
} else {
text = (string)buffer;
}
stdout.print("Response " + text);
} while (true);
Edit I have change the code to print UTF-8 text - without assuming current locale is "UTF-8"-based.
PS 1 This is my guess as it is one gotcha of BSD Sockets (also Winsockets and everything that builds on this) that come to my mind. Please be graceful if the question will be more specific (i.e. it is not the answer to question).
PS 2 In general I would recommend against mixing bytes and chars. While in ASCII-compatible encodings (ISO, UTF-8) sending ASCII subset of chars is safe it will bite when attempt on CJK encodings or if sender will send 'ą' by UTF-8 and sender will treat it as ISO-8859-2 (where this character have different encoding). I assume it is for the toy-examples only. If not you may want to read What Every Programmer Absolutely, Positively Needs To Know About Encodings And Character Sets To Work With Text.

twisted buffer full in tcp connection

I´m having problems with receiving long data (>1024bytes) in a simple twisted server implementation.
From the beginning, I´m developing an ios App that has to synchronize with a twisted server. I prepare the information to send in JSON format. Then I start to send that data in chuncks (right now in chunck of 256bytes + 4 bytes for the command - Yes, I´m implementing my own protocol). The connection is ok, and I receive those packet´s in my server (in the dataReceived function of my own Protocol subclass).
The ios method: NSInteger writtenBytes =[self.outputStream write:[data bytes] maxLength:[data length]] return the written bytes into the stream. For the first 4 packets the value returned is the expected (260 bytes). If I have more available bytes to send, the next time I call that method it returns 0 (which apple documentation says: "If the receiver is a fixed-length stream and has reached its capacity, 0 is returned.").
So I deduce that the input buffer is full. I don´t know how to free that buffer (I don´t know how to reach that buffer). I don't know where is the limit of that buffer (it seems to me almost ridiculous).
This is a basic test of the server (Just the important things for this question with a basic based in strings protocol)
from twisted.internet.protocol import Protocol, Factory
from twisted.internet import reactor
class IphoneSync(Protocol):
def __init__(self):
self.__buffer = ""
def connectionMade(self):
self.transport.write("0:")
self.factory.clients.append(self)
print "clients are ", self.factory.clients
def connectionLost(self, reason):
self.factory.clients.remove(self)
def dataReceived(self, data):
#print "data is ", data
a = data.split(':')
if len(a) > 1:
command = a[0]
content = a[1]
msg = ""
if command == "iam":
#user&Pass checking
msg = "1"
elif command == "msg":
self.__buffer += data
msg = "1: continue"
elif command == "fin":
#procesaremos todo
#Convertir datos en json
#insertar/actualizar data en sqlite
#devolver respuesta
print "buffer is", self.__buffer
msg = "2: procesing"
print msg
self.transport.write(msg)
#for c in self.factory.clients:
#c.message(msg)
def message(self, message):
self.transport.write(message)
#self.transport.write(message + '\n')
factory = Factory()
factory.protocol = IphoneSync
factory.clients = []
dir(factory)
reactor.listenTCP(8000, factory)
print "Iphone Chat server started"
reactor.run()
I saw the LineReceiver class but i´m not sending lines. The transfered data could be very big (10Mb-50Mb). I´m thinking about the Consumer/Producer model, or RPC Protocols like (AMP, or PB) as a solution but i wanted to work with my own protocol.
If someone knows how to help me, i´ll appreciate very much. Thanks anyway.
The connection is ok, and I receive those packet´s in my server (in the dataReceived function of my own Protocol subclass).
Probably not. TCP is a "stream oriented" protocol. Your application's use of it is not in terms of packets but in terms of a sequence of bytes. There is no guarantee whatsoever that dataReceived will be called with the same string that you passed to outputStream write. If you write "hello, world", dataReceived may be called with "hello, world" - or it may be called twice, first with "hello," and then with " world". Or it may be called 12 times: first "h", then "e", then "l", etc.
And if you call outputStream write twice, once with "hello," and once with " world", then it's entirely possible dataReceived will be called just once with "hello, world". Or perhaps twice, but with "h" and then "ello, world".
So this brand new protocol you're inventing (which I see you mentioned you recognized you were doing, but you didn't explain why this is a good idea or an important part of your application, instead of just a large source of potential bugs and a poor use of time :) has to do something called "framing" in order to let you actually interpret the byte sequence being passed around. This is why there are protocols like AMP.
To actually answer your question, outputStream write returns the number of bytes it was actually able to buffer for sending. You must always check its return value and re-try writing any bytes it wasn't able to send, preferably after waiting for notification that there is more buffer space. Buffer space becomes available after bytes using that space are sent across the network and acknowledged by the receiver. This takes time, as networks are not instantaneous. Notification about buffer space being available comes in many forms, the oldest and most widespread of which (but not necessarily the best in your environment), the select(2) system call.
In addition to Jean-Paul Calderone's answer (ensuring that data are being sent completely from the obj-c side by using select or thread), for protocol part I would suggest using length-prefixed string (AKA Netstring) for simple use case.
Here's an implementation. Whenever something is received, you need to call NSBuffer.write then NSBuffer.extract to get available strings.

Erlang exmpp get body from Packet

I need to get body attribute from input xmpp Packet with Erlang/xmpp.
I receive packet:
Record = #received_packet{packet_type=message,
raw_packet=Packet,
from=From,
type_attr=Type} when Type =/= "error
How can i extract body message from Packet?
Thank you.
if you are using exmpp you can use module exmpp_message:get_body/1
http://www.process-one.net/docs/exmpp/devdoc/trunk/exmpp_message.html
Not sure if you already went through exmpp examples. Here is a snippet from echo_client.erl example:
%% Send the same packet back for each message received
echo_packet(MySession, Packet) ->
From = exmpp_xml:get_attribute(Packet, <<"from">>, <<"unknown">>),
To = exmpp_xml:get_attribute(Packet, <<"to">>, <<"unknown">>),
TmpPacket = exmpp_xml:set_attribute(Packet, <<"from">>, To),
TmpPacket2 = exmpp_xml:set_attribute(TmpPacket, <<"to">>, From),
NewPacket = exmpp_xml:remove_attribute(TmpPacket2, <<"id">>),
exmpp_session:send_packet(MySession, NewPacket).

Resources