I retrieve a URL with the following code:
geturl(Url)->
{ok, RequestId}=httpc:request(get,{Url,[{"User-Agent", "Opera/9.80 (Windows NT 6.1; U; ru) Presto/2.8.131 Version/11.10"}]}, [],[{sync, false}]),
M = receive
{http, {RequestId, {_HttpOk, _ResponseHeaders, _Body}}} -> ok
after 20000 ->
not_ok
end.
io:format("httprequest to ~p: ~p~n",[UrlHTTP,M]).
Sometimes I see the following error output:
httprequest to "http://www.afbudsrejser.dk": ok
=ERROR REPORT==== 30-Apr-2014::16:53:49 ===
Received unexpected tcp data on #Port<0.4432>
Data: <<"\" type=\"text/javascript\"></script>\r\n\t</body>\r\n</html>\r\n\n">>
MFA: undefined
Request: undefined
Session: {session,{{"www.afbudsrejser.dk",80},<0.4392.0>},
false,http,#Port<0.4432>,ip_comm,1,pipeline,true}
Status: pipeline
StatusLine: undefined
Profile: httpc_manager
Erlang error received unexpected tcp data on #Port
What is the cause of this error message?
The HTTP server you are connecting to is misbehaving. It declares a Content-Length header that is shorter than the response it actually sends.
This means the last few bytes the server sends (which include the string type="text/javascript"></script>) are not expected by httpc, since they violate the HTTP spec. httpc prints the warning message you're seeing when when it receives unexpected bytes that are not part of a valid HTTP response.
If you cannot fix the server, you can simply ignore this warning. You can also take a look at the relaxed httpc option, which may or may not work around this issue.
Related
This seems to cause strange behaviour when PUT has a body larger than a certain length (in my case it is 902 bytes), i.e. ejabberd trims the body (in my case it receives malformed JSON).
Github reference: https://github.com/processone/ejabberd/blob/master/src/ejabberd_http.erl#L403
If I change the case statement to:
case Method of
_AnyMethod ->
case recv_data(State) of
{ok, Data} ->
LQuery = case catch parse_urlencoded(Data) of
{'EXIT', _Reason} -> [];
LQ -> LQ
end,
{State, {LPath, LQuery, Data}};
error ->
{State, false}
end
end
then the body is parsed correctly.
Is this a configuration issue? How can I force Ejabberd to correctly parse the JSON body?
Looks like you've found a bug.
As you've noticed, for POST requests the function recv_data is called, which checks the Content-Length header and reads that many bytes from the socket. For PUT requests however, it only uses Trail, which is the data that has been already received while reading HTTP request headers. (This happens in the receive_headers function, which sends a length of 0 to the recv function, meaning that it won't wait for any specific amount of data.)
How much of the request body is received is going to depend on the size of the headers, as well as the way the client sends the request. If for example the client first sends the headers in one network packet, and then the request body in the next network packet, ejabberd wouldn't pick up the request body at all.
I used to have a working code in my Admin Panel , checking if a url inputed existed and giving a friendly message to the Administrator in case it did not..
def get_url_response(url)
uri = URI(url)
request = Net::HTTP.get_response(uri)
return request
end
url_response = get_url_response("http://www.example.com").code
if url_response === "200" || url_response === "304"
link_to http://www.example.com, http://www.example.com, target: :blank
else
status_tag("We have a problem ! Response code: "+url_response, :class => 'red')
end
It works great when the address ("http://www.example.com" in the example above) exists, that it to say sends back a 200 code, but as soon as I have a non existing address such as http://www.examplenotexistingotallyfake.com, it should send a 404 code and display ""We have a problem ! Response code:" but it fails with the error message:
SocketError: Failed to open TCP connection to examplenotexistingotallyfake.com:443 (getaddrinfo: Name or service not known)
from /home/mreisner/.rvm/rubies/ruby-2.3.1/lib/ruby/2.3.0/net/http.rb:882:in `rescue in block in connect'
I verified this by opening my Rails console (Rails c) and if I type:
Net::HTTP.get(URI('https://www.examplenotexistingotallyfake.com')).code
I get the same error message:
SocketError: Failed to open TCP connection to examplenotexistingotallyfake.com:443 (getaddrinfo: Name or service not known)
from /home/mreisner/.rvm/rubies/ruby-2.3.1/lib/ruby/2.3.0/net/http.rb:882:in `rescue in block in connect'
How can it work for correct urls and not for non-existing addreses? it should just work but send me back a 404 code, shouldn't it?
I can only see the upgrade to Ubutun 16.04 i made a few days ago, that might have tampered with some critical dns/localhost settings as the source of this issue but not 100% totally sure.
EDIT
After some suggestions, I now try to avoid the app crashing by rescuing this
def get_url_response(url)
begin
uri = URI(url)
request = Net::HTTP.get_response(uri)
return request
rescue SocketError => e
puts "Got socket error: #{e}"
end
end
but the app still crashes with a socket Error message
That's the correct behaviour.
The problem there is that examplenotexistingotallyfake.com doesn't exists in the DNSs entries.
If you look at the description of what the 404: https://en.wikipedia.org/wiki/HTTP_404
to indicate that the client was able to communicate with a given
server, but the server could not find what was requested.
So, in order to get the 404 code you'll need first to be able to communicate with the server in question.
You can double check this behaviour using chrome or even curl, visiting the following urls: examplenotexistingotallyfake.com or google.com/missing
Each will give a different result.
in curl:
$ curl -I examplenotexistingotallyfake.com
curl: (6) Could not resolve host: examplenotexistingotallyfake.com
# google
curl -I google.com/missing
HTTP/1.1 404 Not Found
Content-Type: text/html; charset=UTF-8
Referrer-Policy: no-referrer
Content-Length: 1568
Date: Fri, 12 Jan 2018 09:25:27 GMT
If you want your code to behave in the same way (even though I'd suggest that give the user a different message, you can do the following):
require 'uri'
require 'net/http'
require 'ostruct'
def get_url_response(url)
uri = URI(url)
request = Net::HTTP.get_response(uri)
return request
rescue Errno::ECONNREFUSED => e
OpenStruct.new(code: 404)
end
Its not an issue , the site you are looking for it doesn't exists and can't be reached.
So it will give DNS address is not found issue when you try to hit it directly.Thats why the error:
It directly gives socketError : getaddrinfo: Name or service not known You need to handle this.
But if you want 404 status code, you will get it when the site(address) is present and the page inside that site is not.
To get 404 your address should be valid and error will raise if The requested URL /examples was not found on this server.
I have a voip application for iOS based on pjsip 2.3.0.
The device in question runs iOS 9.3.4 and the problem also seem to happen on iOS 10 beta. It is connected to IPV6 wifi network created from macbook's nat64/dns64.
The client is able to register properly with our server and also has no trouble making outbound call in UDP mode ( Everything works fine in TCP ).
The trouble is with inbound call where it fails intermittently due to the following reason when responding to INVITE with 200 OK message even though it is able to send 100 TRYING successfully just before it.
pjsua_core.c ....
Sent:
2016-07-31T16:49:32.443
<srcipv6:port> -> <dstipv6:port>
1005
SIP/2.0 200 OK Via: SIP/2.0/UDP
--end msg--
tsx0x1401d82a8 ....Error sending Response msg
200/INVITE/cseq=95989621(tdta0x140866000): No route to host
....Transport error, terminating transaction. Err=120065 (No route to host)
pjsua_call.c .Error sending response: No route to host [status=120065]
APIException: API call '::pjsua_call_answer( CALL_REFERENCE,
PJSUA_SIP_RESPONSE_STATUS_OKAY, NULL, NULL )'
failed with result code = 120065 ('No route to host')
Note that this happens only intermittently. Once an inbound call succeeds after sending 200 OK to INVITE then the subsequent calls do succeed. The problem does not exist in IPV4 networks.
I also did a packet capture and found that there is no trace of 200 OK message in it even though pjsua_core.c's trace tells otherwise as seen above.
Sincerely welcome any inputs on how to proceed further or if there are any fixes.
Edit 1 :
I have managed to debug all the way from ::pjsua_call_answer to the method “sendto” () calledin pjlib/src/pj/sock_bsd.c
/*
* Send data.
*/
PJ_DEF(pj_status_t) pj_sock_sendto(pj_sock_t sock,
const void *buf,
pj_ssize_t *len,
unsigned flags,
const pj_sockaddr_t *to,
int tolen)
{
PJ_CHECK_STACK();
PJ_ASSERT_RETURN(len, PJ_EINVAL);
CHECK_ADDR_LEN(to, tolen);
*len = sendto(sock, (const char*)buf, (int)(*len), flags,
(const struct sockaddr*)to, tolen);
if (*len < 0)
return PJ_RETURN_OS_ERROR(pj_get_native_netos_error());
else
return PJ_SUCCESS;
}
sendto here returns -1 when trying to send 200 OK message alone in ipv6.
To learn Erlang I am trying to implement a tiny web server based on gen_tcp. Unfortunately, my code seems to trigger some wired behaviour. To demonstrate the problem I have attached a minimised version of my implementation which is sufficient to reproduce the problem. It is just delivering a static 200 OK, no matter what the HTTP request was.
The problem arises when I try to run ab (Apache HTTP server benchmarking) against my web server (using loopback interface). Without any concurrent requests (-c) everything is running just fine. However, if I use -c 8 or -c 16, the call to gen_tcp:accept/1 seems to fail on some sockets as I see a number of request: closed lines in the shell.
What makes the whole story even weirder is, that I see different behaviours on different operating systems:
OS X+Erlang/OTP 18: ab reports "Connection reset by peer" almost immediately after starting.
Debian+Erlang R15B01: All but two of the HTTP requests seem to run through. But then, ab hangs for a few seconds and reports "The timeout specified has expired, Total of 4998 requests completed", when i run ab with -n 5000. Similarly, 14998 is reported when I run 15000 tests.
This one does not seem to be the problem. I am honestly quite lost and therefore appreciate any help! :) Thanks!
server(Port) ->
Opt = [list, {active, false}, {reuseaddr, true}],
case gen_tcp:listen(Port, Opt) of
{ok, Listen} ->
handler(Listen),
gen_tcp:close(Listen),
ok;
{error, Error} ->
io:format("init: ~w~n", [Error])
end.
handler(Listen) ->
case gen_tcp:accept(Listen) of
{ok, Client} ->
request(Client),
handler(Listen);
{error, Error} ->
io:format("request: ~w~n", [Error])
end.
request(Client) ->
Recv = gen_tcp:recv(Client, 0),
case Recv of
{ok, _} ->
Response = reply(),
gen_tcp:send(Client, Response);
{error, Error} ->
io:format("request: ~w~n", [Error])
end,
gen_tcp:close(Client).
reply() ->
"HTTP/1.0 200 OK\r\n" ++
"Content-Length: 7\r\n\r\n"
"static\n".
When you increase the number of concurrent requests sent with ab -c N it will immediately open multiple TCP sockets to the server.
By default a socket opened with gen_tcp:listen/2 will support only five outstanding connection requests. Increase the number of connection requests outstanding with the {backlog, N} option to gen_tcp:listen/2.
I tested your code on OS X with ab and saw this resolve the prolem with "Connection reset by peer".
I am implementing cowboy websocket. Everything is working fine except when user closes the browser it fires websocket_termination and at server end it generates following error:-
Error in process <0.298.0> on node 'ews_2#servername.com' with exit value: {function_clause,
[{cowboy_req,ensure_response,[{[]},204],[{file,"src/cowboy_req.erl"},{line,1112}]},
{cowboy_protocol,next_request,3,[{file,"src/cowboy_protocol.erl"},{line,545}]}]}
Code in websocket_termination is :-
websocket_terminate(Reason, Req, State) ->
io:format("~nWebsocket connection termination~n"),
ok.
Resolved: Problem was Req was not getting passed and got manipulated between the callbacks... Cowboy needs a proper Req parameter to be passed at the time of connection termination.