I'm trying to implement the websocket protocol and for some reason the iPad isn't cooperating with me. The iPad runs the Hybi-00 specification and the problem lies somewhere in the handshake. I'm getting as far as reading the request and the following magic bytes and responding with what I believe is the correct reply. The iPad isn't agreeing with me and shuts down the connection.
The protocol handler is working for Hybi-06, which is run by most modern browsers.
I'd really like some way to make the iPad tell my why it has decided to shut down the websocket connection so that I can fix the issue. Either that or getting chrome or some other browser to downgrade their websocket version to the same version that iPad supports so that I can get some sort of clue as to what is actually going on.
Is there any way to obtain this information?
Ok, so, even though it would be nice to have the actual error explained, here's the handshake:
GET /myurl HTTP/1.1
Upgrade:WebSocket
Connection:Upgrade
Host:my-host.com
Origin:https://my-host.com
Sec-WebSocket-Key1:1 QB 9 , $169780#8A`#4
Sec-WebSocket-Key2:3 160RhGi7mu 97*66br9
The following eight bytes is the key:
{byte[8]}
[0]: 105
[1]: 199
[2]: 161
[3]: 204
[4]: 5
[5]: 176
[6]: 104
[7]: 199
The server replies with this
HTTP/1.1 101 Web Socket Protocol Handshake
Upgrade:WebSocket
Connection:Upgrade
Sec-WebSocket-Origin:https://my-host.com
Sec-WebSocket-Location:ws://my-host.com/
And the key
{byte[16]}
[0]: 100
[1]: 132
[2]: 37
[3]: 58
[4]: 243
[5]: 195
[6]: 173
[7]: 26
[8]: 180
[9]: 239
[10]: 33
[11]: 56
[12]: 40
[13]: 192
[14]: 123
[15]: 255
This is run over a HTTPS connection, so it's basically data from my debugger since I can't get wireshark properly configured to extract this for the life of me. As far as I can tell, the handshake keys are correct as well.
Look at this thread.
As Cybear points out, I would recommend upgrading to iOS6. Current statistics show around 60% of overall iOS users are on iOS6.
Hybi-06 is an version of the protocol that was just a specification, if i'm not wrong there was not any browser in fact implementing that version. I think the most of the browsers went to Hybi-10 then for Hybi-17 ( Rfc6455), as you say iOS uses Hybi00 at the moment. There are couple of servers supporting that particular handshake. I also know that there are a few supporting every single version of the drafts.
I run XSockets.NET, we chosen to support Hybi-00 and the RFC6455 version, as those are the protocols implemented in the browsers. (Safari still on Hybi00)
As in RFC6455 there is a sections regarding supporting Multiple Protocols ;
https://www.rfc-editor.org/rfc/rfc6455#page-26
If you are implementing somthing, my recomendation is to do the RFC6455
Related
I'm new to Twitter APIs (this is my first experience), and I'm playing with them to monitor an account for new tweets, opening a web page when it happens, but I'm having some doubt on understanding how the allotting works.
Not knowing much, the twitter stream v2 APIs seem the ones fitting my use case, and in the Twitter-API-v2-sample-code git repository there is also a very clear filtered stream nodejs example. In fairness, I had little hassle to implement everything, and my code is not much different from filtered_stream.js source code. Given the provided example, implementation is straightforward: I use https://api.twitter.com/2/tweets/search/stream/rules to setup my rules (an array like [ { 'value': 'from:<myAccount>' } ] and then I start to listen at https://api.twitter.com/2/tweets/search/stream, easy peasy.
What I don't understand is the allotting resources count, because as per Twitter documentation I should be able to make 50 requests every 15 minutes, but I can barely make a couple, thus every time I'm testing my script I have to wait a couple of minutes before restarting.
These are the relevant headers I received after restarting a script running since one hour (the status code at restart was 429):
x-rate-limit-limit: 50
x-rate-limit-remaining: 49
Reset time: +15 minutes from current time
I usually don't have to wait 15 minutes, just a couple usually is fine... And my other note is that i managed to arrive down to 45 x-rate-limit-remaining once or twice, but never lower than that (usually I'm locked out at 47 / 48).
What I don't understand is: I opened one stream, I closed that one stream, and now I'm locked out for a couple of minutes. Allegedly, shouldn't I be able to open up to 50 connection in 15 minutes? (which is actually plenty if I'm just debugging a portion of code). Even the headers says that I have 49 attempts remaining out of 50, the status code 429 seems in pure contradiction with the x-rate-limits ... Sometimes, I cannot even reset the rules and start the stream in the same run, because the stream will return a backoff (status 429) when the rules resetting finishes (get -> set -> delete)...
I could add my code, but it's literally the NodeJS example I cited above, and my problem is not about querying the APIs, but rather not being able to connect for no apparent reason (at least to me). The only thing I could think of is the fact that I use the same Bearer for all requests (as per their example), but I don't see written anywhere it is a problem (I generated it in the developer dashboard, I'm not sure there is an API for that as well).
Edit - adding details
Just to describe my issue, this is the output I get when I start the script the first time:
Headers for stream received (status 200):
- [x-rate-limit-limit]: 50
- [x-rate-limit-remaining]: 49
- [x-rate-limit-reset]: 20/03/2021, 11:05:35
Which make sense, I made one request, remaining count went down by one.
Now, I stopped it, and ran it immediately after (Ctrl + C, run again, let's say two seconds delay), and this is the new output:
Headers for stream received (status 429):
- [x-rate-limit-limit]: 50
- [x-rate-limit-remaining]: 48
- [x-rate-limit-reset]: 20/03/2021, 11:05:35
With the following exception being returned in the body:
{
title: 'ConnectionException',
detail: 'This stream is currently at the maximum allowed connection limit.',
connection_issue: 'TooManyConnections',
type: 'https://api.twitter.com/2/problems/streaming-connection'
}
I understand the server takes a bit to realise I disconnected, but don't I have 50 connections available in a 15 minutes timeframe? I only opened one connection.
Actually, After the time it took to write all of the above (let's say ten minutes), I was able to connect again, receveing with this output:
Headers for stream received (status 200):
- [x-rate-limit-limit]: 50
- [x-rate-limit-remaining]: 47
- [x-rate-limit-reset]: 20/03/2021, 11:05:35
Maybe I'm realising only now and I wrote a useless question, but can I only have one active connection, being able to close it and open it again 50 times in 15 minutes? I understood I could have 50 active connections, but maybe at this point I'm wrong (and the Twitter server indeed takes a few minutes to realise I disconnected).
Issue started about 11:20 UK time, and it affects many big applications like spotify, waze, medium and so on.
It looks like it related with event managers and analytics. Some of the apps that crashed 10 min ago, seems working now.
Can it be solved by changing any facebook dev settings?
Anyone able to solve it without waiting Facebook to solve?
For more info:
https://developers.facebook.com/status/issues/1739188102911114/
https://github.com/facebook/facebook-ios-sdk/issues/1430
https://developers.facebook.com/support/bugs/329763701368293/
4 CoreFoundation 0x00007fff23e43bf8 _CF_forwarding_prep_0 + 120
5 FBSDKCoreKit 0x000000010f736f3a +[FBSDKRestrictiveDataFilterManager updateFilters:] + 62
6 FBSDKCoreKit 0x000000010f739a0c +[FBSDKServerConfigurationManager processLoadRequestResponse:error:appID:] + 1912
7 FBSDKCoreKit 0x000000010f739281 __78+[FBSDKServerConfigurationManager loadServerConfigurationWithCompletionBlock:]_block_invoke + 62
8 FBSDKCoreKit 0x000000010f729110 -[FBSDKGraphRequestMetadata invokeCompletionHandlerForConnection:withResults:error:] + 143
9 FBSDKCoreKit 0x000000010f7269fe __82-[FBSDKGraphRequestConnection processResultBody:error:metadata:canNotifyDelegate:]_block_invoke + 142
10 FBSDKCoreKit 0x000000010f72690d -[FBSDKGraphRequestConnection processResultBody:error:metadata:canNotifyDelegate:] + 636
11 FBSDKCoreKit 0x000000010f72654d __64-[FBSDKGraphRequestConnection completeWithResults:networkError:]_block_invoke + 398
From my side, in the response, there is null value. Yes!!! null is a kind of primitive type in JSON specification.
Then after FB using [NSJSONSerialization JSONObjectWithData: options: error:] to serialize this object, it returns NSNull object without error. According to Apple's doc, NSNull is a singleton object. Yes!!!, it is a valid object, not like nil, which you can guard using if (obj) statement
So, when FB try to use if (serializedObj) { ... } to guard this edge cases in many places, it is actually useless. Here serializedObj is a singleton object, the result is YES then it continues to send message, then, unrecoginized selector . This issue haven't been fixed now. Next time, when the server side sends an another object with null, we will see another disaster.
Updated:
FB is trying to fix this issue
In a production app with the debug information stripped out, how do you convert the output of:
NSLog(#"Stack Trace: %#", [exception callStackSymbols]);
To an legible class and method name? A line number would be a blessing.
Here's the output I'm getting:
0 CoreFoundation 0x23d82f23 <redacted> + 154
1 libobjc.A.dylib 0x23519ce7 objc_exception_throw + 38
2 CoreFoundation 0x23cb92f1 <redacted> + 176
3 MyApp 0x23234815 MyApp + 440341
The final line is the bread and butter line, but when I use dwarf to find the address, nothing exists.
dwarfdump --arch armv7 MyApp.dSYM --lookup 0x00234815 | grep 'Line table'
I've read here that you need to convert the stack address to something else for dwarf or atos:
https://stackoverflow.com/a/12464678/2317728
How would I find the load address or slide address to perform the calculation? Is there not a way to calculate all this prior to sending the stacktrace to the log from within the app? If not, how would I determine and calculate these values after receiving the stack trace? Better yet, is there an easier solution I'm missing?
Note I cannot just wait for crash reports since the app is small and they will never come. I'm planning on sending the stack traces to our server to be fixed as soon as they appear.
EDITORIAL
The crash reporting tools in iOS are very rough, particularly when compared to Android. In android, the buggy lines are sent to Google analytics, you use the map to debug the line--simple (comparatively). For iOS, you are confronted with: a) waiting on user bug reports (not reasonable for a small app), b) sending stack traces to a server where there are scant tools or information on how to symbolicate the stack traces, c) relying on large quasi-commercial 3rd party libraries. This definitely makes it harder to build and scale up--hoping Apple will eventually take notice. Even more hopeful someone has spotted an easier solution I might have missed ;)
Thanks for your help!
A suggestion, you can easily get the method name, exception reason and line number using:
NSLog(#"%# Exception in %s on %d due to %#",[exception name],__PRETTY_FUNCTION__,__LINE__,[exception reason]);
APNS returns the following status codes in error response packets when using the enhanced notification format.
0 - No errors encountered
1 - Processing error
2 - Missing device token
3 - Missing topic
4 - Missing payload
5 - Invalid token size
6 - Invalid topic size
7 - Invalid payload size
8 - Invalid token
255 - None (unknown)
I want to know which of these status codes actually indicate that the issue is actually with the device token (ie: I can mark that particular device token in my database as invalid or inactive). I am sure that I can safely do that when I receive a value of 8 as status code. Also, I am pretty sure about what status codes 2, 4, 5 and 7 indicate and when exactly are they returned.
But for the rest of the status codes, I am pretty much in the dark. Apple's website does not contain much information about them either.
Can anybody please enlighten me about each of these status codes in detail. Many thanks in advance.
You're right about the 2, 4, 5, 7, and 8.
0 is good news obviously.
We're left with:
1 - Processing error
3 - Missing topic
6 - Invalid topic size
10 - Shutdown
255 - None (unknown)
1 is a problem on Apple's side, not related to the token.
3 is, according to Apple :
The topic is currently the bundle identifier of the target application on an iOS device.
6 is related to 3.
10 is out of your control but good to handle as well.
And obviously, 255 will not be able to help you define if the token is the problem.
If your goal is to determine a token that was valid at some point and no longer is, you should really refer to the Feedback Service API.
We have a simple question, but the cause is complicated. We are experienced developers, and have done a lot of research into what may be causing it. We are hoping that MonoTouch developers can work with us to identify what appears to be a common problem that people are having and for which no solution appears to exist yet. We've been working on this for over two weeks, and not been able to resolve it.
The question is: Why is our MonoTouch app breaking in the garbage collector? It is not out of memory.
The situation is that we have an app that checks a web service regularly (perhaps every 5 seconds). After a period of time it fails with a memory management abort. This typically happens after about an hour and a half, but can be anywhere from ten minutes to overnight. This happens on all of our test devices (we have 7 in total covering iOS3 and iOS4, iPod Touch, iPhones and iPads (1&2). After looking on StackOverflow, we have added a System.Gc.Collect in a timer before we take any action. This improved things a little (it takes longer to fail), but it did not go away. It is also worth adding that the memory log from the iPad shows that there are 777 free blocks, and 2041 in use by our app, with a total of 26488 wired pages. Since we've garbage collected, and are not doing anything different to what we did 5 seconds before, it seems odd to run out of memory.
We upgraded to MonoTouch 4.0.1 but that has not fixed it.
StackOverflow questions that might be on the same issue, but not answering it: 5666905 / 4545383 / 5492469 / 5426733
The stack at failure on an iPad2 is below. The failure can happen in the main thread or an http thread, but always goes in this GC_ sequence. I have included the code for the memory manager GC_remap below, with discussion.
Thread 10 Crashed:
0 libsystem_kernel.dylib 0x34b4da1c __pthread_kill + 8
1 libsystem_c.dylib 0x3646a3b4 pthread_kill + 52
2 libsystem_c.dylib 0x36462bf8 abort + 72
3 MyApp 0x004ca92c mono_handle_native_sigsegv (mini-exceptions.c:2249)
4 MyApp 0x004f2208 sigabrt_signal_handler (mini-posix.c:195)
5 libsystem_c.dylib 0x36475728 _sigtramp + 36
6 libsystem_c.dylib 0x3646a3b4 pthread_kill + 52
7 libsystem_c.dylib 0x36462bf8 abort + 72
8 MyApp 0x0061dc94 GC_remap (os_dep.c:2092)
9 MyApp 0x00611678 GC_allochblk_nth (allchblk.c:730)
10 MyApp 0x00611028 GC_allochblk (allchblk.c:561)
11 MyApp 0x0061d0e0 GC_new_hblk (new_hblk.c:253)
12 MyApp 0x006133d0 GC_allocobj (alloc.c:1116)
13 MyApp 0x00617d30 GC_generic_malloc_inner (malloc.c:136)
14 MyApp 0x00617f40 GC_generic_malloc (malloc.c:192)
15 MyApp 0x00618264 GC_malloc_atomic (malloc.c:262)
16 MyApp 0x005a46d4 mono_object_allocate_ptrfree (object.c:4221)
17 MyApp 0x005a4aa0 mono_string_new_size (object.c:4848)
18 MyApp 0x005c1b14 ves_icall_System_String_InternalAllocateStr (string-icalls.c:213)
19 MyApp 0x002d34c4 wrapper_managed_to_native_string_InternalAllocateStr_int + 52
20 MyApp 0x002cff5c string_ToLower_System_Globalization_CultureInfo + 56
21 MyApp 0x003e6ac0 System_Net_WebRequest_GetCreator_string + 40
22 MyApp 0x003e694c System_Net_WebRequest_Create_System_Uri + 48
23 MyApp 0x003e68d8 System_Net_WebRequest_Create_string + 64
24 MyApp 0x004489c4 MyApp_Services_Client_GetResponseContent_string + 152
25 MyApp 0x00446288 MyApp_Services_Client_GetCurrentQuestion_long_long + 916
26 MyApp 0x00196fcc MyApp_Iphone_RootViewController_RetrieveCurrentQuestion + 868
27 MyApp 0x002e6368 System_Threading_Thread_StartUnsafe + 168
28 MyApp 0x00306890 wrapper_runtime_invoke_object_runtime_invoke_dynamic_intptr_intptr_intptr_intptr + 192
29 MyApp 0x004b0274 mono_jit_runtime_invoke (mini.c:5746)
30 MyApp 0x0059f924 mono_runtime_invoke (object.c:2756)
31 MyApp 0x005a1350 mono_runtime_delegate_invoke (object.c:3421)
32 MyApp 0x005ca884 start_wrapper_internal (threads.c:788)
33 MyApp 0x005ca924 start_wrapper (threads.c:830)
34 MyApp 0x005ef4b8 thread_start_routine (wthreads.c:285)
35 MyApp 0x0061f1d0 GC_start_routine (pthread_support.c:1468)
36 libsystem_c.dylib 0x3646a30a _pthread_start + 242
37 libsystem_c.dylib 0x3646bbb4 thread_start + 0
This is the GC_remap code that appears to be the point of failure, from https://github.com/mono/mono/blob/master/libgc/os_dep.c
#ifdef NACL
{
/* NaCl doesn't expose mprotect, but mmap should work fine */
void * mmap_result;
mmap_result = mmap(start_addr, len, PROT_READ | PROT_WRITE | OPT_PROT_EXEC,
MAP_PRIVATE | MAP_FIXED | OPT_MAP_ANON,
zero_fd, 0/* offset */);
if (mmap_result != (void *)start_addr) ABORT("mmap as mprotect failed");
/* Fake the return value as if mprotect succeeded. */
result = 0;
}
#else /* NACL */
result = mprotect(start_addr, len,
PROT_READ | PROT_WRITE | OPT_PROT_EXEC);
#endif /* NACL */
if (result != 0) {
GC_err_printf3(
"Mprotect failed at 0x%lx (length %ld) with errno %ld\n",
start_addr, len, errno);
ABORT("Mprotect remapping failed");
}
GC_unmapped_bytes -= len;
It would appear that the ABORT is caused by the mprotect function failing. We have been unable to get the failure code as the problem does not manifest itself on the simulator. The mprotect function appears to just mark the memory as accessible for read/write/execute. How is the memory manager passing parameters that cause it to fail? Could it be passing an incorrect pointer, or an incorrect length? Or are certain areas or boundaries handled differently on iOS?
The code at https://github.com/mono/mono/blob/master/libgc/allchblk.c for GC_allochblk_nth implies that the GC_remap function is only called if the memory block found was valid. (This file doesn't quite match the line numbers of the stack trace, so presumably it is not exactly the same file.)
http://developer.apple.com/library/ios/#documentation/System/Conceptual/ManPages_iPhoneOS/man2/mprotect.2.html says that it might fail with EACCES, EINVAL, ENOTSUP which are 13, 22, & 45 respectively. One of the reports on SO says that they get an error 12 (ENOMEM). I'm not sure what that means, as mprotect shouldn't be allocating memory, and the documentation doesn't say that is valid.
A more generic documentation at http://linux.die.net/man/2/mprotect indicates that ENOMEM can be caused by "Internal kernel structures could not be allocated. Or: addresses in the range [addr, addr+len] are invalid for the address space of the process, or specify one or more pages that are not mapped." How could this be?
We would much appreciate any suggestions on how we might move this forward. We are not doing anything other than C# code, and are not doing anything other than a periodic https read. What can we do to improve debugging (we can't trace anything as the app is killed by iOS). We have tried creating a simpler demonstration, but it does not fail fast enough to be worth using. If a Novell MonoTouch developer wants our source, we can provide it subject to the obvious confidentiality.
Thanks to your reproduction we have found and corrected a very obscure issue in the garbage collector. It will be included in MonoTouch 4.0.2.
You asked:
"Internal kernel structures could not
be allocated. ... How could this be?
One reason could be the fragmentation of the memory, for more on this topic see for example
How to solve Memory Fragmentation. This may belong to other internal resources like file pointers, too. Therefore the problem is only visible on the resource limited devices and not in the simulator.
To find non obvious resource leaks, too:
Are you using the Static Code Analysis (FxCop) of Visual Studio? Some of the rules give hints to free resources explicitly, e.g. to call Dispose or suggests to implement the IDispose interface.