NSString conversion to lowercase crashes - ios

xcode 4.6 (4H127), xcode 4.6.3 (4H1503): A simple lower/uppercase conversion of a string with an accented char crashes, depending on the setting of Deployment Target. Code snippet:
NSString *lc1 = #"Bosnië-Herzegovina";
NSString *lc2 = [lc1 lowercaseString];
NSString *uc3 = [lc1 uppercaseString];
NSLog( #"\nlc1=%#\nlc2=%#\nuc3=%# ", lc1,lc2,uc3);
The "ë" is simply typed as "opt-u e", the source code file is regular UTF Unicode.
lc1 looks as expected in the debugger. But, lc2 and uc3 strings have "chinese" characters appended at the end, with Deployment Target < 6.1. With 6.1 selected the chinese characters are gone. All that may simply be the UTF compatibility of the debugger, but with deployment target 5.0-5.1 the code snippet crashes even, as shown below, and that is my problem; the strings in my actual application are not in source code but from an SQLite database. So, at this moment I can only build my app for deployment target 6.0+? Am I missing something?
0x1c49a20: incl %eax
0x1c49a21: jmp 0x1c499fb ; CFUniCharMapCaseTo + 1275
0x1c49a23: movl 12(%ebp), %eax
0x1c49a26: movw $105, (%eax)
0x1c49a2b: movw $775, 2(%eax)
0x1c49a31: movl $2, %eax
0x1c49a36: jmp 0x1c49dac ; CFUniCharMapCaseTo + 2220
0x1c49a3b: movl 12(%ebp), %eax
0x1c49a3e: movw $105, (%eax)
0x1c49a43: movw $775, 2(%eax)
0x1c49a49: movw $771, 4(%eax)
0x1c49a4f: movl $3, %eax
0x1c49a54: jmp 0x1c49dac ; CFUniCharMapCaseTo + 2220
0x1c49a59: movl %eax, %edi
0x1c49a5b: movl 1264482(%edi), %eax
0x1c49a61: movl (%eax), %eax
0x1c49a63: movl %eax, (%esp)
0x1c49a66: movl $0, 8(%esp)
0x1c49a6e: movl $48, 4(%esp)
0x1c49a76: calll 0x1bd9980 ; CFAllocatorAllocate
0x1c49a7b: leal 16(%eax), %ecx
0x1c49a7e: movl %ecx, 1379418(%edi)
0x1c49a84: leal 32(%eax), %ecx
0x1c49a87: movl %ecx, 1379422(%edi)
0x1c49a8d: movl 1379410(%edi), %ecx
0x1c49a93: movl (%ecx), %ecx <-- EXC_BAD_ACCESS (code=1,..
0x1c49a95: movl (%ecx), %ecx
Edit:
I tried minimizing the project to show this problem, and... it disappeared. I have a bit of old-style C-code that uses things like malloc, free, freed, memmove, etc. If this bit is simply present, not even called, the problems described occur. My guess now is that some routines are loaded from a library it should not load from. Digging further.

Without exactly answering your question, but attempting to answer as no one else has, it would appear there are no "upper" case associations with those foreign characters.
Could you run a regex, or some kind of string replace to modify all known special characters with a normalized (english) version? Then they would have an uppercase or lowercase conversion.
Of course, this may completely ruin the strings you were reading from the DB if they aren't spelled right.

Well, my hunch that there was a problem with loading from libraries, or the order of loading made me change the order of the frameworks included: under "Build Phases" I spotted "CoreText.framework" as one of the last entries. I moved it to the top spot, and now all works fine for all Deployment Targets, 5.0, 5.1, 6.0, 6.1
I actually looked at the loadmap, that you can generate by setting LD_GENERATE_MAP_FILE to yes, to no avail.
Another pointer was supplied by editing the "Scheme" and switching on "Log library loads" and "Log API Usage", in that you can see that stuff is loaded from various libraries, one of them: CoreText.framework
In the end moving CoreText.framework to the top of the list made it all work.
The "chinese" characters you can still see in the debugger when using Deployment Target 5.0-6.0. With 6.1 even they are gone. I guess they fixed that now.

Related

Stack overflow: Thread 1: EXC_BAD_ACCESS (code=2, address=0x16d09aa00)

Crash description
Recently I'm facing kinda really strange memory issues in one of my iOS/Swift projects. I'm really not sure what's going on and feel it's also not quite easy to describe, but I'll try my best anyway.
It basically behaves like follows:
On a certain code base, the crash always occurs in the same place (100% reproducible)
Changes to the code base, may resolve the issue, but it may also just pop up somewhere else
Crashes only occur on real devices, never inside simulators
Currently the app crashes with following error (results from 3 different runs):
Thread 1: EXC_BAD_ACCESS (code=2, address=0x16d09aa00)
Thread 1: EXC_BAD_ACCESS (code=2, address=0x16af46a00)
Thread 1: EXC_BAD_ACCESS (code=2, address=0x16d526a00)
Reasoning about memory addresses
WWDC session
I found an interesting session (Understanding Crashes and Crash Logs) from WWDC 2018, where one of the guys points out that it's sometimes possible to derive more information from the specific memory addresses, the crashes occur.
Unfortunately the addresses it crashes in my app are somewhat completely different, but maybe we can get clues from them anyway? At least it's interesting, that they're all quite similar, or isn't it?
Changes due to Diagnostic options enabled
Further investigation shows that the first 2 bytes (16) stay always the same, followed by 4 random bytes followd by 3 bytes (a00). When activating diagnositcs (e.g. ASan or Scribble), the last 3 bytes change (e.g. 3a0 or 9e0). But maybe this is only a kind of shift due to more "debug stuff" being added? I'm really not that "memory guy", but just want to provide anything I noticed.
Trying "Diagnostic options"
I tried different Diagnostic options (from schemes), but none of them really changed the crash in any way, or provided any more information.
1. Scribble
Crashes do not reference 0xAA or 0x55, so it's nothing to be catched using Scribble? (Xcode - scribble, guard edges and guard malloc)
2. Malloc Guard Edges
Didn't notice any difference using this either.
3. Zombies
Using this guide.
malloc_info --type 0x16b15e9c0
error: error: Trying to put the stack in unreadable memory at: 0x16b15e920.
4. ASan
Using ASan just puts following entry on top of the stack trace. Unfortunately I didn't find anything helpful related to that.
#0 0x0000000109efbf60 in __asan_alloca_poison ()
5. TSan
Not available on real devices (crashes only occur there)
Recursion / BOF?
Could it be a recursion that is too long, or another kind of stack/heap buffer overflow?
But it seems like the stack size on real devices as well as simulators is exactly the same with 524288 bytes (from Thread.main.stackSize).
So, as it doesn't crash in simulators, it's not a BOF? Or is the architecture difference too big, to make such conclusions here?
Disassembling
I also tried "disassembling".
disassemble -a 0x16d09aa00
error: Could not find function bounds for address 0x16d09aa00
Or disassemble -frame
But my assembler skills are really lacking behind, so currently there is nothing to get for me from that information.
Need help
As you can see I'm really running out of ideas. Either the crashes are really totally weird, or I just do not have enough knowledge/skills to use above tools, to get me any closer to the cause of those issues.
Either way... Any help, hints, ideas or whatever could point me in the right direction is highly appreciated!
Thanks in advance, guys.
Update May 19, 2020
I totally forgot to mention, that we're using ReSwift heavily in our app, and the crashes seem to be related to how we use the Middlewares there, I guess.
I'm also already in contact with the devs there: github.com/ReSwift/ReSwift/issues/271.
Here's finally some code. Unfortunately I can't share all the apps code (which may be necessary!?) and also don't want to overload you with way to much code.
Current issue
Thread 1: EXC_BAD_ACCESS (code=1, address=0x16ed82da0)
UserAccountMiddleware.swift
Note: Using those DispatchQueue.main.async actually makes the crashes go away. They indeed break the current cycle, so maybe there's some kind of recursion or timing issue happening?
func userAccountMiddleware() -> Middleware<AppState> {
return { dispatch, getState in
return { next in
return { action in
switch action {
case _ as ReSwiftInit:
// DispatchQueue.main.async {
dispatch(UserAccountSetAuthToken(authToken: Defaults.customerAuthToken))
dispatch(UserAccountSetAvatar(index: Defaults.avatarIndex))
// }
if let data = Defaults.customer,
let customer = try? JSONDecoder().decode(Customer.self, from: data) {
// DispatchQueue.main.async {
dispatch(UserAccountSetCustomerLoggedIn(customer: customer))
// }
}
// [...]
default:
break
}
next(action)
}
}
}
}
ReSwift Store.swift
// [...]
open func _defaultDispatch(action: Action) {
guard !isDispatching else {
raiseFatalError(
"ReSwift:ConcurrentMutationError- Action has been dispatched while" +
" a previous action is action is being processed. A reducer" +
" is dispatching an action, or ReSwift is used in a concurrent context" +
" (e.g. from multiple threads)."
)
}
isDispatching = true
let newState = reducer(action, state) // Thread 1: EXC_BAD_ACCESS (code=1, address=0x16ed82da0)
isDispatching = false
state = newState
}
// [...]
Xcode console:
(lldb) po state
error: warning: couldn't get required object pointer (substituting NULL): Couldn't load 'self' because its value couldn't be evaluated
error: Trying to put the stack in unreadable memory at: 0x16d95ad00.
Assembler (very last step of crash):
myapp`type metadata accessor for GlobalState:
0x101f6ac10 <+0>: sub sp, sp, #0x30 ; =0x30
-> 0x101f6ac14 <+4>: stp x29, x30, [sp, #0x20] // Thread 1: EXC_BAD_ACCESS (code=1, address=0x16ed82da0)
0x101f6ac18 <+8>: adrp x8, 3620
0x101f6ac1c <+12>: add x8, x8, #0x148 ; =0x148
0x101f6ac20 <+16>: ldr x8, [x8]
0x101f6ac24 <+20>: mov x9, #0x0
0x101f6ac28 <+24>: mov x1, x8
0x101f6ac2c <+28>: str x0, [sp, #0x18]
0x101f6ac30 <+32>: str x1, [sp, #0x10]
0x101f6ac34 <+36>: str x9, [sp, #0x8]
0x101f6ac38 <+40>: cbnz x8, 0x101f6ac54 ; <+68> at <compiler-generated>
0x101f6ac3c <+44>: adrp x1, 2122
0x101f6ac40 <+48>: add x1, x1, #0x1dc ; =0x1dc
0x101f6ac44 <+52>: ldr x0, [sp, #0x18]
0x101f6ac48 <+56>: bl 0x102775358 ; symbol stub for: swift_getSingletonMetadata
0x101f6ac4c <+60>: str x0, [sp, #0x10]
0x101f6ac50 <+64>: str x1, [sp, #0x8]
0x101f6ac54 <+68>: ldr x0, [sp, #0x8]
0x101f6ac58 <+72>: ldr x1, [sp, #0x10]
0x101f6ac5c <+76>: str x0, [sp]
0x101f6ac60 <+80>: mov x0, x1
0x101f6ac64 <+84>: ldr x1, [sp]
0x101f6ac68 <+88>: ldp x29, x30, [sp, #0x20]
0x101f6ac6c <+92>: add sp, sp, #0x30 ; =0x30
0x101f6ac70 <+96>: ret
TL;DR
Just move huge structs to the heap, by wrapping them inside arrays. Using #propertyWrappers, this can be an at least partly elegant solution.
#propertyWrapper
struct StoredOnHeap<T> {
private var value: [T]
init(wrappedValue: T) {
self.value = [wrappedValue]
}
var wrappedValue: T {
get {
return self.value[0]
}
set {
self.value[0] = newValue
}
}
}
// Usage:
#StoredOnHeap var hugeStruct: HugeStruct
https://gist.github.com/d4rkd3v1l/ab582a7cafd3a8b8c164c8541a3eef96
Long version
I'm almost 100% certain now, that this is a stack overflow, as I (finally) managed to reproduce this in a little demo project: https://github.com/d4rkd3v1l/ReSwift-StackOverflowDemo
Now I will just provide some more details and solutions for anyone else may running into this or similar issues.
The stack size on iOS (as of iOS 13) is 512kb and should apply to both, devices and simulators. Why did I say "should"? Because it almost certainly is somewhat different on simulators, as I did not see those crashes there. So maybe Thread.main.stackSize just tells 512kb but is in fact larger? IDK 🤷‍♂️
Here are some indicators, you may face the same issue:
You get EXC_BAD_ACCESS crashes with code 1 or 2**. And the crashes occur in high memory addresses, or at least completely out of where the rest of your app/stack normally "lives". Something like 0x16d95ad00 in my case.
Reducing the stuff you put on the stack (value types, e.g. very very large structs) or breaking the call stack down into smaller pieces (e.g. dispatch async) to give the stack some "time to breathe" prevents this crash.
And here at the latter we're already in the middle of the solution for that issue. As the stack size cannot (and probably even should not) increased, you must reduce the load you put there, like described in the 2nd point.
At least that's the solution we will probably go for. 🤞
*This is true at least for the main thread, other threads may be different.
**I think code 0 is kinda null pointer exceptionand therefore doesn't apply here. Please correct me if I'm wrong about this.

How to solve LLDB error about N_SO in symbol with UID 1

When I launched lldb to debug an iOS application, I got an error that I never had before.
error: Veriff(0x00000001018cc000) N_SO in symbol with UID 1 has
invalid sibling in debug map, please file a bug and attach the binary
listed in this error
Below is the context of the error.
(lldb) process connect connect://localhost:6666
error: Veriff(0x00000001018cc000) N_SO in symbol with UID 1 has invalid sibling in debug map, please file a bug and attach the binary listed in this error
Process 3270 stopped
* thread #1, queue = 'com.apple.main-thread', stop reason = signal SIGSTOP
frame #0: 0x0000000187a1f6b0 libxpc.dylib` _xpc_dictionary_apply_node_f + 108
libxpc.dylib`_xpc_dictionary_apply_node_f:
-> 0x187a1f6b0 <+108>: mov x1, x20
0x187a1f6b4 <+112>: blr x21
0x187a1f6b8 <+116>: tbz w0, #0x0, 0x187a1f6f8 ; <+180>
0x187a1f6bc <+120>: mov x0, x26
0x187a1f6c0 <+124>: cbnz x26, 0x187a1f6a0 ; <+92>
0x187a1f6c4 <+128>: add x22, x22, #0x1 ; =0x1
0x187a1f6c8 <+132>: cmp x22, x23
0x187a1f6cc <+136>: b.lo 0x187a1f698 ; <+84>
Target 0: (Test app) stopped.
Has anyone been able to solve this error?
Does this impact any debugging?
I've never seen that error triggered before. If you can make this binary available to us, please file a bug either with http://bugs.llvm.org or http://bugreporter.apple.com and include the error message and the binary.
The error means lldb can't map symbols from some .o file that was included in your binary back to the .o file they came from (which is where the debug information actually resides.) So that code's debug information will not be available.

how to find variables location in memory without source code?

Basically I want to find the address/location of a variable in gdb?
I know normally the variable are store at rbp but don't know how to locate them using gdb.
I want to find the address/location of a variable in gdb?
That is possible, but the approach is different depending on whether the variable is a global or a local.
I know normally the variable are store at rbp
Local variables are stored at some offset of the frame pointer. %RBP is often used as a frame pointer in unoptimized binaries.
To find such variable, you'll need to know how to read machine code, and then you can find it. GDB will not help you with finding it in code that is compiled without debug info (it can't).
without source code
Source code has nothing to do with this -- GDB never looks at the source code, except to display it to you.
On to concrete example. Suppose you have the following source:
int foo(int *ip) { return *ip + 42; }
int main()
{
int j = 1;
return foo(&j);
}
Compiling this without debug info and without optimizations, results in:
(gdb) disas main
Dump of assembler code for function main:
0x000000000000060d <+0>: push %rbp
0x000000000000060e <+1>: mov %rsp,%rbp
0x0000000000000611 <+4>: sub $0x10,%rsp
0x0000000000000615 <+8>: movl $0x1,-0x4(%rbp)
0x000000000000061c <+15>: lea -0x4(%rbp),%rax
0x0000000000000620 <+19>: mov %rax,%rdi
0x0000000000000623 <+22>: callq 0x5fa <foo>
0x0000000000000628 <+27>: leaveq
0x0000000000000629 <+28>: retq
End of assembler dump.
here you can clearly see that j is being stored at negative offset 4 off %rbp.
You can set a breakpoint on foo, and use GDB to examine its value like so:
(gdb) b foo
Breakpoint 1 at 0x5fe
(gdb) run
Breakpoint 1, 0x00005555555545fe in foo ()
(gdb) up
#1 0x0000555555554628 in main ()
(gdb) x/x $rbp-4
0x7fffffffdbcc: 0x00000001 // indeed that is expected value of j

EXC_BAD_ACCESS in Assembly Code in iOS App

I'm trying to debug an EXC_BAD_ACCESS crash in an iOS App I am working on. Basically, my code calls the function new_dyna_start() which corresponds to the a certain assembly method. Here's the relevant assembly code:
.align 4
42430:
.long _translation_cache_iphone
.align 2
.globl _new_dyna_start
// .type new_dyna_start, %function
_new_dyna_start:
ldr r12, .dlptr
mov r0, #0xa4000000
stmia r12, {r4, r5, r6, r7, r8, r9, sl, fp, lr}
sub fp, r12, #28
add r0, r0, #0x40
bl _new_recompile_block
ldr r0, [fp, #64]
ldr r10, [fp, #400+36] /* Count */
str r0, [fp, #72]
sub r10, r10, r0
ldr r0, 42430b
ldr pc, [r0]
From my (limited) understanding, at line 6 of the method, it calls the C function new_recompile_block(). This method works fine, and I know it finishes because at the end of the function I have
printf("End of loop");
which then appears in the debugger. After the method completes, I'm not entirely sure I understand what happens, but it seems that the assembly method obtains a reference to the C variable translation_cache_iphone. However, at the final line the app crashes oddly. This message appears in Xcode: http://imgur.com/dqKo0
However, if I click on the side to the last method called, I see it is this: http://imgur.com/M5h84
This seems to support my idea that it is the translation_cache_iphone variable causing the crash, as the memory address of the EXC_BAD_ACCESS (0x401000) is the same as translation_cache_iphone. translation_cache_iphone is declared as:
unsigned char* translation_cache_iphone = NULL;
and is initialized by:
translation_cache_iphone = (unsigned char *)(((unsigned long) translation_cache_static_iphone + (4096)) & ~(4095));
Am I right in assuming that this is the problem? Is the problem in the assembly code, or in the C code? I've tried modifying both, but to no avail. The assembly code above is the original.
Here is a link to the full source on Github. Simply compile and run on an iDevice with Xcode and you'll see the exact issues I'm facing. It may be easier to debug that way.
The last two instructions form an indirect jump to the translation_cache_iphone which is thus expected to be executable code. Verify that is the case and that memory permissions are appropriate - in many systems data pages are not executable by default.
This seems to support my idea that it is the translation_cache_iphone variable causing the crash
Yes, I believe that this variable is the problem.
In the assembly code you posted I can see one line that could cause an invalid access to the memory, and it is:
ldr r0, 42430b
ldr pc, [r0]
The first line loads the data from the label 42430 to the register r0. Then, the second line points PC (Program Counter) to the content of r0.
In the beginning of the assembly code you have declared what is the label 42430:
42430:
.long _translation_cache_iphone
Then, when it tries to access this value and execute is as code, it crashes.

Getting calling conventions from DWARF info

I am trying to get information about calling conventions from DWARF info. More specific, I want to get which registers / stack locations are used to pass arguments to functions. My problem is that I am getting somehow wrong information in some cases from DWARF dump. The example I am using is the following "C code":
int __attribute__ ((fastcall)) __attribute__ ((noinline)) mult (int x, int y) {
return x*y;
}
I compile this example using the following command:
gcc -c -g -m32 test.c -o test.o
Now when I use the following command to get the dwarf dump:
dwarfdump test.o
I am getting the following information about this function:
< 2><0x00000042> DW_TAG_formal_parameter
DW_AT_name "x"
DW_AT_decl_file 0x00000001 /home/khaled/Repo_current/trunk/test.c
DW_AT_decl_line 0x00000001
DW_AT_type <0x0000005b>
DW_AT_location DW_OP_fbreg -12
< 2><0x0000004e> DW_TAG_formal_parameter
DW_AT_name "y"
DW_AT_decl_file 0x00000001 /home/khaled/Repo_current/trunk/test.c
DW_AT_decl_line 0x00000001
DW_AT_type <0x0000005b>
DW_AT_location DW_OP_fbreg -16
Looking at the DW_AT_location entries, it is some offset from the frame base. This implies they are memory arguments, but the actual calling convention "fastcall" forces passing them into registers. By looking at the disassembly of the produced object file, I can see they are copied from registers to stack locations at the entry point of the function. Is there a way to know from the dwarf dump --or using any other way-- where the arguments are passed at the call initially?
Thanks,
Because you are using gcc -c -g -m32 test.c -o test.o. Although it is a fastcall function, GCC still needs to generate code to save values from registers to the stack frame at the beginning of the function. Without that, any debugger or gdb cannot debug the program or they will say the argument is being optimized and not shown. It makes debugging impossible.
In x86_64, compiler also uses some registers to pass some arguments by default, even without specifying attribute fastcall for a function. You can also find those registers are being copied to the stack as well.
// x86_64 assembly code
_mult:
Leh_func_begin1:
pushq %rbp
Ltmp0:
movq %rsp, %rbp
Ltmp1:
movl %edi, -4(%rbp)
movl %esi, -8(%rbp)
movl -4(%rbp), %eax
movl -8(%rbp), %ecx
imull %ecx, %eax
If you turn on optimization flag -O, -O2, -O3 (no matter -g or not), you can disassemble and find there is nothing being copied to the stack frame. And when you gdb the optimized executable file, and stop at the beginning of the function to show local variables, gdb will tell you those arguments are being optimized out.
the dwarfdump example of the 32-bit program would look like
0x00000083: TAG_formal_parameter [4]
AT_name( "x" )
AT_decl_file( "test.c" )
AT_decl_line( 1 )
AT_type( {0x0000005f} ( int ) )
AT_location( 0x00000000
0x00000000 - 0x00000003: ecx
0x00000003 - 0x00000018: ecx )
0x00000090: TAG_formal_parameter [4]
AT_name( "y" )
AT_decl_file( "test.c" )
AT_decl_line( 1 )
AT_type( {0x0000005f} ( int ) )
AT_location( 0x0000001e
0x00000000 - 0x00000003: edx
0x00000003 - 0x00000018: edx )
And you can find the generated assembly code is much simple and clean.
_mult:
pushl %ebp
movl %esp, %ebp
movl %ecx, %eax
imull %edx, %eax
popl %ebp
ret $12

Resources