Corrupted memory with DispatchQueue.concurrentPerform - ios

I see a strange behavior with the following code (which runs on Playground).
import Foundation
let count = 100
var array = [[Int]](repeating:[Int](), count:count)
DispatchQueue.concurrentPerform(iterations: count) { (i) in
array[i] = Array(i..<i+count)
}
// Evaluation
for (i,value) in array.enumerated() {
if (value.count != count) {
print(i, value.count)
}
}
The result is different each time, and sometime crashes with memory corruption. It looks like a memory reallocation (of "array") is happening while another thread is accessing the memory.
Is this a bug (of iOS) or an expected behavior? Am I missing something?

This is expected behaviour. Swift arrays are not thread safe; That is, modifying a Swift array from multiple threads concurrently will cause corruption.
I realise that you are just experimenting, but even if arrays were thread safe, this would not be a good use of concurrentPerform and would probably perform worse than a simple for loop given the threading overhead.
Once you introduce an appropriate synchronisation method to guard the array update, such as dispatching that update onto a serial dispatch queue, it will definitely perform more slowly than a simple for loop

Here is the solution. Thank you for quick responses.
import Foundation
let count = 1000
var arrays = [[Int]](repeating:[Int](), count:count)
let dispatchGroup = DispatchGroup()
let lockQueue = DispatchQueue(label: "lockQueue")
DispatchQueue.concurrentPerform(iterations: count) { (i) in
dispatchGroup.enter()
let array = Array(i..<i+count) // The actual code is very complex
lockQueue.async {
arrays[i] = array
dispatchGroup.leave()
}
}
dispatchGroup.wait()
// Evaluation
for (i,value) in arrays.enumerated() {
if (value.count != count) {
print(i, value.count)
}
}

In my case I had to generate 24k elements array with Float multiple times per second and it takes around 40ms for each on old iPhone 6.
Since each element of the array is only assigned once, I've decided to try raw array using pointers:
class UnsafeArray<T> {
let count: Int
let cArray: UnsafeMutablePointer<T>
init(_ size: Int) {
count = size
cArray = UnsafeMutablePointer<T>.allocate(capacity: size)
}
subscript(index: Int) -> T {
get { return cArray[index] }
set { cArray[index] = newValue }
}
deinit {
free(cArray)
}
}
Then I used it like this:
let result = UnsafeArray<Float>(24000)
DispatchQueue.concurrentPerform(iterations: result.count, execute: { i in
result[i] = someCalculation()
})
And it worked! Now it takes 9-16ms. Also, I have no memory leaks using this code.

Related

Some classes returned by objc_getClassList aren't accessible

My code is crashing (rarely) while accessing classes retrieved by objc_getClassList
Crash message:
objc[48740]: Attempt to use unknown class 0x600000ac2640.
var numClasses: Int32 = 0
var newNumClasses = objc_getClassList(nil, 0)
var classes: UnsafeMutablePointer<AnyClass>?
while (numClasses < newNumClasses) { // calling objc_getClassList until all returned classes are fitting into the buffer
numClasses = newNumClasses
if let classes = classes {
classes.deallocate()
}
classes = UnsafeMutablePointer<AnyClass>.allocate(capacity: Int(numClasses))
let buffer = AutoreleasingUnsafeMutablePointer<AnyClass>(classes)
newNumClasses = objc_getClassList(buffer, numClasses);
}
for i in 0..<Int(newNumClasses) {
let someclass: AnyClass = classes![i]
let s = "\(someclass)" // crash happens here
// not very costly computations here
}
Mostly it happens when iteration is performed for 80% or so. Total number of classes is around 50000. It reproduces more often on slower devices/simulators. No other signs of heap corruption are present. Memory graph is fine.
Classes around the broken one looks somewhat like this: fir_420DF5A9-4BF1-468C-8919-FFAA7D864314_GDTCCTUploadOperation (this is a name of the class. Looks firebasy).
EDIT: Alternatively
var count: UInt32 = 0
guard let classList = objc_copyClassList(&count) else {
return []
}
return UnsafeBufferPointer(start: classList, count: Int(count)).filter { (type: AnyClass) in
return class_conformsToProtocol(type, protocolName)
}
Produces the same result (again, very rarely)
Please let me know if anyone has an idea how to filter out dead/inaccessible classes.
Thank you.

DispatchQueue threads don't always set the correct results

Am trying on coding game MineSweeper, the following code is to set the numbers around landmines. For a test, I choose the minimum level 9 x 9 with 10 landmines.
For a faster performance, I tried to use more threads when setting numbers, but one day I found that it doesn't always give the correct number arrangements, I made a loop to create it 1000 times and found that 20 ~ 40 out of 1000 are wrong.
Here are several wrong results, "*" represents landmine, "0" means no landmine around
wrong mine: index 1 should be "*"
10212*100
112*21211
0011101*1
000000111
000011100
01123*200
01*2**200
023432100
01**10000
wrong num: index 0 should be "1"
0*212*100
112*21211
0011101*1
000000111
000011100
01123*200
01*2**200
023432100
01**10000
wrong num: index 73 should be "1"
1*212*100
112*21211
0011101*1
000000111
000011100
01123*200
01*2**200
023432100
00**10000
Without using DispatchQueue or set DispatchSemaphore's value to 1, it gives the correct number arrangement in 1000%1000.
1*212*100
112*21211
0011101*1
000000111
000011100
01123*200
01*2**200
023432100
01**10000
Here is the sample code:
// actually indexes created randomly every time
let minesIndexArr = [59, 74, 1, 12, 50, 56, 75, 58, 5, 25]
var defaultCellArr = [String]()
var totalCellArr = [String]()
var count = 0
for _ 1...81 {
defaultCellArr.append("0")
}
runLoop()
func runLoop() {
if count == 1000 {
return
}
totalCellArr = defaultCellArr
setNums()
}
func setNums() {
let group = DispatchGroup()
let queue = DispatchQueue(label: "com.setnums", attributes: .concurrent)
let semaphore = DispatchSemaphore(value: 10)
for i in 0..<self.totalCellArr.count {
semaphore.wait()
group.enter()
queue.async(group: group, execute: {
if self.minesIndexArr.firstIndex(of: i) != nil{
self.totalCellArr[i] = "*"
}else{
var num = 0
let neighbourIndexes = self.getNeighbourIndex(i)
for v in neighbourIndexes.values {
if self.minesIndexArr.firstIndex(of: v) != nil {
num += 1
}
}
self.totalCellArr[i] = String(num)
}
group.leave()
semaphore.signal()
})
}
group.notify(queue: DispatchQueue.main) {
printMap()
count += 1
self.runLoop()
}
}
tl;dr
You are using this non-zero semaphore to do parallel calculations, constraining the degree of concurrency to something reasonable. I would recommend concurrentPerform.
But the issue here is not how you are constraining the degree of the parallelism, but rather that you are using the same properties (shared by all of these concurrent tasks) for your calculations, which means that one iteration on one thread can be mutating these properties while they are being used/mutated by another parallel iteration on another thread.
So, I would avoid using any shared properties at all (short of the final array of boards). Use local variables only. And make sure to synchronize the updating of this final array so that you don't have two threads mutating it at the same time.
So, for example, if you wanted to create the boards in parallel, I would probably use concurrentPerform as outlined in my prior answer:
func populateBoards(count: Int, rows: Int, columns: Int, mineCount: Int, completion: #escaping ([Board]) -> Void) {
var boards: [Board] = []
let lock = NSLock()
DispatchQueue.global().async {
DispatchQueue.concurrentPerform(iterations: count) { index in
let board = Board(rows: rows, columns: columns, mineCount: mineCount)
lock.synchronize {
boards.append(board)
}
}
}
DispatchQueue.main.async {
lock.synchronize {
completion(boards)
}
}
}
Note, I'm not referencing any ivars. It is all local variables, passing the result back in a closure.
And to avoid race conditions where multiple threads might be trying to update the same array of boards, I am synchronizing my access with a NSLock. (You can use whatever synchronization mechanism you want, but locks a very performant solution, probably better than a GCD serial queue or reader-writer pattern in this particular scenario.) That synchronize method is as follows:
extension NSLocking {
func synchronize<T>(block: () throws -> T) rethrows -> T {
lock()
defer { unlock() }
return try block()
}
}
That is a nice generalized solution (handling closures that might return values, throw errors, etc.), but if that is too complicated to follow, here is a minimalistic rendition that is sufficient for our purposes here:
extension NSLocking {
func synchronize(block: () -> Void) {
lock()
block()
unlock()
}
}
Now, I confess, that I'd probably employ a different model for the board. I would define a Square enum for the individual squares of the board, and then define a Board which was an array (for rows) of arrays (for columns) for all these squares. Anyway, this in my implementation of the Board:
enum Square {
case count(Int)
case mine
}
struct Board {
let rows: Int
let columns: Int
var squares: [[Square]]
init(rows: Int, columns: Int, mineCount: Int) {
self.rows = rows
self.columns = columns
// populate board with all zeros
self.squares = (0..<rows).map { _ in
Array(repeating: Square.count(0), count: columns)
}
// now add mines
addMinesAndUpdateNearbyCounts(mineCount)
}
mutating func addMinesAndUpdateNearbyCounts(_ mineCount: Int) {
let mines = (0..<rows * columns)
.map { index in
index.quotientAndRemainder(dividingBy: columns)
}
.shuffled()
.prefix(mineCount)
for (mineRow, mineColumn) in mines {
squares[mineRow][mineColumn] = .mine
for row in mineRow-1 ... mineRow+1 where row >= 0 && row < rows {
for column in mineColumn-1 ... mineColumn+1 where column >= 0 && column < columns {
if case .count(let n) = squares[row][column] {
squares[row][column] = .count(n + 1)
}
}
}
}
}
}
extension Board: CustomStringConvertible {
var description: String {
var result = ""
for row in 0..<rows {
for column in 0..<columns {
switch squares[row][column] {
case .count(let n): result += String(n)
case .mine: result += "*"
}
}
result += "\n"
}
return result
}
}
Anyway, I would generate 1000 9×9 boards with ten mines each like so:
exercise.populateBoards(count: 1000, rows: 9, columns: 9, mineCount: 10) { boards in
for board in boards {
print(board)
print("")
}
}
But feel free to use whatever model you want. But I'd suggest encapsulating the model for the Board in its own type. It not only abstracts the details of the generation of a board from the multithreaded algorithm to create lots of boards, but it naturally avoids any unintended sharing of properties by the various threads.
Now all of this said, this is not a great example of parallelized code because the creation of a board is not nearly computationally intensive enough to justify the (admittedly very minor) overhead of running it in parallel. This is not a problem that is likely to benefit much from parallelized routines. Maybe you'd see some modest performance improvement, but not nearly as much as you might experience from something a little more computationally intensive.

multithread task, Threads in ViewController? [duplicate]

I have to make a simple task - singleview app with two buttons and three treads -
Start Button :
create a thread T1 - the GPS location of the device is collected repeatedly every 3.0 seconds, and the results (as a string) are handed over to T3.
create w thread T2 - the percentage usage of the device's battery is collected repeatedly every B seconds, and the results (as a string) are handed over to T3.
Data from T1 and T2 should be stored in array with limited positions (maxData).
In Thread T3, if array reaches maxData data should be send to the server (URL) via HTTP. Then clear array
STOP button - just stop all three threads
I am a total noob in iOS threading. Unfortunately, App MUST use threads even if it is not necessary. In my previous question, Rob Napier suggested to use DispatchSourceTimers.
I'm close to find a proper solution, but sth is still wrong. When i clear array and post, single data from DispatchSourceTimer is LOST. I guess, that sth is wrong with concurrentDataQueue.async(flags: .barrier), and i have to block reading of array somehow (?)
Class BackgroundThreadTimer is just wrapper for DispatchSourceTimers taken from
medium article(RepeatingTimer)
//partly Pseudocode
var counter = 0
var data = [String]()
var maxData = 5
var counterInt = 0
var concurrentDataQueue = DispatchQueue(label: "dataCleanAndPostQueue", attributes: .concurrent)
var timerLocalization = BackgroundThreadTimer(timeInterval: 2)
timerLocalization.eventHandler = {
//do sth and receive String x with actual localization
//...
concurrentDataQueue.async(flags: .barrier) {
appendAndPost(phoneInfo: String(counterInt) + ": " + x)
counterInt += 1
}
}
var timerBattery = BackgroundThreadTimer(timeInterval: 3)
timerBattery.eventHandler = {
//do sth and receive String x with actual battery level
//...
concurrentDataQueue.async(flags: .barrier) {
appendAndPost(phoneInfo: String(counterInt) + ": " + x)
counterInt += 1
}
}
func appendAndPost(phoneInfo: String) {
if data.count < maxData {
data.append(phoneInfo)
} else {
let arrayString = data.joined(separator: "; ")
DispatchQueue.global(qos: .background).async {
//post arrayString to http
print(arrayString)
}
data = [String]()
}
}
//when start pressed
timerLocalization.resume()
timerBattery.resume()
//when stop pressed
timerLocalization.suspend()
timerBattery.suspend()
Could sb help? Is my
diagnosis proper? And how to fix this.
In appendAndPost, you are appending result only if the count is less than maxData.
But let’s imagine that you called this where you already had five items in the array. In that scenario, you start the request, but you’re never doing anything with the supplied value.
I would advise appending the value regardless, and sending if the count hits the threshold:
func appendAndPost(phoneInfo: String) {
data.append(phoneInfo)
if data.count >= maxData {
// create your request and send it
...
// reset `data`
data = []
}
}

two DispatchSourceTimers sharing one resource error, multithread task

I have to make a simple task - singleview app with two buttons and three treads -
Start Button :
create a thread T1 - the GPS location of the device is collected repeatedly every 3.0 seconds, and the results (as a string) are handed over to T3.
create w thread T2 - the percentage usage of the device's battery is collected repeatedly every B seconds, and the results (as a string) are handed over to T3.
Data from T1 and T2 should be stored in array with limited positions (maxData).
In Thread T3, if array reaches maxData data should be send to the server (URL) via HTTP. Then clear array
STOP button - just stop all three threads
I am a total noob in iOS threading. Unfortunately, App MUST use threads even if it is not necessary. In my previous question, Rob Napier suggested to use DispatchSourceTimers.
I'm close to find a proper solution, but sth is still wrong. When i clear array and post, single data from DispatchSourceTimer is LOST. I guess, that sth is wrong with concurrentDataQueue.async(flags: .barrier), and i have to block reading of array somehow (?)
Class BackgroundThreadTimer is just wrapper for DispatchSourceTimers taken from
medium article(RepeatingTimer)
//partly Pseudocode
var counter = 0
var data = [String]()
var maxData = 5
var counterInt = 0
var concurrentDataQueue = DispatchQueue(label: "dataCleanAndPostQueue", attributes: .concurrent)
var timerLocalization = BackgroundThreadTimer(timeInterval: 2)
timerLocalization.eventHandler = {
//do sth and receive String x with actual localization
//...
concurrentDataQueue.async(flags: .barrier) {
appendAndPost(phoneInfo: String(counterInt) + ": " + x)
counterInt += 1
}
}
var timerBattery = BackgroundThreadTimer(timeInterval: 3)
timerBattery.eventHandler = {
//do sth and receive String x with actual battery level
//...
concurrentDataQueue.async(flags: .barrier) {
appendAndPost(phoneInfo: String(counterInt) + ": " + x)
counterInt += 1
}
}
func appendAndPost(phoneInfo: String) {
if data.count < maxData {
data.append(phoneInfo)
} else {
let arrayString = data.joined(separator: "; ")
DispatchQueue.global(qos: .background).async {
//post arrayString to http
print(arrayString)
}
data = [String]()
}
}
//when start pressed
timerLocalization.resume()
timerBattery.resume()
//when stop pressed
timerLocalization.suspend()
timerBattery.suspend()
Could sb help? Is my
diagnosis proper? And how to fix this.
In appendAndPost, you are appending result only if the count is less than maxData.
But let’s imagine that you called this where you already had five items in the array. In that scenario, you start the request, but you’re never doing anything with the supplied value.
I would advise appending the value regardless, and sending if the count hits the threshold:
func appendAndPost(phoneInfo: String) {
data.append(phoneInfo)
if data.count >= maxData {
// create your request and send it
...
// reset `data`
data = []
}
}

Performance issue while finding min and max with functional approach

I have an array of subviews and I want to find the lowest tag and the highest tag (~ min and max). I tried to play with the functional approach of Swift and optimized it as much as my knowledge allowed me, but when I do this:
let startVals = (min:Int.max, max:Int.min)
var minMax:(min: Int, max: Int) = subviews.filter({$0 is T2GCell}).reduce(startVals) {
(min($0.min, $1.tag), max($0.max, $1.tag))
}
I still get worse performance (approximately 10x slower) than good ol' for cycle:
var lowest2 = Int.max
var highest2 = Int.min
for view in subviews {
if let cell = view as? T2GCell {
lowest2 = lowest2 > cell.tag ? cell.tag : lowest2
highest2 = highest2 < cell.tag ? cell.tag : highest2
}
}
To be totally precise I am also including snippet of the measuring code. Note that the "after-recalculations" for human readable times is done outside of any measurement:
let startDate: NSDate = NSDate()
// code
let endDate: NSDate = NSDate()
// outside of measuring block
let dateComponents: NSDateComponents = NSCalendar(calendarIdentifier: NSCalendarIdentifierGregorian)!.components(NSCalendarUnit.CalendarUnitNanosecond, fromDate: startDate, toDate: endDate, options: NSCalendarOptions(0))
let time = Double(Double(dateComponents.nanosecond) / 1000000.0)
My question is - am I doing it wrong, or this use case is simply not suitable for functional approach?
EDIT
This is is 2x slower:
var extremes = reduce(lazy(subviews).map({$0.tag}), startValues) {
(min($0.lowest, $1), max($0.highest, $1))
}
And this is only 20% slower:
var extremes2 = reduce(lazy(subviews), startValues) {
(min($0.lowest, $1.tag), max($0.highest, $1.tag))
}
Narrowed and squeezed down to very nice performance times, but still not as fast as the for cycle.
EDIT 2
I noticed I left out the filter in previous edits. When added:
var extremes3 = reduce(lazy(subviews).filter({$0 is T2GCell}), startValues) {
(min($0.lowest, $1.tag), max($0.highest, $1.tag))
}
I'm back to 2x slower performance.
In optimized builds, reduce and for should be completely equivalent in performance. However, in unoptimized debug builds, a for loop may beat the reduce version, because reduce will not be specialized and inlined. The filter can be removed, eliminating an unnecessary extra array creation, however that array creation is going to be pretty fast (all it is doing is copying pointers into memory) so that is not really a big deal, eliminating it is more for clarity.
However, I believe part of the problem is that in your reduce, you are calling the .tag property on AnyObject, whereas in your for loop version, you are calling T2GCell.tag. This could make a big difference. You can see this if you break out the filter:
// filtered will be of type [AnyObject]
let filtered = subviews.filter({$0 is T2GCell})
let minMax:(min: Int, max: Int) = filtered.reduce(startVals) {
// so $1.tag is calling AnyObject.tag, not T2GCell.tag
(min($0.min, $1.tag), max($0.max, $1.tag))
}
This means .tag is going to be dynamically bound at runtime, potentially a slower operation.
Here's some sample code that demonstrates the difference. If you compile this will swiftc -O you'll see the statically-bound (or rather not-quite-so dynamically-bound) reduce and the for loop perform pretty much the same:
import Foundation
#objc class MyClass: NSObject {
var someProperty: Int
init(_ x: Int) { someProperty = x }
}
let classes: [AnyObject] = (0..<10_000).map { _ in MyClass(Int(arc4random())) }
func timeRun<T>(name: String, f: ()->T) -> String {
let start = CFAbsoluteTimeGetCurrent()
let result = f()
let end = CFAbsoluteTimeGetCurrent()
let timeStr = toString(Int((end - start) * 1_000_000))
return "\(name)\t\(timeStr)µs, produced \(result)"
}
let runs = [
("Using AnyObj.someProperty", {
reduce(classes, 0) { prev,next in max(prev,next.someProperty) }
}),
("Using MyClass.someProperty", {
reduce(classes, 0) { prev,next in
(next as? MyClass).map { max(prev,$0.someProperty) } ?? prev
}
}),
("Using plain ol' for loop", {
var maxSoFar = 0
for obj in classes {
if let mc = obj as? MyClass {
maxSoFar = max(maxSoFar, mc.someProperty)
}
}
return maxSoFar
}),
]
println("\n".join(map(runs, timeRun)))
Output from this on my machine:
Using AnyObj.someProperty 4115µs, produced 4294310151
Using MyClass.someProperty 1169µs, produced 4294310151
Using plain ol' for loop 1178µs, produced 4294310151
Can't reproduce your exact example, but you can try moving away the filter. The following code should be functionally equivalent to your last attempt.
var extremes4 = reduce(subviews, startValues) {
$1 is T2GCell ? (min($0.lowest, $1.tag), max($0.highest, $1.tag)) : $0
}
Thus you don't iterate twice on subviews. Notice I removed lazy since it appears you always use the entire list.
By the way, IMHO, functional programming can be a very useful approach, but I would think twice before sacrificing code clarity for the only purpose of a fancy functional approach. Thus if a for loop is clearer, and even faster ... just use it ;-) That said, is good for you to experiment with different ways to approach the same problem.
One issue I could think of is that the looping is done twice. First the filter returns an filtered array and then a looping in reduce.
filter(_:)
Returns an array containing the elements of the array for which a provided closure indicates a match.
Declaration
func filter(includeElement: (T) -> Bool) -> [T]
Discussion
Use this method to return a new array by filtering an existing array. The closure that you supply for includeElement: should return a Boolean value to indicate whether an element should be included (true) or excluded (false) from the final collection:
While in the second case there is only one loop.
I am not sure if there is any difference of execution time for 'is' as 'as?' operator.

Resources