I'm trying to test an async function that creates a WKWebsiteDataStore with some HTTPCookies.
import Foundation
import WebKit
#MainActor
class HttpCookieUtility {
func createWebsiteDataStore(httpCookies: [HTTPCookie]) async -> WKWebsiteDataStore {
let websiteDataStore = WKWebsiteDataStore.nonPersistent()
for cookie in httpCookies {
await websiteDataStore.httpCookieStore.setCookie(cookie)
}
return websiteDataStore
}
}
The unit test code is here:
import XCTest
#testable import MyFramework
final class HttpCookieUtilityTests: XCTestCase {
func test_websiteDataStore_is_created_from_cookies() async throws {
var httpCookies = [HTTPCookie]()
for index in 1...20 {
let properties: [HTTPCookiePropertyKey: Any] = [.domain: "www.example.com", .path: ".", .name: "name-\(index)", .value: "value-\(index)"]
let httpCookie = HTTPCookie(properties: properties)!
httpCookies.append(httpCookie)
}
let websiteDataStore = await HttpCookieUtility().createWebsiteDataStore(httpCookies: httpCookies)
XCTAssertNotNil(websiteDataStore)
}
}
The unit test passes. However, when I turn on the Thread Sanitizer in the scheme's Diagnostic settings, I see a series of warnings like the following (not all output include here for brevity):
WARNING: ThreadSanitizer: data race (pid=47736) Read of size 8 at
0x7b6400040310 by main thread:
#0 (1) suspend resume partial function for HttpCookieUtilityTests.test_websiteDataStore_is_created_from_cookies()
HttpCookieUtilityTests.swift:18 (MyFrameworkTests:x86_64+0x1e793)
#1 swift::runJobInEstablishedExecutorContext(swift::Job*) :2 (libswift_Concurrency.dylib:x86_64+0x2a4b5)
Previous write of size 8 at 0x7b6400040310 by thread T7:
#0 HttpCookieUtilityTests.test_websiteDataStore_is_created_from_cookies()
HttpCookieUtilityTests.swift:18 (MyFrameworkTests:x86_64+0x1e613)
#1 swift::runJobInEstablishedExecutorContext(swift::Job*) :2 (libswift_Concurrency.dylib:x86_64+0x2a4b5)
Location is heap block of size 1032 at 0x7b6400040100 allocated by
thread T7:
#0 malloc :2 (libclang_rt.tsan_iossim_dynamic.dylib:x86_64+0x533ac)
#1 swift::StackAllocator<1000ul, &(swift::TaskAllocatorSlabMetadata)>::getSlabForAllocation(unsigned
long) :2 (libswift_Concurrency.dylib:x86_64+0x2f19a)
#2 swift::runJobInEstablishedExecutorContext(swift::Job*) :2 (libswift_Concurrency.dylib:x86_64+0x2a4b5)
Thread T7 (tid=397262, running) is a GCD worker thread
SUMMARY: ThreadSanitizer: data race HttpCookieUtilityTests.swift:18 in
(1) suspend resume partial function for
HttpCookieUtilityTests.test_websiteDataStore_is_created_from_cookies()
When I run the same code from a normal application, I don't see the same ThreadSanitizer warnings. I'm assuming that running XCTests that require the main thread are not supported or are somehow problematic. But I wanted to outrule issues with the actual code. I'm also just wondering if async/await is creating unexpected complications.
Also, I tried this alternate implementation which does not have the ThreadSanitizer issue, and is also significantly faster (presumably because the setCookie operations can process concurrently), but it specifically avoids using the modern concurrency support (aside from the continuation).
class HttpCookieUtility {
func createWebsiteDataStore(httpCookies: [HTTPCookie]) async -> WKWebsiteDataStore {
return await withCheckedContinuation{ continuation in
DispatchQueue.main.async {
let websiteDataStore = WKWebsiteDataStore.nonPersistent()
let waitGroup = DispatchGroup()
for cookie in httpCookies {
waitGroup.enter()
websiteDataStore.httpCookieStore.setCookie(cookie, completionHandler: {
waitGroup.leave()
})
}
waitGroup.notify(queue: DispatchQueue.main) {
continuation.resume(returning: websiteDataStore)
}
}
}
}
}
Note that I removed the #MainActor attribute from the HttpCookieUtility class declaration.
Related
I'm build a softphone for iOS using Swift and PJSIP.
The PJSIP documentation states: "PJLIB API should be called from a registered thread, otherwise it will raise assertion such as "Calling pjlib from unknown/external thread...". With GCD, we cannot really be sure of which thread executing the PJLIB function. Registering that thread to PJLIB seems to be a simple and easy solution, however it potentially introduces a random crash which is harder to debug. Here are few possible crash scenarios:
PJLIB's pj_thread_desc should remain valid until the registered thread stopped, otherwise crash of invalid pointer access may occur, e.g: in pj_thread_check_stack().
Some compatibility problems between GCD and PJLIB, see #1837 for more info.
If you want to avoid any possibility of blocking operation by PJLIB (or any higher API layer such as PJMEDIA, PJNATH, PJSUA that usually calls PJLIB), instead of dispatching the task using GCD, the safest way is to create and manage your own thread pool and register that thread pool to PJLIB. Or alternatively, simply use PJSUA timer mechanism (with zero delay), see pjsua_schedule_timer()/pjsua_schedule_timer2() docs for more info."
So what I did was to use the class below found here on another thread in Stack Overflow:
class Worker {
private var thread: Thread?
private let semaphore = DispatchSemaphore(value: 0)
private let lock = NSRecursiveLock()
private var queue = [() -> Void]()
public func enqueue(_ block: #escaping () -> Void) { locked { queue.append(block) }
semaphore.signal()
if thread == nil { thread = Thread(block: work) thread?.start()
}
}
private func work() { while true { semaphore.wait()
let block = locked { queue.removeFirst() } block()
}
}
return block()
}
}
So every time I have to call any function from pjsip I use
worker.enqueue { }
For example:
worker.enqueue {
var threadError: NSError = NSError()
if !createPjSipThread(error: &threadError) {
print(threadError)
}
}
worker.enqueue { var error = NSError()
if !PjApp.shared().start(appDir: "", error: &error {
print(error)
}
}
Without the worker thread, the app crashes constantly (specially on TestFlight/Production Build).
Any ideias on what I'm missing or doing wrong? Thanks in advance.
I am trying to implement upload mechanism for my application. However, I have a concurrency issue I couldn't resolve. I sent my requests using async/await with following code. In my application UploadService is creating every time an event is fired from some part of my code. As an example I creation of my UploadService in a for loop. The problem is if I do not use NSLock backend service is called multiple times (5 in this case because of loop). But if I use NSLock it never reaches the .success or .failure part because of deadlock I think. Could someone help me how to achieve without firing upload service multiple times and reaching success part of my request.
final class UploadService {
/// If I use NSLock in the commented lines it never reaches to switch result so can't do anything in success or error part.
static let locker = NSLock()
init() {
Task {
await uploadData()
}
}
func uploadData() async {
// Self.locker.lock()
let context = PersistentContainer.shared.newBackgroundContext()
// It fetches data from core data to send it in my request
guard let uploadedThing = Upload.coreDataFetch(in: context) else {
return
}
let request = UploadService(configuration: networkConfiguration)
let result = await request.uploadList(uploadedThing)
switch result {
case .success:
print("success")
case .failure(let error as NSError):
print("error happened")
}
// Self.locker.unlock()
}
}
class UploadExtension {
func createUploadService() {
for i in 0...4 {
let uploadService = UploadService()
}
}
}
A couple of observations:
Never use locks (or wait for semaphores or dispatch groups, etc.) to attempt to manage dependencies between Swift concurrency tasks. This is a concurrency system predicated upon the contract that threads can make forward progress. It cannot reason about the concurrency if you block threads with mechanisms outside of its purview.
Usually you would not create a new service for every upload. You would create one and reuse it.
E.g., either:
func createUploadService() async {
let uploadService = UploadService()
for i in 0...4 {
await uploadService.uploadData(…)
}
}
Or, more likely, if you might use this same UploadService later, do not make it a local variable at all. Give it some broader scope.
let uploadService = UploadService()
func createUploadService() async {
for i in 0...4 {
await uploadService.uploadData(…)
}
}
The above only works in simple for loop, because we could simply await the result of the prior iteration.
But what if you wanted the UploadService keep track of the prior upload request and you couldn’t just await it like above? You could keep track of the Task and have each task await the result of the previous one, e.g.,
actor UploadService {
var task: Task<Void, Never>? // change to `Task<Void, Error>` if you change it to a throwing method
func upload() {
…
task = Task { [previousTask = task] in // capture copy of previous task (if any)
_ = await previousTask?.result // wait for it to finish before starting this one
await uploadData()
}
}
}
FWIW, I made this service with some internal state an actor (to avoid races).
Since creating Task {} is part of structured concurrency it inherits environment (e.g MainThread) from the scope where it was created,try using unstructured concurrency's Task.detached to prevent it from runnning on same scope ( maybe it was called on main thread ) - with creating Task following way:
Task.detached(priority: .default) {
await uploadData()
}
I have a async function func doWork(id: String) async throws -> String. I want to call this function from a concurrent dispatch queue like this to test some things.
for i in 1...100 {
queue.async {
obj.doWork(id: "foo") { result, error in
...
}
}
}
I want to do this because queue.async { try await obj.doWork() } is not supported. I get an error:
Cannot pass function of type '#Sendable () async throws -> Void' to parameter expecting synchronous function type
But the compiler does not provide me with a completion handler version of doWork(id:). When I call it from Obj C, I am able to use the completion handler version: [obj doWorkWithId: #"foo" completionHandler:^(NSString * val, NSError * _Nullable error) { ... }]
How do I do something similar in Swift?
You can define a DispatchQueue. This DispatchQueue will wait for the previous task to complete before proceeding to the next.
let queue = DispatchQueue(label: "queue")
func doWork(id: String) async -> String {
print("Do id \(id)")
return id
}
func doWorksConcurrently() {
for i in 0...100 {
queue.async {
Task.init {
await doWork(id: String(i))
}
}
}
}
doWorksConcurrently()
You are initiating an asynchronous task and immediately finishing the dispatch without waiting for doWork to finish. Thus the dispatch queue is redundant. One could do:
for i in 1...100 {
Task {
let results = try await obj.doWork(id: "foo")
...
}
}
Or, if you wanted to catch/display the errors:
for i in 1...100 {
Task {
do {
let results = try await obj.doWork(id: "foo")
...
} catch {
print(error)
throw error
}
}
}
Now, generally in Swift, we would want to remain within structured concurrency and use a task group. But if you are trying to mirror what you'll experience from Objective-C, the above should be sufficient.
Needless to say, if your Objective-C code is creating a queue solely for the purpose for calling the completion-handler rendition of doWork, then the queue is unnecessary there, too. But we cannot comment on that code without seeing it.
Is it possible to set a condition on the next queue of DispatchQueue? Supposed there are 2 API calls that should be executed synchronously, callAPI1 -> callAPI2. But, callAPI2 should be only executed if callAPI1 returning true. Please check code below for more clear situation:
let dispatchQueue: DispatchQueue = DispatchQueue(label: "queue")
let dispatchGroup = DispatchGroup()
var isSuccess: Bool = false
dispatchGroup.enter()
dispatchQueue.sync {
self.callAPI1(completion: { (result) in
isSuccess = result
dispatchGroup.leave()
}
}
dispatchGroup.enter()
dispatchQueue.sync {
if isSuccess { //--> This one always get false
self.callAPI2(completion: { (result) in
isSuccess = result
dispatchGroup.leave()
})
} else {
dispatchGroup.leave()
}
}
dispatchGroup.notify(queue: DispatchQueue.main, execute: {
completion(isSuccess) //--> This one always get false
})
Currently above code always returning isSuccess as false despite on callAPI1's call returning true, which cause only callAPI1's is called.
All non-playground code typed directly into answer, expect little errors.
It appears that you are trying to make an asynchronous call into a synchronous one, and the way you are attempting this simply will not work. Assuming callAPI1 is asynchronous then after:
self.callAPI1(completion: { (result) in
isSuccess = result
}
the completion block has (in all probability) not yet been run, you cannot test isSuccess immediately, as in:
self.callAPI1(completion: { (result) in
isSuccess = result
}
if isSuccess
{
// in all probability this will never be reached
}
Wrapping the code into a synchronous block will have no effect whatsoever:
dispatchQueue.sync
{
self.callAPI1(completion: { (result) in
isSuccess = result
}
// at this point, in all probability, the completion block
// has not yet run, therefore...
}
// at this point it has also not run
A sync dispatch just runs its block on a different queue and waits for it to complete; if that block contains asynchronous code, as yours does, then it is not magically made synchronous - it executes asynchronously as normal, the synchronously dispatched block terminates, the sync dispatch returns, and your code continues. The sync dispatch has no real effect (apart from running the block on a different queue while blocking the current one).
If you need to sequence a number of asynchronous calls you can do it a number of ways. One method is to simply chain the calls through the completion blocks. Using this approach your code becomes:
self.callAPI1(completion: { (result) in
if !result { completion(false) }
else
{
self.callAPI2(completion: { (result) in
completion(result)
}
}
}
Using Semaphores
If you have a long sequence of such calls using the above pattern then the code can become very nested, in such a case instead of nesting you can use semaphores to sequence the calls. A simple semaphore can be used to block (thread) execution, using wait(), until it is signalled (by an unblocked thread), using signal().
Notice the emphasis here on blocking, once you introduce the ability to block execution all sorts of issues have to be considered: among them are UI responsiveness - blocking the UI thread is not good; deadlock - for example if the code that will issue semaphore wait and signal operations is executing on the same thread then after a wait there will be no signal...
Here is a sample Swift Playground script to demonstrate using semaphores. The pattern follows your original code but uses a semaphore in addition to your boolean.
import Cocoa
// some convenience functions for our dummy callAPI1 & callAPI2
func random(_ range : CountableClosedRange<UInt32>) -> UInt32
{
let lower = range.lowerBound
let upper = range.upperBound
return lower + arc4random_uniform(upper - lower + 1)
}
func randomBool() -> Bool
{
return random(0...1) == 1
}
class Demo
{
// grab the global concurrent utility queue to schedule our work on
let workerQueue = DispatchQueue.global(qos : .utility)
// dummy callAPI1, just pauses and then randomly return success or failure
func callAPI1(_ completion : #escaping (Bool) -> Void) -> Void
{
// do the "work" on workerQueue, which is concurrent so other work
// can be executing, or *blocked*, on the same queue
let pause = random(1...2)
workerQueue.asyncAfter(deadline: .now() + Double(pause))
{
// produce a random success result
let success = randomBool()
print("callAPI1 after \(pause) -> \(success)")
completion(success)
}
}
func callAPI2(_ completion : #escaping (Bool) -> Void) -> Void
{
let pause = random(1...2)
workerQueue.asyncAfter(deadline: .now() + Double(pause))
{
let success = randomBool()
print("callAPI2 after \(pause) -> \(success)")
completion(success)
}
}
func runDemo(_ completion : #escaping (Bool) -> Void) -> Void
{
// We run the demo as a standard async function
// which doesn't block the main thread
workerQueue.async
{
print("Demo starting...")
var isSuccess: Bool = false
let semaphore = DispatchSemaphore(value: 0)
// do the first call
// this will asynchronously execute on a different thread
// *including* its completion block
self.callAPI1
{ (result) in
isSuccess = result
semaphore.signal() // signal completion
}
// we can safely wait for the semaphore to be
// signalled as callAPI1 is executing on a different
// thread so we will not deadlock
semaphore.wait()
if isSuccess
{
self.callAPI2
{ (result) in
isSuccess = result
semaphore.signal() // signal completion
}
semaphore.wait() // wait for completion
}
completion(isSuccess)
}
}
}
Demo().runDemo { (result) in print("Demo result: \(result)") }
// For the Playground
// ==================
// The Playground can terminate a program run once the main thread is done
// and before all async work is finished. This can result in incomplete execution
// and/or errors. To avoid this we sleep the main thread for a few seconds.
sleep(6)
print("All done")
// Run the Playground multiple times, the results should vary
// (different wait times, callAPI2 may not run). Wait until
// the "All done"" before starting next run
// (i.e. don't push stop, it confuses the Playground)
Or...
Another approach to avoid the nesting is to design functions (or operators) which take two async methods and produce a single one by implementing the nesting pattern. Long nested sequences can then be reduce to more linear sequences. This approach is left as an exercise.
HTH
ThreadSanitizer detects a data race in the following Swift program run on macOS:
import Dispatch
class Foo<T> {
var value: T?
let queue = DispatchQueue(label: "Foo syncQueue")
init(){}
func complete(value: T) {
queue.sync {
self.value = value
}
}
static func completeAfter(_ delay: Double, value: T) -> Foo<T> {
let returnedFoo = Foo<T>()
let queue = DispatchQueue(label: "timerEventHandler")
let timer = DispatchSource.makeTimerSource(queue: queue)
timer.setEventHandler {
returnedFoo.complete(value: value)
timer.cancel()
}
timer.scheduleOneshot(deadline: .now() + delay)
timer.resume()
return returnedFoo
}
}
func testCompleteAfter() {
let foo = Foo<Int>.completeAfter(0.1, value: 1)
sleep(10)
}
testCompleteAfter()
When running on iOS Simulator, ThreadSanitizer does not detect a race.
ThreadSanitizer output:
WARNING: ThreadSanitizer: data race (pid=71596)
Read of size 8 at 0x7d0c0000eb48 by thread T2:
#0 block_destroy_helper.5 main.swift (DispatchTimerSourceDataRace+0x0001000040fb)
#1 _Block_release <null>:38 (libsystem_blocks.dylib+0x000000000951)
Previous write of size 8 at 0x7d0c0000eb48 by main thread:
#0 block_copy_helper.4 main.swift (DispatchTimerSourceDataRace+0x0001000040b0)
#1 _Block_copy <null>:38 (libsystem_blocks.dylib+0x0000000008b2)
#2 testCompleteAfter() -> () main.swift:40 (DispatchTimerSourceDataRace+0x000100003981)
#3 main main.swift:44 (DispatchTimerSourceDataRace+0x000100002250)
Location is heap block of size 48 at 0x7d0c0000eb20 allocated by main thread:
#0 malloc <null>:144 (libclang_rt.tsan_osx_dynamic.dylib+0x00000004188a)
#1 _Block_copy <null>:38 (libsystem_blocks.dylib+0x000000000873)
#2 testCompleteAfter() -> () main.swift:40 (DispatchTimerSourceDataRace+0x000100003981)
#3 main main.swift:44 (DispatchTimerSourceDataRace+0x000100002250)
Thread T2 (tid=3107318, running) created by thread T-1
[failed to restore the stack]
SUMMARY: ThreadSanitizer: data race main.swift in block_destroy_helper.5
Is there anything suspicious with the code?
The comment from #Rob made me think again about the issue. I came up with the following modification for static func completeAfter - which ThreadSanitizer is happy with *):
static func completeAfter(_ delay: Double, value: T) -> Foo<T> {
let returnedFoo = Foo<T>()
let queue = DispatchQueue(label: "timerEventHandler")
queue.async {
let timer = DispatchSource.makeTimerSource(queue: queue)
timer.setEventHandler {
returnedFoo.complete(value: value)
timer.cancel()
}
timer.scheduleOneshot(deadline: .now() + delay)
timer.resume()
}
return returnedFoo
}
This change ensures that all accesses to timer will be executed in queue queue, which tries to synchronise the timer that way. Even though, this same solution in my "real" code didn't work with this solution, it's probably due to other external factors.
*) We should never think, our code has no races, just because ThreadSanitizer doesn't detect one. There may be external factors which just happen to "erase" a potential data race (for example, dispatch lib happens to execute two blocks with a conflicting access on the same thread - and no data race can happen)