I make calls to the iCloud database in two different ways in my app:
1.
Calling a CKDatabase convenience method works perfectly in development (simulator) and production (device) environments:
let privateDatabase = CKContainer.defaultContainer().privateCloudDatabase
privateDatabase.fetchRecordWithID(recordID, completionHandler: { (record, error) -> Void in
// handle record and errors
})
2.
Adding CKOperations to the main queue works perfectly in the development environment, and works for a while when I test in the production environment - but then arbitrarily, after a few hours of intermittent testing, this call to the database just starts hanging - no errors are produced, and no completion code is executed. (The fetchRecordWithID call still works perfectly all the time.)
let op1 = CKModifyRecordsOperation(recordsToSave: records, recordIDsToDelete: nil)
op1.modifyRecordsCompletionBlock = { (savedRecords, deletedRecordIDs, error) -> Void in
// handle records and errors
}
op1.database = privateDatabase
// let op2, op3...
NSOperationQueue.mainQueue().addOperations([
op1,
op2,
op3,
], waitUntilFinished: false)
In the Usage panel in the CloudKit dashboard, nothing is even remotely approaching a limit. What other variables am I not considering - or what other tests can I run to figure out what's going wrong?
Try setting the op1, op2 and op3.qualityOfService = .UserInitiated
That setting has been set to a 'lower' standard value in iOS 9.
The documentation states for the qualityOfService:
The relative amount of importance for granting system resources to the
operation. Service levels affect the priority with which an operation
object is given access to system resources such as CPU time, network
resources, disk resources, and so on. Operations with a higher quality
of service level are given greater priority over system resources so
that they may perform their task more quickly. You use service levels
to ensure that operations responding to explicit user requests are
given priority over less critical work. This property reflects the
minimum service level needed to execute the operation effectively. The
default value of this property is
NSOperationQualityOfServiceBackground and you should leave that value
in place whenever possible. When changing the service level, use the
minimum level that is appropriate for executing the corresponding
task. For example, if the user initiates a task and is waiting for it
to finish, assign the value NSOperationQualityOfServiceUserInitiated
to this property. The system may give the operation a higher service
level to the operation if the resources are available to do so.
Related
In summary:
The cascade effect nature of the Cold Stream, from Inactive to Active, Is in itself an "alien" execution (alien to the reactive design) that MUST BE EXECUTED WITHIN THE SYNCRHONIZED REGION, and this is unavoidable, going against Item 79 of Effective Java.
Effective Java: Item 79:
"..., to avoid deadlock and data corruption, never call an alien
method from within a synchronized region. More generally, keep the
amount of work that you do from within synchronized to a minimum."
never call an alien
method from within a synchronized region
An add(Consumer<T> observer) AND remove(Consumer<T> observer) WILL BE concurrent (because of switchMaps that react to asynchronous changes in values/states), BUT according to Item 79, it should not be possible for a subscribe(Publisher p); method to even exist.
Since a subscribe(publisher) MUST WORK as a callback function that reacts to additions and removals of observers...
private final Object lock = new Object();
private volatile BooleanConsumer suscriptor;
public void subscribe(Publisher p) {
syncrhonized (lock) {
suscriptor = isActive -> {
if (isActive) p.add(this);
else p.remove(this);
}
}
}
public void add(Consumer<T> observer) {
syncrhonized (lock) {
observers.add(observer);
if (observer.size() > 0) suscriptor.accept(true);
}
}
I would argue that using a volatile mediator is better than holding on to the Publisher directly, but holding on to the publisher makes no difference at all, because by altering its state (when adding ourselves to the publisher) we are triggering the functions (other possible subscriptions to publishers) within it!!!, There really is no difference.
Doing it via misdirection is the proper answer, and doing so is the main idea behind the separation of concerns principle!!
Instead, what Item 79 is asking, is that each time an observer is added, we manually synchronize FROM THE OUT/ALIEN-SIDE, and deliberately check whether a subscription must be performed.
synchronized (alienLock) {
observable.add(observer);
if (observable.getObserverSize() > 0) {
publishier.add(observable);
}
}
and each time an observer is removed:
synchronized (alienLock) {
observable.remove(observer);
if (observable.getObserverSize() == 0) {
publishier.remove(observable);
}
}
Imagine those lines repeated EACH and EVERY TIME a node forks or joins on to a new one (in the reactive graph), it would be an insane amount of boilerplate defeating the entire purpose.
Reading carefully the item, you can see that the rule is there to prevent a "something" done wrong by the user that hangs the thread preventing accesses.
And this answer will be part me trying to justify why this is possible but also a non-issue in this case.
Binary Events.
In this case an event that involves only 2 states, isActive == true || false;
This means that if the consumption gets "hanged" on true, the only other option that may be waiting is "false", BUT even worst...
IF one of the two becomes deadlocked, it means the entire system is deadlocked anyways... in reality the issue is outside the design, not inside.
What I mean is that out of the 2 possible options: true or false. the time it takes for either of them to execute is meaningless since the ONLY OTHER OPTION IS STILL REQUIRED TO WAIT regardless.
Enclosed functionality of the lock.
Even if subscribe(Publisher p) methods can be concatenated, the only thing the user has access to, IS NOT THE lock per se, but the method.
So even If we are executing "alien methods" with functions, inside our synchronized bodies, THOSE FUNCTIONS ARE STILL OURS, and we know what they have inside them and how they work and what exactly they will perform.
In this case the only uncertainty in the system is not what the functions will do, but HOW MANY CONTATENATIONS THE SYSTEM HAS.
What's wrong in my code:
Finally, the only thing (I see) wrong, is that observers and subscriptions MOST DEFINITEY WORK IN SEPARATE LOCKS, because observers MUST NOT, under any circumstance should allow themselves to get locked while a subscription domino effect is taking place.
I believe that's all...
We use Xodus for a remote probe project in order to store temporary data before sending them to the centralized database. Thus, we have several stores which can grow or decrease accordingly to the environment (traffic, network connection, etc...). Thanks to the garbage collector, we expected to see decrease in the database file size but for the moment, it has only increased.
We tried several garbage collector configurations to trigger it as frequently as possible. For example, we have :
conf.setGcFileMinAge(1);
conf.setGcFilesInterval(1);
conf.setGcMinUtilization(1);
Without visible effects...
After the store has been emptied, we expected to see reducing or deletion of .xd files but the database keeps growing and growing.
EDIT :
I try to see GC effects with a simpler code as below :
Environment exodus = Environments.newInstance(dbPath);
final Transaction xtxn = exodus.beginExclusiveTransaction();
Store store = exodus.openStore("testStore", StoreConfig.WITHOUT_DUPLICATES, xtxn);
xtxn.commit();
Thread.sleep(10 * 1000); // Wait to do actions after first background cleaning cycle
// Fill store, then clear it
exodus.executeInExclusiveTransaction(tx -> {
for(int i = 1; i <= 1000000; i++) {
store.putRight(tx, LongBinding.longToEntry(i), StringBinding.stringToEntry(dbPath));
}
});
clearStore(exodus, store);
exodus.gc();
Thread.sleep(5 * 60 * 1000); // Wait to see GC doing the work
boolean clearStore(final Environment exodus, final Store store) {
Transaction tx = exodus.beginExclusiveTransaction();
try(Cursor cursor = store.openCursor(tx)) {
boolean success = true;
while(cursor.getNext() && success) {
success &= cursor.deleteCurrent();
}
if(success) {
tx.commit();
return true;
} else {
log.warn("failed to delete entry {}", cursor.getKey());
tx.abort();
return false;
}
} catch(Exception e) {
tx.abort();
return false;
}
}
If I remove the first "sleep", Garbage Collector is doing the work, the database file size is reduced as expected, everything is ok.
But if I keep the first "sleep", Garbage Collector never seems to be called.
It's like the first background cleaning cycle is ok, but not the following ones...
I keep default configuration in this example.
There is the Environment.gc() method. The javadoc for the method is as follows:
Says environment to quicken background database garbage collector activity. Invocation of this method doesn't have immediate consequences like freeing disk space, deleting particular files, etc.
I wouldn't recommend modifying default GC settings. EnvironmentConfig.setGcMinUtilization() can be used to keep the database more compact than it would be by default, or to decrease GC load (e.g., in parallel with batch updates). Basically, higher required minimum utilization (less admissible free space) results in higher GC load.
GC cleans the database file by file, selecting files with least utilization first. When a file is cleaned it is not deleted immediately, two conditions should be satisfied:
A delay configured by EnvironmentConfig.getGcFilesDeletionDelay() should pass. It's 5 seconds by default.
Any transaction (even read-only), created before the moment when the file is cleaned, should be finished (committed or aborted).
I'm doing some performance experiments and I keep getting this error with my Xcode UI Tests since no new UI Test statement is hit:
UITesting Failure - Failed to get snapshot within 15.0s
How can I change this 15.0s variable to something longer? Is there some configuration or setting that I can change?
You can use expectations to allocate an arbitrary amount of time to run any test case. You just create an additional expectation right before 15.0s has passed, and repeat this process as many times as necessary. Here's a short code sample to illustrate:
var timeToDelay = 60.0
repeat {
let delay = min(13.0, timeToDelay)
timeToDelay -= delay
let date = Date().addingTimeInterval(delay)
let predicate = NSPredicate(format: "now() > %#", argumentArray: [date])
self.expectation(for: predicate, evaluatedWith: [], handler: nil)
self.waitForExpectations(timeout: 14.0, handler: nil)
} while timeToDelay > 0
Place this right before you see the testing failure and replace timeToDelay with the amount of additional time you need (in seconds).
I don't think this is possible. Apple has a system limit of 20 seconds for any app to start up, if it takes longer than that your app will be killed. Source
Because UI tests require bootstrapping your app, Apple has given itself ~5 seconds to perform all of it's setup as well. So essentially you have a 15 second limit to launch your app.
Because it's not possible to override the system's 20 second limit, it's also not going to be possible to override XCTest's 15 second limit.
I've started working with CloudKit and finally started using subclassed NSOperation for most of my async stuff.
How ever, I have two questions.
How can I mark an operation as failed? That is, if operation A fails, I don't wan't its dependent operations to run. Can I just not mark it as isFinished? What happens to the unexecuted items already in the queue?
What would be the recommended route to take if I would like something like a try, catch, finally. The end goal is to have one last operation that can display some UI with information of success or report errors back to the user?
isFinished means your operations complete execution, you can cancel an operation but that means your operation is canceled and that could be done without even executing the operation and you can check that by calling isCanceled, if you want spefically failure and success flags after executing an NSOperation then in subclass add isFailure property and check in dependent operation before executing, and cancel that if isFailure is set to true.
You can add dependency on operation and check there status, and if all are successful just update UI on main thread or report and error.
Update
You can keep and array of dependent operations and when on failure you can cancel those operations.
Add the operations to a queue. When one of them fails call cancel on the queue. Future CKOperations will error on start with "operation was cancelled before it started" and any block operations won't even run.
One way is to use KVO on the queue's operationCount (Example on Github) and wait until it is zero. Then if you got an error at any stage (and captured it) you can make a final callback operation ended with the error. However, usually you will end up wanting to display a different message depending on the operation that caused the error, in which case it is best to handle them when they happen, rather than wait until the end when you then need to figure out which error came from which operation.
Overview:
When an operation fails (based on your business logic) and you want to abort all dependant operations then you can cancel dependant operations manually
Order in which you cancel operation is important as cancelling an operation would allow for the dependant operations to start (as the dependancy condition has been broken).
In order to all this you need to have variables holding each of those dependant operations, so that you cancel them in the order you intend.
Code:
var number = 0
let queue = OperationQueue()
let b = BlockOperation()
let c = BlockOperation()
let d = BlockOperation()
let a = BlockOperation {
if number == 1 {
//Assume number should not be 1
//If number is 1, then it is considered as a failure
//Cancel the remaining operations manually in the reverse order
//Reverse order is important because if you cancelled b first, it would start c
d.cancel()
c.cancel()
b.cancel()
}
}
b.addDependency(a)
c.addDependency(b)
d.addDependency(c)
queue.addOperation(a)
queue.addOperation(b)
queue.addOperation(c)
queue.addOperation(d)
I've implemented a Web role that writes to a queue. This is working fine. Then I developed a Worker role to read from the queue. When I run it in debug mode from my local machine it reads the messages from the queue fine, but when i deploy the Worker role it dos'nt seem to be reading the queue as the message eventually end up in the dead letter queue. Anyone know what could be causing this behavior? Below are some bit that might be key in figuring this thing out
queueClient = QueueClient.Create(queueName, ReceiveMode.PeekLock);
var queueDescription = new QueueDescription(QueueName)
{
RequiresSession = false,
DefaultMessageTimeToLive = TimeSpan.FromMinutes(2),
EnableDeadLetteringOnMessageExpiration = true,
MaxDeliveryCount = 20
};
Increase the QueueDescription.DefaultmessageTimeToLive to ~10 mins.
This property dictates how much time a message should live in the Queue - before being processed (Message.Complete() is called). If it remains in the queue for more than 2 mins - it will be automatically moved to DeadLetterQueue (as you had Set EnableDeadLetteringOnMsgExp to true).
TTL is useful in these messaging scenarios
if a message is not being processed after N mins after it arrived -then it might not be useful to process it any more
if the message was attempted to process many times and was never completed (Reciever - msg.Complete()) - this might be needing special processing
So - to be safe have a bit higher value of DefaultMsgTTL.
Hope it Helps!
Sree