Should any reads (SELECT) made to Table1 by another thread, after the transaction is started but before the UPDATE is executed, be blocked, or does the UPDATE need to first begin execution?
var _con = new SqliteConnection( "Data Source=" + FileName );
_con.Open();
SqliteCommand _cmd = _con.CreateCommand();
_cmd.CommandType = CommandType.Text;
_con.BeginTransaction(IsolationLevel.Serializable);
// UPDATE
_cmd.CommandText = "UPDATE Table1 SET field1 = 'a' WHERE Id = 1";
_cmd.ExecuteReader();
_cmd.Transaction.Commit();
In SQLite, all transactions (explicit and automatics ones) are serializable.
By default, SQLite's transactions are DEFERRED, which means that read/write locks are taken only when the database file actually needs to be read/written (and writes usually happen only when the caches is flushed when the transaction commits).
In journal rollback mode, readers and a writer block each other; in WAL mode, readers and a writer can access the DB concurrently, but there can still be only one writer.
However, Xamarin does not use the defaults; BeginTransaction() immediately starts a transaction in EXCLUSIVE mode, which means that it conflicts with all other concurrent transactions. (WAL mode makes no difference.)
Apparently, you are not supposed to use any concurrency in Xamarin.
Related
Stored procedures in Cosmos DB are transactional and run under isolation snapshop with optimistic concurrency control. That means that write conflicts can occur, but they are detected so the transaction is rolled back.
If such a conflict occurs, does Cosmos DB automatically retry the stored procedure, or does the client receive an exception (maybe a HTTP 412 precondition failure?) and need to implement the retry logic itself?
I tried running 100 instances of a stored procedures in parallel that would produce a write conflict by reading the a document (without setting _etag), waiting for a while and then incrementing an integer property within that document (again without setting _etag).
In all trials so far, no errors occurred, and the result was as if the 100 runs were run sequentially. So the preliminary answer is: yes, Cosmos DB automatically retries running an SP on write conflicts (or perhaps enforces transactional isolation by some other means like locking), so clients hopefully don't need to worry about aborted SPs due to conflicts.
It would be great to hear from a Cosmos DB engineer how this is achieved: retry, locking or something different?
You're correct in that this isn't properly documented anywhere. Here's how OCC check can be done in a stored procedure:
function storedProcedureWithEtag(newItem)
{
var context = getContext();
var collection = context.getCollection();
var response = context.getResponse();
if (!newItem) {
throw 'Missing item';
}
// update the item to set changed time
newItem.ChangedTime = (new Date()).toISOString();
var etagForOcc = newItem._etag;
var upsertAccecpted = collection.upsertDocument(
collection.getSelfLink(),
newItem,
{ etag: etagForOcc }, // <-- Pass in the etag
function (err2, feed2, options2) {
if (err2) throw err2;
response.setBody(newItem);
}
);
if (!upsertAccecpted) {
throw "Unable to upsert item. Id: " + newItem.id;
}
}
Credit: https://peter.intheazuresky.com/2016/12/22/documentdb-optimistic-concurrency-in-a-stored-procedure/
SDK does not retry on a 412, 412 failures are related to Optimistic Concurrency and in those cases, you are controlling the ETag that you are passing. It is expected that the user handles the 412 by reading the newest version of the document, obtains the newer ETag, and retries the operation with the updated value.
Example for V3 SDK
Example for V2 SDK
How does one run SQL queries with different column dimensions in parallel using dask? Below was my attempt:
from dask.delayed import delayed
from dask.diagnostics import ProgressBar
import dask
ProgressBar().register()
con = cx_Oracle.connect(user="BLAH",password="BLAH",dsn = "BLAH")
#delayed
def loadsql(sql):
return pd.read_sql_query(sql,con)
results = [loadsql(x) for x in sql_to_run]
dask.compute(results)
df1=results[0]
df2=results[1]
df3=results[2]
df4=results[3]
df5=results[4]
df6=results[5]
However this results in the following error being thrown:
DatabaseError: Execution failed on sql: "SQL QUERY"
ORA-01013: user requested cancel of current operation
unable to rollback
and then shortly thereafter another error comes up:
MultipleInstanceError: Multiple incompatible subclass instances of TerminalInteractiveShell are being created.
sql_to_run is a list of different sql queries
Any suggestions or pointers?? Thanks!
Update 9.7.18
Think this is more a case of me not reading documentation close enough. Indeed the con being outside the loadsql function was causing the problem. The below is the code change that seems to be working as intended now.
def loadsql(sql):
con = cx_Oracle.connect(user="BLAH",password="BLAH",dsn = "BLAH")
result = pd.read_sql_query(sql,con)
con.close()
return result
values = [delayed(loadsql)(x) for x in sql_to_run]
#MultiProcessing version
import dask.multiprocessing
results = dask.compute(*values, scheduler='processes')
#My sample queries took 56.2 seconds
#MultiThreaded version
import dask.threaded
results = dask.compute(*values, scheduler='threads')
#My sample queries took 51.5 seconds
My guess is, that the oracle client is not thread-safe. You could try running with processes instead (by using the multiprocessing scheduler, or the distributed one), if the conn object serialises - this may be unlikely. More likely to work, would be to create the connection within loadsql, so it gets remade for each call, and the different connections hopefully don't interfere with one-another.
It looks like previously working approach is deprecated now:
unsupported.dbms.executiontime_limit.enabled=true
unsupported.dbms.executiontime_limit.time=1s
According to the documentation new variables are responsible for timeouts handling:
dbms.transaction.timeout
dbms.transaction_timeout
At the same time the new variables look related to the transactions.
The new timeout variables look not working. They were set in the neo4j.conf as follows:
dbms.transaction_timeout=5s
dbms.transaction.timeout=5s
Slow cypher query isn't terminated.
Then the Neo4j plugin was added to model a slow query with transaction:
#Procedure("test.slowQuery")
public Stream<Res> slowQuery(#Name("delay") Number Delay )
{
ArrayList<Res> res = new ArrayList<>();
try ( Transaction tx = db.beginTx() ){
Thread.sleep(Delay.intValue(), 0);
tx.success();
} catch (Exception e) {
System.out.println(e);
}
return res.stream();
}
The function served by the plugin is executed with neoism Golang package. And the timeout isn't triggered as well.
The timeout is only honored if your procedure code invokes either operations on the graph like reading nodes and rels or explicitly checks if the current transaction is marked as terminate.
For the later, see https://github.com/neo4j-contrib/neo4j-apoc-procedures/blob/master/src/main/java/apoc/util/Utils.java#L41-L51 as example.
According to the documentation the transaction guard is interested in orphaned transactions only.
The server guards against orphaned transactions by using a timeout. If there are no requests for a given transaction within the timeout period, the server will roll it back. You can configure the timeout in the server configuration, by setting dbms.transaction_timeout to the number of seconds before timeout. The default timeout is 60 seconds.
I've not found a way how to trigger timeout for a query which isn't orphaned with a native functionality.
#StefanArmbruster pointed a good direction. The timeout triggering functionality can be got with creating a wrapper function in Neo4j plugin like it is made in apoc.
I am working on a seat booking module where the user checks whether the seats are available in exam and if yes he can proceed to payment page an then enter card details and pay. Now what I want is that when the payment page is opened seats available value should be reduced by 1 for some time(till session expires or user navigates to some other page).
If I do it using database transaction for the create action(MVC) it won't be possible as when the controller's action(In grails) will be triggered and database changes would be saved without any possibility of rollback.
Is it feasible to acquire seat lock using db or should I use some cron job to check it. Please suggest.
I'm assuming that an exam should never be over-booked. In other words, that exactness is more important than concurrency. In such a case, pessimistic locking strategy is the best bet. Note: by default, Grails uses optimistic locking.
You can use database pessimistic locking to assist with allocating seats, but on it's own, database locking is ill-suited for business transaction locking. As you implied, the lock is released when the transaction commits. So what you can do is implement your business transaction locking on top of database pessimistic locking.
Allocating/locking seats
I recommend using a service to allocate/deallocate seats so that all of the locking code is done in one place.
import groovy.time.TimeDuration
class ExamService {
TimeDuration allocationExpiration = new TimeDuration(0, 30, 0, 0) // 30 minutes
SeatsAllocation allocateSeats(long examId, long userId, int numberOfSeats) {
def exam = Exam.lock(examId)
if(exam.seatsAvailable >= numberOfSeats) {
exam.seatsAvailable = exam.seatsAvailable - numberOfSeats
exam.save()
return new SeatsAllocation(
exam: exam,
user: User.get(userId),
numberOfSeats: numberOfSeats,
purchased: false,
expiration: new Date() + allocationExpiration).save()
} else {
return null
}
}
}
In this example, allocateSeats() is called when the user decides to purchase a certain number of seats, but has not actually made the purchase yet. In this stage of the transaction, a SeatsAllocation represents a user's intent to purchase. The allocation has an expiration date which must be checked at the exact moment the purchase is made. If the allocation has expired, the purchase should be disallowed. The purchasing code is not shown, but basically make sure to set SeatsAllocation.purchased to true.
Deallocating/unlocking seats
This is a good place for the cron/Quartz job you mentioned. The job would retrieve the expired SeatsAllocations, make the seats available, and finally delete the allocations. In the following example, I have the code as part of the service, which the Quartz job can call.
import groovy.time.TimeDuration
class ExamService {
...
def deallocateExpiredSeats() {
SeatsAllocation.where {
purchased == false
expiration < new Date()
}.list([lock: true])
.each {
it.exam.seatsAvailable = it.exam.seatsAvailable + it.numberOfSeats
it.exam.save()
it.delete()
}
}
}
Note that using a scheduled job presents the problem of a delay between the SeatsAllocations expiring and those seats being made available. If the job runs too frequently it will conflict with the pessimistic locking used in allocateSeats(), reducing concurrency even further. On the other hand, if the job runs too infrequently, seats which technically are available cannot be allocated to potential customers because they'll be held hostage by the SeatsAllocations. You can choose the frequency based on the app's load.
Consider the following code:
if (!serpKeyword) {
serpKeyword = new SerpKeyword(
keyword: searchKeyword,
geoKeyword: geoKeyword,
concatenation: concatenation,
locale: locale
)
serpKeyword.save(flush: true, failOnError: true)
}
serpService.submitKeyword(serpKeyword, false)
Here's the submitKeyword method:
#Transactional(propagation = Propagation.REQUIRES_NEW)
boolean submitKeyword(keywordToSubmit, boolean reset) {
def keyword = SerpKeyword.get(keywordToSubmit.id)
No error is raised when I call serpKeyword.save, but when I get into the submitKeyword method, SerpKeyword.get(keywordToSubmit.id) returns null. What could be preventing this from saving?
Edit
Changing REQUIRES_NEW to REQUIRED seems to do the trick. Here's what I think is happening.
The code that calls serpService.submitKeyword is located within a service method. From what I understand, service method's have a default propagation strategy of REQUIRED. Since all this database writes are happening within the context of a transaction, the writes are queued up in the database, but not actually executed against the database until the transaction is completed, according to the docs:
Note that flushing is not the same as committing a transaction. If
your actions are performed in the context of a transaction, flushing
will execute SQL updates but the database will save the changes in its
transaction queue and only finalize the updates when the transaction
commits.
We call serpService.submitKeyword before our transaction is actually finished. That method starts a completely new transaction where our serpKeyword is not available. Changing it to REQUIRED works because we are now operating within the context of our parent transaction.
I think some part of the stack is being lazy. I've ran into this behavior with Hibernate, if that's what you're using. I'm not sure it's an approved maneuver, but you could clear the session before calling submitKeyword like so:
long keywordId = serpKeyword.id
SerpKeyword.withSession{it.clear()}
serpService.submitKeyword(keywordId, false)
And then change the method to:
#Transactional(propagation = Propagation.REQUIRES_NEW)
boolean submitKeyword(keywordId, boolean reset) {
def keyword = SerpKeyword.get(keywordId)
Then I bet the .get() will work. If you have other objects in the session you need. You will need to lift those out by storing their id's and .get()ing them as well.