Just noticed that https://github.com/bubibubi/EntityFrameworkCore.Jet supports connection pooling. I am currently using https://github.com/bubibubi/JetEntityFrameworkProvider and would like to use connection pooling in the EF6 version until I have time to port the app to Core.
Can I setup connection pooling for JetEntityFrameworkProvider? If so, how? Are there any things to watch for if I do turn on connection pooling?
Actually you can't. There is a connection pooling implementation in EF Core.
In that case there the ADO .Net provider for Access is implemented in a different project and there and has been projected since the beginning. That's not the case of EF 6 provider where the ADO .Net provider has been written step by step every time I found a Microsoft Access weird behaviour.
I don't know your case but I'm not using EF Core because of the missing lazy loading implementation. Reading the backlog I think that has been implemented some days ago.
Related
I'm in process of upgrading our client-server ERP app to multi-tier. We want to offer our customers the possibility having their databases in cloud(hosted in our server).So, clients are written in Delphi, server is a http IOCP server written also in Delphi (from mORMot framework), for dbs we use Firebird embedded.
Our customers(let's say 200), can have 25-30 Firebird databases (total 5000-6000 databases), accessed by 4-5 user per each customer. This is not happening all at once. One user can work in one db, other 2 users can work in another db, but all the dbs should be available and online. So, I can have 800-1000 users working at 700-900 dbs. Databases are not big, typically 20-30 MB each but can go to 200 MB.
This is not data sharding so please don't suggest to merge all databases together, I really need them individually with possibility to backup/restore/replace each one of them.
So, I need multiple pools of connections - for every database I need a pool of let's say 2 connections. I read about Firedac connection pooling. It seems that TFDManager should be perfect for me. I define multiple "ConnectionDef"s with "Pooled=true" and it can maintain multiple pools of connections (each connection lasting until some minutes of inactivity).
Questions:
I have to create all "ConnectionDef"s before server starts serving requests?
Can TFDManager "handle" requests (and time-out connections on inactivity), while in other thread I need to create a new database , so automatically I need to create a new pool of connections and start serving requests from newly created database. Practically can I call FDManager.AddConnectionDef(..) while other pools are in use?
AFAIK Firebird embedded does not have any "connection". As its name states, it is embedded within the same process, so there is no connection pool needed. Connection pools are needed when several clients connect/disconnect to the same DB over the network, whereas here all is embedded, and you would have direct access to the Firebird engine.
As such, you may:
Define one "connnection" per Firebird embedded database;
Protect your SOA code via a mutex (aka critical section) per DB. In fact, mORMot's HTTP IOCP server would run the incoming requests from a thread pool, so ensure that all DB access is safely made.
Ensure you use at least Firebird 2.5 since the embedded version is told to be threadsafe only since revision 2.5 (see the release notes).
Instead of FireDAC, consider using ZDBC/Zeos (in latest 7.2/7.3 branch), which has nice features, together with the native mORMot SynDB libraries.
Looking at Firedac sources, seems that all about adding connection definitions and acquiring connections in pooled mode is thread-safe.
Adding a connection definition or matching one is guarded by a TMultiReadExclusiveWriteSynchronizer and acquiring a connection from the pool is guarded by a TCriticalSection.
So, answers:
I don't have to create all "ConnectionDef"s before server starts serving requests.
Yes, I can call safely FDManager.AddConnectionDef(..) while other pools are in use.
Using Firedac, acquiring a connection for any of those databases will be guarded by one TCriticalSection. The solution proposed by #Arnaud Bouchez presents a more grained access by creating one TCriticalSection per database and I think will scale better, but you should be aware of a bug when using multiple TCriticalSection, especially that all will be initiated at once:
https://www.delphitools.info/2011/11/30/fixing-tcriticalsection/
In that article present a very simple fix for this bug.
Am working on a enterprise application where multiple clients will work on the system. Used three tier architecture and in database layer we are using enterprise library 5 for all the database operation. I have declared a class level variable like this :-
SqlDatabase sqldtabase =
EnterpriseLibraryContainer.Current.GetInstance<SqlDatabase>("Beta");
and we are using sqldatabase variable for retrieving data from db and calling various Sp.
My query is how enterprise library does connection management and how it does connection pooling as well. Do we need to explicitly open or close connection or enterprise library take care of it.
If you're using ExecuteReader, for example, you'll need to explicitly close the connection.
You can change the pool size, for example set the max size in the web.config. The actual management is controlled by SQL Server.
Do you have some example code that you're not sure about?
I have a Client/Server application written Delphi. Essentially all the application is doing is transferring xml data streams between a server application and connected clients. I am currently using the Indy TIdTCPServer component. But the server side application keeps crashing on some of my installments. And it has been extremely difficult to debug. So I am wondering if there is some "architecture" I should be utilizing which does all the tcp/ip connection management and database connection pooling, allowing me to concentrate on the business logic.
Here are more details:
clients must maintain a "persistent" connection. There are times when the server must notify and send data to all connected clients.
clients are connecting from laptop computers using wireless aircards. So network "drops" are pretty common.
Backend database is SqlServer.
There can be upward of 100 computers simultaneously connected at a time.
When the server gets a new connection (TCPServer.OnConnect) I instantiate my own object containing it own SqlServer database connection. When tcp connections are dropped I in turn free these objects (and associated database connection).
Client application have a TTimer built into them. They routinely send heartbeats to the server. And if they "drop"/"lose" their connection they automatically establish a new connection once the network is back.
Anyone have any suggestions on the best approach/architecture here?
I presume the Indy component would work, but at the same time feel I am "reinventing the wheel" with respect to managing the connections.
Three component sets I am aware of that will take care of the nitty gritty technical aspects of client server applications for you:
kbmMW: http://components4developers.com/
Asta: http://www.astatech.com/index.asp
RemObjects: http://www.remobjects.com/
You may have to rework your applications to take advantage of the way these component sets work, but assuming you have properly separated layers that shouldn't be too much of a hassle and will buy you the advantage of well tested and widely used code for your client server work.
If you want some light TCP/IP components, take a look at our SynCrtSock unit.
You'll find low-level classes to create IP Client and Servers.
We implemented both TCP/IP and UDP/IP in one of our applications.
There is also a THttpServer class, which implement a HTTP/1.1 server. Therefore it follows the HTTP/1.1 connection management. There is also an optional compression, and using HTTP/1.1 on a port other than 80 is not a bad idea. And what is good with HTTP/1.1 is that it can pass through firewalls, and can be easily be VPNed or hosted on another HTTP server (like IIS or Apache) with a proxy. There is even a FastCGI class, if you need such a server under a linux-based solution.
Of course, a THttpClientSocket class does the same on the client class.
We use these classes to add HTTP/1.1 connection to our Open Source SQLite3 RESTful framework - http://synopse.info/forum/viewforum.php?id=2
See http://synopse.info/fossil/artifact?name=722e896e3d7aad1fe217b0e2e7903483e66d66d1 for the SynCrtSock unit. Open source, work from Delphi 7 to Delphi 2010.
Misha Charrett's CSI Application Framework covers pretty much exactly what you're asking for.
It's an open source Delphi framework that at its heart is a distributed message passing and threading framework that allows XML message passing from both client to server and server to client.
It can handle disconnections/reconnections, high client numbers and there's an optional virtual database library that will handle SQL server (or you could just use same SQL Server access you're using now).
It's not particularly well known yet but I can tell you that it's been actively developed over the last few years and that the author Misha is very keen to assist anyone who's interested in using it in their application.
Well, it would probably require a complete rewrite of much of your C/S code, but instead of using the Indy components, you could try to use a COM+ solution instead. Basically, you would create a COM+ component that will be installed on the server and your client applications will connect to this client and call the functions of this component directly. It will have transaction management which will be handled by Windows itself and the same is true about handling transactions. It's also technically possible to create events, which would allow the server to do callbacks to the client, although that would make things a bit more complicated.
I don't think this solution would work out for you, though, unless you have a lot of experience with COM development in Windows and/or you're brave enough to try something different.
In the past, I had a similar problem where hundreds of clients had to connect to a single server, doing all kinds of database transactions. It has a steep learning curve but me and my team managed to get things working and once we understood the technique, it resulted in a very stable and reliable solution which did manage to have up to 500 users simultaneously doing updates and other actions in a one-time extreme stress-test. But again, the learning curse is steep, so it might not be the solution you're looking for.
(Still, COM+ will use a lot of functionality that's build-in into Windows, like transaction management, database pooling and whatever more.)
If you use Indy each connection will equal a thread.
Anyway, I suggest for connecting to MSSQL to use SDAC from Devart http://www.devart.com/sdac/ and for the connection layer to use HPScktSrvr based on I/O Completion Port from http://www.torry.net/authorsmore.php?id=7131 (I don't know though what changes it will need for TThread changes in newer VCL).
You build your client class arround THPServerClient, you set your new class as the server ClientClass and the framework will create automatically new clients for you.
You may also want to have a look at the ICS/Midware combo: http://www.overbyte.be/
I want to develop windows service in Delphi that communicates with an Oracle server via ADO.
Which guidelines should I follow?
I already know ADO, but I don't have experience with the windows services.
Thanks in advance.
I think Tutorial for making a Service Application using Delphi is good point to start.
And If you want to connect database in an service app. you can create second thread in OnStart event of TService class.
As a hint: I did have lot's of problems when developing a service using ADO to an Oracle server. After some testing the problem lied in the fact that the ADO driver from Oracle was unable to handle concurrent connections in multiple threads. In the end we used a single thread to do all ADO connections and sync the resulting datasets to the threads that did the processing.
We have a Delphi 7 application that runs as an ISAPI extension in IIS6. The code use ADO to connect to a MS SQL 2000 database and performs many reads on the database (no writes). If I watch the audit login and logout events in SQL profiler I can see that numerous requests to the app result in only 1 audit login event. However, if I run that same code from outside IIS (i.e. a test app calling the same method in the dll) I see many login and logout events. My guess is that IIS is performing some automatic connection pooling without my doing anything. I'd like to see the same behavior when I run the dll from outside of IIS for performance reasons - the app is almost 100% slower in this situation. How can I get ADO connection pooling when the dll runs outside of IIS?
EDIT - I'm actually using the SQL ole provider. The connection string looks like this:
Provider=SQLOLEDB.1;Initial Catalog=%s;Data Source=%s;Password=%s;User ID=%s;Pooling=True;Min Pool Size=5;Max Pool Size=50;Connection Lifetime=120
I tried adding the Pooling=True attribute but this doesn't change things. Also, I learned that audit login and logout events don't necessarily change for connection pooling so I started using the Logins/sec, Logouts/sec and User Connections performance counters (SQLServer:GeneralStatistics) to determine if connection pooling occurs. From inside IIS I see many logins/sec and no logouts/sec. Outside of IIS I see many logins and logouts per second and user connections fluctuates (it holds steady in IIS).
It's hard to say based on the info given, but connection pooling is definitely based off of the connection string - if the connection string is exactly the same then the connection can be pooled... it sounds like your outside application may be altering the connection string?
IIS isn't pooling ADO connections. It is likely caching the ISAPI dll though. Are you starting/stopping your outside application continuously? Or is it a single run causing multiple login events?