At the heart of Petapoco.cs there is the function OpenShareConnection.
I believe this cannot take advantage of the Connection Pool in SQL Azure.
I am monitoring my connections and the connection count grows above the pool limit.
Anyone has done some improvements?
Here is the OpenShareConnection (from Petapoco open source):
public void OpenSharedConnection()
{
if (_sharedConnectionDepth == 0)
{
//read the connection string from web.config and
//create a new connection
_sqlConnection = _factory.CreateConnection();
_sqlConnection.ConnectionString = _connectionString;
// Wrap this method with a retry policy.
_sqlConnection.Open();
_sqlConnection = OnConnectionOpened(_sqlConnection);
if (KeepConnectionAlive)
_sharedConnectionDepth++; // Make sure you call Dispose
}
_sharedConnectionDepth++;
}
https://learn.microsoft.com/en-us/dotnet/framework/data/adonet/sql-server-connection-pooling
As far as I can see Petapoco is fighting the basic premise behind ADO connection pooling not creating a new SQLConnection(string connectionString) and then performing a .Close() method when the connection is closed.
Not surprising there are nothing but crickets chirping on this 4 yr. 9 month old question.
Related
I use mysql-native. This driver is suppport vibed's connection pool. On dlang newsgroup mysql-native developer Nick Sabalausky wrote:
"If you're using a connection pool, you shouldn't need to worry about closing the connection. The whole point is that the connections stay open until you need to use one again. When your program ends, then connections will close by themselves."
"You create the pool once (wherever/whenever you want to). Then, every time you want to use the database you obtain a connection by calling MySqlPool.lockConnection."
"Calling 'close' will always close the connection. If you got you connection from the pool, then it will automatically return to the pool when you're no longer using it. No need to do anything special for that."
The question about how pool should be done? I have read about singleton pattern and can't unserstand is it this case.
I wrote next code:
database class:
import std.stdio;
import std.string;
import mysql;
import vibe.d;
import config;
import user;
class Database
{
Config config;
MySqlPool mydb;
Connection connection;
this(Config config)
{
this.config = config;
mydb = new MySqlPool(config.dbhost, config.dbuser, config.dbpassword, config.dbname, config.dbport);
}
void connect()
{
if(connection is null)
{
connection = mydb.lockConnection();
}
scope(exit) connection.close();
}
}
users class/struct:
module user;
import mysql;
import vibe.d;
struct User
{
int id;
string login;
string password;
string usergroup;
}
void getUserByName(string login)
{
User user;
Prepared prepared = prepare(connection, `SELECT id, login, password, usergroup from users WHERE login=?`); // need to get connection accessible here to make request to DB
prepared.setArgs(login);
ResultRange result = prepared.query();
if (result.empty)
logWarn(`user: "%s" do not exists`, login);
else
{
Row row = result.front;
user.id = row[0].coerce!(int);
user.login = row[1].coerce!string;
user.password = row[2].coerce!string;
user.usergroup = row[3].coerce!string;
logInfo(`user: "%s" is exists`, login);
}
}
The problem that I can't understand what is proper way to getting access to connection instance. It seems that it's very stupid ideas to create every new database connection class inside users structure. But how to do it's in better way? To make Connection connection global? Is it's good? Or there is more correct way?
scope(exit) connection.close();
Delete that line. It's closing the connection you just received from the pool before the connect function returns. All you're doing there is opening a connection just to immediately close it again.
Change getUserByName to take a connection as an argument (typically as the first argument). Typically, whatever code needs to call getUserByName should either open a connection, or get a connenction from the pool via lockConnection, and then pass that connection to getUserByName and whatever other DB-related functions it needs to use. Then, after your code is done calling getUserByName (and whatever other DB functions it needs to call), you either just don't worry about the connection anymore and let your vibed fiber finish (if you're using vibed and got the connection from a pool) or you close the connection (if you did NOT get the connection from a vibed pool).
One way to do it is to pass the connection to your functions that need it. So you would refactor your getUserByName() to take connection as an argument.
Another alternative is to use the DAO pattern . Constructor of your DAO class would take the connection as one of the main parameters, and all the methods would use it to do the DB operation.
Problem description:
Lets have a service method which is called from controller:
class PaymentService {
static transactional = false
public void pay(long id) {
Member member = Member.get(id)
//long running task executing HTTP request
requestPayment(member)
}
}
The problem is if 8 users hit the same service in the same time and the time to execute the requestPayment(member) method is 30 seconds, the whole application gets stucked for 30 seconds.
The problem is even bigger than it seems, because if the HTTP request is performing well, nobody realizes any trouble. The serious problem is that availability of our web service depends on the availability of our external partner/component (in our use-case payment gateway). So when your partner starts to have performance issues, you will have them as well and even worse it will affect all parts of your app.
Evaluation:
The cause of problem is that Member.get(id) reserves a DB connection from pool and it keeps it for further use, despite requestPayment(member) method never needs to access DB. When next (9-th) request hits any other part of the application which requires DB connection (transactional service, DB select, ...) it keeps waiting (or timeouts if maxWait is set to lower duration) until the pool has an available connection, which can last even 30 seconds in our use case.
The stacktrace for the waiting thread is:
at java.lang.Object.wait(Object.java:-1)
at java.lang.Object.wait(Object.java:485)
at org.apache.commons.pool.impl.GenericObjectPool.borrowObject(GenericObjectPool.java:1115)
at org.apache.commons.dbcp.PoolingDataSource.getConnection(PoolingDataSource.java:106)
at org.apache.commons.dbcp.BasicDataSource.getConnection(BasicDataSource.java:1044)
at org.springframework.jdbc.datasource.DataSourceUtils.doGetConnection(DataSourceUtils.java:111)
Or for timeout:
JDBC begin failed
org.apache.commons.dbcp.SQLNestedException: Cannot get a connection, pool error Timeout waiting for idle object
at org.apache.commons.dbcp.PoolingDataSource.getConnection(PoolingDataSource.java:114)
at org.apache.commons.dbcp.BasicDataSource.getConnection(BasicDataSource.java:1044)
Caused by: java.util.NoSuchElementException: Timeout waiting for idle object
at org.apache.commons.pool.impl.GenericObjectPool.borrowObject(GenericObjectPool.java:1167)
at org.apache.commons.dbcp.PoolingDataSource.getConnection(PoolingDataSource.java:106)
... 7 more
Obviously the same issue happens with transactional service, however it makes much more sense since the connection is reserved for the transaction.
As a temporary solution its possible to increase the pool size with maxActive property on datasource, however it doesn't solve the real problem of holding an unused connection.
As a permanent solution its possible to enclose all DB operations to transactional behavior (withTransaction{..}, #Transactional), which returns the connection back to pool after commit (or to my surprise also withNewSession{..} works). But we need to be sure that the whole call chain from controller up to the requestPayment(member) method doesn't leak the connection.
I'd like to be able to throw an exception in the requestPayment(member) method if the connection is "leaked" (similar to Propagation.NEVER transactional behavior), so I can reveal the issue early during test phase.
After digging in the source code I've found the solution:
class PaymentService {
static transactional = false
def sessionFactory
public void pay(long id) {
Member member = Member.get(id)
sessionFactory.currentSession.disconnect()
//long running task executing HTTP request
requestPayment(member)
}
}
The above statement releases the connection back to pool.
If executed from transactional context, an exception is thrown (org.hibernate.HibernateException connnection proxy not usable after transaction completion), since we can't release such a connection (which is exactly what I needed).
Javadoc:
Disconnect the Session from the current JDBC connection. If the
connection was obtained by Hibernate close it and return it to the
connection pool; otherwise, return it to the application.
This is used by applications which supply JDBC connections to
Hibernate and which require long-sessions (or long-conversations)
I'm trying to implement connection pooling for a JSF 2.1 application which has a H2 database and Jetty 9 Web server embedded in it. I have two options to implement connection pooling for the h2 database. The options being let Jetty implement connection pooling for me, or I define a application scoped managed bean which creates connection pool. I would like to know which would be a better approach in handling connection pooling?
Connection pooling using Application scoped managed bean:
JdbcConnectionPool cp = JdbcConnectionPool.create(
"jdbc:h2:~/test", "sa", "sa");
for (String sql : args) {
Connection conn = cp.getConnection();
conn.createStatement().execute(sql);
conn.close();
}
cp.dispose();
Either approach of connection pooling is fine. There are many connection pool implementations (each one with advantages and disadvantages), use whatever you feel like using.
If you have a list of statements to execute, then I wouldn't open a new connection for each statement. Instead, execute all statements with the same connection (and statement):
JdbcConnectionPool cp = JdbcConnectionPool.create(
"jdbc:h2:~/test", "sa", "sa");
...
Connection conn = cp.getConnection();
Statement stat = conn.createStatement();
for (String sql : args) {
stat.execute(sql);
}
conn.close();
...
cp.dispose();
The connection pool can be started / stopped:
Outside the web application, as a resource (that's a bit more complicated in my view), for example as described in the article "Database Connection Pooling with Tomcat". You will find similar documentation for Jetty.
Using a ServletContextListener (also described in the H2 documentation). In my view, this is a bit simpler. The disadvantage is that the connection pool can not be used by multiple web applications.
I have asp.net mvc app that shows varios events. All events stored in a database. But now, I have to load data from the database and remote program. This program have external service (this is simple program that listening specific TCP port and recieve a query and send xml back).
And, I wrote simple page for test that connects to external program. The code got from MSDN:
static string Connect(String server, String message)
{
try
{
// Create a TcpClient.
Int32 port = 9197;
TcpClient client = new TcpClient(server, port);
// Translate the passed message into ASCII and store it as a Byte array.
Byte[] data = System.Text.Encoding.ASCII.GetBytes(message);
// Get a client stream for reading and writing.
// Stream stream = client.GetStream();
NetworkStream stream = client.GetStream();
// Send the message to the connected TcpServer.
stream.Write(data, 0, data.Length);
// Receive the TcpServer.response.
// Buffer to store the response bytes.
data = new Byte[256];
// String to store the response ASCII representation.
String responseData = String.Empty;
// Read the first batch of the TcpServer response bytes.
Int32 bytes = stream.Read(data, 0, data.Length);
responseData = System.Text.Encoding.ASCII.GetString(data, 0, bytes);
stream.Close();
client.Close();
return responseData;
}
catch (ArgumentNullException e)
{
//
}
catch (SocketException e)
{
//
}
}
This is my action:
public ActionResult GetData()
{
string query = "some query";
var response = Connect("192.168.0.1", query);
var model = ParseResponse(response);
return View(model);
}
I think this solution will reduce the perfomance.
What is best practicies to use TCPClient in ASP.NET MVC 3 app?
What you think about my code?
Any suggestions are welcome.
I think this solution will reduce the perfomance.
Well. All/most database operations are done over sockets. And you do not notice that, do you?
The most likely performance issues are:
Your server
Server location
Connection setup
The only thing I would do now is to build in checks in the client to monitor the response time and write to a log (or send an email) when the response times are too high.
Don't try to optimize performance until that happen.
Solutions for the above mentioned issues:
Refactor and optimize
Either put the server on the same lan or create a cache proxy server.
Use connection pooling instead of disconnecting the connections every time.
I think this solution will reduce the perfomance.
It's as any other remote request that your server does - an I/O intensive operation. So you could use an asynchronous controller and the asynchronous versions of the TcpClient methods. This way you won't be jeopardizing any worker threads on your server during the execution of the remote request.
I'm in a hoo-ha with my boss as I can't shift to using newer technologies until I have proof of some outstanding issues. One of the main concerns is how repositories deal with connections. One of the supposedly largest overheads is connecting and disconnecting to/from the database. If I have a repository where I do the following:
public ContractsControlRepository()
: base(ConfigurationManager.ConnectionStrings["AccountsConnectionString"].ToString()) { }
with the class like so:
public class ContractsControlRepository : DataContext, IContractsControlRepository
with functions like:
public IEnumerable<COContractCostCentre> ListContractCostCentres(int contractID)
{
string query = "SELECT C.ContractID, C.CCCode, MAC.CostCentre, C.Percentage FROM tblCC_Contract_CC C JOIN tblMA_CostCentre MAC ON MAC.CCCode = C.CCCode WHERE C.ContractID = {0}";
return this.ExecuteQuery<COContractCostCentre>(query, contractID);
}
Now if in my controller action called _contractsControlRepository.ListContractCostCentres(2) followed immediately by another call to the repository, does it use the same connection? When does the connection open in the controller? When is it closed?
Cheers
EDIT
I'm using hand-written LINQ as suggested by Steve Sanderson in his ASP.NET MVC book.
EDIT EDIT
To clarify, I'm using LINQ as my ORM, but I'm using raw SQL queries (as shown in the extract above) for querying. For example, here's a controller action:
public ActionResult EditBusiness(string id)
{
Business business = _contractsControlRepository.FetchBusinessByID(id);
return View(business);
}
I'm not opening/closing connections.
Here's a larger, more complete extract of my repo:
public class ContractsControlRepository : DataContext, IContractsControlRepository
{
public ContractsControlRepository()
: base(ConfigurationManager.ConnectionStrings["AccountsConnectionString"].ToString()) { }
public IEnumerable<COContractCostCentre> ListContractCostCentres(int contractID)
{
string query = "SELECT C.ContractID, C.CCCode, MAC.CostCentre, C.Percentage FROM tblCC_Contract_CC C JOIN tblMA_CostCentre MAC ON MAC.CCCode = C.CCCode WHERE C.ContractID = {0}";
return this.ExecuteQuery<COContractCostCentre>(query, contractID);
}
Then ContractsControlRepository is instantiated in my controller and used like _contractsControlRepository.ListContractCostCentres(2). Connections aren't opened manually, DataContext deals with that for me.
Without knowing the details of your ORM and how it connects the SQL database drivers will connection pool. When a connection is closed it is released back to the pool and kept open for X number of seconds (where X is configurable). If another connection is opened and all the parameters match (the server name, the application name, the database name, the authentication details etc.) then any free, but open connections in the pool will get reused instead of opening a brand new connection.
Having not read the book in question I don't know what "manual linq" actually is. If it's manual means you're getting the tables back youself then obviously you're doing the connection open/close. Linq to SQL will use a new connection object when a statement is finally executed at which point connection pooling comes into play - which means a new connection object may not be an actual new connection.