Connection Pooling in Grails 2.2.x? - grails

I could not find how connection pooling is activated in Grails.
In my DataSource.groovy I have the following:
url = "jdbc:mysql://localhost/myapp?useUnicode=yes&characterEncoding=UTF-8"
username = "root"
password = ""
dialect = org.hibernate.dialect.MySQL5InnoDBDialect
pooled = true
properties {
maxActive = -1
minEvictableIdleTimeMillis=1800000
timeBetweenEvictionRunsMillis=1800000
numTestsPerEvictionRun=3
testOnBorrow=true
testWhileIdle=true
testOnReturn=true
validationQuery = "select 1"
}
Is there something else I have to do to activate connection pooling than setting pooled = true?

No, the pooled attribute configures pooling. If it's set to false then you will create a new connection each time, and this can take hundreds of milliseconds, so it's not advised.
You would set pooled = false when using JNDI however, since the JNDI datasource does its own pooling, so there's no need to pool twice.

Below is a sample from Grails DataSource documentation page:
dataSource {
pooled = true
dbCreate = "update"
url = "jdbc:mysql://localhost:3306/my_database"
driverClassName = "com.mysql.jdbc.Driver"
dialect = org.hibernate.dialect.MySQL5InnoDBDialect
username = "username"
password = "password"
properties {
jmxEnabled = true
initialSize = 5
maxActive = 50
minIdle = 5
maxIdle = 25
maxWait = 10000
maxAge = 10 * 60000
timeBetweenEvictionRunsMillis = 5000
minEvictableIdleTimeMillis = 60000
validationQuery = "SELECT 1"
validationQueryTimeout = 3
validationInterval = 15000
testOnBorrow = true
testWhileIdle = true
testOnReturn = false
jdbcInterceptors = "ConnectionState;StatementCache(max=200)"
defaultTransactionIsolation = java.sql.Connection.TRANSACTION_READ_COMMITTED
}}
In addition to setting pooled to true, you can also tweak settings on how many connections are initialized up front (initialSize) and the maximum number of connections you would want to maintain when the load gets higher (maxActive).

Related

Artillery + Playwright, StatsD data not being ingested in InfluxDB correctly by Telegraf (Template not working)

I have some tests written using Artillery + Playwright and I am using the publish-metrics plugin with type influxdb-statsd. I then have the following telegraf.config
[[outputs.influxdb_v2]]
urls = ["http://${INFLUX_DB2_HOST_ADDRESS}:8086"]
token = "${INFLUX_DB2_TOKEN}"
organization = "${INFLUX_DB2_ORGANIZATION}"
bucket = "${INFLUX_DB2_BUCKET}"
[[inputs.statsd]]
protocol = "udp"
max_tcp_connections = 250
tcp_keep_alive = false
service_address = ":8125"
delete_gauges = true
delete_counters = true
delete_sets = true
delete_timings = true
metric_separator = "_"
parse_data_dog_tags = true
datadog_extensions = true
datadog_distributions = false
Data from artillery is sent in this format to statsD
artillery.browser.page.FID.compliance-hub_dashboard.min:3.2|g
artillery.browser.page.FID.compliance-hub_dashboard.max:3.2|g
artillery.browser.page.FID.compliance-hub_dashboard.count:2|g
artillery.browser.page.FID.compliance-hub_dashboard.p50:3.2|g
I would like to set up a telegraf template so that in Influx DB
artillery.browser.page.FID.compliance-hub_dashboard is a measurement and min, max, count and p50 are fields.
How do I do that?
I tried:
templates = [
"measurement.measurement.measurement.measurement.measurement.field",
]
but it's not working. :(
What I see in InfluxDb is a measurement of artillery_browser_page_FID_compliance-hub_dashboard_min with a field of value = 3.2

Telegraf http listener v2: unable to send JSON with string values

I'm trying to send this very simple JSON string to Telegraf to be saved into InfluxDB:
{ "id": "id_123", "value": 10 }
So the request would be this: curl -i -XPOST 'http://localhost:8080/telegraf' --data-binary '{"id": "id_123","value": 10}'
When I make that request, I get the following answer: HTTP/1.1 204 No Content Date: Tue, 20 Apr 2021 13:02:49 GMT but when I check what was written to database, there is only value field:
select * from http_listener_v2
time host influxdb_database value
---- ---- ----------------- -----
1618923747863479914 my.host.com my_db 10
What am I doing wrong?
Here's my Telegraf config:
[global_tags]
[agent]
interval = "10s"
round_interval = true
metric_batch_size = 1000
metric_buffer_limit = 10000
collection_jitter = "0s"
flush_interval = "10s"
flush_jitter = "0s"
precision = ""
hostname = ""
omit_hostname = false
# OUTPUTS
[[outputs.influxdb]]
urls = ["http://127.0.0.1:8086"]
database = "telegraf"
username = "xxx"
password = "xxx"
[outputs.influxdb.tagdrop]
influxdb_database = ["*"]
[[outputs.influxdb]]
urls = ["http://127.0.0.1:8086"]
database = "httplistener"
username = "xxx"
password = "xxx"
[outputs.influxdb.tagpass]
influxdb_database = ["httplistener"]
# INPUTS
## system
[[inputs.cpu]]
percpu = true
totalcpu = true
collect_cpu_time = false
report_active = false
[[inputs.disk]]
ignore_fs = ["tmpfs", "devtmpfs", "devfs", "iso9660", "overlay", "aufs", "squashfs"]
[[inputs.mem]]
[[inputs.swap]]
[[inputs.system]]
## http listener
[[inputs.http_listener_v2]]
service_address = ":8080"
path = "/telegraf"
methods = ["POST", "PUT"]
data_source = "body"
data_format = "json"
[inputs.http_listener_v2.tags]
influxdb_database = "httplistener"
Use json_string_fields = ["id"]

grails oracle connection timeout

I have below datasource config in grails.
dataSource_staging_oracle {
dbCreate = "none"
pooled = true
url = "jdbc:oracle:thin:#//my-box-oracle.com:1521/DB1"
driverClassName = "oracle.jdbc.OracleDriver"
username = "USER_1"
password = "encryptedpassword"
passwordEncryptionCodec = PropertiesCodec
}
dataSource_prod_oracle {
dbCreate = "none"
pooled = true
autoReconnect = true
url = "jdbc:oracle:thin:#//my-oracle-prod-box.com:1521/DB2"
driverClassName = "oracle.jdbc.OracleDriver"
username = "user_2"
password = "encrypted_password"
passwordEncryptionCodec = PropertiesCodec
}
Application is able to fetch the DB connection and work fine. However after 1-2 hours of processing, I see connection closed exceptions.
ERROR util.JDBCExceptionReporter - Closed Connection
I believe this has to do with datasource config properties. What grails settings help me in refreshing the connections? I already have set autoReconnect and pooled to true
I use grails 2.3.3
Datasource properties will inform the database what the middleware server will expect from the Database. If your Oracle server does not agree with the settings Grails would like to use, expect sometimes cryptic error messages.
Using pooled connections tells Oracle to hold a number of connections open for the application. If the application is running 1-2 hours and continuously processing, I expect the pool may be running out of valid connections. For some background please view the links included below.
Check out the Grails configuration docs, section 4.4, to see advanced datasource configuration options. Do you have a validation query defined? (eg select 1 from dual;)
Check out this AskTom post for a clear explanation of connection pooling.
Previous StackOverflow question with MySQL but similar error.
Below config helped me fix my problem.
dataSource_staging {
dbCreate = "none"
pooled = true
autoReconnect = true
url = "jdbc:oracle:thin:#//my-box-oracle.com:1521/DB1"
driverClassName = "oracle.jdbc.OracleDriver"
username = "USER_1"
password = "encryptedpassword"
passwordEncryptionCodec = PropertiesCodec
properties {
validationQuery = "SELECT 1 FROM DUAL"
testOnBorrow = true
testWhileIdle = true
testOnReturn = false
timeBetweenEvictionRunsMillis = 5000
minEvictableIdleTimeMillis = 60000
maxAge = 10 * 60000
maxWait = 10000
maxIdle = 25
maxActive = 50
initialSize = 5
}
}

How to use multiple data sources in Grails 1.3.3

I want to user two data source in single environment.
dataSource {
dbCreate = "update" // one of 'create', 'create-drop','update'
pooled = true
driverClassName = "com.mysql.jdbc.Driver"
username = "root"
password = ""
url = ""
dialect = "org.hibernate.dialect.MySQL5InnoDBDialect"
properties {
maxActive = 75
maxIdle = 10
minIdle = 2
initialSize = 2
maxWait = 30000
validationQuery="select 1"
testOnBorrow=true
testWhileIdle=true
timeBetweenEvictionRunsMillis=60000
}
}
Prior to Grails 2.0 multiple data source support is provided by the Datasources plugin.

Grails maxRows/queryTimeout warning

I seem to randomly get the warning below in my Grails 2.2.4 application. It doesn't look like it is causing any issues, but it is still concerning.
I tried to prevent this warning by modifying my datasource properties in my DataSource.groovy file:
dataSource {
pooled = true
properties {
maxWait = 10000 // 10 seconds
minEvictableIdleTimeMillis = 1000 * 60 * 30 // 30 minutes
numTestsPerEvictionRun = 3
testOnBorrow = true
testOnReturn = false
testWhileIdle = false
timeBetweenEvictionRunsMillis = 1000 * 60 * 30 // 30 minutes
validationQuery = "SELECT 1"
}
}
And when that didn't work I tried to set the properties in my BootStrap.groovy file:
def init = { servletContext ->
def ctx = Holders.getApplicationContext()
def dataSource = ctx.dataSourceUnproxied
println "configuring database connection pool"
dataSource.setMinEvictableIdleTimeMillis(1000 * 60 * 30)
dataSource.setTimeBetweenEvictionRunsMillis(1000 * 60 * 30)
dataSource.setNumTestsPerEvictionRun(3)
dataSource.setTestOnBorrow(true)
dataSource.setTestWhileIdle(false)
dataSource.setTestOnReturn(false)
dataSource.setValidationQuery("SELECT 1")
}
Neither attempt prevents the warning. The author of this post said he had success setting the properties directly in the tomcat config, but I need a more generic solution that will work from the command line and in other servers.
2013-09-25 15:07:51,027 [http-bio-8080-exec-9] WARN jdbc.AbstractBatcher - exception clearing maxRows/queryTimeout
java.sql.SQLException: org.apache.commons.dbcp.DelegatingPreparedStatement with address: "com.mysql.jdbc.JDBC4PreparedStatement#13ed0db0: EXCEPTION: com.mysql.jdbc.exceptions.jdbc4.MySQLNonTransientConnectionException: No operations allowed after statement closed." is closed.
at org.apache.commons.dbcp.DelegatingStatement.checkOpen(DelegatingStatement.java:137)
at org.apache.commons.dbcp.DelegatingStatement.getMaxRows(DelegatingStatement.java:237)
at ace_2.DefsUploadController.upload(DefsUploadController.groovy:16)
at grails.plugin.cache.web.filter.PageFragmentCachingFilter.doFilter(PageFragmentCachingFilter.java:195)
at grails.plugin.cache.web.filter.AbstractFilter.doFilter(AbstractFilter.java:63)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:724)
After connections to my MySQL database timed out I would see that exact exception as well. I updated the DataSource configuration using the suggestion in the Grails docs as a guideline and I have yet to see any exceptions from closed connections.
Here are my current settings:
properties {
initialSize=5
maxActive=50
minIdle=5
maxIdle=25
maxWait = 10000
maxAge = 10 * 60000
timeBetweenEvictionRunsMillis = 5000
minEvictableIdleTimeMillis = 60000
validationQuery = "SELECT 1"
validationQueryTimeout = 3
validationInterval = 15000
testOnBorrow = true
testWhileIdle = true
testOnReturn = false
jdbcInterceptors = "ConnectionState;StatementCache(max=200)"
defaultTransactionIsolation = java.sql.Connection.TRANSACTION_READ_COMMITTED
}

Resources