I am working on porting a Rails app to JRuby and HSQLDB. My goal is to embed a database and the site within a single JAR file for deployment at customer sites. I have the site working quite well from the JAR, with a few notable problems.
When I do the following with a pretty mundane ActiveRecord model:
#total = SessionLog.count(:id)
I get the following exception:
ActiveRecord::StatementInvalid (ActiveRecord::ActiveRecordError: Not
in aggregate function or group by clause: org.hsqldb.Expression#7be117eb
in statement [SELECT count(session_logs.id) AS count_id
FROM session_logs WHERE (created_at >= '2010-02-06' AND created_at <=
'2010-03-09' AND session_type = 'tunnel_client') ORDER BY id DESC ]:
SELECT count(session_logs.id) AS count_id FROM session_logs WHERE
(created_at >= '2010-02-06' AND created_at <= '2010-03-09' AND
session_type = 'tunnel_client') ORDER BY id DESC )
It seems clear to me that the COUNT statement is causing the trouble in HSQLDB, but I'm not sure what the solution is to fix this. SQLite3 and MySQL both process this SQL statement without issue.
I'm open to using a different database other than HSQLDB, but it needs to be embeddable into our application on the JVM. That is the appeal of HSQLDB.
You've probably found a bug in the ActiveRecord adapter - activerecord-jdbchsqldb-adapter I assume.
Can you try run the SQL directly in some non-ruby SQL session? Then maybe you can see where it's going wrong and submit a bug or (better), submit a patch.
You can try H2 Database, wire it like so. From wikipedia:
The database engine is written by Thomas Mueller. He also developed the Java database engine Hypersonic SQL [1]. In 2001, the Hypersonic SQL was stopped, and the HSQLDB Group was formed to continue work on the Hypersonic SQL code. The name H2 stands for Hypersonic 2, however H2 does not share any code with Hypersonic SQL or HSQLDB. H2 is built from scratch.
Related
I have rails code that is consuming an oracle view/function.
This is my code:
def run_query
connection.exec_query(
"SELECT * FROM TABLE(FN_REQ(#{demo_type_param},#{demo_tid_param}}))")
end
When run Brakeman analyzer it warns of possible "sql injection attack"
I need to understand if this is a valid warning, if so, how do I remediate it?
Since this is a function & not an actual table, I am not sure what's the right way.
If it was a normal model, i would have just followed this pattern:
Model.where("mycolumn1= ? AND mycolumn2= ?", demo_type_param, demo_tid_param).first
Yes, it is real. Almost every time, you build any SQL query from simply concatenating variables, you are vulnerable to SQL injection. Generally, an SQL injection happens each time when data inserted into the query can look like valid SQL and can result in additional queries executed.
The only solution is to manually enforce appropriate escaping or to use prepared statements, with the latter being the preferred solution.
With ActiveRecord / Rails, you can use exec_query with binds directly
sql = 'SELECT * FROM TABLE(FN_REQ(?,?))'
connection.exec_query(sql, 'my query', [demo_type_param, demo_tid_param])
Here, Rails will prepare the statement on the database and add the parameters to it on execution, ensuring that everything is correctly escaped and save from SQL injection.
Running Rails 4.0.13 with TinyTDS connected to Microsoft SQL Server 2012, I'm trying to run the following query:
sql = Model.where(:foo => bar).select(:open, :high, :low, :close).to_sql
Model.connection.execute(sql)
The problem is, the generated sql is
"SELECT open, high, low, close FROM [models]"
Which gives me an error as the column names open and close are protected.
TinyTds::Error: Incorrect syntax near the keyword 'open'
If I use #pluck, I can see the correct SQL is generated (with column names escaped):
"SELECT [models].[open], [models].[high], [models].[low], [models].[close] FROM [models]"
However, this produces an array, which is not what I want.
My question is how can i get #select to correctly quote the column names?
Thank you
I don't think you can make the select method to protect your column names when using symbols (maybe because different DBMS use different quoting identifiers), but you could pass your selection as a string :
sql = Model.where(:foo => bar).select("[open], [high], [low], [close]").to_sql
Model.connection.execute(sql)
I attempted to submit a bug report to Rails, however in doing so I saw the problem did not appear to exists using SQLite test case, this leads me to believe the issue is with the SQL Server Adapter.
Since I am on Rails 4 and not the latest version of the adapter I left it and wrote the following (horrible) method as wrapping the column names was not enough, I needed to prefix the table to prevent ambiguous column names. Yuck
def self.quote(*columns, klass)
columns.map { |col| "[#{klass.table_name}].[#{col}]" }.join(', ')
end
Recently I was tasked with creating a SQL Server Job to automate the creation of a CSV file. There was existing code, which was using an assortment of #temp tables.
When I set up the job to execute using BCP calling the existing code (converted into a procedure), I kept getting errors:
SQLState = S0002, NativeError = 208
Error = [Microsoft][SQL Native Client][SQL Server]Invalid object name #xyz
As described in other post(s), to resolve the problem lots of people recommend converting all the #tempTables to #tableVariables.
However, I would like to understand WHY BCP doesn't seem to be able to use #tempTables?
When I execute the same procedure from within SSMS it works though!? Why?
I did do a quick and simple test using global temp tables within a procedure and that seemed to succeed via a job using BCP, so I am assuming it is related to the scope of the #tempTables!?
Thanks in advance for your responses/clarifications.
DTML
You are correct in guessing that it's a scope issue for the #temp tables.
BCP is spawned as a separate process, so the tables are no longer in scope for the new processes. SSMS likely uses sub-processes, so they would still have access to the #temp tables.
A simple Rails 3 application tries to talk to SQL Server 2000 using activerecord-jdbc-adapter. I tried both microsoft jdbc driver and jtds driver. seems to connect to database OK.
when it is time to SHOW data I get this error:
ActiveRecord::StatementInvalid in PencilsController#show
ActiveRecord::JDBCError: 'ROW_NUMBER' is not a recognized function name.: SELECT t.* FROM (SELECT ROW_NUMBER() OVER(ORDER BY [pencils].id) AS _row_num, [pencils].* FROM [pencils] WHERE [pencils].[id] = 1) AS t WHERE t._row_num BETWEEN 1 AND 1
The real problem here is the DB do not support proper LIMIT and OFFSET functions. Rails 2 would have the same problem.
For one of my old projects I had to use Sybase15, which is quite similar to old SQL Server. To make limit and offset work with that DB I had to write my own adapter:
https://github.com/arkadiyk/ar-sybase-jdbc-adapter .
It uses scrollable cursors to simulate offset. You can try to use it as it is with SQL SERVER 2000 or feel free to clone it and modify for your specific needs.
Update:
The ROW_NUMBER function is called at https://github.com/jruby/activerecord-jdbc-adapter/blob/master/lib/arjdbc/mssql/limit_helpers.rb line 82 (SqlServerReplaceLimitOffset)
There is no replacement for this function. There are other ways of implementing OFFSET but there is no straight forward one.
This is kind a old, but if anyone is passing through here, i put together another solution that uses activerecord-sqlserver-adapter that can be used to connect a rails 3.2 app to sqlserver 2000
https://bitbucket.org/jose_schmidt/rails-sqlserver-adapter-sql-server-2000-friendly
I'm trying to figure out where a whole pile of extra queries are being generated by my rails app. I need some ideas on how to tackle it. Or, if someone can give me some hints, I'd be grateful.
I get these:
SQL (1.0ms) SELECT name
FROM sqlite_master
WHERE type = 'table' AND NOT name = 'sqlite_sequence'
SQL (0.8ms) SELECT name
FROM sqlite_master
WHERE type = 'table' AND NOT name = 'sqlite_sequence'
SQL (0.8ms) SELECT name
FROM sqlite_master
WHERE type = 'table' AND NOT name = 'sqlite_sequence'
repeated over and over on every request to the DB (as much as 70 times for a single request)
I tried installing a plugin that traced the source of the queries, but it really didn't help at all. I'm using the hobofields gem, dunno if that is what's doing it but I'm somewhat wedded to it at the moment
Any tips on hunting down the source of these extra queries?
Have a look at ActiveRecord gem in connection_adapters/sqlite_adapter.rb on line 173 (I'm using ActiveRecord 3.0.7) you have a function called tables that generates the exact query you posted:
SELECT name
FROM sqlite_master
WHERE type = 'table' AND NOT name = 'sqlite_sequence'
In development mode this function gets called for every table from your database, on every request.
In production this function gets called only once, when the server starts, and the result is cached.
In development for each request rails looks into your database to see what columns each table has so it can generate methods on your models like "find_by_name". Since in production is unlikely that your database changes, rails does this lookup only on server start.
It's very difficult to tell this without look into the Code.
but i am sure you write your query in a certain loop
for i in 0..70
SqliteMaster.find(:all, :conditions=>["type=? and not name=?", 'table', 'sqlite_sequesnce'])
end
So my advice is that to check all the methods which gets called after requesting certain method and look whether the query called in a loop.
I've just seen this appearing in my logs when I do a search with the metasearch gem but ONLY in development mode.
I believe that it is caused by the plugin acts-as-taggable-on.
It will check if the table or the cache column exists.