I'm in the process of setting up the backend environment for a mobile app on my local machine and a Postgres function call within a Ruby model errors out the request. Here's the log:
D, [2015-08-05T23:10:06.029187 #1732] DEBUG -- : Game Load (2.0ms) SELECT field1, field2, field3
FROM valuestable
WHERE ARRAY[field1] && (
SELECT custom_function(3, '{}', 'ipad')
);
E, [2015-08-05T23:10:06.029262 #1732] ERROR -- : PG::UndefinedFunction: ERROR: function custom_function(integer, unknown, unknown) does not exist
LINE 4: SELECT custom_function(3, '{}', 'ipad')
^
HINT: No function matches the given name and argument types. You might need to add explicit type casts.
SELECT field1, field2, field3
FROM gamestable
WHERE ARRAY[field1] && (
SELECT custom_function(3, '{}', 'ipad')
);
We use Postgres 9.4.1/Redis/Rails/AWS.
The custom_function in the above sql query is defined in a class that got auto created during a migration. I see it in one of the time-stamped .rb files in the db/migrate folder of the said ruby app. However, I do not know how to "include/import" that into my model so this error can be prevented. I have an sql file called user_functions.sql authored by one of my ex-colleagues in which the custom_function (from the above error) is referenced.
Also, another interesting point is this application runs perfectly on both Staging and Production. It's only on the local machine that it doesn't work. I have no idea if this is some kind of a one-time setup on the backend run directly on AWS or if it's simply a step I'm missing while setting up my local backend.
Meta Information: I'm originally an iOS dev, trying to figure out some backend logic in the absence of a real backend person.
What I tried so far:
Making sure the version number of Postgres matches on server and local backend
Trying to use ActiveRecord::Base.connection.execute(IO.read("/path/to/user_functions.sql")) just before custom_function is first called.
Look what arguments custom function is expecting and cast them.
like this: instead of SELECT custom_function(3, '{}', 'ipad') use SELECT custom_function(3, '{}'::json, 'ipad'::text) or SELECT custom_function(3, '{}'::text, 'ipad'::text) cos many types can go inside the quotes
Related
create or replace procedure output_message(message varchar)
returns varchar not null
language sql
as
begin
return message;
end;
call output_message('Hello World');
error I'm getting:
Database Error in model my_first_dbt_model (models/example/my_first_dbt_model.sql)
001003 (42000): SQL compilation error:
syntax error line 2 at position 7 unexpected 'create'.
syntax error line 3 at position 0 unexpected 'returns'.
compiled SQL at target/run/dbt_project/example/my_first_dbt_model.sql
You cannot put code for a stored procedure in a model file (any .sql file in your models directory). dbt assumes that a model file contains a single select statement; it then wraps that select statement in DDL (usually a create table my_model as (...) statement), depending on your materialization config for that model.
Macros are more flexible. Generally, Stored Procedures aren't part of the dbt workflow, but there are some valid use-cases for them (along with UDFs).
You can create a output_message.sql file in your macros directory, and use the code you provided, wrapping it in a macro definition, like:
{% macro output_message(msg) %}
create or replace procedure output_message(message varchar)
returns varchar not null
language sql
as
begin
return message;
end;
call output_message(msg);
{% endmacro %}
You can then call that macro in a post-hook or by using run-operation:
dbt run-operation output_message --args "{'msg': 'Hello World!'}"
You can customize your code using materialization link and I have followed a link to create ddl objects. You need to write macros for that.
To run a one-off procedure of loading data that is already formatted for loading into postgresql
BEGIN;
INSERT INTO "public"."optionsets" [...]
COMMIT;
Rails has the command that allows to execute raw sql
ActiveRecord::Base.connection.execute()
However, the syntax alludes to placing the syntax within the parenthesis and the documentation confirms this: "Executes the SQL statement in the context of this connection". The following fail
ActiveRecord::Base.connection.execute(/Users/main/optionsets.sql)
SyntaxError ((irb):3: unknown regexp options - ma)
ActiveRecord::Base.connection.execute("/Volumes/main/optionsets.sql")
ActiveRecord::StatementInvalid (PG::SyntaxError: ERROR: syntax error at or near "/")
LINE 1: /Users/main/optionsets.sql
location = "/Volumes/main/optionsets.sql"
ActiveRecord::Base.connection.execute(location)
ActiveRecord::StatementInvalid (PG::SyntaxError: ERROR: syntax error at or near "/")
LINE 1: /Users/main/optionsets.sql
How can this be effected instead by calling an .sql file /[path_to]/optionsets.sql ?
Reading the file is not effected in the above scenarios, so it must be executed beforehand.
file = File.read('/[path_to]/optionsets.sql')
ActiveRecord::Base.connection.execute(file)
If run from the command line, then Rails also has a "dbconsole" command with a "db" alias, so you can do something like this
rails db < /Volumes/main/optionsets.sql
https://github.com/rails/rails/blob/bbd6d82dc4e00b00f07193a0c7ac3e266bce410a/guides/source/command_line.md#binrails-dbconsole
ActionView::Template::Error (PG::UndefinedFunction: ERROR: operator
does not exist: double precision ~~ unknown
2016-04-10T23:45:59.506005+00:00 app[web.1]: LINE 1: ... =
"trackers"."category_id" WHERE (categories.tag LIKE '1.%'...
this is the error i get when i try to run this line of code here
Tracker.group(:category_id).joins(:category).where('categories.tag LIKE ? AND user_id = ?', "#{tag.to_i}.%", current_user.id)
tag is of type float, and i typecast it to an integer in order to check for tags 1.1, 1.2, 1.3 etc
so in the example above I type cast tag with value 1.0 to be 1, so i can search for tags that are like 1.1, 1.2 etc
I am using postgres on heroku that gives this error. locally i use sqlite3 and it works just fine.
how can i overcome this?
Since you're in rails, sort out the dynamic-ness in rails first then send that to the ORM. The syntax you provided already accepts any parameters (eg: WHERE tag between ? and ?), so before you request the data from the ORM, sort out in rails the high and lows. The query is already setup for something to be dynamic.
I am getting the following error:
2013-11-14 11:57:33,994 [TestScheduler_QuartzSchedulerThread] ERROR Common.Loggi
ng.Factory.AbstractLogger.Error(:0) - An error occurred while scanning for the n
ext trigger to fire.
Quartz.JobPersistenceException: Couldn't acquire next trigger: Cannot insert the
value NULL into column 'SCHED_TIME', table 'quartz.dbo.QRTZ_FIRED_TRIGGERS'; co
lumn does not allow nulls. INSERT fails.
The statement has been terminated. ---> System.Data.SqlClient.SqlException: Cann
ot insert the value NULL into column 'SCHED_TIME', table 'quartz.dbo.QRTZ_FIRED_
TRIGGERS'; column does not allow nulls. INSERT fails.
The statement has been terminated.
I am using the example from HERE and I think I need to understand how jobs work with ADO.Net Jobstore better. My code works perfectly with RAMJobStore. Is there something else I have to do to get ADO to scan for triggers?
Turns out when I looked closer at the errors it was referencing the older version of Quartz.net.
I switched to Postgres in development and now following error breaks a test case:
PG::Error: ERROR: column "id" of relation "assets_content_nodes" does not exist
: SELECT pg_get_serial_sequence($1, $2)
Same error happens in development and production environments but it is not a show stopper and does not affect the import.
assets_content_nodes is a join table and I'd rather not add the 'id' column.
Any ideas why this may be happening?
This is fixed in activerecord-import -v0.3.1 (or possibly an earlier version, but it was broken in 0.2.11 and earlier). You no longer need an id column on a table to do an import.
You might also try Upsert.
If you're importing pets:
require 'csv'
require 'upsert'
u = Upsert.new Pet.connection, Pet.table_name
CSV.foreach('pets.csv', headers: :first_row) do |row|
selector = { name: row['name'] }
setter = row
u.row selector, setter
end
My tests show it's 20–30% faster than activerecord-import. Since it's "upserting," it's generally OK if the process gets interrupted and you just start over from the beginning.