I use next settings to connect to remote DB:
class MyModel < ActiveRecord::Base
self.table_name = 'users'
establish_connection(
adapter: "mysql2",
host: "host_ip",
encoding: "koi8u",
username: "custom_name",
password: "password",
database: "db_name")
end
It connects well but when I make a query, I receive something like Address: "п⌠п╟п╪п╟п?п╦п? п░п╩я▄я├п╣я│я┌"
I also try another variant of connection like:
connection = Mysql2::Client.new( host: 'host_name',
username: 'user',
password: 'password',
port: 3306,
database: 'db_name',
encoding: 'koi8u',
reconnect: true
)
In this case I receive connection object, but can't make a query... When I use
connection.query("SELECT * FROM users") it returns me the connection object...
Also when I check encoding it returns:
MyModel.first.Address.encoding.name
MyModel Load (6.8ms) SELECT `users`.* FROM `users` ORDER BY `users`.`login` ASC LIMIT
1
=> "UTF-8"
When in settings I use koi8u !
I was checking charset on MySQL server in this DB - it is set to koi8u !
Any ideas? I need to connect to this DB with normal charset
It seems to me that your database locale is different then you expect it to be
Did you changed it or set it in any stage?
It is not enough to declare about it in the connection string
You can find up here how to check what is your database character set
How do I see what character set a MySQL database / table / column is?
Related
This is a piece of code in place. When I add this to the cron with timeout the entire array gets saved twice. When I remove timeout nothing gets saved
In this scenario we would want to save the array results (coming in from an api) with over 100k records to be saved to the db. I have used bulk insert and TinyTds gems here
ActiveRecord::Base.establish_connection(adapter: 'sqlserver', host: "xxx", username: "xxx", password: "xxx", database: "xxx", azure: true, port: 1433, timeout: 5000)
class Report < ActiveRecord::Base
self.primary_key = 'id'
end
my_array = [] #count of 100000 records
Report.bulk_insert(:account_owner_id) do |worker|
my_array.drop(2).each do |arr|
worker.add account_owner_id: arr[0]
end
end
You can try removing timeout and adding ignore: true to your bulk insert as shown here. There may be an insert that is failing.
Report.bulk_insert(:account_owner_id, ignore: true) do |worker|
How do we connect two databases with a single sql query? I prefer using sql here since to improve the performance.
config/database.yml
example:
adapter: postgresql
encoding: unicode
database: example_dev
example_report:
adapter: postgresql
encoding: unicode
database: example_report_dev
Now in one of my model I would like to write a query to fetch data from both these databases
SELECT example_dev.*, example_report_dev.*
FROM example_dev.myTable AS firstdb
INNER JOIN example_report_dev.myTable AS seconddb
ON firstdb.id = seconddb.id
Thank you
One approach is to use postgres's dblink.
From the docs:
dblink executes a query (usually a SELECT, but it can be any SQL
statement that returns rows) in a remote database.
First, you'll need to enable dblink in your database with:
CREATE EXTENSION dblink;
Then, in your application, you can execute a raw query statement with ActiveRecord::Base.connection.execute.
I created this test setup here:
Database db1 contains a table tbl with field1 and field2
columns.
Database db2 contains a table tbl with field1 and
field2 columns.
Both have 5 rows.
My database.yml:
development:
adapter: postgresql
encoding: unicode
database: db1
From rails console:
ActiveRecord::Base.connection.execute("
SELECT * FROM tbl
UNION ALL
SELECT * FROM dblink('dbname=db2','SELECT * FROM tbl') AS tbl2(field1 varchar, field2 int);
").to_a
# [{"field1"=>"one", "field2"=>1}, {"field1"=>"two", "field2"=>2}, {"field1"=>"three", "field2"=>3}, {"field1"=>"four", "field2"=>4}, {"field1"=>"five", "field2"=>5}, {"field1"=>"one", "field2"=>1}, {"field1"=>"two", "field2"=>2}, {"field1"=>"three", "field2"=>3}, {"field1"=>"four", "field2"=>4}, {"field1"=>"five", "field2"=>5}]
It's just a possible approach. You can setup dblink connection string to point to a remote server (for instance: 'dbname=yourdb port=5432 host=yourhost user=youruser password=yourpwd)
Be advised that this is not the rails way. Raw query is not related to a model. I suggest you to choose this only for specific tasks, like running a report.
EDIT
If you want one query for each database, and don't want to link it to your model, you can use ActiveRecord::Base.establish_connection, like this:
conn1 = {
adapter: 'postgresql',
encoding: 'utf8',
database: 'db1'
}
conn2 = {
adapter: 'postgresql',
encoding: 'utf8',
database: 'db1'
#, more config here - other host, for instance #
}
arr1 = ActiveRecord::Base.establish_connection(conn1).connection.execute("select * from tbl").to_a
# => [{"field1"=>"one", "field2"=>1}, {"field1"=>"two", "field2"=>2}, {"field1"=>"three", "field2"=>3}, {"field1"=>"four", "field2"=>4}, {"field1"=>"five", "field2"=>5}] => [{"field1"=>"one", "field2"=>1}, {"field1"=>"two", "field2"=>2}, {"field1"=>"three", "field2"=>3}, {"field1"=>"four", "field2"=>4}, {"field1"=>"five", "field2"=>5}]
arr2 = ActiveRecord::Base.establish_connection(conn2).connection.execute("select * from tbl").to_a
# => [{"field1"=>"one", "field2"=>1}, {"field1"=>"two", "field2"=>2}, {"field1"=>"three", "field2"=>3}, {"field1"=>"four", "field2"=>4}, {"field1"=>"five", "field2"=>5}] => [{"field1"=>"one", "field2"=>1}, {"field1"=>"two", "field2"=>2}, {"field1"=>"three", "field2"=>3}, {"field1"=>"four", "field2"=>4}, {"field1"=>"five", "field2"=>5}]
You'll get two arrays arr1 and arr2 with both data.
You can try the octopus gem out: https://github.com/thiagopradi/octopus.
Octopus is a better way to do Database Sharding in ActiveRecord. Sharding allows multiple databases in the same rails application. While there are several projects that implement Sharding (e.g. DbCharmer, DataFabric, MultiDb), each project has its own limitations. The main goal of octopus project is to provide a better way of doing Database Sharding.
I have a task to query in a hive db from ruby code. I am planning to use rbhive gem, but from it documentation, I am not able to get how to pass username, password, db name, etc when connecting to hive server.
Here is my code:
res = RBHive.connect('host_address', 10_000) do |connection|
connection.fetch 'show databases;'
end
It just shows:
Connecting to host_server on port 10000
Executing Hive Query: show databases;
and it hangs there indefinitely.
That may be surprising, but with Hive 1.0.0 I managed to connect using
RBHive.tcli_connect('host', 10000, {transport: :sasl, sasl_params:{}}) do |connection|
Please use
options = { username: 'username', password: 'password' }
RBHive.tcli_connect('host', 'port', { transport: :sasl, sasl_params: options}) do |connection|
results = connection.execute "SHOW DATABASES"
end
database.yml looks like
development:
adapter: mysql2
host: localhost
database: database1
username: root
password:
external:
adapter: mysql2
host: localhost
database: database2
username: root
password:
I have a class ExternalDatabaseConnection
class ExternalDatabaseConnection < ActiveRecord::Base
self.abstract_class = true
establish_connection(:external) #connect to external DB specified in database.yml
attr_protected
end
I'm using AR to query on my external DB.
database2(external DB) is set dynamically from user end and I will not have any information about the tables and attributes.
database1(internal DB) which contains information about tables and some attributes.
So basically I have to handle things dynamically.
Works fine.
ExternalDatabaseConnection.table_name = "set_table1" #dynamic
ExternalDatabaseConnection.create(...) #creates a record.
This doesn't.
ExternalDatabaseConnection.table_name = "set_different_table" #dynamic
ExternalDatabaseConnection.create(...)
throws
unknown attribute: attribute_name
Both the things happen in loop, so every-time it works good for initial table set, for the second table, it throws an error.
Debugging:
Note:
ExternalDatabaseConnection.table_name = "payments"
ExternalDatabaseConnection.column_names
=> ["id", "customer_id", "patent_id", "amount", "currency", "date"]
ExternalDatabaseConnection.table_name = "patents"
=> "patents"
2.0.0-p451 :006 > ExternalDatabaseConnection.column_names
=> ["id", "customer_id", "patent_id", "amount", "currency", "date"]
ExternalDatabaseConnection.all
ExternalDatabaseConnection Load (0.4ms) SELECT `patents`.* FROM `patents`
=> [#<ExternalDatabaseConnection id: 1, number: "1111", description: "https://www.google.co.in", registry_date: "2014-08-20", documnetdir: nil, currency: #<BigDecimal:a8f2adc,'0.1212E2',18(27)>>, #<ExternalDatabaseConnection id: 4, number: "1", description: "jhufgtrdfyuh", registry_date: nil, documnetdir: "", currency: #<BigDecimal:a8f2258,'0.0',9(18)>>, ....]
There is no change in attributes, may be this is causing a problem.
If any information is missing, comment up.
ruby 2.0.0p451
Rails 3.2.17
edit1:
Tries:
Before setting table name
1) ExternalDatabaseConnection.table_name = nil .
2) ExternalDatabaseConnection.clear_active_connections! and clear_all_connections!
3) tried with set_table_name :table1
edit2:
ClassName.column_names is not getting changed even after setting different table_name for the Class
I have tried this following way. It may help
:set_table_name method is used to change the table name dynamically. You can set the table name and get the attributes name instead of columns name.
For payments table
ExternalDatabaseConnection.set_table_name :payments
ExternalDatabaseConnection.first.attribute_names
Then you have to reset the column information
ExternalDatabaseConnection.reset_column_information
For patents:
ExternalDatabaseConnection.set_table_name :patents
ExternalDatabaseConnection.first.attribute_names
I use symfony in version 1.4.9 with doctrine.
symfony doctrine:build-schema works and create a schema in config/doctrine/schema.yml
symfony doctrine:build-model fails with the following message:
>> doctrine generating model classes
>> file+ /tmp/doctrine_schema_92251.yml
Invalid schema element named "class" at path "doctrine"
the tmp/doctrine_schema_922... is this:
doctrine:
class: sfDoctrineDatabase
param:
dsn: 'mysql:host=localhost;dbname=xxxxx'
username: xxxxx
password: xxxxx
attributes:
use_dql_callbacks: true
use_native_enum: true
Products:
connection: doctrine
tableName: products
columns:
product_id:
type: integer
...
any ideas whats wrong?
I don't think the first section of your schema is valid, I have never seen this before.
doctrine:
class: sfDoctrineDatabase
param:
dsn: 'mysql:host=localhost;dbname=xxxxx'
username: xxxxx
password: xxxxx
attributes:
use_dql_callbacks: true
use_native_enum: true
The database connection string should go in your database.yml file. In fact, this whole section resembles something that would go in the database.yml file. The remaining code in your schema looks good.