I have a rails application and I am trying to load data into it via PostgreSQL's COPY command. The file is CSV. The table maps to a Rails model and I need to retain the primary-key ID column so I can use model.for_each to play with the data--the table will have 5M+ rows.
My question is, how can I structure my CSV so that I can load data into the table and still allow the ID column to be there? It's a bummer because the only reason I need it is for the for_each method.
Regardless, I tried sending NULLs, as in:
NULL,col1,col2,col3,col4, etc.
But it doesn't work.
Thanks.
Passing in null for the primary key will never work no matter what options you have set for null string. You would be telling the backend to set the primary key to null which it will never allow no matter what the command to insert might be.
I really have no idea what you mean by retaining the primary key. That is something that is going be retained no matter what do. If you mean letting the DB pick the value for you and the primary key is a serial (auto-increment) then explicitly name all the columns but the primary key:
COPY country (colA, colB, colC) FROM '/usr1/proj/bray/sql/country_data'; -- leave out pkey
It also might be quicker to read the documentation on what null string options you would like to use instead of guessing possible values:
http://www.postgresql.org/docs/9.0/static/sql-copy.html
The default when using WITH CSV is an unquoted empty string, such as:
,col1,col2,col3,,col5
Which would create a record that looks like:
NULL,'col1','col2','col3',NULL,'col5'
Related
I have a User's information which I have stored in a Hash. I have his permission information stored in a Set.
I want to test for a condition on the Set, retrieve the Key and then fetch the actual Value using the retrieved Key from the Hash.
How can I do this?
Do I have to use Lua scripting to achieve the above goal or is there any other way?
You'll have to do it using multiple requests in that case. While Lua appears to be the way, your requirements - namely fetch a key based on the value of another key - rule out a single script since key names should be passed explicitly.
I had the same requirement and I couldn't find any other way to do this. Redis pipeline and transaction functions are other tools to execute multiple commands in single step but for your situation where you want to get a value and execute a command based on it, the only way remains is LUA!!
you can use a hash to handle user information and his permissions (in the same hash). In this case, when you have a userId you can retrieve the permissions with one shot by using userId and permissionKey. See example below, I put the permissions as string (in your case put it as Set):
redis> HSET userId permissionKey "perm1, perm2, perm3"
redis> HGET userId permissionKey
"perm1, perm2, perm3"
Hope this help you.
I need some architectural advice. I'm more into java, but trying to get up to speed with Ruby-on-rails. In the app I am building I need a convenient place to store some dictionary values that will be later used in various places of the application. These will be usually key value pairs - e.g. list of values to be used in select list.
The main objective is to keep this logic in one place of the application.
I am considering following options:
Store values in the database - i'm kind of reluctant from that, as values won't change very often.
Put all of the values in one class. In JAVA I'd have some static properties in one class holding this values (e.g. call Utils.getStates() will return list of states). How to do it ruby way?
Have some .yml file with values - read from the values. How to do it? I guess I have to parse the file in the initializer, but is there any tutorial how to do it?
Precise example? Let's say that have a model that have a field called "Type". Type can be: ['Type A', 'Type B', 'Type C'...]. And of course, for each type I want to have key and value.
I'd appreciate some suggestions about how you solve this problem in your apps.
Thanks,
Maciek
How often does the list change? Is it acceptable to have developers involved each time a value changes (updating code, re-deploying the app)? If the answer is no then store the values in a database.
Is the list of values reuseable? Then a gem or a yaml file with an initializer might be a good choice.
Is it just a small list and does not change often? Then you might want to consider a constant.
I think in Rails any data that would change at runtime and needs to be persisted, would normally be stored in the database. I think that would be the "rails way". You could save the data to yaml or json file, but that would not follow the normal flow of the MVC pattern that is so common in rails
i have this query:
insert into orders (customers_id, customers_name) values ('51064', 'Šample Šample')
If i execute this query from PHP, my database record become
[51604, '?ample ?ample'] (if executed with MySQLi) or
[51604, 'Šample Šample'] (with mysql_query)
I also noticed that if i use the value from $_GET
insert into orders (customers_id, customers_name) values ('51064', {$_GET['name']})
it becomes [51064, 'Åample Åample']
BUT if i insert manually the query using software like 'Navicat' it saves the query with the correct character (so i think that the charset is the right one)
I need to save the character Š (and many others) in the right way from PHP.
Set the charset to be used when sending data back and forth from the database server.
$mysqli->set_charset("utf8")
This is preferred over
// Will not affect $mysqli->real_escape_string();
$mysqli->query("SET NAMES utf8");
See http://www.php.net/manual/en/mysqli.set-charset.php and http://www.php.net/manual/en/mysqlinfo.concepts.charset.php
set Mysql encoding before using the DB.
mysql_query("SET NAMES 'utf8'");
OR use the utf8 utf8_encode() utf8_decode() functions
I am using propel orm as database abstraction layer.
I want to define different schema.xml files for diffenrent modules.
E.g. user.xml.schema for handling users and roles, or app.schema.xml for the application model.
My problem is, I want to reference to the user table of the user.schema.xml. I can handle this by the tag, but I want to use another table prefix for the user schema tables.
Running propel-gen creates two sql files (one for the user.schema and one for the app.schema), but the problem is, that the user-schema tables are generated twice. First with correct table prefix of the user schema file and the others with the prefix of the app schema file.
The foreign key also references to the wrong tables (that from the table with the prefix of the app.schema.xml).
I do not know any way to prevent this behaviour.
Any hints?
You cannot add different table prefixes to one database, and there is no way to add a tablePrefix attribute on the table tag. Let me explain a bit more, I know, you can specify a tablePrefix per XML schema even for a same database, but it leads to errors if you try to add relationship.. I don't know whether it's a bug or not, AFAIK the tablePrefix should be defined at the table level… Without relationship, you'll get a clean SQL file (or two if you don't set the propel.packageObjectModel build property to true.
You can read: http://www.propelorm.org/reference/schema.html. So I think, it's not possible to do what you would like to do unfortunately.
BTW, what you want to achieve is called multi component data model in the Propel doc.
A Cobol program with file-control like so:
SELECT D-FLAT-FILE ASSIGN TO DFLAT-FILE
ORGANIZATION IS INDEXED
ACCESS MODE IS SEQUENTIAL
FILE STATUS IS RECORD-STAT
RECORD KEY IS D_KEY OF D-FLAT-FILE DESCENDING WITH DUPLICATES.
SELECT C-MAST-FILE ASSIGN TO CMAST-FILE
ORGANIZATION IS INDEXED
ACCESS MODE IS DYNAMIC
FILE STATUS IS RECORD-STAT
RECORD KEY IS C_KEY OF C-MAST-FILE.
reads a record from the first flat file like so:
PROCESSING.
READ D-FLAT-FILE NEXT RECORD
AT END ....END READ.
and reads a record on the second DYNAMIC file like so:
READ C-MAST-FILE RECORD
INVALID KEY
GO TO PROCESSING.
All works well except for 1 case. If the 1st record from the 1st flat file does not match any records on the 2nd dynamic file, the program goes into an infinite loop instead of doing GO TO PROCESSING. I checked the manuals, all as per manual (it is the VAX Cobol). What am I missing?
Best practice is to use a different FILE STATUS variable for each file. In your case you haven't shown us enough context to see the problem. But if you are in a loop looking at RECORD-STAT, then it is possible that the failed read from C-MAST-FILE is giving you grief.