While importing a csv file to mongodb , a field having two new lines in it. When I am importing it , it is giving an error showing "csv file ends while inside quoted fields". can anyone help me how to sort this problem
Related
I have 20 files (csv files) in the same bucket. I am able to read all the file in one go and load on to bigquery. But when there is some data type mismatches, im able to get that row into invalidDataTag where as i am unable to find the file name that has the error record.
inputFilePattern is gs://bucket-name/* this picks up all the files that are present under the bucket. and reading the files as below
PCollection<String> sourceData = pipeline.apply(Constants.READ_CSV_STAGE_NAME, TextIO.read().from(options.getInputFilePattern()));
Is there a way where i can find the file name that has the error row in it ?
My suggestion would be to add a column to the BigQuery table that indicates which file the record came from.
I am running Neo4J on Docker within Vagrant.
I am attempting to LOAD CSV WITH HEADERS from a file within the /import/ directory (I had to move my file there) via a cURL request. My request looks something like this:
"LOAD CSV WITH HEADERS FROM \"file:///insert-neo4j.csv\" AS row ...
This provides me with the following error:
{"results":[],"errors [{"code":"Neo.ClientError.Statement.ExternalResourceFailed","message":"Couldn't load the external resource at: file:/var/lib/neo4j/import/insert-neo4j.csv"}]}
It is often suggested to me that I append the following to my '/conf/neo4j.conf' file, however this file DOES NOT EXIST, and creating it manually does not seem to work...
dbms.directories.import=import
dbms.security.allow_csv_import_from_file_urls=true
So I created the file /conf/neo4j.conf with the above variables, and I also tried adding these as environment variables to my docker-compose file. I seem to continuously have no luck uploading via CSV this way.
My questions are:
Is there anything blatantky wrong with this implementation?
Why does my /conf/neo4j.conf file NOT exist and how can I get it created?
Thank you
(p.s. my insert-neo4j.csv has -rwxr-xr-x)
The error message indicates it found the file but there is an error in the CSV ... most likely the formatting. Check this and if you can't see it, please post a few rows, including the header, of it so we might help.
I generate a CSV text file in Rails like this:
CSV.generate(col_sep: ';') do |csv|
sheet.add_row ['1st line']
sheet.add_row ['2nd line']
end
When I open the text file the two lines are there as expected. Unfortunately this file now should be used by a program that reads the file and I get an error message, that the second line is missing. I have a sample file that looks exactly like the file I generated which works fine but my file can't be read properly. It also has the same encoding. Any suggestions where to look? Anything concerning line breaks?
I'm not sure this is a question that can be answered as asked. You said that a 3rd party program is having trouble reading a text file generated by Ruby, but provided no information on that error and how you think Ruby is related to this error.
Could you please update your original post with the plaintext version of your CSV file and what program you're trying to open it in?
I'm working on a project with some supplied CSV files that I need to parse and do some manipulation on. One is throwing this error when I try to load it into a file using CSV.read('path/file.csv')
CSV::MalformedCSVError: Unquoted fields do not allow \r or \n (line 7911).
Now when looking at the file, the last line is just blank. It's a \n character. I feel like this should not break the CSV read but it is. Now, I could just check the end of the CSV documents and strip any access return carriages/new lines since that seems like it'll work but it doesn't seem like the correct way. Anybody have some advice?
Edit: Using Ruby 2.0.0 and Rails 4.0.5
Hi um facing an issue of creating a blob to a file and save it to a binary column in a postgre sql using rails . Still dont have an idea of how to start it. I would be happy if any one can tell me a way to do it.
file.each_line do |line|
line = Iconv.conv('utf-8', 'ISO-8859-1', line)
I want to save a file as Binary data (as a binary large object) and the file contains string
if you are looking for a gem that does this, carrier-wave should be helpflul: https://github.com/diogob/carrierwave-postgresql