I'm building a Rails server with the model stored in MongoDB using Mongoid.
There are cases where a user can attempt to add a document to the mongo database with a duplicate index value. Is there a way to retrieve the MongoDB error code (in this case 11000) without parsing the error message so that I can make my exception handling more robust?
EDIT: Title had Mongoid::Errors instead of Moped::Errors
I developed the mongoid_token gem and encountered this exact problem, since the core functionality of this gem relies on being able to identify if a particular field (in this case, the token) is the cause of the key duplication.
If all you're after is the error code, the yes - you can get this. However, if you need more precise details (such as the field name), you will need parse the error description.
Also, if you're testing for duplicate keys, I think you'll need to check for both error codes 11000 and 11001 (duplicate key on update). A partial list of the mongoDB error codes is here.
I've paraphrased some of the code from the gem below:
begin
#... do whatever
rescue Moped::Errors::OperationFailure => e
description = e.details['err']
if [11000, 11001].include?(e.details['code'])
# Duplicate key error
end
end
Related
I am getting this type of error in the logs :
Parameters: {"id"=>"4", "step"=>{"documents_attributes"=>{"0"=>
{"file"=>"\x89PNG\r\n\u001A\n\u0000\u0000\u0000\rIHDR\u0000\..."}}}}
def update
#step = Step.find_by(id: params[:id])
if #step.update(steps_params)
render :json => #step
else
render :json => { :responseStatus => 402,
:responseMessage => #step.errors.full_messages.first}
end
end
During update, it rollbacks without giving any error (not execute else condition)
ArgumentError (invalid byte sequence in UTF-8):
(0.2ms) ROLLBACK
How can I fix or handle this type of request?
Your question is how to handle this type of request or error. So here is my suggestion of a general strategy.
First, do your homework. You could easily find this past question, for example. If you have tried the way already but found it did not work, you should have described what you did and what did not work in your question.
Now, I am assuming you can reproduce the case or at least you can expect you will encounter the same problem in near future (or you can wait till then) so you will have a more chance to pin down the problem next time. If you know what parameters caused the error, I guess you can reproduce the case in your development environment. However, if not, it is more tricky to pin down — it heavily depends how much information about the error and input you have and what development environment you can use, and my answer does not cover the case.
The first objective should be to pin down which command (method) exactly in your code caused an error. Did it happen just inside Rails or did your DB raise an error?
In your specific case, did it occur at Step.find_by or #step.update or else? What is steps_params? It seems like a method you have defined. Are you sure steps_params is working as expected? (You may be sure, but we don't know…)
A convenient way to find it out is simply to insert logger.debug (or logger.error) etc before and after each sentence. In doing it, it is recommended to split a sentence into smaller units in some cases. For example, steps_params and update() should be separated, such as (in the simplest case),
logger.debug 'Before steps_params'
res_steps_params = steps_params
logger.debug 'Before update'
res_update = #step.update(res_steps_params)
logger.debug 'Before if'
if res_update
# ……
Obviously you can (and perhaps should) log more detailed information, such as, res_steps_params.inspect, and you may also enclose a part with a begin-rescue clause so that you can get the detailed infromation about the exception and log it. Also, I can recommend to split update into 2 parts – substitutions and save – to find out exactly what action and parameter cause a problem.
Once you have worked out which of DB or Rails or something before (like HTTP-server or Client-browser) is to blame and which parameter causes a problem, then you can proceed to the next stage. The error message suggests it is a character-encoding issue. Is the character encoding of a string invalid (as a UTF-8), or wrongly recognised by Rails (which might be not a fault of Rails but of the client), or not recognised correctly by the DB?
Wherever the problem lies, it is usually (though not always!) possible to fix or circumvent character-encoding problems with Ruby (Rails). The Ruby methods of String#encode, String#encoding, and String#force_encoding would be useful to diagnose and perhaps fix the problem.
As an added note, it can be useful, if possible in your environment, to browse the logfile of your DB (PostgreSQL?) to find out which query passed from Rails to the DB caused a problem (if a query was indeed passed to them!). Alternatively, Rails Gem SQL Query Tracker might be handy to know what queries your Rails app create (though I have never used it and so can't tell much.)
At the end of the day, when a code misbehaves mysteriously, I am afraid only the sure way to solve is to narrow down the problematic clause or parameter step by step. Good luck!
In my Rails 5 app, I'm using the Ahoy gem which uses a jsonb field (called properties) that I don't know how to query.
A sample of the data in the properties field:
"{\"id\":\"11\"}"
What I'm trying to do is:
Find the records created in the last seven days (using the time field)
Group the records by the id in the properties field
Order the records based on which id had the most records
I tried this:
#foo = Ahoy::Event.where(name: "Viewed Product").where(time: 7.days.ago..Time.now).group("properties(id)").order('COUNT("properties(id)") DESC')
but I received the error, PG::UndefinedColumn: ERROR: column "properties(id)" doe not exist
I found this somewhat similar question in the issues for the gem, so I tried breaking the query into parts:
#foo = Ahoy::Event.where(name: "Viewed Product").where(time: 7.days.ago..Time.now)
#foo.group("properties REGEXP '[{,]\"id\"...).count
but then my IDE gives the error unterminated string meets the end of file.
Can someone please help me determine what I'm doing wrong?
If you are using PostgreSQL, I believe that you would have to refer to json columns like this:
.group("properties ->> 'id'")
I want provide as much feedback as possible to users trying to import data from csv. The problem is on the cases I need to rescue CSV::MalformedCSVError because the exception message is in english(I can't use english) and there is no other way to distinguish one error from another.
What can I do to show the exact problem to the user?
I noticed you tagged this as Rails so that means you have I18n. Why don't you do something like the following:
begin
# csv parsing code
rescue CSV::MalformedCSVError => ex
raise(CSV::MalformedCSVError.new(I18n.t("csv_parser.malformed_csv_error"))
end
Note: this does assume CSV::MalformedCSVError inherits from StandardError which might not be the case but you get the idea, raise an exception and set the message to some translated I18n string.
UPDATE:
If you wanted even more detail you could match against the exception message and have translations for each message type, while capturing the data you want from the error message string - for instance line number etc. I have no idea right now what the error messages look like but say you have something like "error in column 45" then you could do the following
begin
# csv import code
rescue CSV::MalformedCSVError => ex
err_message = case ex.message
when /column (\d+)/
I18n.t("csv_error.column_error_message", column: Regexp.last_match[1])
else
I18n.t("csv_error.generic_message")
end
raise(CSV::MalformedCSVError.new(err_message))
end
UPDATE: The GEM
I've taken this monkey patch and turned it into a gem. Check it out, along with a test suite:
https://github.com/jhubert/csv-i18n
Original Answer
I bumped into this problem and decided that unless I was ok with showing unhelpful error messages to my users, I could go with two options:
Introduce a new class like FriendlyCSV that wrapped the CSV parser, examined the exception message and returned a relevant error message in the right language.
Monkey-patch the CSV library to return translated error messages.
The first way is probably the proper way to go, since I control the source code and can use whatever class I want to process the incoming CSV files.
However, I'm far too reckless for that and I liked the idea that ANY CSV parsing done anywhere in the system returned translated messages, without other people needing to be aware of the FriendlyCSV class.
So, I went with #2.
The Monkey Patch
You can find the patch here:
https://gist.github.com/jhubert/7d75586857d41fb4c45c4491363636e9
The core behavior is that we're overwriting the shift method, which is where the bulk of the string parsing is. We're attempting to translate the error message and then returning the original exception.
def shift
super
rescue ::CSV::MalformedCSVError => exception
raise $!, translated_exception_message(exception.message), $!.backtrace
end
Hope that helps!
I have adopted a Rails app that has a lot of complex relationships. We started getting ActiveRecord::RecordNotFound error and am trying to track it down.
Is there a way to wrap relationships in a begin ... rescue block to determine which one is causing us problems similar to what they are doing here for a find method? Or is there a way to log that exact SQL call that is getting the RecordNotFound error?
Edit
I am not able to find post I was referencing but I really just need to find the relationship that is busted. In my logs, I just see that it is rendering the template for 'not_found' records but I'm not sure what is causing it.
Without your relationships & logs, everything is conjecture; but if you're getting a RecordNotFound error - it basically means you're trying to load a record which doesn't exist in your db
Is there a way to wrap relationships in a begin ... rescue block to
determing which one is causing us problems similar to what they are
doing here for a find method
No - a better way to debug is to find the logs in your /log/development.log file & then display the response here.
A pointer is that I don't think the relationship will be the issue. The relationship will just return null if nothing is there; the RecordNotFound error will be a result of ActiveRecord not being able to find the requested resource
EG
#post = Post.find 13 #-> RecordNotFound if a post with id 13 does not exist
If you post your logs & code, it will be the best way to help us solve your issue!
Firstly, sorry about the essay, I'm trying not to provide any actual application code and hope there is some logical explanation just because the problem is so weird.
I have spent the past 2 days debugging some code that works 95% of the time, the error seemed simple to debug at first but now I find myself scratching my head with no answer(s).
Some background
We are running:
Ruby 1.8.6, Rails 2.3.2 & Postgresql 8
We still have to migrate to 2.3.8, so unless the solution lies within upgrading to rails 2.3.8 don't advise me on it ;)
In a nutshell
We have a script that scans through CSV files. I store all the relevant columns into hashes by row and then proceed to iterate through the hashes afterwards, calling various ActiveRecord models and methods to store the data into our database.
The problem
The most important data that needs to get saved have method calls that are wrapped inside a Transaction block, so if any errors are raised, no data should get inserted into our core tables.
The table in question has 2 foreign keys that BOTH need to be present in order for the rails application to function as expected.
Like I said, ~95% of the time while processing our data, these values get inserted correctly.
The other ~5% of the time the one foreign key value does not get saved at all, and for no good reason.
I have saved the script flow and object/variable output into a log file and stepped through/scrutinised the log file for the instances where the foreign key was missing.
The objects/variables that I use as foreign keys were all reported as 'saved' by ActiveRecord in each instance where I inspected the objects after being 'saved'.
One thing that might be worth noting is that the value that gets saved into the foreign key column in question gets computed outside of the Transaction block - but I don't see why that would be a problem, seeing as I can output and use the value further down the line.
Simplified code flow
#Returns ActiveRecord object
fk2_source_object = method_to_compute_fk2(x, y)
#logger.info fk2_source_object.inspect
begin
Transaction do
begin
fk1_source_object = FK1Model.new
#etc, etc.
fk1_source_object.save!
#logger.info fk1_source_object.inspect
object_in_question = ObjectInQuestion.new
object_in_question.fk1_source_object_id = fk1_source_object.id
#FK2 >> This is the value that does not reflect in the database!? even if i could inspect it and see an id after calling .save!.
object_in_question.fk2_source_object_id = fk2_source_object.id
begin
object_in_question.save!
#At this point it shows that the object has saved, all values have been set, but it does not reflect in the database?
#logger.info object_in_question.inspect
rescue
#Logger.error "error message"
raise ActiveRecord::Rollback
end
rescue
#Logger.error "error message"
raise ActiveRecord::Rollback
end
end
rescue Exception => e
#logger.error e.inspect
end
#etc, etc.
I'm currently grasping at straws, going to wrap the entire section into the Transaction block.
I cannot recreate the error on my development box, quite frankly if i rerun the csv files on the server, the values get inserted correctly the second / third time. (So it's not a data source problem)
I'm starting to worry that it may be a rails / postgresql 8 problem?
Am I losing the plot or what can be the causes of this?