I am working on building a Ruby on Rails v2.3.8 real estate application. I've written a rake task to pull in the property listings and photos. The app uses rets4r (specifically josephholsten-rets4r) to talk to a RETS-based server and Paperclip is used to manage the property photos and push them up to S3.
The problem I'm seeing is the rake task never completes. It will always crash at some point and it's never the same spot. One time it crashed after 45 min, after some minor changes (added some begin/rescue blocks to try and track down the error), it seems to run longer (2-12 hours), but still crashes.
The error I get is always "Rake aborted" and then "thread [memory address] tried to join itself." Looking into the logs provides nothing helpful. Just the standard Paperclip log messages. I'm also not deliberately trying to do anything multi-threaded in the rake task.
Update: I was using the gem version 1.1.17 of rets4r. However, I notice on the Github page for the project, there have been quite a few commits since that version was tagged. One of those commits talks about removing the use of threads.
Related
I am following Hartl's Rails Tutorial and using Rails 6. I keep getting persistent errors that seem to pop up at random whenever I run tests - random because the consecutive tests sometimes indicate errors in different areas. These tests are also very very slow - > 30 minutes sometimes. Has anyone encountered this? What could I be doing wrong? And now for the red herring: I am using Win 8.1 :)
The common thing about these errors messages is that they all contain a "RuntimeErroer: database is locked" message. Here's one of them:
ERROR["test_email_validation_should_reject_invalid_addresses",
#<Minitest::Reporters::Suite:0x000000000c9b29c0 #name="UserTest">, 608.7059665989946]
test_email_validation_should_reject_invalid_addresses#UserTest (608.71s)
RuntimeError: RuntimeError: database is locked
I've been fighting this same error for quite sometime. It actually go so bad it roadblocked me from really moving forward in Hartl's Rails Tutorial.
Edit: Found a much better answer that simply resolves the problem instead of playing with settings I probably don't really understand.
See -> https://stackoverflow.com/a/62730905/10463184
My only contribution, as a windows user, I found that commenting out the entire line in test/test_help.rb...
parallelize(workers: :number_of_processors, with: :threads)
Resolved the issue. Trying the setting suggested at the link resulted in a "UNIXServer is required (LoadError)" error.
Here's a solution to one of my challenges - the interminablly slow test speed. In the config/database.yml I added (to the test.sqlite part) the following lines:
database: db/test.sqlite3
pool: <%= ENV.fetch("RAILS_MAX_THREADS") { 5 } %>
timeout: 3000
The test duration dropped from minutes to mere seconds. Victory!
Alas, the "RuntimeError: database is locked" messages are still there, however.
In windows you can try this program http://www.nirsoft.net/utils/opened_files_view.html to find out the process is handling db file. Try closed that program for unlock database
In Linux and macOS you can do something similar, for example, if your locked file is development.db:
$ fuser development.db
This command will show what process is locking the file:
> development.db: 5430
Just kill the process...
kill -9 5430
...And your database will be unlocked.
Or
it should be a database's internal problem...
For me it has been manifested after trying to browse database with "SQLite manager"...
So, if you can't find another process connect to database and you just can't fix it, just try this radical solution:
Provide to export your tables (You can use "SQLite manager" on Firefox)
If the migration alter your database scheme delete the last failed migration
Rename your "database.sqlite" file
Execute "rake db:migrate" to make a new working database
Provide to give the right permissions to database for table's importing
Import your backed up tables
Write the new migration
Execute it with "rake db:migrate"
I found myself in the same situation.
In my case this error happened because the tests run in parallel by default on MiniTest, which means that they also run in parallel in rails. When this happens and there are tests that make transactions on the tests db, the db locks itself causing the error.
The solution that worked for me was setting the attribute "use_transactional_tests" to false in the test class.
You can see the proper usage and an example in the docs.
In case anyone else makes the obvious mistake that I did, be sure to exit out of any rails console sessions in your terminals before trying to delete on your live site or in testing.
I am running into a very odd issue and coming up short with identifying the issue. I don't even know if it is a front-end, server or Heroku issue.
I am managing a Rails App (with React front-end), hosting on Heroku. It is a workforce management and dispatching app. It has been working fine the whole time, until earlier today, when I made a huge feature deployment. The deployment was successful, and I tested the old & new features fine on all of our testing computers. After deployment, I ran heroku run rake db:migrate and heroku restart. During this time, we have customers using the web app on their local computers, in different time zones.
Few minutes later, I got a call from multiple customers, each mentioning that at least one of their dispatching features is not working, all of which were older features/components that shouldn't have been affected. Some say they cant see their workers. One say they cant see jobs on a particular day. Some say they cant see jobs on another. When I check the heroku logs, I see no errors whatsoever. Routes are called fine and things are passed correctly back to the front-end. When logging into their app with master credentials, we also were unable to replicate ANY of these issues on ANY of our computers. All of our customers uses Chrome, but checking on both Mac and Windows reveal no issues from our ends. Yet, the issue still remains.
We then decided to revert our deployment and deployed an earlier version. Same thing. Everything works on ours, and customers continue to say these features/components just appear...blank. We tried heroku run rake tmp:clear to clear heroku cache, still no luck.
A couple months back I finally upgraded a Rails app on Heroku to the Cedar stack and Rails 3.2. Since it was suggested in the Cedar documentation, I created an initializer with this one line Rack::Timeout.timeout = 20. Everything worked fine for a long time, but in the past week I've suddenly begun getting numerous Timeout::Error: execution expired errors and associated NoMethodError: undefined method 'result' for #<Timeout::Error: execution expired> errors.
All of the errors are in mundane parts of the site that display basic content from the database, there's no interactive action going on that could take anywhere close to 20 seconds to process under normal conditions.
Is this the kind of thing that happens when traffic increases to the point where I need more dynos? Any advice you can give as to the source of these mysterious errors would be appreciated.
A few recent changes that I made to the site:
Upgraded from Heroku Dev db to Heroku Basic db
Explicitly defined the Ruby version in my gemfile ruby '1.9.2'
Added IronWorker (from iron.io) to do some background tasks (for admins only, not the everyday users who are seeing the errors) I'm setting it up in config/application.rb with the following line: config.iw_client = IronWorkerNG::Client.new(:token => iw_conf['iron']['token'], :project_id => iw_conf['iron']['project_id'])
Update: I checked New Relic, and there's an extreme issue with the Heroku Basic DB performance. Over the past 24 hours, the database response time has hovered towards the higher end of 1000ms to 6000ms. Is this an error on my part, or is it an issue with Heroku?
I'm completely new to Ruby on Rails, have never worked on it, but I am supposed to take over somebody else's old project. He designed a webapp that logs into a website/server online, extracts data from it daily and performs calculations. However, the application has not been running for quite some time, so now when it tries to display statistics, the page crashes. It uses data from a 5 week period and currently only has data for 2 days.
I need to manually insert data for the missing weeks in order to get it up and running again, but the problem is I don't know how to find/access its database, nor how exactly to use Ruby on Rails. I found several files in the db directory of his project, however they were mostly .sqlite33 files or just 'files'. I downloaded sqlite precompiled binary for Windows and tried to use it, but not sure what to do. I also downloaded and tried using SQTView to open the files to change the tables, but I have had no luck.
How can I tell which file is the main database and how do I edit it? And what are .sqlite33 files? Thanks.
EDIT
The application produces a
TypeError in HomeController#index
"nil can't be coerced into Float"
It links the error to a line in the code, but the only reason the error happens is because there is no data in the table for the time period that the variable in this line of code tries to use. That is why I want to manually update it with values.
I think that Sqliteman might do the trick and thank-you for explaining all of this to me. I just have to confirm it works after editing.
EDIT2
Okay, so I had a small revelation while experimenting on his app. It turned out that the page crashed because of empty values for full weeks. If I used the app to gather data for at least one day per week up until this week, then the app worked.
Is there a way I can populate all the missing days without having to manually click the days in its calendar because it takes a good 20-30 seconds for it to load?
Thank you very much for your help.
A Rails application's database is defined in config/database.yml - though the database itself and the scheme are in the db folder. In database.yml you'll find three database configurations - development, test and production - so make sure to choose the right file.
I've never heard of .sqlite33 files, but to edit SQLite files manually I recommend SQLiteMan. But I don't recommend editing it manually.
What I do recommend is to check if you are running the app in development or production mode, as the two mods use different databases. Try to run it in the other mode.
If that doesn't work, try saving a copy of the database files, and running the commands:
bundle install
bundle exec rake assets:precompile
rake db:migrate
rake db:migrate RAILS_ENV="production"
(if you're using Linux, you might want to also run the first command with sudo)
Those commands make sure all the required gems are installed, that the precompiled assets are up to date, and that the database fits the schema. If you are running that application on the same machine and with the same database as the ones the other guy was using, than those commands shouldn't be necessary, but if you have moved the application to your own machine, or used an used an outdated db file, than those commands might fix the crash.
At any rate, if you tell us what error the application is producing we might be able to provide more assistance.
I am getting up and running with Spree in production on a 256MB Slicehost slice with MySQL, Mongrel, and Nginx. I am getting this error every time I try to access any URL in the app:
SystemStackError (stack level too deep):
compass (0.8.17) lib/compass/app_integration/rails/action_controller.rb:7:in `process'
vendor/plugins/haml/rails/./lib/sass/plugin/rack.rb:44:in `call'
(Some more test url requests are in this pastie)
What are the advanced (but free and open source) debugging strategies/tools I should use while in production mode so I can do better at solving this problem?
It's slightly different each time depending on the controller. I have tried increasing my stack size, but that doesn't help because there's probably just some loop I can't find.
The question is, how do I get a more detailed report of the exact output while in production?
I am using Deprec and Capistrano to execute all server side commands. I am running a mongrel cluster of 3. The log files I see are:
current/log$ ls
development.log
testsite.com-error.log
mongrel.8000.log
mongrel.8001.log
mongrel.8002.log
mongrel.log
production.log
testsite.com-access.log
I'm just looking for some advanced debugging strategies/tools while in production mode so I can do better at solving this problem. Maybe some cap tasks for streaming/debugging (or conditionally break-pointing!) the production app (really, a staging app) locally (do people do that?), or something that spits out every single thing that executes in code?
Thanks
I'd check all of your gems and plugins and any other dependencies your application might have: I'd guess that you're missing something (or have the wrong version of something) and it's failing badly enough that it can't even spit out a can't find require message.
check your action controller. On line 7 is most likely where the loop occurs.
Seems you have initiated some endless recursion on line 7 of your controller. Check if you are calling the same method inside the method.