I have a scraper set up to use delayed_job so that it runs in the background.
class Scraper
def do_scrape
# do some scraping stuff
end
handle_asynchronously :do_scrape
end
Now I can comment out the handle_asynchronously line, open the console and run the scraper just fine. It does exactly what I expect it to do.
However, when I try to fire the scrape as a delayed job, it doesn't seem to do anything at all. Further to that, it doesn't seem to log anything important either.
Here's how my log looks from enqueueing a job to running rake jobs:work.
County Load (1.0ms) SELECT "counties".* FROM "counties" WHERE "counties"."name" = 'Fermanagh' LIMIT 1
(0.1ms) BEGIN
SQL (20.5ms) INSERT INTO "delayed_jobs" ("attempts", "created_at", "failed_at", "handler", "last_error", "locked_at", "locked_by", "priority", "run_at", "updated_at") VALUES ($1, $2, $3, $4, $5, $6, $7, $8, $9, $10) RETURNING "id" [["attempts", 0], ["created_at", Mon, 30 May 2011 21:19:25 UTC +00:00], ["failed_at", nil], ["handler", "---
# serialized object omitted for conciseness
nmethod_name: :refresh_listings_in_the_county_without_delay\nargs: []\n\n"], ["last_error", nil], ["locked_at", nil], ["locked_by", nil], ["priority", 0], ["run_at", Mon, 30 May 2011 21:19:25 UTC +00:00], ["updated_at", Mon, 30 May 2011 21:19:25 UTC +00:00]]
(0.9ms) COMMIT
Delayed::Backend::ActiveRecord::Job Load (0.4ms) SELECT "delayed_jobs".* FROM "delayed_jobs" WHERE (locked_by = 'host:David-Tuites-MacBook-Pro.local pid:7743' AND locked_at > '2011-05-30 17:19:32.116511') LIMIT 1
(0.1ms) BEGIN
SQL (0.3ms) DELETE FROM "delayed_jobs" WHERE "delayed_jobs"."id" = $1 [["id", 42]]
(0.4ms) COMMIT
As you can see, it seems to just inset a job and then delete it straight away? This scraping method should take at least a few minutes.
The worst part is, it was working perfectly last night and I can't think of a single thing I'm doing differently. I tried fixing the gem to a previous version incase it was updated recently but doesn't seem to have fixed the problem.
Any ideas?
Have you configured your delayed job to delete failed jobs? Look for the following setting in your initializer:
Delayed::Worker.destroy_failed_jobs = true
If yes then set it to false and look into the delayed_jobs table for the exception due to which it failed and debug further.
Related
I am keeping an eye on my rails server and my rails console. Everything looks great in my server.
When I go to create a Like, my server saves all the stuff I'm looking to save.
Notice that I am passing in an author_id, a likable_id, and a likable_type. This association is polymorphic.
Like Create (0.6ms) INSERT INTO "likes" ("author_id", "likeable_type", "likeable_id", "created_at", "updated_at") VALUES ($1, $2, $3, $4, $5) RETURNING "id" [["author_id", 1], ["likeable_type", "Pin"], ["likeable_id", 34], ["created_at", "2020-12-18 04:52:49.973371"], ["updated_at", "2020-12-18 04:52:49.973371"]]
↳ app/controllers/api/likes_controller.rb:8
(1.9ms) COMMIT
↳ app/controllers/api/likes_controller.rb:8
Rendering api/likes/show.json.jbuilder
Rendered api/likes/_like.json.jbuilder (0.4ms)
Rendered api/likes/show.json.jbuilder (2.4ms)
Completed 200 OK in 75ms (Views: 8.1ms | ActiveRecord: 20.6ms)
Great! I get the 200OK! Awesome.
However, when I check Like.all in the console, It looks like the Like is being created with none of the params I passed into it above.
[55] pry(main)> Like.all
Like Load (0.3ms) SELECT "likes".* FROM "likes"
=> [#<Like:0x00007fbef776c7e0
id: 6, #<--- Only the id for the like is being shown :(
created_at: Fri, 18 Dec 2020 04:30:50 UTC +00:00,
updated_at: Fri, 18 Dec 2020 04:30:50 UTC +00:00>]
[56] pry(main)>
I found the answer. For whatever reason, my schema was not bugged out and somehow had extra text. I dropped the table and re-migrated.
I have 2 models, Lanzadera and Addict.
Lanzadera has_many :addicts
Addict belongs_to :lanzadera
I have a list of Lanzaderas with a "Sign up" button next to them, that will render the Addict new form.
The user will then fill out a form with the Addict attributes, and that addict should be listed in that particular lanzadera. So, it's all about listing addicts in lanzaderas.
Everything looks great, I can see lanzadera_id in the logs, but when I say #lanzadera.addicts.count it will put 0. That means, addicts are being created but are not being assigned to it's lanzadera.
lanzadera_id is being permitted in addict_params in the controller strong parameters.
My question is, how can I pass lanzadera_id in my form when creating a new Addict, so that the addict gets listed within that Lanzadera?
Started POST "/lanzaderas/1/addicts" for 127.0.0.1 at 2014-07-10 19:25:41 +0200
Processing by AddictsController#create as HTML
Parameters: {"utf8"=>"✓", "authenticity_token"=>"mJ7pl2xsjhwhdGM2RZBDA+fmD75tpZcsPIaeSwYaBhE=", "addict"=>{"name"=>"Kike", "email"=>"kikeisasi#gmail.com", "city"=>"65", "postal"=>"56478", "street"=>"Aiboa 19, 4", "lanzadera_id"=>""}, "commit"=>"Crear Addict", "lanzadera_id"=>"1"}
(0.2ms) BEGIN
SQL (0.7ms) INSERT INTO "addicts" ("city", "created_at", "email", "name", "postal", "street", "updated_at") VALUES ($1, $2, $3, $4, $5, $6, $7) RETURNING "id" [["city", "65"], ["created_at", Thu, 10 Jul 2014 17:25:41 UTC +00:00], ["email", "kikeisasi#gmail.com"], ["name", "Kike"], ["postal", 56478], ["street", "Aiboa 19, 4"], ["updated_at", Thu, 10 Jul 2014 17:25:41 UTC +00:00]]
My guess is that you are missing the landazera_id in the Addicts table in your database. When it begins the INSERT INTO line it doesn't list the id as one of the columns. If you haven't, make sure 'landazera_id' migrated into the table. Rails doesn't do this automatically based on your has_many, belongs_to associations (or at least I've never figured out how to make it automatic) so you'll have to make the migration yourself.
rails g migration add_landazera_id_to_addicts landazera_id:integer
Check the migration in your text editor to make sure it generated correctly. It should include:
def change
add_column :addicts, :landazera_id, :integer
end
When i try to do
rake db:reset
or
rake db:drop
rake db:create
rake db:schema:load
rake db:seed
I randomly get a NoMethodError: undefined method 'fields' for nil:NilClass when the seed occurs.
Here's one:
[1m[35m (186.0ms)[0m INSERT INTO "schema_migrations" (version) VALUES ('20130318105449')
[1m[36m (187.0ms)[0m [1mINSERT INTO "schema_migrations" (version) VALUES ('20130319154146')[0m
[1m[35m (189.0ms)[0m INSERT INTO "schema_migrations" (version) VALUES ('20130322132730')
[1m[36m (104.0ms)[0m [1mINSERT INTO "schema_migrations" (version) VALUES ('20130322142814')[0m
NoMethodError: undefined method `fields' for nil:NilClass: SELECT COUNT(*)
FROM pg_class c
LEFT JOIN pg_namespace n ON n.oid = c.relnamespace
WHERE c.relkind in ('v','r')
AND c.relname = 'schema_migrations'
AND n.nspname = ANY (current_schemas(false))
Here's an other one:
[1m[35m (39.0ms)[0m BEGIN
[1m[36mCategory Exists (107.0ms)[0m [1mSELECT 1 AS one FROM "categories" WHERE "categories"."hash" = 'cat1' LIMIT 1[0m
[1m[35mSQL (124.0ms)[0m INSERT INTO "categories" ("category_id", "created_at", "label", "hash", "updated_at") VALUES ($1, $2, $3, $4, $5) RETURNING "id" [["category_id", 58], ["created_at", Mon, 25 Mar 2013 15:07:53 CET +01:00], ["label", "Cat1"], ["hash", "cat1"], ["updated_at", Mon, 25 Mar 2013 15:07:53 CET +01:00]]
[1m[36m (117.0ms)[0m [1mCOMMIT[0m
[1m[35m (162.0ms)[0m BEGIN
[1m[36mCategory Exists (136.0ms)[0m [1mSELECT 1 AS one FROM "categories" WHERE "categories"."hash" = 'cat2' LIMIT 1[0m
[1m[35mSQL (104.0ms)[0m INSERT INTO "categories" ("created_at", "label", "hash", "updated_at") VALUES ($1, $2, $3, $4) RETURNING "id" [["created_at", Mon, 25 Mar 2013 15:07:53 CET +01:00], ["label", "Cat2"], ["hash", "cat2"], ["updated_at", Mon, 25 Mar 2013 15:07:53 CET +01:00]]
[1m[36m (99.0ms)[0m [1mCOMMIT[0m
[1m[35m (108.0ms)[0m BEGIN
[1m[36mCategory Exists (173.0ms)[0m [1mSELECT 1 AS one FROM "categories" WHERE "categories"."hash" = 'cat3' LIMIT 1[0m
NoMethodError: undefined method `fields' for nil:NilClass: SELECT 1 AS one FROM "categories" WHERE "categories"."hash" = 'cat3' LIMIT 1
I really do not understand where this can be coming from. Sometimes i don't even have an error and everything is perfectly inserted.
After doing some tests it seems that it depends on the speed of insertion. If i execute this from my development server, it almost always succeed. If i execute this remotly from my computer, it almost always fails.
I'm using ruby 1.9.3 locally and ruby 2 on my development server. On both i'm using rails4 (edge) and my driver is postgresql (pg, no version specified)
I'm experiencing this problem as well on rails4 (edge) and postgres. I'm running into it on a solr:reindex, so I don't think its necessarily related to insertions in any way. I'll update this when I figure out whats going wrong.
Edit:
Try updating to rails revision 1a838ccda4a31bb023985f6c977e6bc3e238cda9, it solved my problem.
This is the github issue on it: https://github.com/rails/rails/issues/9710
I found it after a quick git bisect. It's definitely fixed in git now.
I had an annoying problem with a freshly generated scaffold. Everything would run nicely, but while I could enter data, everytime I tried to save he would commit nil successfully. For all fields.
Looks like this:
Started POST "/spraches" for 127.0.0.1 at 2012-08-26 23:34:03 +0200
Processing by SprachesController#create as HTML
Parameters: {"utf8"=>"✓", "authenticity_token"=>"nMf1+rBegI9BKoSAekzq8iLPQPAHpiiPk2DlmhEQxsQ=", "sprache"=>{"name"=>"23", "level"=>"23", "zertifikat"=>"124", "zertifikat_anders"=>"213", "zertifikat_note"=>"f23"}, "commit"=>"Create Sprache"}
[1m[36m (1.0ms)[0m [1mBEGIN[0m
[1m[35mSQL (29.0ms)[0m INSERT INTO "spraches" ("created_at", "level", "name", "updated_at", "zertifikat", "zertifikat_anders", "zertifikat_note") VALUES ($1, $2, $3, $4, $5, $6, $7) RETURNING "id" [["created_at", Sun, 26 Aug 2012 21:34:03 UTC +00:00], ["level", nil], ["name", nil], ["updated_at", Sun, 26 Aug 2012 21:34:03 UTC +00:00], ["zertifikat", nil], ["zertifikat_anders", nil], ["zertifikat_note", nil]]
[1m[36m (1.0ms)[0m [1mCOMMIT[0m
Redirected to http://localhost:3000/spraches/1
Completed 302 Found in 35ms (ActiveRecord: 31.0ms)
The reason become obivous to me after looking at the code in detail: I named my table in German (Sprache -> Language) and Rails tried to set it to plural. However, the controller only had it in the singular form.
#sprach = Sprache.new(params[:sprach])
I addded an "e" in the params to match the incoming code. Works.
The reason become obivous to me after looking at the code in detail: I named my table in German (Sprache -> Language) and Rails tried to set it to plural. However, the controller only had it in the singular form.
#sprach = Sprache.new(params[:sprach])
I addded an "e" in the params to match the incoming code. Works.
I have model Document:
class Document < ActiveRecord::Base
belongs_to :company
validates :name, :presence => true
end
And two classes inherited from Document:
License:
class License < Document
mount_uploader :file, DocumentUploader
end
And Certificate
class Certificate < Document
mount_uploader :file, DocumentUploader
end
And when I try to do current_company.licenses.create(...) or same action for the certificate, always all params are saved, besides file, which always is nil
Also I've tried to mount file inside of Document model... Help me please.
Here is logs:
Started POST "/companies/1/verified" for 127.0.0.1 at Mon Mar 19 09:33:41 +0200 2012
Processing by CompaniesController#verified as HTML
Parameters: {"verified"=>{"certificate"=>{"name"=>"Certificate", "file"=>"test.png"}, "insured"=>"2000000", "suppliers"=>"", "license"=>{"name"=>"License", "file"=>"test.png"}}, "authenticity_token"=>"0hIn41Tjonm/AXZBKM1PE/tjQxJDLqZaojMTHDoZq2k=", "id"=>"1", "utf8"=>"✓", "commit"=>"Update verifications"}
Company Load (0.7ms) SELECT "companies".* FROM "companies" WHERE "companies"."id" = 1 LIMIT 1
(0.1ms) BEGIN
SQL (0.8ms) INSERT INTO "documents" ("company_id", "created_at", "file", "name", "type", "updated_at", "verified") VALUES ($1, $2, $3, $4, $5, $6, $7) RETURNING "id" [["company_id", 1], ["created_at", Mon, 19 Mar 2012 07:33:41 UTC +00:00], ["file", nil], ["name", "License"], ["type", "License"], ["updated_at", Mon, 19 Mar 2012 07:33:41 UTC +00:00], ["verified", false]]
(0.6ms) COMMIT
(0.1ms) BEGIN
SQL (0.5ms) INSERT INTO "documents" ("company_id", "created_at", "file", "name", "type", "updated_at", "verified") VALUES ($1, $2, $3, $4, $5, $6, $7) RETURNING "id" [["company_id", 1], ["created_at", Mon, 19 Mar 2012 07:33:41 UTC +00:00], ["file", nil], ["name", "Certificate"], ["type", "Certificate"], ["updated_at", Mon, 19 Mar 2012 07:33:41 UTC +00:00], ["verified", false]]
(0.4ms) COMMIT
Redirected to http://localhost:3000/company/profile
Completed 302 Found in 18ms (ActiveRecord: 3.3ms)
Thanks.
Are you sure you've set the form that you've uploading the file with to have a multipart payload?
If not the file won't get sent on submission, and rails will only receive the textual form data. Inside the form_helper tag you'll need to add.
:html => {:multipart => true}