I've a problem with a slow post request in a RoR app.
the project is a distribuited system wher the client send a lot of photos and the server save those in DB.
I use MySql server 5.1, both client and the server are in the same local network and they are programmer in RoR....
the client send 10 photo in a single request. the 10 photos are send in an array structure in yaml format.
the request is:
res = Net::HTTP.post_form(uri, :mac => 'String', :value => 'Yaml_array_images_data', :num => 10)
and the log says:
`Started POST "/events/save_photo" for 192.168.0.113 at 2012-03-30 09:45:10 +0200
Processing by EventsController#save_photo as */*
Parameters: {"data_type"=>"image", "value"=>"--- \n- !binary |\n /9j/2wCEAAoH
BwgHBgoICAgLCgoLDhgQDg0NDh0VFhEYI...", "created_at"=>"--- \n- 2012-03-30 09:26:29 +02:00 \n-....", "mac"=>"00606E91E5D2"}
(0.3ms) SET PROFILING=1
Device Load (1.2ms) SELECT SQL_NO_CACHE `devices`.* FROM `devices`
WHERE `devices`.`mac` = '00606E91E51E' LIMIT 1
CACHE (0.0ms) SELECT `devices`.* FROM `devices` WHERE `devices`.
`mac` = '00606E91E51E' LIMIT 1
(0.3ms) BEGIN
SQL (2.9ms) INSERT INTO `multimedia` (`created_at`, `data`, `eve
nt_id`, `updated_at`) VALUES ('2012-03-30 07:45:00', x'f22b19237b63f1c8c40da49ae
5eb68969dd3cf28193ba6a3704fe2f286494439eea4f9071bb6ef9de530ccaaab235a402e2d94aad
2f4f1b90df022c23078f543bc6ca1c6fe8b17af45eae865bb....)
(0.3ms) SET PROFILING=1
Device Load (1.2ms) SELECT SQL_NO_CACHE `devices`.* FROM `devices`
WHERE `devices`.`mac` = '00606E91E51E' LIMIT 1
CACHE (0.0ms) SELECT `devices`.* FROM `devices` WHERE `devices`.
`mac` = '00606E91E51E' LIMIT 1
(0.3ms) BEGIN
SQL (2.9ms) INSERT INTO `multimedia` (`created_at`, `data`, `eve
nt_id`, `updated_at`) VALUES ('2012-03-30 07:45:00', x'f22b19237b63f1c8c40da49ae
5eb68969dd3cf28193ba6a3704fe2f286494439eea4f9071bb6ef9de530ccaaab235a402e2d94aad
2f4f1b90df022c23078f543bc6ca1c6fe8b17af45eae865bb...)
(44.0ms) COMMIT
(0.3ms) BEGIN
SQL (0.8ms) INSERT INTO `events` (`created_at`, `data_type`, `devic
e_id`, `element_id`, `multimedia_id`, `name`, `status`, `updated_at`, `value`) V
ALUES ('2012-03-30 07:45:00', 'image', 19, 413397, 1066303, 'photo', NULL, '2012
-03-30 07:45:11', '--- 0\n...\n')
(38.2ms) COMMIT
(0.3ms) BEGIN
SQL (2.9ms) INSERT INTO `multimedia` (`created_at`, `data`, `eve
nt_id`, `updated_at`) VALUES ('2012-03-30 07:45:01', x'f22b19237b63f1c8c40da49ae
5eb68969dd3cf28193ba6a3704fe2f28...)
....
for 10 times insert a record in multimedia and one in events table
...
(33.0ms) COMMIT
(0.3ms) BEGIN
(0.4ms) SET PROFILING=1
(1.3ms) SELECT SQL_NO_CACHE 1 FROM `devices` WHERE (`devices`.`mac
` = BINARY '00606E91E51E' AND `devices`.`id` != 19) LIMIT 1
(0.7ms) UPDATE `devices` SET `elem_photo_id` = 413408, `updated
_at` = '2012-03-30 07:45:12' WHERE `devices`.`id` = 19
(34.5ms) COMMIT
Rendered events/save_photo.html.erb (0.2ms)
Completed 200 OK in 1957ms (Views: 10.2ms | ActiveRecord: 1466.6ms)`
My first problem is the difference by total time and the sum between views time and activerecord time.
After that this times are in the log of the server, so those are the execution time of the request. If I take the time, in the client, from send request to receive respond, it's 3,6 seconds! Why? I don't think that in local network are needed 2 second to transfer 200k!
there's a lot of things happening between client sending request and server sending response
to name a few:
client resolves dns(usually it's cached)
client initiates http connection
client actually posts the data
whatever server you have there accepts post request and enqueues it to be processed by your rails stack
request goes through all the rack middlewares you have before actually hitting your controller
your request is processed by controller and response is generated
response is then transferred to client and browser parses and renders it
what you see in your server logs is probably only point 6, everything else depends on your app setup and properties of your response(html with lots of css, javascripts, images, etc)
regarding your problem with time differences in this line:
Completed 200 OK in 1957ms (Views: 10.2ms | ActiveRecord: 1466.6ms)
detailed time consumption(in parenthesis) is not entirely detailed. it is missing GC time(which probably was invoked if you send files as yaml-formatted parameters), it is also missing time spent in your before filters and action itself.
Related
I've updated a Rails '6.0.3' app to use hotwire-rails rather than turbolinks and am getting a 304 for the response with no update to the view within the turbo_frame_tag
I created a new Rails app from scratch and implemented hotwire successfully so I'm guessing the issue is with residual Javascript in my existing app.
Server request
The server request matches that of the new app so it doesn't appear to be the issue, note the 200 status code.
Started GET "/models/1/edit" for ::1 at 2021-01-18 13:33:22 +1100
Processing by ModelsController#edit as TURBO_STREAM
Parameters: {"id"=>"1"}
User Load (0.2ms) SELECT "users".* FROM "users" WHERE "users"."id" = $1 ORDER BY "users"."id" ASC LIMIT $2 [["id", 2], ["LIMIT", 1]]
Model Load (0.1ms) SELECT "models".* FROM "models" WHERE "models"."id" = $1 LIMIT $2 [["id", 1], ["LIMIT", 1]]
↳ app/controllers/models_controller.rb:15:in `edit'
Rendering models/edit.haml
Rendered models/edit.haml (Duration: 1.1ms | Allocations: 1500)
Completed 200 OK in 4ms (Views: 1.3ms | ActiveRecord: 0.3ms | Allocations: 4244)
Browser
The browser response seems to be identical except for the status code being 304 rather than 200 and obviously no view change rendered.
Steps to recreate
Existing Rails '6.0.3' app
Update Gemfile to remove tubolinks and webpacker
trash app/javascript
trash node_modules/**
rails hotwire:install
Extra info
When I ran rails hotwire:install I get a File unchanged! The supplied flag value not found! app/assets/config/manifest.js message - I've checked the file and it's identical to the one generated in the new app.
I've got a RefineryCMS 3 application and in the backend the thumbnails for uploaded images are not showing up.
If I run the app in production mode on my development machine then the thumbnails show up. On the production machine the rails app is ran through passenger & apache and it fails.
I have copied my development copy to the production server for a test and ran the refinery project in development mode and I still get the same issue
rails s -b 192.168.200.215
Here is the standard output when I try and load the images area in refinery admin;
Started GET "/refinery/images" for 192.168.200.4 at 2016-03-01 15:32:25 +1100
Processing by Refinery::Admin::ImagesController#index as HTML
Parameters: {"locale"=>:en}
Refinery::Authentication::Devise::User Load (0.2ms) SELECT "refinery_authentication_devise_users".* FROM "refinery_authentication_devise_users" WHERE "refinery_authentication_devise_users"."id" = ? ORDER BY "refinery_authentication_devise_users"."id" ASC LIMIT 1 [["id", 1]]
Refinery::Authentication::Devise::Role Load (0.1ms) SELECT "refinery_authentication_devise_roles".* FROM "refinery_authentication_devise_roles" INNER JOIN "refinery_authentication_devise_roles_users" ON "refinery_authentication_devise_roles"."id" = "refinery_authentication_devise_roles_users"."role_id" WHERE "refinery_authentication_devise_roles_users"."user_id" = ? [["user_id", 1]]
Rendered /var/lib/gems/2.1.0/gems/refinerycms-acts-as-indexed-2.0.1/app/views/refinery/acts_as_indexed/admin/_search_header.html.erb (0.2ms)
Rendered /var/lib/gems/2.1.0/gems/refinerycms-core-3.0.1/app/views/refinery/admin/_search_header.html.erb (4.4ms)
(0.2ms) SELECT COUNT(*) FROM "refinery_images"
CACHE (0.0ms) SELECT COUNT(*) FROM "refinery_images"
Refinery::Image Load (0.3ms) SELECT "refinery_images".* FROM "refinery_images" ORDER BY updated_at DESC LIMIT 20 OFFSET 0
Refinery::Image::Translation Load (0.3ms) SELECT "refinery_image_translations".* FROM "refinery_image_translations" WHERE "refinery_image_translations"."refinery_image_id" IN (2)
Rendered /var/lib/gems/2.1.0/gems/refinerycms-images-3.0.1/app/views/refinery/admin/images/_grid_view.html.erb (10.3ms)
Rendered /var/lib/gems/2.1.0/gems/refinerycms-images-3.0.1/app/views/refinery/admin/images/_images.html.erb (14.5ms)
Rendered /var/lib/gems/2.1.0/gems/refinerycms-images-3.0.1/app/views/refinery/admin/images/_records.html.erb (33.7ms)
Rendered /var/lib/gems/2.1.0/gems/refinerycms-acts-as-indexed-2.0.1/app/views/refinery/acts_as_indexed/admin/_search.html.erb (2.2ms)
Rendered /var/lib/gems/2.1.0/gems/refinerycms-core-3.0.1/app/views/refinery/admin/_search.html.erb (4.8ms)
Rendered /var/lib/gems/2.1.0/gems/refinerycms-images-3.0.1/app/views/refinery/admin/images/_actions.html.erb (17.0ms)
Rendered /var/lib/gems/2.1.0/gems/refinerycms-images-3.0.1/app/views/refinery/admin/images/index.html.erb within layouts/refinery/admin (61.1ms)
Rendered /var/lib/gems/2.1.0/gems/refinerycms-core-3.0.1/app/views/refinery/_html_tag.html.erb (0.2ms)
Rendered /var/lib/gems/2.1.0/gems/refinerycms-core-3.0.1/app/views/refinery/admin/_javascripts.html.erb (159.9ms)
Rendered /var/lib/gems/2.1.0/gems/refinerycms-core-3.0.1/app/views/refinery/admin/_head.html.erb (232.9ms)
Rendered /var/lib/gems/2.1.0/gems/refinerycms-core-3.0.1/app/views/refinery/_site_bar.html.erb (3.4ms)
Rendered /var/lib/gems/2.1.0/gems/refinerycms-core-3.0.1/app/views/refinery/admin/_menu_item.html.erb (28.4ms)
Rendered /var/lib/gems/2.1.0/gems/refinerycms-core-3.0.1/app/views/refinery/admin/_menu.html.erb (31.9ms)
Rendered /var/lib/gems/2.1.0/gems/refinerycms-core-3.0.1/app/views/refinery/_no_script.html.erb (0.3ms)
Rendered /var/lib/gems/2.1.0/gems/refinerycms-core-3.0.1/app/views/refinery/_message.html.erb (0.1ms)
Completed 200 OK in 375ms (Views: 362.0ms | ActiveRecord: 1.1ms)
Started GET "/system/images/W1siZiIsIjIwMTYvMDMvMDEvN3lmN3ZpNDRtZV9hYWEuanBnIl0sWyJwIiwidGh1bWIiLCIxMzV4MTM1I2MiXV0/aaa.jpg?sha=7a0325c4ef12ef3d" for 192.168.200.4 at 2016-03-01 15:32:27 +1100
When it tries to get the /system/images/.... it fails in the web browser console. If I try and access that directly with the following
http://192.168.200.215:3000/system/images/W1siZiIsIjIwMTYvMDMvMDEvN3lmN3ZpNDRtZV9hYWEuanBnIl0sWyJwIiwidGh1bWIiLCIxMzV4MTM1I2MiXV0/aaa.jpg
I get the following error in the browser
Internal Server Error
The production server didn't have imagemagick installed.
sudo apt-get install imagemagick
This fixed it.
i want to optimize a post http request. My system is composed by a server and a client.
The client send an image and the server save it in the database.
this is the request:
res = Net::HTTP.post_form(uri, :mac => 'String', :value => 'ImageData(stringa di 22k circa)')
The server make this operation with the database:
Started POST "/events/save_photo" for 192.168.0.113 at 2012-03-30 09:27:27 +0200
Processing by EventsController#save_photo as */*
Parameters: {"data_type"=>"image", "value"=>"!binary |\n /9j/2wCEAAoHBwgHBgoICAgLCgoLDhgQDg0NDh0VFhE...(very long parameter", "created_at"=>"2012-03- 30 09:26:29 +02:00", "mac"=>"00606E91E5D2"}
WARNING: Can't verify CSRF token authenticity
`[[ (0.3ms)] [1mSET PROFILING=1]
[[Device Load (1.2ms)] SELECT SQL_NO_CACHE `devices`.* FROM `devices` WHERE `devices`.`mac` = '00606E91E5D2' LIMIT 1
[[CACHE (0.0ms)] [1mSELECT `devices`.* FROM `devices` WHERE `devices`.`mac` = '00606E91E5D2' LIMIT 1]
[[(0.4ms)] BEGIN
[[SQL (2.3ms)] [INSERT INTO `multimedia` (`created_at`, `data`) VALUES ('2012-03-30 07:26:29', x'f22b19237b63f1c8c40da49ae5eb68969dd3cf28193ba...',)]
[[ (61.9ms)] COMMIT
[[ (0.2ms)] [BEGIN]
[[SQL (0.6ms)] INSERT INTO `events` (`created_at`, `data_type`, `device_id`, `element_id`, `multimedia_id`, `name`, `status`, `updated_at`, `value`) VALUES ('2012-03-30 07:26:29', 'image', 23, 163785, 1065577, 'photo', NULL, '2012-03-30 07:27:28', '--- 0\n...')
[[ (36.6ms)] [COMMIT]
[[35m (0.3ms)] BEGIN
[[ (0.4ms)] [SET PROFILING=1]
[[ (1.1ms)] SELECT SQL_NO_CACHE 1 FROM `devices` WHERE (`devices`.`mac` = BINARY '00606E91E5D2' AND `devices`.`id` != 23) LIMIT 1
[[ (0.8ms)] [UPDATE `devices` SET `elem_photo_id` = 163804, `updated_at` = '2012-03-30 07:27:30' WHERE `devices`.`id` = 23]
[[ (78.9ms)] COMMIT
Rendered events/save_photo.html.erb (0.2ms)
Completed 200 OK in 358.4ms (Views: 11.6ms | ActiveRecord: 246.2ms)`
In my opinion this time is not acceptable. i would that this operation become more fast!! is it possible?
make attention that i don't use Base64, this method make the image string longer, and i must save memory
As you can see most of the time is spent writing to the Database (246.2ms) - which database are you using?
If you need a fast database you could use a non-relational database like redis or mongoDB. When you say you must save 'memory' do mean disk space or RAM? Because such non-relational databases store the data in the RAM (don't worry there are persistant possibilities ).
I am using Clearance gem. It's "half" working.
I can do all basic actions (sign up, sign in, :authorize).
But, when I do a POST action to a controller, it just signs me out ...
Here what's in the log :
Started POST "/monkeys" for 127.0.0.1 at Fri Jun 03 06:22:00 -0400 2011
Processing by MonkeysController#create as HTML
Parameters: {"commit"=>"Save monkey", "authenticity_token"=>"[FILTERED]", "utf8"=>"\342\234\223", "monkey"=>{"name"=>"fghfgh", "description"=>"fghfgh"}}
User Load (0.1ms) SELECT `users`.* FROM `users` WHERE `users`.`remember_token` = 'c1f450a81fde65282e73ce41f69095b66d782e34' LIMIT 1
SQL (0.1ms) BEGIN
AREL (0.3ms) UPDATE `users` SET `remember_token` = 'b9c2cac3d446bee370e40619c6c8cd42c457bb13', `updated_at` = '2011-06-03 10:22:00' WHERE `users`.`id` = 4
SQL (1.1ms) COMMIT
Redirected to http://localhost:3000/sign_in
When that happens, I am actually signed in.. what might I have missed?
Thanks
It was actually because of my cookie domain, for some reason, localhost is not right. So i added a simple dns called "monkey.local" on my machine, and now I access the application with it and all works fine.
I also believe I can use 127.0.0.0 and it would work. I'll try it at home.
I'm building my first webapp, following the Michael Hartl Rails Tutorial. After finishing Chapter 10, I find that my application works locally but does not work when I push it to Heroku. The development log output reads:
Started GET "/" for 98.154.183.51 at Thu Jan 06 01:07:15 -0800 2011
Processing by PagesController#home as HTML
Rendered layouts/_header.html.erb (28.3ms)
Rendered pages/home.html.erb within layouts/application (30.9ms)
Completed in 35ms
User Load (29.6ms) SELECT "users".* FROM "users" WHERE ("users"."id" = 101) LIMIT 1
CACHE (0.0ms) SELECT "users".* FROM "users" WHERE ("users"."id" = 101) LIMIT 1
Rendered layouts/_header.html.erb (149.3ms)
CACHE (0.0ms) SELECT "users".* FROM "users" WHERE ("users"."id" = 101) LIMIT 1
Rendered layouts/_sidebar.html.erb (8.1ms)
Rendered layouts/_footer.html.erb (0.6ms)
Rendered sessions/new.html.erb within layouts/application (342.4ms)
Completed 200 OK in 421ms (Views: 390.1ms | ActiveRecord: 29.9ms)
Whereas the heroku production log reads:
Started GET "/" for 98.154.183.51 at Thu Jan 06 01:07:15 -0800 2011
Processing by PagesController#home as HTML
Rendered layouts/_header.html.erb (28.3ms)
Rendered pages/home.html.erb within layouts/application (30.9ms)
Completed in 35ms
ActionView::Template::Error (PGError: ERROR: invalid input syntax for integer: "nil"
: SELECT "users".* FROM "users" WHERE ("users"."id" = 'nil') LIMIT 1):
I'm not sure why the Heroku-deployed app looks for a user_id of "nil", whereas my local app looks for one of "101." This is my first app, so I'm not sure where to even begin the troubleshooting. I've tried pushing my local db to the Heroku one via heroku rake db:push, but I still get the same errors. Any advice? Thanks in advance for your help.
Make sure you
Migrated your database on Heroku with heroku rake db:migrate
Have a valid user in your database (you could e.g. create one with the Rails console on Heroku)
It seems your code is trying to load a User instance when you have none in your database. If your app works once you have some users in your database, fix your code to act appropriately if no users are present.
You might want to run you rake db:seed also if you have setup some initial values.