We're using the autoscaler on Heroku. Almost all the requests are very quick (0-500ms), but periodically we have an image upload that takes a long time (3000-9000ms). Autoscaling doesn't help at all in this case. Is there a best practice handle this? For instance, is there a way to omit a particular URL from the autoscaler? Or do you handle this by setting up a parallel app just to handle these transactions?
Here is an example "slow" transaction (with identifiers anonymized) -- all times are intact:
May 22 11:46:19 production heroku/router: at=info method=POST path="/api/users/cases/24070/case_posts.json" host=www.website.com request_id=XXX...XXX fwd="xxx.xxx.xxx.xxx" dyno=web.1 connect=0ms service=10483ms status=200 bytes=1762 protocol=https
Started POST "/api/users/cases/24070/case_posts.json" for xxx.xxx.xxx.xxx at 2018-05-22 18:46:10 +0000
Processing by Users::CasePostsController#create as JSON
Parameters: {
"image"=>#<ActionDispatch::Http::UploadedFile:0x00007f3068cb98d8 #tempfile=#<Tempfile:/tmp/RackMultipart20180522-11-1x9ayll.JPG>,
#original_filename="IMG_1325.JPG",
#content_type="image/jpeg",
#headers="Content-Disposition: form-data; name=\"image\"; filename=\"IMG_1325.JPG\"\r\nContent-Type: image/jpeg\r\n">,
"step"=>"pack out",
...
}
Rendered users/case_posts/_case_post.json.jbuilder (1.0ms)
Rendering users/case_posts/show.json.jbuilder
Rendered users/case_posts/_case_post.json.jbuilder (0.4ms)
Rendered users/case_posts/show.json.jbuilder (1.0ms)
Completed 200 OK in 9120ms (Views: 1.8ms | ActiveRecord: 29.8ms)
Related
So, long story short - I'm comfortable with HTML, CSS, SQL a bit of Javascript etc as a hobby, but am far from a 'proper' developer. Recently a friend paid a developer to build a web app for them (the friend have no IT skills at all). The developer (who is no longer on the scene) left them with a working demo in Heroku and a GitHub repo which I have now cloned. My friend wanted some of the cosmetic bits changing, which I offered to help with, but in order to do this, I had to create my own version of the app on Heroku from my forked repo.
I couldn't get this to work, kept getting an error message when trying to deploy the app, despite the code being completely unchanged from the parent. I should mention now that the app has been written in Ruby, with which I have zero experience. But I thought I'd have a go anyway...
After much Googling, I managed to get the app to successfully deploy by changing the version of #rails/webpacker in the package.json file. I then changed any references to originalApp.herokuapps.com to myNewVersion.herokuapps.com.
After this, I had to set some Config Vars in Heroku (relating to gmail and stripe) to get things up and running. This all seemed to work ok...
So I'm now at the point where a user can sign up on the app and subscribe to the service using Stripe (test) and is successfully sent a confirmation email.
However, once a user is logged in, anytime they try to go to any other page, for example the 'About Us' or 'Contact Us' page, it immediately redirects (or stays) on the same page. There's no other pages the logged-in user can access. However, if I log out, then the 'guest' user can navigate to the other pages freely.
Example:
If I'm logged in, the URL shows https://myproject.herokuapp.com/start and the contents of the page are what the user was presented with after signing up and paying the Stripe fee.
If they then click on, for example, the 'About Us' link, the log shows this (I've edited the URLs to show myproject instead of the real project name):
2022-01-05T22:16:29.805824+00:00 app[web.1]: I, [2022-01-05T22:16:29.805699 #4] INFO -- : [0efc978f-35cf-4d11-ae66-edbdb5b26ec4] Started GET "/about-us" for 90.197.167.151 at 2022-01-05 22:16:29 +0000
2022-01-05T22:16:29.806538+00:00 app[web.1]: I, [2022-01-05T22:16:29.806485 #4] INFO -- : [0efc978f-35cf-4d11-ae66-edbdb5b26ec4] Processing by PagesController#about_us as HTML
2022-01-05T22:16:29.809496+00:00 app[web.1]: D, [2022-01-05T22:16:29.809430 #4] DEBUG -- : [0efc978f-35cf-4d11-ae66-edbdb5b26ec4] [1m[36mUser Load (0.7ms)[0m [1m[34mSELECT "users".* FROM "users" WHERE "users"."id" = $1 ORDER BY "users"."id" ASC LIMIT $2[0m [["id", 1], ["LIMIT", 1]]
2022-01-05T22:16:29.810151+00:00 app[web.1]: I, [2022-01-05T22:16:29.810102 #4] INFO -- : [0efc978f-35cf-4d11-ae66-edbdb5b26ec4] Redirected to https://myproject.herokuapp.com/start
2022-01-05T22:16:29.810227+00:00 app[web.1]: I, [2022-01-05T22:16:29.810201 #4] INFO -- : [0efc978f-35cf-4d11-ae66-edbdb5b26ec4] Filter chain halted as :check_subscription rendered or redirected
2022-01-05T22:16:29.810322+00:00 app[web.1]: I, [2022-01-05T22:16:29.810300 #4] INFO -- : [0efc978f-35cf-4d11-ae66-edbdb5b26ec4] Completed 302 Found in 4ms (ActiveRecord: 0.7ms | Allocations: 964)
2022-01-05T22:16:29.835064+00:00 app[web.1]: I, [2022-01-05T22:16:29.834991 #4] INFO -- : [a7c07664-645b-449c-a9a8-a50a2c060f3b] Started GET "/start" for 90.197.167.151 at 2022-01-05 22:16:29 +0000
2022-01-05T22:16:29.835712+00:00 app[web.1]: I, [2022-01-05T22:16:29.835663 #4] INFO -- : [a7c07664-645b-449c-a9a8-a50a2c060f3b] Processing by CheckoutsController#start as HTML
2022-01-05T22:16:29.836943+00:00 app[web.1]: I, [2022-01-05T22:16:29.836887 #4] INFO -- : [a7c07664-645b-449c-a9a8-a50a2c060f3b] Rendering checkouts/start.html.erb within layouts/application
2022-01-05T22:16:29.839055+00:00 app[web.1]: D, [2022-01-05T22:16:29.838940 #4] DEBUG -- : [a7c07664-645b-449c-a9a8-a50a2c060f3b] [1m[36mUser Load (0.7ms)[0m [1m[34mSELECT "users".* FROM "users" WHERE "users"."id" = $1 ORDER BY "users"."id" ASC LIMIT $2[0m [["id", 1], ["LIMIT", 1]]
2022-01-05T22:16:29.839685+00:00 app[web.1]: I, [2022-01-05T22:16:29.839609 #4] INFO -- : [a7c07664-645b-449c-a9a8-a50a2c060f3b] Rendered checkouts/start.html.erb within layouts/application (Duration: 2.6ms | Allocations: 401)
2022-01-05T22:16:29.841324+00:00 app[web.1]: I, [2022-01-05T22:16:29.841270 #4] INFO -- : [a7c07664-645b-449c-a9a8-a50a2c060f3b] Rendered shared/_navbar.html.erb (Duration: 0.9ms | Allocations: 905)
2022-01-05T22:16:29.841552+00:00 app[web.1]: I, [2022-01-05T22:16:29.841516 #4] INFO -- : [a7c07664-645b-449c-a9a8-a50a2c060f3b] Completed 200 OK in 6ms (Views: 4.1ms | ActiveRecord: 0.7ms | Allocations: 2433)
2022-01-05T22:16:29.816017+00:00 heroku[router]: at=info method=GET path="/about-us" host=myproject.herokuapp.com request_id=0efc978f-35cf-4d11-ae66-edbdb5b26ec4 fwd="90.197.167.151" dyno=web.1 connect=0ms service=6ms status=302 bytes=1393 protocol=https
As said, I'm not familiar with Ruby at all, but I suspect it's something to do with the lines that mention:
Redirected to https://myproject.herokuapp.com/start
Filter chain halted as :check_subscription rendered or redirected
You can see it's trying to go to the /about-us page but it remains on /start.
I believe the app uses something called 'Devise' for the user registration process.
Given the code is the same as the parent repo that I forked from, I'm guessing the issue is either a setting within Heroku or some kind of variable that I'm missing or have not updated. If anyone can advise me on next steps with regards to research or a solution, I'd be very grateful. I appreciate I'm no expert, but I'm just trying to do a friend a favour after they've been left hanging by another dev. Many Thanks.
Update:
application_controller.rb
before_action :authenticate_user!, :halt_if_mobile
before_action :check_subscription, except: %i[ home start ]
def halt_if_mobile
if browser.device.mobile?
render inline: "Mobile access is currently disabled"
return
end
end
def check_subscription
return unless current_user
return if controller_path =~ /devise/ || controller_path =~ /registrations/
if !current_user.active
redirect_to start_path, notice: 'Please update your payment settings'
return
end
end
end
I am attempting to adapt this wenderlich tutorial to Swift 3 syntax to create an iOS frontned with rails backend hosted on Heroku.
Let me know if there is any other code that would be good to share, but it seems like an auth issue. Here's where the authorization header is set for the request:
let basicAuthString = "\(HTTPHelper.API_AUTH_NAME):\(HTTPHelper.API_AUTH_PASSWORD)"
let utf8str = basicAuthString.data(using: String.Encoding.utf8)
let base64EncodedString = utf8str?.base64EncodedString(options: Data.Base64EncodingOptions())
request.addValue("Basic \(base64EncodedString)", forHTTPHeaderField: "Authorization")
I am having trouble telling how to debug these requests to begin with. If I step through the code in the debugger, the request object seems pretty flat, and if I print the request I just see the URL.
The request to the heroku server (also built from that tutorial) returns a 401, and it doesn't seem to be entering any of the rails methods. It just returns a 401 unauthorized:
2017-03-24T20:13:47.022176+00:00 heroku[router]: at=info method=POST path="/api/signin" host=young-retreat-61850.herokuapp.com request_id=9d58e8f4-ee9f-4c73-babd-2a2ee5c82a4f fwd="73.83.200.10" dyno=web.1 connect=1ms service=13ms status=401 bytes=499 protocol=https
2017-03-24T20:13:47.014605+00:00 app[web.1]: Started POST "/api/signin" for 73.83.200.10 at 2017-03-24 20:13:47 +0000
2017-03-24T20:13:47.020239+00:00 app[web.1]: Processing by ApiController#signin as */*
2017-03-24T20:13:47.020279+00:00 app[web.1]: Parameters: {"email"=>"dsadsadsa", "password"=>"[FILTERED]", "api"=>{"email"=>"dsadsadsa", "password"=>"[FILTERED]"}}
2017-03-24T20:13:47.020671+00:00 app[web.1]: Filter chain halted as #<Proc:0x007f6ae4d6f140#/app/vendor/bundle/ruby/2.1.0/gems/actionpack-4.1.5/lib/action_controller/metal/http_authentication.rb:71> rendered or redirected
2017-03-24T20:13:47.020791+00:00 app[web.1]: Completed 401 Unauthorized in 0ms (ActiveRecord: 0.0ms)
2017-03-24T20:13:47.142925+00:00 app[web.1]: Started POST "/api/signin" for 73.83.200.10 at 2017-03-24 20:13:47 +0000
2017-03-24T20:13:47.148098+00:00 app[web.1]: Processing by ApiController#signin as */*
2017-03-24T20:13:47.148134+00:00 app[web.1]: Parameters: {"email"=>"dsadsadsa", "password"=>"[FILTERED]", "api"=>{"email"=>"dsadsadsa", "password"=>"[FILTERED]"}}
2017-03-24T20:13:47.156450+00:00 app[web.1]: Filter chain halted as #<Proc:0x007f6ae4d6f140#/app/vendor/bundle/ruby/2.1.0/gems/actionpack-4.1.5/lib/action_controller/metal/http_authentication.rb:71> rendered or redirected
2017-03-24T20:13:47.156647+00:00 app[web.1]: Completed 401 Unauthorized in 8ms (ActiveRecord: 0.0ms)
2017-03-24T20:13:47.157852+00:00 heroku[router]: at=info method=POST path="/api/signin" host=young-retreat-61850.herokuapp.com request_id=bc270aa1-aeed-40b8-b9a0-3c09a68dec8d fwd="73.83.200.10" dyno=web.1 connect=2ms service=24ms status=401 bytes=458 protocol=https
Should it seem strange that the nested "api" key has the same params? I am unclear on which direction to take for debugging these requests. I can't use a service like Charles because they are https right?
Feel like a fool...
the base64EncodedString was an optional.
fix:
request.addValue("Basic \(base64EncodedString!)", forHTTPHeaderField: "Authorization")
I was able to get more details about the request with this URLRequest method:
print("request header:\(request.allHTTPHeaderFields)")
I'm running a rails app that writes files to Amazon S3. Now I have uniqorn running as server and see all lots of request coming in from different amazon AWS IP-addresses.
Here is a part of my log file:
Started GET "/" for 54.245.168.11 at 2016-10-02 18:30:50 +0000
Processing by StaticPagesController#home as */*
Rendered static_pages/home.html.erb within layouts/application (0.2ms)
Rendered layouts/_shim.html.erb (0.1ms)
Rendered layouts/_header.html.erb (0.3ms)
Completed 200 OK in 3ms (Views: 3.0ms | ActiveRecord: 0.0ms)
Started GET "/" for 54.252.254.235 at 2016-10-02 18:30:51 +0000
Processing by StaticPagesController#home as */*
Rendered static_pages/home.html.erb within layouts/application (0.2ms)
Rendered layouts/_shim.html.erb (0.0ms)
Rendered layouts/_header.html.erb (0.3ms)
Completed 200 OK in 3ms (Views: 3.0ms | ActiveRecord: 0.0ms)
Started GET "/" for 54.232.40.107 at 2016-10-02 18:30:52 +0000
Processing by StaticPagesController#home as */*
Rendered static_pages/home.html.erb within layouts/application (0.7ms)
Rendered layouts/_shim.html.erb (0.2ms)
Rendered layouts/_header.html.erb (1.8ms)
Completed 200 OK in 12ms (Views: 10.6ms | ActiveRecord: 0.0ms)
Started GET "/" for 54.252.79.139 at 2016-10-02 18:30:52 +0000
Processing by StaticPagesController#home as */*
Rendered static_pages/home.html.erb within layouts/application (0.7ms)
Rendered layouts/_shim.html.erb (0.3ms)
Rendered layouts/_header.html.erb (1.5ms)
Completed 200 OK in 13ms (Views: 11.6ms | ActiveRecord: 0.0ms)
Started GET "/" for 54.250.253.203 at 2016-10-02 18:30:52 +0000
Processing by StaticPagesController#home as */*
Rendered static_pages/home.html.erb within layouts/application (0.5ms)
Rendered layouts/_shim.html.erb (0.1ms)
Rendered layouts/_header.html.erb (0.3ms)
Completed 200 OK in 4ms (Views: 3.2ms | ActiveRecord: 0.0ms)
Started GET "/" for 54.243.31.203 at 2016-10-02 18:30:54 +0000
Processing by StaticPagesController#home as */*
Rendered static_pages/home.html.erb within layouts/application (0.5ms)
Rendered layouts/_shim.html.erb (0.0ms)
Rendered layouts/_header.html.erb (0.3ms)
Completed 200 OK in 4ms (Views: 3.5ms | ActiveRecord: 0.0ms)
I can't find any info on this online. My application isn't live yet, so nobody looks at it. Why is AWS doing this?
These are Route 53 Health Checks, which are a different type of health check than the health checks configured in Elastic Load Balancer (ELB), and different from the EC2 instance health and reachability checks that EC2 automatically performs on every EC2 instance.
Take a look at the published AWS IP Address Ranges, and you'll find that these match up to the address blocks specifically assigned to Route 53 Health Checks, quite nicely.
For example, this request...
Started GET "/" for 54.232.40.107 at 2016-10-02 18:30:52 +0000
...matches this entry...
{
"ip_prefix": "54.232.40.64/26",
"region": "sa-east-1",
"service": "ROUTE53_HEALTHCHECKS"
},
...because 54.232.40.64/26 is the CIDR spec for the IP address block from 54.232.40.64 through 54.232.40.127.
If you configured a Route 53 health check against your stack, then you probably have your answer, at this point.
Route 53 Health Checks are often used to drive the logic of complex DNS configurations, such as in cases where there are multiple candidate servers geographically distributed, and you want to control the answer returned by DNS based on the health of the servers... but they can also be used for monitoring latency and application performance (measuring TCP connect time and time-to-first-byte, as well as availability), tied into CloudWatch alarms, and used to drive scaling and other logic. If you configured anything like this, the question may be solved.
But if you didn't, this could be a case of another AWS user who has configured health checks against an IP address they were formerly using, or has configured the checks against your IP address or hostname by simple mistake.
If that is the case, you'll need to inspect the request headers from the incoming request. You'll find something like this:
User-Agent: Amazon Route 53 Health Check Service; ref:xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx; report http://amzn.to/zzzzzzz
With these values in hand, along with the other headers and any relevant log entries, you can visit the "report" URL provided in the headers you captured, (it's similar to the one I've shown above, captured from my logs, but sanitized since it seems to be generic, but I'm not certain -- the link in my logs redirects me here) and submit a request for AWS to work with you to to eliminate the unwanted health checks.
But first, try this...
Take the value immediately after "ref:" and create a URL from it, using this template:
https://console.aws.amazon.com/route53/healthchecks/home#/details/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx
(Replace xxxx with your ref:, excluding the actual string ref:).
If the health check is from your own AWS account, and you're logged into the console, and have permission to view it, then this crafted link should bring the health check right up for you to view. If you get an error that you are "not authorized to access this resource," then the health check is not from your AWS account, or you aren't a privileged user on the AWS account and thus may lack the ability to view it.
I have also changed the domain of the cookie so that it would work on subdomains ("Switched from domain.com cookie to .domain.com cookie
The issue is users can't log in. Unless they clear their cookie or log in on incognito mode.
After a user logs in, they are just taken back to the sign in page without any errors.
Logs are showing no errors other than a 302, here's an example:
Started POST "/users/sign-in" for 174.235.55.66 at 2015-05-16 18:38:20 +0000
May 16 11:38:21 abc app/web.1: Processing by Users::SessionsController#create as HTML
May 16 11:38:21 abc app/web.1: Parameters: {"utf8"=>"✓", "user"=>{"email"=>"email#domain.com", "password"=>"[FILTERED]", "remember_me"=>"0"}}
May 16 11:38:21 abc heroku/router: at=info method=POST path="/users/sign-in" host=www.bidvine.com request_id=f3f8fd11-acc5-443a-866d-46c554bcabb1 fwd="70.53.68.47,173.245.55.66" dyno=web.1 connect=1ms service=454ms status=302 bytes=794
May 16 11:38:21 abc app/web.1: Redirected to http://www.bidvine.com/admin
May 16 11:38:21 abc app/web.1: Completed 302 Found in 392ms (ActiveRecord: 35.2ms)
May 16 11:38:21 abc app/web.2: Processing by Admin::RequestsController#index as HTML
May 16 11:38:21 abc app/web.3: Started GET "/users/sign-in" for 173.245.55.66 at 2015-05-16 18:38:21 +0000
Started GET "/home" for 117.55.68.xxx at 2012-11-05 22:56:06 +0900
Processing by PageController#home as HTML
Parameters: {"id"=>"420"}
Started POST "/login" for 61.211.5.107 at 2012-11-05 23:25:52 +0900
Processing by UsersController#login as HTML
Parameters: {"utf8"=>"✓", "authenticity_token"=>"xxxxxxx=", "username"=>"***DELETED***", "password"=>"***DELETED***", "commit"=>"login"}
Rendered users/login.html.haml within layouts/application (6.7ms)
Completed 200 OK in 142ms (Views: 103.5ms | ActiveRecord: 0.0ms)
I am using passenger 3.0.11.
I have encountered the problem that the log is not written.
It's happen when the "Completed" is not output.
After "Completed" is not output, log is not written for 20-40 minutes.
Does anyone have idea?
This usually happens when you have a memory issue. Ruby cannot write to log because it has run out of memory.