I am having some trouble working with the MicrosoftGraph Webhook. I think that I am not properly returning HTTP response, because when I put a binding.pry into my controller, after making the request it hits the controller.
However, the request seems to timeout before hitting my controller (see stack trace below). Here is my controller code and subscription request code, and stack trace. I really appreciate the help.
def self.create_subscription(user)
callback = Proc.new do |r|
r.headers['Authorization'] = "Bearer #{user.outlook_token.access_token}"
r.headers['Content-Type'] = 'application/json'
r.headers['X-AnchorMailbox'] = user.email
end
path = 'subscriptions'
data = {
changeType: "created, updated",
notificationUrl: "https://my_url/api/watch/outlookNotification",
resource: "me/mailFolders('Inbox')/messages",
expirationDateTime:"2017-08-19T06:23:45.9356913Z",
clientState: "subscription-identifier"
}
graph = MicrosoftGraph.new(base_url: 'https://graph.microsoft.com/v1.0',
cached_metadata_file: File.join(MicrosoftGraph::CACHED_METADATA_DIRECTORY, 'metadata_v1.0.xml'),
&callback)
response = graph.service.post(path, data.to_json)
end
def outlook_webhook
#token = params[:validationToken]
head 200, content_type: "text/plain", content_length: 7
response.body = #token
end
Completed 500 Internal Server Error in 14613ms (ActiveRecord: 478.4ms)
OData::ClientError (400 InvalidRequest: "Subscription validation
request timed out." from
"https://graph.microsoft.com/v1.0/subscriptions"):
app/models/outlook_wrapper.rb:44:in create_subscription'
app/controllers/business/messages_controller.rb:18:inindex'
Rendered
/Users/.rbenv/versions/2.1.2/lib/ruby/gems/2.1.0/gems/actionpack-4.2.4/lib/action_dispatch/middleware/templates/rescues/_source.erb
(7.9ms) Rendered
/Users/.rbenv/versions/2.1.2/lib/ruby/gems/2.1.0/gems/actionpack-4.2.4/lib/action_dispatch/middleware/templates/rescues/_trace.text.erb (1.9ms) Rendered
/Users/.rbenv/versions/2.1.2/lib/ruby/gems/2.1.0/gems/actionpack-4.2.4/lib/action_dispatch/middleware/templates/rescues/_request_and_response.text.erb
(3.8ms) Rendered
/Users/.rbenv/versions/2.1.2/lib/ruby/gems/2.1.0/gems/actionpack-4.2.4/lib/action_dispatch/middleware/templates/rescues/diagnostics.text.erb
(176.2ms)
Started POST
"/api/watch/outlookNotification?validationToken=YTJhMTk5MDQtYjdmOC00ZjYxLWIzOGEtMDczM2FjMTAxZTBj"
for 52.161.110.176 at 2017-08-23 12:29:05 -0400 Processing by
InboundEmailsController#outlook_webhook as HTML Parameters:
{"validationToken"=>"YTJhMTk5MDQtYjdmOC00ZjYxLWIzOGEtMDczM2FjMTAxZTBj"}
Completed 200 OK in 35ms (ActiveRecord: 0.0ms)
A Content-Length of 7 is not large enough to accommodate the validation token.
As far as the request timing out, when testing locally, is notificationUrl a publicly accessible URL? If you're using something like http://localhost/api/watch/outlookNotification, then Microsoft will not know how to access your webhook since localhost is not a publicly accessible host.
Even if you do provide a publicly accessible host, you'll still need to poke a hole in your firewall. My suggestion is to use ngrok when testing locally and use an environment variable to set the host.
Run ngrok http 3000 in a console and it will give you a URL that looks something like https://d83375e5.ngrok.io. While ngrok is running, it will forward HTTP and HTTPS requests from d83375e5.ngrok.io to your computer on port 3000.
If you use a notificationUrl like https://${ENV['WEBHOOK_HOST']}/api/watch/outlookNotification, then in development, set the environment variable WEBHOOK_HOST to the host provided by ngrok. In production, you don't need to run ngrok; just set WEBHOOK_HOST to the publicly accessible host for your server.
I may be a little late to the party, but I ran into this same issue. The development environment in rails blocks multiple requests which is why you are getting a timeout. Since you are waiting for a response from the post while Graph is blasting a request back, the request will be blocked until you receive a response from your first post.
As it seems you have discovered, when running in prod this is not an issue as the production environment settings do not block multiple requests.
Related
Please bare with me for a question for which it's nearly impossible to create a reproducible example.
I have an API setup with FastAPI using Docker, Serverless and deployed on AWS API Gateway. All routes discussed are protected with an api-key that is passed into the header (x-api-key).
I'm trying to accomplish a simple redirect from one route to another using fastapi.responses.RedirectResponse. The redirect works perfectly fine locally (though, this is without api-key), and both routes work perfectly fine when deployed on AWS and connected to directly, but something is blocking the redirect from route one (abc/item) to route two (xyz/item) when I deploy to AWS. I'm not sure what could be the issue, because the logs in CloudWatch aren't giving me much to work with.
To illustrate my issue let's say we have route abc/item that looks like this:
#router.get("/abc/item")
async def get_item(item_id: int, request: Request, db: Session = Depends(get_db)):
if False:
redirect_url = f"/xyz/item?item_id={item_id}"
logging.info(f"Redirecting to {redirect_url}")
return RedirectResponse(redirect_url, headers=request.headers)
else:
execution = db.execute(text(items_query))
return convert_to_json(execution)
So, we check if some value is True/False and if it's False we redirect from abc/item to xyz/item using RedirectResponse(). We pass the redirect_url, which is just the xyz/item route including query parameters and we pass request.headers (as suggested here and here), because I figured we need to pass along the x-api-key to the new route. In the second route we again try a query in a different table (other_items) and return some value.
I have also tried passing status_code=status.HTTP_303_SEE_OTHER and status_code=status.HTTP_307_TEMPORARY_REDIRECT to RedirectResponse() as suggested by some tangentially related questions I found on StackOverflow and the FastAPI discussions, but that didn't help either.
#router.get("/xyz/item")
async def get_item(item_id: int, db: Session = Depends(get_db)):
execution = db.execute(text(other_items_query))
return convert_to_json(execution)
Like I said, when deployed I can successfully connect directly to both abc/item and get a return value if True and I can also connect to xyz/item directly and get a correct value from that, but when I pass a value to abc/item that is False (and thus it should redirect) I get {"message": "Forbidden"}.
In case it can be of any help, I try debugging this using a "curl" tool, and the headers I get returned give the following info:
Content-Type: application/json
Content-Length: 23
Connection: keep-alive
Date: Wed, 27 Jul 2022 08:43:06 GMT
x-amzn-RequestId: XXXXXXXXXXXXXXXXXXXX
x-amzn-ErrorType: ForbiddenException
x-amz-apigw-id: XXXXXXXXXXXXXXXX
X-Cache: Error from cloudfront
Via: 1.1 XXXXXXXXXXXXXXXXXXXXXXXXX.cloudfront.net (CloudFront)
X-Amz-Cf-Pop: XXXXX
X-Amz-Cf-Id: XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
So, this is hinting at a CloudFront error. Unfortunately I don't see anything slightly hinting at this API when I look into my CloudFront dashboard on AWS, there literally is nothing there (I do have permissions to view the contents though...)
The API logs in CloudWatch look like this:
2022-07-27T03:43:06.495-05:00 Redirecting to /xyz/item?item_id=1234...
2022-07-27T03:43:06.495-05:00 [INFO] 2022-07-27T08:43:06.495Z Redirecting to /xyz/item?item_id=1234...
2022-07-27T03:43:06.496-05:00 2022-07-27 08:43:06,496 INFO sqlalchemy.engine.Engine ROLLBACK
2022-07-27T03:43:06.496-05:00 [INFO] 2022-07-27T08:43:06.496Z ROLLBACK
2022-07-27T03:43:06.499-05:00 END RequestId: 6f449762-6a60189e4314
2022-07-27T03:43:06.499-05:00 REPORT RequestId: 6f449762-6a60189e4314 Duration: 85.62 ms Billed Duration: 86 ms Memory Size: 256 MB Max Memory Used: 204 MB
I have been wondering if my issue could be related to something I need to add to somewhere in my serverless.yml, perhaps in the functions: part. That currently looks like this for these two routes:
events:
- http:
path: abc/item
method: get
cors: true
private: true
request:
parameters:
querystrings:
item_id: true
- http:
path: xyz/item
method: get
cors: true
private: true
request:
parameters:
querystrings:
item_id: true
Finally, it's probably good to note that I have added custom middleware to FastAPI to handle the two different database connections I need for connecting to other_items and items tables, though I'm not sure how relevant this is, considering this functions fine when redirecting locally. For this I implemented the solution found here. This custom middleware is the reason for the redirect in the first place (we change connection URI based on route with that middleware), so I figured it's good to share this bit of info as well.
Thanks!
As noted here and here, it is mpossible to redirect to a page with custom headers set. A redirection in the HTTP protocol doesn't support adding any headers to the target location. It is basically just a header in itself and only allows for a URL (a redirect response though could also include body content, if needed—see this answer). When you add the authorization header to the RedirectResponse, you only send that header back to the client.
A suggested here, you could use the set-cookie HTTP response header:
The Set-Cookie HTTP response header is used to send a cookie from the
server to the user agent (client), so that the user agent can send it back to
the server later.
In FastAPI—documentation can be found here and here—this can be done as follows:
from fastapi import Request
from fastapi.responses import RedirectResponse
#app.get("/abc/item")
def get_item(request: Request):
redirect_url = request.url_for('your_endpoints_function_name') #e.g., 'get_item'
response = RedirectResponse(redirect_url)
response.set_cookie(key="fakesession", value="fake-cookie-session-value", httponly=True)
return response
Inside the other endpoint, where you are redirecting the user to, you can extract that cookie to authenticate the user. The cookie can be found in request.cookies—which should return, for example, {'fakesession': 'fake-cookie-session-value-MANUAL'}—and you retrieve it using request.cookies.get('fakesession').
On a different note, request.url_for() function accepts only path parameters, not query parameters (such as item_id in your /abc/item and /xyz/item endpoints). Thus, you can either create the URL in the way you already do, or use the CustomURLProcessor suggested here, here and here, which allows you to pass both path and query parameters.
If the redirection takes place from one domain to another (e.g., from abc.com to xyz.com), please have a look at this answer.
I am working on a project whereby we have sites (developed with ruby on rails) hosted on an Ubuntu server using tomcat. We want these sites to make HTTP calls to a service developed using Nancy. We have this working locally whereby the service is hosted on a machine that we can call within our network. We cannot however get it working when live. Here is an example call:
def get_call(routePath)
started_at = Time.now
enc_url = URI.encode("#{settings.service_endpoint}#{routePath}")
uri = URI.parse(enc_url)
http = Net::HTTP.new(uri.host, uri.port)
req = Net::HTTP::Get.new(uri.request_uri)
resp = http.request(req)
logger.bench 'SERVICE - GET', started_at, routePath
return resp if response_ok?(resp)
end
When working locally the settings are as follows:
settings.service_endpoint = http://10.10.10.27:7820
routePath = /Customers
When we upload it to the server we use the following:
settings.service_endpoint = http://127.0.0.1:24099
routePath = /Customers
We currently get the following error:
SocketError at /register
initialize: name or service not know
with the following line being highlighted:
resp = http.request(req)
Are we completely wrong with the IP being called. Should it be 127.0.0.1, localhost. 10.10.10.27 or something entirely different? The strange thing is we can do a GET call via telnet in our Ubuntu server (telnet 127.0.0.1 24099) so that must mean the server can make the calls but the site hosted on the server cannot. Do we need to include a HTTP proxy (have read some reference to that but dont really know if its needed).
Apologies if its obvious but we have never tried anything like this before so its all very perplexing. Any further information required just let me know.
We changed the service_endpoint to localhost and it worked. Not sure if this is because it didnt like "http://" or some other reason. Any explanation as to why this is the case would be much appreciated, just so we know. Thanks!
I am currently trying to write an auxiliary module for Metasploit. The module basically tries multiple default credentials to get access to the router's management page. The authentication is done via web, i.e. HTTP POST.
Currently, the module works as expected for plain HTTP connections, i.e. unsecured connections, however every connection attempt via HTTPS (port 443), returns nil. Below is the function used within the Metasploit class to retrieve the login page:
def get_login_page(ip)
begin
response = send_request_cgi(
'uri' => '/',
'method' => 'GET'
)
# Some models of ZyXEL ZyWALL return a 200 OK response
# and use javascript to redirect to the rpAuth.html page.
if response && response.body =~ /changeURL\('rpAuth.html'\)/
vprint_status "#{ip}- Redirecting to rpAuth.html page..."
response = send_request_cgi(
'uri' => '/rpAuth.html',
'method' => 'GET'
)
end
rescue ::Rex::ConnectionError
vprint_error "#{ip} - Failed to connect to Web management console."
end
return response
end
When trying to connect via HTTPS, the first send_request_cgi call returns nil. No exception are caught or thrown. I have tried with 3 different hosts to make sure the issue was not with a specific endpoint. All my 3 attempts failed to return a response. At every attempt, I set the RPORT option to 443;
RHOSTS 0.0.0.0 yes The target address range or CIDR identifier
RPORT 443 yes The target port
Note that I have replaced the real IP with 0.0.0.0. Using a web browser, I can actually connect to the router via HTTPS with no issue (other than having to add an exception since the certificate is untrusted) and am presented the login page. With Wireshark, I tried to look at the generated traffic. I can clearly see that nothing is sent by the router. I notice the 3-way handshake being completed and the HTTP GET request being made:
GET / HTTP/1.1
Host: 0.0.0.0
User-Agent: Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1)
Content-Type: application/x-www-form-urlencoded
Content-Length: 0
There are 3-4 ACK after and then a FIN/PUSH sent by the server.
Based on this page on Metasploit's GitHub, I was under the impression that connections to HTTPS websites were handled by the underlying framework. I have not seen any articles/tutorial/source that leads me to believe otherwise. The doc about the send_request_cgi does not specify any specific requirement to establish a HTTPS connection. Other posts did not had the exact same issue I'm having. At this point I suspect either the OS, the framework or me forgetting to enable something. Other modules I have looked at either only targets HTTP websites - which I doubt - or do not have any special handling for HTTPS connections.
Any help determining the cause would be greatly appreciated.
Version of Metasploit:
Framework: 4.9.3-2014060501
Console : 4.9.3-2014060501.15168
Version of OS:
SMP Debian 3.14.5-1kali1 (2014-06-07)
As per this post on SecurityStreet, the solution was to set SSL to true in the DefaultOptions in the initialize function:
def initialize
super(
...
'DefaultOptions' =>
{
...
'SSL' => true
}
)
...
end
Connections to routers using HTTPS worked afterwards.
I'm working on a plugin for the Redmine platform and I would like to attach a file to a document (basically uploading a file) using a link instead of a form, to do this I'm creating POST requests inside a method.
I followed the instructions here, I set the content type to application/octet-stream as requested then I put the file content in the request body.
I read a lot of posts on this website and I know this has been frequently asked but I can't manage to do my request correctly tough, I'm still getting the error. Here is my code:
uri = URI.parse("http://<my_server_IP_address>:3000/uploads.js")
http = Net::HTTP.new(uri.host, uri.port)
request = Net::HTTP::Post.new(uri.path, initheader = {'X-CSRF-Token' => form_authenticity_token, 'Content-Type' => 'application/octet-stream'})
file = File.new("/home/testFile.txt", 'rb')
request.body = file.read
#response = http.request(request)
As you can see, I set the CSRF token in the header using the form_authenticity_token method but I'm still getting a 422 error.
Filter chain halted as :verify_authenticity_token rendered or redirected
Completed 422 Unprocessable Entity in 4.7ms (ActiveRecord: 0.0ms)
I also tried to put skip_before_filter :verify_authenticity_token at the beggining of my controller although it's not recommended, but it's not working neither.
Do you have an idea what's wrong here?
Note: I'm working with Rails 3.2.16, Ruby 1.9.3-p392 and Redmine 2.4.2
Did you mean to POST to "uploads.js" not "uploads.json"?
uri = URI.parse("http://<my_server_IP_address>:3000/uploads.js")
The docs indicate you POST to either uploads.json or uploads.xml seemingly based on the content format you want to receive in response.
(would have made a comment to your question, but I don't yet have the karma for that)
Rails 3.1.3, ruby 1.9.3p374
I am trying to POST from a controller (which receives data via POST from client and then does some processing first) to another controller in the app, and am getting Timeout::Error.
I have tried using Net::HTTP (long form below, also did shortcut form):
uri = URI.parse(credit_payments_url)
http = Net::HTTP.new(uri.host, uri.port)
request = Net::HTTP::Post.new(uri.request_uri)
auth_token = params[:authenticity_token]
request.set_form_data({:cc => "test", :authenticity_token => auth_token })
response = http.request(request)
And HTTParty:
auth_token = params[:authenticity_token]
HTTParty.post(credit_payments_url, :body => {:cc => "test", :authenticity_token => auth_token})
In both cases, I get the Timeout::Error, and also see this in the server output:
Started POST "/payments/credit" for 127.0.0.1 at 2013-02-19 17:39:35 -0600
Processing by PaymentsController#credit as HTML
Parameters: {"cc"=>"test", "authenticity_token"=>"px+YzdbEfC5p2i3e5yjNT4EQy4WMA9aEWY/v2tfdFhA="}
WARNING: Can't verify CSRF token authenticity
credit_payments_url is the correct url and there is a corresponding route. I've been getting the CSRF warning so I added :authenticity_token from the original request but the CSRF warning still shows up. I'm not sure if that has anything to do with the POST timing out.
I feel like there may be some basic network or configuration issue causing the POST to not work, but can't quite tell what it is. Any ideas?
First - Probably, you have just one worker, busy in this request, and therefore unable to respond the second request. Try to make the post inside a thread, or use more than one worker.
Second - Why are you posting to the app itself? Why don't you dry the code, extracting the needed code from the other action to a method, and calling it in both places?