I'm running an IIS web site (built using ASP.NET/MVC) that among other things collects files from multiple agents that anonymously upload the files via BITS.
I need to make sure that only files uploaded from known sources as well as matching certain predefined file name pattern will be accepted by IIS. All other BITS upload attempts must be cancelled.
As I understand, BITS uses an ad hoc protocol over HTTP 1.1 using "BITS_POST" verb. So, ideally, I'd like to hook into IIS, analyze a BITS_POST request info and if it does not satisfy my pre-conditions, drop the request.
I've tried to create and register a filter implementing IActionFilter.OnActionExecuting, but it seems that my filter does not receive BITS_POST requests.
I'd be glad to hear if somebody have implemented similar BITS related solutions and how this was done. Anyway, other ideas are welcome too.
Regards,
Natan
I have never worked with BITS, frankly i dont know what is it.
What i usually do is such situations is implement an HTTP module. On its begin request event, you can iterate through incoming HTTP request data and decide to stop processing the request if data is not complying with requirements. You have full access to HttpContext.Current.Request object from HTTP module code.
With HTTP modules, you can execute .NET code even before entering the ASP.NET pipeline.
Related
I have spent my afternoon getting very excited about the container-native serverless platform 'fn project' - http://fnproject.io/.
I love the idea of the FaaS model but have no intention of locking myself into a particular cloud vendor for most of the lifetime of an app - and several other reasons including the desire to spin up the entire app on a small server anywhere if I choose.
fn project seems great for my needs until I finish perusing the documentation and all the relevant blog posts and suddenly think 'what? Wait....what??? Where are the http operations?'.
I cannot find a single reference anywhere that states if it is even possible to to have http triggers for different http operations (ie POST, PUT, PATCH, DELETE), let alone how I would do it.
I want to build REST api's (or certainly at the very least json-serving http-based RPC apis - if it doesn't have hypermedia links it isn't REST ;) but let's not get into that one in this thread)
Am I missing something here (certainly the correct bit of documentation)??
Can anybody please enlighten me as to how I would do this, or even tell me if I have totally misunderstood what I should use this for?
My excitement has gone soft for now but I'm hoping someone that will change with the right information.
It feels odd that I can't find anyone else complaining about this, so I think that indicates my misunderstanding perhaps.
Other solutions such as OpenFaaS look interesting but I dont wan't to have to learn how to deploy kubernetes and docker swarms if I can avoid it :)
I'm not an expert, but as of now it seems not possible to specify the http method inside the trigger. Check latest trigger spec : as you can see, there is no notion of http method here.
However, handling different HTTP methods can be done inside the function itself.
For example, in Java (with fdk-java v1.0.80), you can use com.fnproject.fn.api.httpgateway.HTTPGatewayContext as the first parameter of the function, as described in the section "Accessing HTTP Information From Functions" of the documentation :
In Fn for Java, when your function is being served by an HTTP trigger (or another compatible HTTP gateway) you can get access to both the incoming request headers for your function by adding a 'com.fnproject.fn.api.httpgateway.HTTPGatewayContext' parameter to your function's parameters.
Using this allows you to :
...
Access the method and request URL for the trigger
...
You can then retrieve the HTTP method by calling getMethod() on the HTTPGatewayContext passed as parameter.
In other languages (with others fdk), it's possible to do the same :
in Go : example calling RequestMethod() on context
in Ruby : class HTTPContext
in Python : class HTTPGatewayContext
in Node : class HTTPGatewayContext
From this different contexts, you'll then be able to get method parameter passed when fn invoke --method=[GET|POST|...] (via fn-http-method header).
The main drawback here is that all HTTP methods should be handled in the same function. Nonetheless, you can structure your code to have only one class per method.
After some further thought it seems fairly clear now what my actual misunderstanding was....
When I have built Serverless framework services in the past (or built and deployed Lambda functions using terraform) I have been deploying to AWS and so have been using AWS's API Gateway offering (their product is actually called API gateway but its important to recognise that API Gateway is a distributed systems / micro-sevices design pattern).
API gateway makes it possible to route specific http request types including the method (GET,POST,PUT,DELETE) to the desired functions.
Platforms such as Fn project and OpenFaaS do not provide an out of the box api gateway solution and it seems we would need to take care of this ourselves.
These above mentioned platforms are about deployment of functions. We find the other bits via our product of choice.
I've read about push-promise in HTTP/2 specs and several other tutorials, and have an idea as a concept.
I've read here in SO why bundling won't be as relevant in upcoming days. So, if I have to incorporate push promise into applications, where is the ideal place to do this. Should it be just before redirecting to the view from the Action method? Or, in the script in the view? As far as I've searched I couldn't find any examples.
Please someone share their experience implementing in the real code. Does it seem like an overhead if you have to support both the protocols?
Also, if I'm using IIS 10, then is there any configuration changes that I should do to support both protocols? [As far as I've read, we don't have to. But always better heed to some experts.]
So, if I have to incorporate push promise into applications, where is the ideal place to do this. Should it be just before redirecting to the view from the Action method? Or, in the script in the view?
I did it in the controller action method while experimenting, but if you have common resources you may want to move it somewhere more fundamental/shared in the pipeline. Anywhere that has access to the HttpResponse object should work. As I noted here, you'll want to use the PushPromise overload that takes in an HTTP method and headers if what you're pushing will vary based on any request headers, e.g. accept-encoding (compression).
Does it seem like an overhead if you have to support both the protocols?
Also, if I'm using IIS 10, then is there any configuration changes that I should do to support both protocols?
You do not need to do anything explicitly to support both protocols; IIS will take care of it. Per David So of Microsoft, "provided the client and server configuration supports HTTP/2, then IIS will use HTTP/2 (or fallback to HTTP/1.1 if not possible)". This is true even if you're using server push: "If the underlying connection doesn’t support push (client disabled push, or HTTP/1.1 client), the call does nothing and returns success, so you can safely call the API without needing to worry about whether push is allowed."
Incidentally, if you want to disable HTTP/2 on Windows Server 2016, you can do so via the registry.
In addition to checking IIS logs, as David So suggested, you can verify HTTP/2 is being used by right-clicking on the headers row (Name, Status, Type, etc.) in Chrome's Network tab and checking off "Protocol"; you'll see "h2" for HTTP/2 responses. You can verify push promises are working by looking at the Chrome HTTP/2 internals page (chrome://net-internals/#http2) and looking at the "Pushed" and "Pushed and claimed" columns for your domain.
I've searched for this a bit on Stack, but I cannot find a definitive answer for https, only for solutions that somehow include http or unencrypted parameters which are not present in my situation.
I have developed an iOS application that communicates with MySQL via Apache HTTPS POSTS and php.
Now, the server runs with a valid certificate, is only open for traffic on port 443 and all posts are done to https://thedomain.net/obscurefolder/obscurefile.php
If someone knew the correct parameters to post, anyone from anywhere in the world could mess up the database completely, so the question is: Is this method secure? Let it be known nobody has access to the source code and none of the iPads that run this software are jailbreaked or otherwise compromised.
Edit in response to answers:
There are several php files which alone only support one specific operation and depend on very strict input formatting and correct license key (retreived by SQL on every query). They do not respond to input at all unless it's 100% correct and has a proper license (e.g. password) included. There is no actual website, only php files that respond to POSTs, given the correct input, as mentioned above. The webserver has been scanned by a third party security company and contains no known vulnerabilities.
Encryption is necessary but not sufficient for security. There are many other considerations beyond encrypting the connection. With server-side certificates, you can confirm the identity of the server, but you can't (as you are discovering) confirm the identity of the clients (at least not without client-side certficates which are very difficult to protect by virtue of them being on the client).
It sounds like you need to take additional measures to prevent abuse such as:
Only supporting a sane, limited, well-defined set of operations on the database (not passing arbitrary SQL input to your database but instead having a clear, small list of URL handlers that perform specific, reasonable operations on the database).
Validating that the inputs to your handler are reasonable and within allowable parameters.
Authenticating client applications to the best you are able (e.g. with client IDs or other tokens) to restrict the capabilities on a per-client basis and detect anomalous usage patterns for a given client.
Authenticating users to ensure that only authorized users can make the appropriate modifications.
You should also probably get a security expert to review your code and/or hire someone to perform penetration testing on your website to see what vulnerabilities they can uncover.
Sending POST requests is not a secure way of communicating with a server. Inspite of no access to code or valid devices, it still leaves an open way to easily access database and manipulating with it once the link is discovered.
I would not suggest using POST. You can try / use other communication ways if you want to send / fetch data from the server. Encrypting the parameters can also be helpful here though it would increase the code a bit due to encryption-decryption logic.
Its good that your app goes through HTTPS. Make sure the app checks for the certificates during its communication phase.
You can also make use of tokens(Not device tokens) during transactions. This might be a bit complex, but offers more safety.
The solutions and ways here for this are broad. Every possible solution cannot be covered. You might want to try out a few yourself to get an idea. Though I Suggest going for some encryption-decryption on a basic level.
Hope this helps.
How do I play a track from a SoundCloud URL, which, for example, I got from the xml response from a query
<stream-url>https://api.soundcloud.com/tracks/31164607/stream</stream-url>
I should have thought that it would have been as easy as:
https://api.soundcloud.com/tracks/31164607/stream&client_id=my_client_id
yet I get
<error>401 - Unauthorized</error>
All I want to do is consume it in a Silverlight MediaElement, so all I need is set some url to the MediaElement's Source property.
I've checked an application that I wrote about 2 years ago, and THEN, accessing the stream url was as easy as this for a public track:
http://api.soundcloud.com/tracks/18163056/stream&consumer_key=MY_CONSUMER_KEY
however this no longer seems to work.
For example, all I had to do then in C# was:
MediaElement me = new MediaElement();
me.Source= new Url("http://api.soundcloud.com/tracks/18163056/stream&consumer_key=MY_CONSUMER_KEY");
me.Play();
Any hints would be appreciated.
I had a reply on a Microsoft forum that seems to imply that SoundCloud might not be possible to stream to Windows 8 Metro devices without consuming the whole stream before playback starts - which is quite worrying and would seem to imply that to make authentication possible, it would have to be done entirely in the url querystring insterad of using the header:
(The following reply is the answer to the following question: 'I am able to access an audio stream by http using the MediaElement, however I need to access it via https in which I need to add the oAuth info to the header of the initial request.
How is this done when using a MediaElement, and if it cannot be done, what is the workaround for consuming an audio feed in Metro 8 that requires header authentication to stream?')
"Direct access to the underlying network stream is not currently permitted by the MediaElement. Because of this there is currently no way to modify the header of the HTTP request to include any additional authentication information. That said, you do have control over the URL. You could theoretically setup an HTTP proxy service that translated the HTTP GET request parameters into the necessary oAuth credentials. Keep in mind that this is just a theoretical workaround. You may find different behavior in practice. Another theoretical workaround would be to handle the oAuth yourself via a raw stream socket and pass the retuned media data to the MediaElement via "Set Source" and a "Random Access Stream". Please keep in mind that this method has major limitations. in order to use a "Random Access Stream" with the ME you need to make sure all of the data is available before passing it to the ME."
The proxy service is not scalable for an application that is merely distributed for free as every stream would need to come via the proxy. And the raw stream socket, although getting around this, would mean that playback could not start until the whole file had downloaded - and this goes against all current UX (User Experience) guidelines.
So once again, if anyone has any tips, or info about how the whole authentication thing can be achieved in a querystring instead of using headers, I'd appreciate it!
I'm a little confused about whether you're referring to a public or a private track? If it's a public track, then you shouldn't need to send any authentication information, just your client id.
When I request https://api.soundcloud.com/tracks/31164607/stream?client_id=YOUR_CLIENT_ID then I get a 302 redirect to the proper mp3 stream.
Remember, adding parameters to a URL must start with a ? not &. This could (more than likely) be the reason why you are getting a 401 (SC is not picking up the client_id).
After authentication the link like this
http://api.soundcloud.com/tracks/103229681/stream?consumer_key=d61f17a08f86bfb1dea28539908bc9bf
is working fine. I am using Action Script.
I'm following up on Tom's reply because he calls attention to url character specificity. My HTTP requests randomly started failing today, and I was prefacing my client_Id with a ?. As soon as I changed that single ? to &, it started working. So in my case, SC wasn't picking up my client_Id because I used the wrong character. I think depending on where in the request we're talking about specifically, it's worth noting that differences between ? and & do make a difference.
I'm working on a Firefox add-on. At one point in the add-on, it needs to contact my server, give it a URL, then the server will process the URL (it's not a quick or easy process) and send back a link to the resulting file (on Amazon S3). However, since Firefox add-ons are mostly open-source by nature, how can I protect that one call to the server from being discovered by malicious people and then abused? (I only want users of my add-on to be able to make that call to the server...)
I was thinking about using some sort of key or something, but since Firefox add-ons are all open-source that wouldn't really work. I also thought about writing a binary component, but I have almost no experience in C++ (but maybe for something simple like this I could learn) - but then how could I protect calls to the binary component? Also, how could I protect from network sniffers and such? Would it be possible to somehow include encryption of some sort in that binary component so network sniffers cannot detect what is being sent to the server? (I found this question: Client-Server security and authentication but it was a bit over my head.)
When you distribute an extension you are basically giving people the car keys, so it's hard to stop them from taking the car for a joyride. The most secure solution is the one you suggested: use a binary component and authenticate the connection with username/password or PKI certificates. If you use HTTPS then the connection will be encrypted. This isn't totally secure (the binary component could be reverse-engineered by a determined hacker) but it will certain deter casual attacks. Note, however, that implementing, building and maintaining a binary component is significantly harder than the JavaScript equivalent.
An alternative would be to put the user through a one-time registration process when they install the extension. This could even happen in the background, and the user would get some credentials (e.g. a PKI key pair) at the end that they would use for subsequent communication with the server. If you're worried about people overusing the server, then presumably doing so via the extension wouldn't be great either. This way you can track the usage of each user (via the extension or not) and limit it as necessary.
In the end I don't think that there is much you can do. You could have a binary library in your add-on and call it via js-ctypes (it is easier to implement and maintain than a binary XPCOM component). But nothing will stop people from taking your extension and changing its JavaScript logic in any way that they like. You can make it harder by using some logic like this:
Client to server: "Extension version 1.2.3 wants to make a request."
Server to client: "Take your chrome://... file, bytes M to N, what SHA1 checksum do you get for them?"
Client to server: "SHA1 checksum is ..., please process URL ... now."
But this is also everything but fail-proof, it merely makes abusing your service somewhat harder. Same goes for automatic registration - if your extension can register then an attacker can create an account as well.