Have a client running a website on Cloud Run and post-release 2 issues came up:
CR uses a default 302 redirect from HTTP to HTTPS - is there any way of changing those to 301 permanent redirects (temporary redirects are a rather poor choice for SEO)
I know it's not possible to remove the Cloud Run assigned URLs like *.a.run.app, but is there any hack for adding a noindex directive on them (either via meta tags or the HTTP response header - but those would need to apply to *.a.run.app URLs, and not the custom production domain)? Alternatively, adding a separate /robots.txt file only for *.a.run.app URLs?
I was told a dev tried adding code to noindex the *.a.run.app URLs but that did not work due to Cloud Run's limitation itself.
I would appreciate any help on these two.
I believe both of these can be achieved in your application code.
Check this article for HTTP redirection implementation.
As for noindex it should also be possible to determine request host and then respond with X-Robots-Tag header to disable indexing.
Related
From the Electron renderer, I am accessing a local GraphQL endpoint served by a Django instance on my computer, which I'd like to do over HTTP, not HTTPS. But Electron's Chromium seems to intercept my fetch request and preemptively return a 307 redirect.
So if my fetch request is POST to http://local.myapp.com:3000/v1/graphql, then Chromium returns a 307 and forces a redirect to https://local.myapp.com:3000/v1/graphql, which fails because my server is listening on port 3000 and for my use case I can't do a local cert for local.myapp.com.
Theoretically the first insecure request should be hitting an nginx docker container listening on port 3000 without any SSL requirement. And nginx is proxying the request to a Hasura container. But I'm not even seeing the requests in the nginx access logs, so I'm pretty sure the request is being intercepted by Chromium.
I believe this StackOverflow comment summarizes well why this is happening: https://stackoverflow.com/a/34213531
Although I don't recall ever returning a Strict-Transport-Security header from my GraphQL endpoint or Django server.
I have tried the following code without success to turn off this Chromium behavior within my Electron app:
import { app, } from 'electron'
app.commandLine.appendSwitch('ignore-certificate-errors',)
app.commandLine.appendSwitch('allow-insecure-localhost', )
app.commandLine.appendSwitch('ignore-urlfetcher-cert-requests', )
app.commandLine.appendSwitch('allow-running-insecure-content', )
I have also tried setting the fetch options to include {redirect: 'manual'} and {redirect: 'error'}. I can prevent the redirect but that doesn't do me any good because I need to make a successful request to the endpoint to get my data.
I tried replacing the native fetch with electron-fetch (link) and cross-fetch (link) but there seems to be no change in behavior when I swap either of those out.
Edit: Also, making the request to my GraphQL outside of Electron with the exact same header and body info works fine (via Insomnia).
So I have a couple of questions:
Is there a way to programmatically view/clear the list of HSTS domains that is being used by Chromium within Electron?
Is there a better way to accomplish what I'm trying to do?
I think the issue might be from the server, most servers don't allow HTTP in any possible way, they'll drop the data transfer and redirect you to HTTPS and there's a clear reason why they would do that.
Imagine you have an app that connects through HTTPS to send your API in return for some data, if someone just changed the https:// to http:// that'd mean the data will be sent un-encrypted and no matter what you do with your API key, it'll be exposed, that's why the servers don't ever allow any HTTP request, they don't accept even a single bit of data.
I could think of two solutions.
Chromium is not the reason for the redirect, our Django instance might be configured as production or with HTTPS listeners.
Nginx might be the one who's doing the redirecting (having a little bit of SSL def on the configuration)
Last but not least, just generate a cert with OpenSSL (on host http://local.myapp.com:3000/) note: include the port and use that on your Django instance. You can trust the certificate so that it could work everywhere on your computer.
I'm forcing https to access my website, but some of the contents must be loaded over http (for example video contents can not be over https), but the browsers block the request because of mixed-contents policy.
After hours of searching I found that I can use Content-Security-Policy but I have no idea how to allow mixed contents with it.
<meta http-equiv="Content-Security-Policy" content="????">
You can't.
CSP is there to restrict content on your website, not to loosen browser restrictions.
Secure https sites given users certain guarantees and it's not really fair to then allow http content to be loaded over it (hence the mixed content warnings) and really not fair if you could hide these warnings without your users consent.
You can use CSP for a couple of things to aid a migration to https, for example:
You can use it to automatically upgrade http request to https (though browser support isn't universal). This helps in case you missed changing a http link to https equivalent. However this assumes the resource can be loaded over https and sounds like you cannot load them over https so that's not an option.
You can also use CSP to help you identify any http resources on you site you missed by reporting back a message to a service you can monitor to say a http resource was attempted to be loaded. This allows you identify and fix the http links to https so you don't have to depend on above automatic upgrade.
But neither is what you are really looking for.
You shouldn't... but you CAN, the feature is demonstrated here an HTTP PNG image converted on-the-fly to HTTPS.
<meta http-equiv="Content-Security-Policy" content="upgrade-insecure-requests">
There's also a new permissions API, described here, that allows a Web server to check the user's permissions for features like geolocation, push, notification and Web MIDI.
I'm working on a project to require HTTPS everywhere among a suite of MVC and WebAPI applications. I'm trying to understand the trade-offs between clicking the "Require SSL" checkbox in IIS & using a URL Rewrite zmodule vs. using a RequireHttpsAttribute in my global filters and modifying my web.config.
I've found the following guides detailing each approach:
https://webmasters.stackexchange.com/questions/28057/iis-7-require-ssl-automatically-redirect-to-https
http://tech.trailmax.info/2014/02/implemnting-https-everywhere-in-asp-net-mvc-application/
Explain the mechanism can be lengthy, so I will just list the most significant differences in behaviour:
do "Require SSL" in IIS:
The context basically expalin what it do, it's "Require" not "Enforce", which means, if people trying to access your website content through http, the server will just respond with a 403 error, which is usually not a desired behavior, but this may help some certain situation
using URL rewrite module:
The module itself can do quite some different thing, but I assume you are just going to do the regular https redirect. Which means, if user trying to hit ANY content of the site through http, the server will do a 301 or 302 redirect to the https version of same url. This is usually a good option since it doesn't affect any usability of the website.
Global RequireHttpsAttribute action filter: This do similar thing to option number 2, it will do a 302 redirect for any http request that is hitting an ACTION. The main difference is that this only applies to all actions in your controllers, Which means, if someone trying to just get a image or css file through http on your website, this option will let it through and not do any enforcement. This leave you the capability to serve static contents through http, which can be useful in some specific circumstances
Just one extra thing worth mention, the 301 and 302 redirect is not going too well with http POST, so if your user trying to do a post through http, the request body will get lost (thanks to the info from #ChrisPratt).
Typically the folks managing the infrastructure are responsible for making sure things are on https. Typically they aren't very good at this so that is where the RequireHttpsAttribute kicks in as it can encforce https requests at a code level thereby enforcing the HTTPS-only attribute.
In practice it isn't so great as many production setups -- including stackoverflow.com's -- see https terminated in an edge device before being unwrapped and handed to the back-end apps as http and the require https attribute isn't quite nuanced enough to understand this distinction.
The best bet in general is to configure the edge device providing the public http interface to take HTTPS and only HTTPS. Then setup secondary virtual sites [or whatever is vendor appropriate] to redirect all traffic to the cannonical HTTPS url. I'd be a bit nervous about relying upon the RequireHttpsAttribute unless it will be a small app handling it's own requests. That still leaves open holes in terms of artifacts and other things that might not be coming off of a controller.
I have added a custom domain to my Heroku application and it works fine, but the application still responds to {mysubdomain}.herokuapp.com.
To prevent duplicate content I would like to stop having my application respond to the subdomain. Is there some setting in Heroku which does this for me, or do I need to code a 301 redirect?
Another option is to use the rel="canonical" link tag. This tells search engines which URL to use for content that may appear on multiple URLs:
<link rel="canonical" href="http://www.example.com/correct_url">
Here's what google has to say: http://support.google.com/webmasters/bin/answer.py?hl=en&answer=139394
(Your use case is explicitly mentioned at the bottom.)
You would need a 301 redirect. Heroku will always respond to the .herokuapp.com domain of your app
I created the hide_heroku gem to handle this- it uses X-Robots-Tag HTTP headers to prevent search engines from indexing anything under *.herokuapp.com
I don't believe it's possible to remove the Heroku-provided domain name, either via their web interface or the command-line client. If you're concerned about it, redirect or add a robots.txt to your site that blocks when accessed via .herokuapp.com (I don't know how to do that offhand, sorry).
I suspect Google is reasonably smart about indexing Heroku sites and handles the dual-domain issue itself, but that's just a guess.
My ASP.NET MVC3 site, www.mysite.com, pulls images from images.mysite.com. When I'm not logged into my site and using SSL, it works flawlessly. However, when logged in, it get the
Only secure content is displayed.
message in IE9. I understand that. What's the best way to deal with switching URL's for my images? Should I check to see if I'm currently using SSL and point my images to https://images.mysite.com, otherwise http://images.mysite.com?
EDIT: This is an e-commerce site, so most of the time the site is browsed unsecured. But after login, I still need to pull some of those same images, and of course if they browse back to a regular catalog page, it would need to access images. Perhaps I will just have to always use https://images.mysite.com. Just seemed like overkill.
I believe the problem only happens when you're in a secure page accessing content over http. So, for pages that can be seen both in http or https, might be as easy as always using https to get the images, regardless if you're in http or https.
You will always get that message if you are pulling content from a non-SSL site when viewing over SSL. If you site is mostly SSL protected, just always pull images from https://images.mysite.com as you do not get the error if you pull SSL content into a non-SSL site.
Otherwise, you will need to know which pages are only viewable over SSL and which ones are not, and link appropriately.
Lastly, if you site is available over both, you will probably need to look at the HTTPS server variable to determine if you are on SSL or not and use this to determine your link (http or https).
Did you try prefixing with ~instead of ../ or /?
This worked for me.