I'm creating signed URLs and sending them to an external system for later use. Unfortunately there are some length parameters that do not allow extremely long strings to be passed along. Recently, it appears that signed URLs were reformatted and extended which subsequently broke my app.
Is there some method for generating a shorter URL from S3? I would prefer not to rely on a third party URL shortening service for a number of reasons (it's an extra request at URL generation and it adds a point-of-failure).
You can build a very simple URL Shortener directly within Amazon S3:
Turn on static website hosting. This will give you a URL to access your bucket, eg: mybucket.s3-website-ap-southeast-2.amazonaws.com
Create a zero-length object in S3 with a 'short name', eg pic1.jpg
Add metadata to the zero-length object:
Key: Website Redirect Location
Value: The Long URL
Then, when you access the zero-length object (eg mybucket.s3-website-ap-southeast-2.amazonaws.com/pic.jpg) it will actually redirect to the Long URL stored in the metadata.
Simple, with no database required!
Related
At the moment my Rails 6 React app has user uploaded images (avatars, profile wallpapers, etc) stored in S3, inside a public bucket for local development (not facilitated by active storage because it was not playing nice with vips for image processing). The reason its set to public was for ease of set up, now that all of the functionality is complete, for the stagging (and soon production) I would like to add sensible bucket policies. I don't currently have CloudFront set up but I do intend to add that in the near term, for right now I'm using the bucket asset URL to serve assets. I have created two separate buckets, one for images that will be displayed in the app and one for content that is never to be publicly displayed which will be used for internal purposes.
The question I have is, for the content that is in the bucket reserved for viewable content, do I have to make it public (disable that setting in the AWS console that disables public access), then create a policy that allows GET request from wherever (*), then restriction POST, PUT, DELETE, requests to the arn ID of the EC2 instance that's hosting the rails application. The AWS documentation has confused me, it gives me the impression that you never want to enable public access to a bucket, and that policies alone are how you surface bucket content. When I take that approach I keep getting access denied in the UI when I have attempted to do that.
EDIT:
I'm aware that signed URLs can be used, but it is my current understanding that there is a nontrivial speed hit to the UX of the application if you have to generate a signed URL for every image (this app is image heavy). There are also SEO concerns given that all the image URLs would effectively be temporary.
Objects in Amazon S3 are private by default. You can grant access to an object in several ways:
A Bucket Policy that can grant access to everyone ('Public'), or to specific IP addresses or users
An IAM Policy on an IAM User or IAM Group that grants access to that user or group -- however, they would need to access via an AWS SDK so that they can authenticate the call (eg when an application makes a request to S3, it would make an authenticated API call)
An Access Control List (ACL) on the object, which can make the object public without requiring the bucket to be public
By using an Amazon S3 pre-signed URL, which is a time-limited URL that provides temporary access to a private object
Given your use-case, an S3 pre-signed URL would be the best choice since the content is kept private but the application can generate a link that provides temporary access to the object. This can also be done with CloudFront.
Generating the pre-signed URL only takes a few lines of code and does not involve an API call to AWS. It is simply creating a hash of the request using your Secret Key, and then appending that hash as a 'signature'. Therefore, there is effectively no speed impact of generating pre-signed URLs for all of your images.
I don't see how SEO would be impacted by using pre-signed URLs. Only actual web pages (HTML) are tracked in SEO -- images are not relevant. Also, the URLs point to the normal image, but have some parameters at the end of the URL so they could be tracked the same as a non-signed URL.
No it does not have to be public. If you don't want to use CloudFront, the other option is to use S3 presigned RLs.
I have 10 websites hosting in my IIS. A user clicks on a link in site 1, the secret value generated in server-side (of site 1), then I want to send this secret value (encrypt or hash) in URL (query string) to central site and decrypt/validate the value there, to be sure a user is coming from site 1 to proceed (and not from another websites).
I think it won't be something fixed string so users can make this URL themselves:
https://centralSite.com/process?param1=....¶m2=....&source_website=site1
It should be something like a one-time dynamic generated token.
My question is :
Can I achieve this situation in only ONE HTTP GET request?
Should I use encryption/decryption or hashing/validating (all server-side)?
P.S: I can't use POST or other HTTP verbs here.
P.S: I know about HTTP referrer, but it can be spoofed or be null in some situations.
P.S: All websites are serving on HTTPS protocol.
I have a model called Campaign and every Campaign has one attachment.
I use S3 ActiveStorage storage and I need a PERMANENT URL for my Campaign images.
I currently generate URLs like:
campaign.image.service_url
But this link expires in 5 minutes. I need non-expire links. (Config settings only let me get a URL that expires in 1 week, it does not solve my problem again)
How can I get URLs of my images?
EDIT
Solution:
I use CloudFront as CDN. This is the solution I found:
https://domainName+/campaign.image.key
this gives a link to an image file that does not expire.
Check the docs https://api.rubyonrails.org/classes/ActiveStorage/Variant.html#method-i-service_url
You are not supposed to expose service_url directly:
Returns the URL of the variant on the service. This URL is intended to be short-lived for security and not used directly with users. Instead, the service_url should only be exposed as a redirect from a stable, possibly authenticated URL. Hiding the service_url behind a redirect also gives you the power to change services without updating all URLs. And it allows permanent URLs that redirect to the service_url to be cached in the view.
Use url_for(variant) (or the implied form, like +link_to variant+ or +redirect_to variant+) to get the stable URL for a variant that points to the ActiveStorage::RepresentationsController, which in turn will use this service_call method for its redirection.
So use url_for(campaign.image) (or url_for(campaign.image.some_variant)) instead.
The URL that is not expire is simple and without any params:
http[s]://[bucket-name.s3].amazonaws.com/pathtofile/file.extention
You can get this URL from the AWS SDK by using S3::Objects :public_url method
With active storage you can do
url ="#{campaign.image.service.bucket.url}/#{campaign.image.blob.key}"
Then you would need to configure the public access settings on the S3 bucket.
This tutorial for building API's with Devise recommends using headers to send over the login email and API token vs. embeding them as URL parameters.
Rather than sending the data over parameters, we're expecting the client application to send it via two headers: "X-API-EMAIL" and "X-API-TOKEN"; this cleans up the endpoint URIs.
Can someone elaborate on what it means to "clean up" theWhat are the the advantages of requiring authentication via headers vs. having the client embed them as parameters in the URL?
I think by "clean up" they just mean that the URLs are tidier and only contain information about the resource being requested/updated/...
It's common to use this approach -- it can be argued that it's conceptually nicer to keep what you are requesting somewhat separate from the details of your credentials for making the request. The "Authorization" HTTP header is a standard one to use for credentials (e.g. HTTP Basic Auth http://en.wikipedia.org/wiki/Basic_access_authentication#Client_side, AWS API http://docs.aws.amazon.com/AmazonS3/latest/dev/RESTAuthentication.html#ConstructingTheAuthenticationHeader and many more).
It also removes the possibility of clashes between your credential parameters and other parameters. For a trivial example, imagine you use password for your credential password and also want a normal parameter called password for a particular API call (when you're updating another user perhaps). You can't have that - one of them has to be named differently. Ok, you can easily do that, but it's a little artificial, and if you instead supply the password for the credentials in the Authorization header, you are free to have a parameter called password which relates to the actual request you are making (e.g. the new password you're setting for some other user).
Hi I am a web application in ROR and shortening url using bit.ly. whenever i create an url using bit.ly i could custom url like http://bit.ly/19Mk8Oj now i want to remove bit.ly and needs to add my own custom url like ferdy.ly/sdf2323 how to do that?
when i Google about this and found the following url http://support.bitly.com/knowledgebase/articles/76741-how-do-i-set-up-a-custom-short-domain-. now i have a registered domain for this from http://libyanspider.com/m and need help to integrate the custom domain with my bitly account? and my application is a feedback engine wherein we are charging user for subscription so i choose business account and received a mail from bit.ly that per month i need to pay $1995.. is custom domain name in bit.ly will cost this much?
what you want is a url shortener algorithm.
Simple explanation with least efforts:
Have a table that you would store your URLs in for lookup, and the id should be auto-incremented(This is default with AR in Rails), convert the id to base 36 with ruby
6788999.to_s(36) #=> "41ifb"
Then you can have a URL as:
foo.com/41ifb
When the request for the shortened URL hits the controller(which you can basically even use bare Routing for) convert the param to an integer:
"41ifb".to_i(36) #=> 6788999
This is a simple basic URL shortener service