I have application where we have posts to which we upload photos. I have implemented S3 uploading module using carrier-wave and fog integration which is successful. But when images are uploaded along with versions the original file also getting stored in the same directory.
Is there any way to configure a separate folder inside my bucket to store only original images and rest of the images separately.
I also searched and learned that operating with multiple buckets is not yet possible with carrier-wave.
Kindly please direct me on this. Thanks in advance.
Related
I have CSV files in a directory of an S3 bucket. I would like to use all of the files as a single table in Dremio, I think this is possible as long as each file has the same header/columns as the others.
Do I need to first add an Amazon S3 data source using the UI or can I somehow add one as a Source using the Catalog API? (I'd prefer the latter.) The REST API documentation doesn't provide a clear example of how to do this (or I just didn't get it), and I have been unable to find how to get the "New Amazon S3 Source" configuration screen as shown in the documentation, perhaps because I've not logged in as an administrator?
For example, let's say I have a dataset split over two CSV files in an S3 bucket named examplebucket within a directory named datadir:
s3://examplebucket/datadir/part_0.csv
s3://examplebucket/datadir/part_1.csv
Do I somehow set the S3 bucket/path s3://examplebucket/datadir as a data source and then promote each of the files contained therein (part_0.csv and part_1.csv) as a Dataset? Is that sufficient to allow all the files to be used as a single table?
It turns out that this is only possible for admin users, normal users can't add a source. To do what I have proposed above you put the files into an S3 bucket which has already been configured as a Dremio source by an admin user. Then you promote the files or folder as a data source using the Dremio Catalog API.
I need help moving the images I have from Parse to S3 on AWS. I have viewed numerous supposed guides and GitHub projects, but everything stops short at giving you all the information. One even says, you need GCS bucket set up, but gives no details on how to set up one. Just someone please help me with this. I have the S3 File Adapter in my index.js all set up for the app, but none of the images are there, they are still hosted in parse.
If you are referring to old images that where hosted with parse.com that you want to move across to your own environment then it can be done with the utility tool.
Get all files across all classess in a Parse database. Print file URLs
to console OR transfer to S3, GCS, or filesystem. Rename files so that
Parse Server no longer detects that they are hosted by Parse. Update
MongoDB with new file names.
https://github.com/parse-server-modules/parse-files-utils
Moving forward if you have setup your S3 bucket correctly all new images from your app will be stored there.
https://github.com/ParsePlatform/parse-server/wiki/Configuring-File-Adapters
I am developing an iOS app that uses a large amount of images that are needed for animations for short videos. I want to save my application assets as static files in cloud and once they are needed download them using secure API call (either JSON, XML or any other alternative for that matter).
What is the best option for that. I have checked Parse, Dropbox, iCloud, Google Drive, but I am puzzled since I see only instructions for dynamic data that lets users access content they have created and not static assets.
What would be best option for that?
If you just want an easy way to serve static files I would take a look at Amazon S3. You can just upload files through the online console and then get the public URL to those files to use in your app. You can also use the S3 API to upload files through your web service or iOS app.
Hope this helps!
I'd go for Parse (basically because it is fast to learn and develop), you can create a table with the images and change the writing permissions if you are afraid somebody could modify the table.
Another option that you can check it's the special Config table so you can upload custom files (zip files i.e.) and download them in demand.
For my Rails application, I download a bunch of files from a remote URL to my application. I would like to directly upload them to Amazon S3, without needing a form to do the upload, since I will temporarily cache the file I downloaded on the EC2 instance.
I would also like to retain the links to the files I uploaded so I can download them later.
I am essentially reposting the files I downloaded.
I looked around, but most of the solution seem to involve form uploading to S3 with a user.
Is there s direct upload solution?
You can upload directly to S3 using the AWS SDK for Ruby. The easiest way is:
require 'aws-sdk'
s3 = Aws::S3::Resource.new(region:'us-west-2')
obj = s3.bucket('bucket-name').object('key')
obj.upload_file('/path/to/source/file')
Or you can find a couple other options here.
You can simply use EvaporateJS to achieve this. You can also take advantage of sending ajax request to update file name to the database after each file upload. Though javascript exposes few details your bucket is not vulnerable to hack as S3 service provide a bucket policy.
Just set the <AllowedOrigin>*</AllowedOrigin> to <AllowedOrigin>specificwebsite.com</AllowedOrigin> in production mode.
I've successfully uploaded, via Paperclip, images to the RackSpace cloudfile storage, and they appear correctly within subsequent webpages when I'm using the CDN url.
However, I can't figure out why Paperclip is not showing the files if I do not use CDN enabled cloud files.
Paperclip returns (via its 'url' method for the attachment) the more usual /attachments/fred/1/image/123.jpg path, however that results in a broken image as there is no actual file stored at that url - its in RS cloud file storage.
I'm not sure whether
a) Paperclip is supposed to give me a url to non-cdn location
b) Paperclip provides a url which results in it then responding at that url to provide the raw image data
c) Something completely different to a) and b)
If someone could please shed some light on what url I'm supposed to get back from Paperclip for non CDN enabled RackSpace stored files I think it would help steer me in the correct direction.
Thanks.
Rackspace Cloud Files has the ability to download files without a CDN, but after researching it paperclip and fog do not currently support this.
Paperclip supports both CDN access as well as downloading files using a temporary url (sans CDN). I was going to suggest using a temporary url, however, the get_http_url method paperclip uses to retrieve this url isn't currently implemented for Rackspace.
I have created a fog issue to address this https://github.com/fog/fog/issues/2103.