I have a user submission form that includes images. Originally I was using Carrierwave, but with that the image is sent to my server for processing first before being saved to Google Cloud Services, and if the image/s is/are too large, the request times out and the user just gets a server error.
So what I need is a way to upload directly to GCS. Active Storage seemed like the perfect solution, but I'm getting really confused about how hard compression seems to be.
An ideal solution would be to resize the image automatically upon upload, but there doesn't seem to be a way to do that.
A next-best solution would be to create a resized variant upon upload using something like #record.images.first.variant(resize_to_limit [xxx,xxx]) #using image_processing gem, but the docs seem to imply that a variant can only be created upon page load, which would obviously be extremely detrimental to load time, especially if there are many images. More evidence for this is that when I create a variant, it's not in my GCS bucket, so it clearly only exists in my server's memory. If I try
#record.images.first.variant(resize_to_limit [xxx,xxx]).service_url
I get a url back, but it's invalid. I get a failed image when I try to display the image on my site, and when I visit the url, I get these errors from GCS:
The specified key does not exist.
No such object.
so apparently I can't create a permanent url.
A third best solution would be to write a Google Cloud Function that automatically resizes the images inside Google Cloud, but reading through the docs, it appears that I would have to create a new resized file with a new url, and I'm not sure how I could replace the original url with the new one in my database.
To summarize, what I'd like to accomplish is to allow direct upload to GCS, but control the size of the files before they are downloaded by the user. My problems with Active Storage are that (1) I can't control the size of the files on my GCS bucket, leading to arbitrary storage costs, and (2) I apparently have to choose between users having to download arbitrarily large files, or having to process images while their page loads, both of which will be very expensive in server costs and load time.
It seems extremely strange that Active Storage would be set up this way and I can't help but think I'm missing something. Does anyone know of a way to solve either problem?
Here's what I did to fix this:
1- I upload the attachment that the user added directly to my service provider ( I use S3 ).
2- I add an after_commit job that calls a Sidekiq worker to generate the thumbs
3- My sidekiq worker ( AttachmentWorker ) calls my model's generate_thumbs method
4- generate_thumbs will loop through the different sizes that I want to generate for this file
Now, here's the tricky part:
def generate_thumbs
[
{ resize: '300x300^', extent: '300x300', gravity: :center },
{ resize: '600>' }
].each do |size|
self.file_url(size, true)
end
end
def file_url(size, process = false)
value = self.file # where file is my has_one_attached
if size.nil?
url = value
else
url = value.variant(size)
if process
url = url.processed
end
end
return url.service_url
end
In the file_url method, we will only call .processed if we pass process = true. I've experimented a lot with this method to have the best possible performance outcome out of it.
The .processed will check with your bucket if the file exists or not, and if not, it will generate your new file and upload it.
Also, here's another question that I have previously asked concerning ActiveStorage that can also help you: ActiveStorage & S3: Make files public
I absolutely don't know Active Storage. However, a good pattern for your use case is to resize the image when it come in. For this
Let the user store the image in Bucket1
When the file is created in Bucket1, an event is triggered. Plug a function on this event
The Cloud Functions resizes the image and store it into Bucket2
You can delete the image in Bucket1 at the end of the Cloud Function, or keep it few days or move it to cheaper storage (to keep the original image in case of issue). For this last 2 actions, you can use Life Cycle to delete of change the storage class of files.
Note: You can use the same Bucket (instead of Bucket1 and Bucket2), but an event to resize the image will be sent every time that a file is create in the bucket. You can use PubSub as middleware and add filter on it to trigger your function only with the file is created in the correct folder. I wrote an article on this
I would like to know how can we save an image that has been segmented (using fuzzy c-means method) in MATLAB where the end product are images of each cluster group. I would like to save the images to use later.
I assume you just want to save an image, this should be independent on how you produces that.
If I understood correctly you just have to use the function
imwrite(M, filename)
Where M is the matrix containing your image data. You just need to do this for each matrix/image you have.
Then you can reload the image from filename using imread.
imread(filename)
Note that if you want to specify the format for imwrite and not obtain it via the filename extension, you just to add an additional parameter as follow:
imwrite(M, filename, format)
I've build a processor that determine which image format is best to compress my model attached thumbnail. The processor simply build images in png and jpg and check which one is the smallest.
Since Paperclip use the original thumbnail format to build it's thumbnail style URL I had to create a field in my model to store the format of each of the styled thumbnail.
ex: thumbnail_small_content_type, would be "image/png"
In my processor I tried to save the format by using the Paperclip:attachment method : instance_write.
#attachment.instance_write "#{#style_name}_content_type", "image/#{optimised_format}"
Strangely it work perfectly when a create a new model, but failed to work when I use the paperclip method reprocess! to crop my image. Any idea how I could workaround that limitation?
In paperclip when you save an image with lots of style and sizes it also saves the original.
But in my app it's not necessary to save the original, just the style will do, what I was wondering was how do you not save the original. Just store it in memory or in a temporary area and then not save it once the style have been generated.
Ideally it would not save original at all. I guess one solution would be to save the original, process the style and delete afterwards, however I'm trying to save on bandwidth and deleting the original after it has been saved kindof defeats the point.
Cheers!
check this: How do I tell paperclip to not save the original file?
This worked for me
def destroy_original
File.unlink(self.photo.path)
end
Taken from here: http://tekn0t.net/delete-original-image-when-using-papercliprai
Edit: the provided link is no longer valid. Here is a valid one from the same author: https://gist.github.com/tekn0t/755593
I am developing a photo gallery which will read/write EXIF tags. I will put photo title in the EXIF tag DocumentName and description in EXIF tag ImageDescription. I also plan to use the geo-tags. But what about photo tags/categories? I want to store a space-separated string of the tags a photo is tagged with in my system. Is there any EXIF tag that is normally used for this type of information? I could write my own (i.e. PhotoTags) but I guess that is not really of any use except for internally in my system (where this information is stored in a database anyway).
I think that will want to look into IPTC and/or XMP (not entirely clear over their relation) for storing metadata about images (such as keywords/tags, title, description and so on).
There's no EXIF tag for this, but there is an IPTC "keywords" tag. JPEG can handle IPTC records, so there should be no problem.