It seems paperclip and/or delayed_paperclip saves a temporary file on local web server fs before upload it to S3 or Rackspace Cloud File (i'm using fog.io).
If you have a worker for delayed_job on a separated server, this worker won't be able to see this tmp file on the web sever.
Or you could have one worker in each web server, but then worker1 on webserver1 can't see files on webserver2. You would need a queue/server affinity, right?
So the question is: how should I use delayed_paperclip with load balancer?
Thank you!
Related
A quick question: In Java/Spring/Spring Boot world I know it is possible to have your server load their configuration from a "config server" instead of files in the instance or environment variables.
I am talking of spring-cloud-config-server.
Would it be possible to do the same thing in Rails?
Is there an extension or gem that would allow me to pull config from a KV storage or maybe from a remote storage (S3 for example) on instances starting?
Thanks in advance.
I have a WebSphere application cluster on SUSE15 with v9 and java8.
2 load balancer and 4 IHS and 4 WAS. it designs for file uploading.
file uploading through load balancers is very very slow and always failed.
as I investigated, the problem is LB, because if I skip them(Load balancers) and directly connect to IHS or WAS, uploading a file, works as expected.
I couldn't find the reason and related load balancer's config.
does anyone know the reason and the answer?
I have successfully uploaded and deployed my grails application on amazon elastic beanstalk with Tomcat 8 and Java 8 on linux ec2 and web app is up and running. It works well when doing REST API calls to and from RDS database. I have a api to upload file to the server from mobile app and from web app frontend. When running this grails app in localhost its works great for this api and uploads files successfully to user.home/{myapplicationDirectory}/somefile path in my Windows OS. But after running this app in elastic beanstalk and trying to upload image from mobile gives NPE as FileNotFoundException
FileNotFoundException occurred when processing request: [POST] /api/images/add
/usr/share/tomcat8/sdpl/images/260519011919.zip (No such file or directory)
Stacktrace follows:
java.io.FileNotFoundException: /usr/share/tomcat8/sdpl/images/260519011919.zip (No such file or directory)
at java.io.FileOutputStream.open(FileOutputStream.java:270)
I have a service to get application data storage directory with this method
def String getApplicationPath() {
return System.getProperty("user.home") + File.separator + "images" + File.separator;
}
Hi as I don't see your full application I don't want to be too presumptuous but as you're using AWS Beanstalk you should consider local file storage to always be a temporary storage. Your server could be terminate and restarted by beanstalk if it stops responding or fails any health checks.
You have other options available, again I don't know if you considered them and have a good reason for using the local file system so forgive me if that's the case, though if not, you could use S3 for the storage of images, then you don't have to worry about disk space, and the images could automatically be then served via AWS's CDN - Cloudfront, thus also reducing load on your app.
Alternatively, when you really want to store these images in the filesystem, you can look at using EFS, the Elastic File System. Your EBS instance could mount the filesystem on startup so it will be always available whenever your instance(s) start.
I didn't suggest using a standard EBS volume, as you can only ever attach a volume to a single instance, if you used EFS, you don't have to worry about space and it can be mounted to multiple instances so is a little more flexible.
When I enter a URL into my browser, and it sends a request to the server (in this case, the host is salty-headland-18854.herokuapp.com), are all my Rails files stored at this host?
How does Puma come into play? I understand Puma to be a web server and I've included its gem. What is the distinction here between the host and Puma?
Does Heroku have a bunch of physical computers somewhere, that house my literal .rb files?
Visual of my question
It depends on the files, but pretty much the answer is yes. Heroku pulls down your app from the git repository specified (or you push it to Heroku's git repository) and Heroku has all the files from that. Without a security exploit, no one is able to access their source.
You can think of a Heroku Dyno as a Linux server with the complete Rails application. When it is launched for the first time, a linux server is built and the Rails application copied to it and the app server launched.
You code actually lives in the Github repository you have linked to Heroku. Persistant data is stored either in the database or an external file service such as S3.
It is possible to run commands on the Dynos using heroku run
Reference
We have a big problem with downloads when the size is over than 1gb.
We are using Rails 2.3.5, passenger 2.2.9 on Amazon EC2 2gb with 2gb of Ram and Fedora 10.
Files are stored on /mnt/files, project is on /mnt/www/project
We tried to send files with Nginx and x-accel-redirect and also Apache with x-sendfile.
We can download only and always 1.09gb instead of 1.54gb!!
We can download files without problems where size is less than 1gb
If we link same file (that is not corrupted) in rails public dir, we can download without any problem.
X-Accel-Redirect or X-SendFile are configured correctly, tested and checked a lot of time.
So:
Nginx with x-accel-redirect [fail]
Apache with x-send-file [fail]
Send File without x-accel-redirect or x-sendfile on nginx or apache [fail]
Linking file in public and direct download [works]
Any suggestion?
Thanks!!!
If you're looking to restrict access to these downloads, have you tried the Access Key module?