As part of my web application I have a program that runs periodically during low activity times. However I want to allow the functionality to the user to be able to start the program manually. How can I do this? Where should I put my ruby program within the rails framework and how do I offer a link to commence the program then redirect to another page while the test is running. Thanks.
The code could be put in any controller but if you want the program to run quietly behind the scene without blocking your browser (pretty logic spec), you should trigger it using some tool like DelayedJob or Resque.
Great Railscasts here.
Related
I'm learning full stack development through the free course at appacademy. If you are familiar with it. am almost finished with the rails section and have done also done the SQL & Ruby sections. I have yet to start JavaScript or React.
To test my abilities, I want to create an app. It will mostly be backend + HTML since I haven't covered front end yet.
The app function: ability for someone to provide a list of URLs for reddit posts and then track number of upvotes. I want to do this by scraping the reddit content using this. This is just to test my ability and not a real use case.
I only want to scrape once a day to keep the scraping function minimal. I also want to add a refresh button next to each post, so the user can refresh when they want to.
Questions:
Ill be creating a database that stores the value of upvotes. However, from what I have learned with the program, I do not know how I can use my ruby app to keep running my daily scraper function. If I create a file scraper.rb with the function that is on a time loop, how do I put this file in my ror created app? do i put it in the models, views or controllers folder? Will it run automatically if I run my ruby server? or is this part entirely separate? do I run two ruby apps at once? One for backends and one the actual page
For the refresh button, I think I would need to include a call to the scraper function in my controller before rendering the refreshed text. Does that sound right?
From your description, I would say that the scraper should be a model method and it needs to be wrapped around a rake task.
There are many ways you can achieve the periodic execution. Some of them are:
Whenever gem
Plain old cron job for running the rake task
If you deploy it in a PaaS like heroku, render.com or something similar, there are addons which can be configured to do the cron,like heroku scheduler or render.com cron.
In a rails web app, if I write messages to a queue like rabbitmq, how will clients be notified when a producer sends a message to the queue?
I'm guessing I have to create a seperate process that runs in the background to respond to messages correct? i.e. this code is outside of the scope of a web application.
If this is the case, is it possible to re-use the models/libs that are in the rails application already? do I have to copy this code in 2 places then?
It looks like your application requires what's usually called a background or worker process. This is a fairly common requirement for any moderately complex web application.
I'm guessing I have to create a seperate process that runs in the background to respond to messages correct?
Yes - you're right about this. Whilst it's perfectly possible to use threads to handle the background tasks (in your case, reading and processing messages from RabbitMQ), the standard and recommended route for a Rails application is to run a separate background process.
If this is the case, is it possible to re-use the models/libs that are in the rails application already?
Absolutely. The simplest possible way to get this working is by using Rails' built in runner command.
Another option is to create a ruby script which loads up your Rails application. For example, you could create the file my_script.rb in the root of your project, which might look something like this:
# Load my application:
require File.join(File.dirname(__FILE__), 'config/environment.rb')
# Now you can access your Rails environment as normal:
MyModel.all.each { |x| x.do_something }
If your needs become more complex, or you find that you need to run more than one background process to keep up with the volume of data you need to process, you might want to look at one of the many available libraries and frameworks which can help with this.
Once you've created your background process, you'll need a way to run it continuously when you deploy it to your production server. Whilst it's possible to use libraries like daemons, as suggested by ctcherry, I would recommend using a dedicated tool like upstart (if deploying to ubuntu) or runit. A good summary of the most popular options is available here.
You are correct, you do need a background process. And you can keep the code for that process in the lib folder of the Rails project if you like, I have done that before without issue, and it keeps related code together which is nice.
I used this library to create my long running process, it was quite simple:
http://daemons.rubyforge.org/
In order to re-use models from your rails application you can run a require on the config/environment.rb file to get everything loaded. (Set RAILS_ENV as an environment variable first to select the correct envrionement) From that point the script behaves as though you are inside a rails console session.
I am writing an application that uses FireWatir to do a bunch of different actions. The problem is that I want to trigger these actions from many separate ruby files.
So for example, one ruby script will launch a new FireFox browser instance, than a totally different script will have that instance goto a specific website, and another will log into gmail.
I want all of these scripts to affect the same browser window. That way I can have one script take me to a specific website, and wait for another script to be triggered to do something else.
Please tell me that this is possible.
Chad,
I think that is possible. I am not sure that it's necessary or efficient, but I know that it's possible. The key is to make sure that you attach to the right browser instance. If you will only have one, that could be much simpler.
If you identify the problem that you are trying to solve with these multiple scripts then maybe one or more of the experienced framework designers can point you to existing solutions to the problem. There are some pretty awesome solutions that exist already. At the end of the day, we face the same issues.
Good luck,
Dave
I ended up getting around this issue by using socketing. Had a ruby script acting as the server that was waiting for requests from another group of ruby scripts that could be triggered whenever.
Firstly let me mention that I'm new to web-frameworks.
I have to write my first web-app for a Uni project. I spent two weeks learning Grails and Django. Started working with Rails yesterday and loved it. So I've decided to go with it and discard my work in the other frameworks.
About the app
It's supposed to be a Twitter app that utilizes Twitter's Streaming API to record tweets which match a set of specified filters. (I'm going to use the Tweetstream gem which takes care of connecting to Twitter and capturing matching tweets).
The app's web interface should have the following functionality -
Creating new requests The user inputs a set of filter parameters (keywords to track) & URL/username/password of an existing PostgreSQL or MySQL database.
When a request is created, the web-app spawns a background ruby process. This process connects to Twitter via the Tweetstream gem. It also connects to the database specified by the user to stores received tweets.
View/terminate of existing requests
The user should be able to see a list of requests that are running as background processes by visiting a URL such as /listRequests.
See further details about a process/terminate the process
The user should be able to go to URL such as /requests/1/detail to view some details (e.g how long request has been running, number of tweets captured, etc). The user should also be able to terminate the process.
My inexperience is showing as I'm unable to comprehend -
what my models should be (maybe Request should be a model. Tweet doesn't need to be a model as it's not being stored locally)
how I'm going to connect to remote databases.
how I can create background processes (backgroundrb??) and associate them with request objects so that I can terminate then when the user asks.
At the end of the day, I've got to build this myself, so I'm not asking for you to design this for me. But some pointers in the right direction would be extremely helpful and appreciated!
Thanks!
Hmm.
Since the web app is just a thin wrapper around the heavy-lifting processes, it might be more appropriate to just use something like Sinatra here. Rails is a big framework that pulls in lots of stuff that you won't need for this project, even though it will work.
Does the "background process" requirement here strictly mean a separate process, or does it just mean concurrency? TweetStream uses the EventMachine gem to handle updates as they come, which uses a separate thread for each connection. It would be quite possible to spawn the TweetStream clients from a simple Sinatra web app, keep them in a big array, have them all run concurrently with no trouble, and simply run stop on a given client when you want it to stop. No need for a database or anything.
I'm not sure exactly what your prof is looking for you to do here, but MVC doesn't really fit. It's better to work with the requirements than to mush it into a design pattern that doesn't match it :/
Even so, I <3 Rails! Definitely get on that when you're working primarily with objects being represented in a database :)
Quite a project. Most of what will be challenging is not related to rails itself, but rather the integration with background processes. backgroundrb is a bit out of fashion. The last commit on the main github project is over a year ago, so it's likely not up to snuff for Rails 3. Search around and evaluate your options. Resque is popular, but I'm not sure if your real-time needs match with its queue-based structure.
As for your app, I see only a single model, but don't call it request. That's a reserved name in rails. Perhaps a Search model, or something along that line.
Connecting to different databases is straight forward but will require direct configuration of your ActiveRecord class during operation rather than using database.yml.
It seems to me that Rails shines when building web-centric applications such as the Depot online-store app presented in the Agile book or the Basecamp project management system. The crux of these applications is their web interface, and Rails gives you the tools to create these webapps quickly, easily, and elegantly.
But what if you have an application with a command line interface for which you'd like to add a front-end? For example, imagine an application which processes third-party EDI files - it gets them from an FTP server, parses them, inserts the data into a database and sends out reports via emails. The schedule of when the app should run, which reports should be generated, who should get the email, etc. is all configured in a database. And although the main point of the application is the processing of EDI files and you want to run it via cron, you'd like to build a web page to allow users to manage which reports they're receiving, how often they receive them, etc.
Both the command line app and the web page need to access the same data in the database. They should be able to share models and business logic. If the command line app already contains a library with a data access layer, is there a way to leverage this library within Rails? Or, conversely, is there a way to reuse Rails' models and controllers in an outside app? Or does it not make sense to use Rails as the web framework in this scenario?
You could always just include the ActiveRecord library (and ActiveSupport) in a regular ruby application, if you are just looking to abstract away the model layer for a db centric app.
Actually, most rails apps have a web interface and also a command-line interface - through rake.
The classical example are batch jobs that have to be run periodically, though cron or whenever. But it is also possible to run those tasks manually.
If I were you, I'd try building some rake tasks inside lib/tasks in order to map those command-line commands.