I have 2 separate rails applications (app a, app b). Both of these apps maintain a customer list. I would like to run a rake task once a day and have app b pull in select customers from app a.
This is the way I have attempted to solve this. If I am going down the wrong road please let me know.
I am using JBuilder to generate the JSON
My issue is with how to have App B set an id in app A, so that the system knows the customer has already been transfered over.
Im assuming I have to do a put request similar to what I have done to get the customers list, but I am having issues getting that to work.
App A
Customers Model
scope :for_export, :conditions => {:verified => true, :new_system_id => nil ...}
Customers Controller
skip_before_filter :verify_authenticity_token, :only => [:update]
#
def index
#customers = Customer.for_export
end
def update
#customer = Customer.find(params[:id])
if #customer.update_attributes(params[:customer])
render :text => 'success', :status => 200
end
end
App B
rake task
task :import_customers => :environment do
c = Curl::Easy.new("http://domain.com/customers.json")
c.http_auth_types = :basic
c.username = 'username'
c.password = 'password'
c.perform
a = JSON.parse(c.body_str)
a.each do |customer|
customer = Customer.create(customer)
#put request back to server a to update field
end
end
end
end
What I have is currently working, Im just not sure if this is the correct method, and also how to initiate a put request to call the update method in the customer controller.
Thanks!
Ryan
I'm sorry that I'm not answering your question, but I am giving you an alternative. What you are trying to create sounds a lot like an ETL job. You may want to consider having a batch job move a copy of your customers table from app a over to app b periodically, and then have another batch job import that table into app b's database. I know, it's a little clunky, but it's a very popular and reliable pattern to solve your problem.
Also, if both apps are in the same data center, then you may want to create a read-only database view of app a's customer data and then have app b read that using SQL calls. It's a slightly cheaper and easier way to integrate the two apps than the option that I listed above.
Good luck!
Related
I am trying to create my own rss reader app in ruby on rails. I want to be able to store various news stories in my database that I can pull from later to display each story with its headline, image, summary, etc. in a nice layout. I am working with the feedjira library and am also pretty new to RoR. I know that these two commands in the rails console fetch rss feeds and somehow parse them:
urls = %w[http://feedjira.com/blog/feed.xml https://github.com/feedjira/feedjira/feed.xml]
feeds = Feedjira::Feed.fetch_and_parse urls
While these two commands work on rss feeds, I was wondering how I could configure my database/model and then save the news entries I get from Feedjira into the db. I tried watching the railscast on this issue but it seemed a bit out of date. Any help on this issue would be immensely appreciated! Thanks in advance!
Here's one way:
Create a model such as this:
class Entry < ActiveRecord::Base
attr_accessible :guid, :source_site_id, :url, :title, :summary, :description, :published_at
def self.update_from_feed(feed_name)
feed = Feed.find_by_name(feed_name)
feed_data = Feedjira::Feed.fetch_and_parse(feed.feed_url)
add_entries(feed_data.entries, feed)
end
private
def self.add_entries(entries, feed)
entries.each do |entry|
break if exists? :entry_id => entry.id
create!(
:entry_id => entry.id,
:feed_id => feed.id,
:url => entry.url,
:title => entry.title.sanitize,
:summary => entry.summary.sanitize,
:description => entry.content.sanitize,
:published_at => entry.published
)
end
end
end
end
You can then call this from the cli / cron or whatever with, for example:
rails runner -e development 'Entry.update_from_feed("feedname")'
This runs the update_from_feed method in the context of your Rails app using a separate rails instance (a bit like rails console), but doesn't impact the running Rails instance.
In this example, there's a separate model which has name and feed_urls, so there's a lookup of the url based on the provided name.
This code doesn't use the ability of Feedjira to check for updates, so dupe checking is baked in.
(This guthub issue says to avoid using the #update method.
Note that the use of break assumes that new entries are always added to the top of the feed. If you don't trust the feed, then replace break if with unless. The url can be used as an alternative unique id.
Edit:
Here's a version of the update_from_feed method that takes advantage of Feedjira's ability to process multiple feeds:
def self.update_all
feed_urls = Feed.pluck :feed_url
feeds = Feedjira::Feed.fetch_and_parse(feed_urls)
feed_urls.each do |feed_url|
feed = Feed.find_by_feed_url(feed_url)
add_entries(feeds[feed_url].entries, feed)
end
end
pluck returns all the rows of the specified column(s) (:feed_url in this case) in an array. Equally you could change it to accept an array of names, from which it looks up an array of URLs to pass to feedjira.
Finally, if you wanted a self-looping method, you could include:
def self.update_all_periodically(frequency = 15.minutes)
loop do
update_all_from_feed
sleep frequency.to_i
end
end
Then this:
rails runner -e development 'Feed.update_all_periodically'
won't return until you break the process, and will update all feeds at the default frequency, or that specified as an optional argument.
If you wanted to run the updates asynchronously in your main Rails process, then a background worker such as Sidekiq, Resque or DelayedJob will do the... job. :)
Scheduling the fetching and parsing of al these feeds can be incredibly hard and time consuming, which means you shoud absolutely not do it from inside the Rails app itself. At best, you should do it using an 'offline' script.
You could also simply rely on existing APIs like Superfeedr and its rack middleware.
I am facing an issue with accessing a particular variable of a method say A , in another method say B in the controller.. The size of the object(variable) is too big since it contains the results of a service call made.. My usecase is like on selecting an option from a drop down box, it redirects to a method B in controller and the same object(variable) should be parsed. How can I access the variable in the other method?
I tried storing in a cookie and since the size is too big I am getting Cookie Overflow exception. I am not using a DB. So I guess using memcache won't work. Also tried storing it as hidden field in view and passed its value as a data through ajax call. But I am getting it as a string. Tried to specify datatype as json and several other ways.. but of no use..Using ##var also din work..Not sure why..
Code:
On change of the drop down:
$(document).ready(function(){
$('#filter_service').change(function() {
$.ajax({type: "GET",
url: "/device_troubleshootings/query_operation",
data: { filter_service: $('# filter_service').val()},
});
});
});
Service call:
def log_results
//Service call
#get_log_results = LogQueryService.client.get_logs(Com::Amazon::Logqueryservice::DeviceSerialNumberQuery.new(:search_text => # search , :index => 'dms', :index_type => '_all', :from_time_stamp => #from_time_stamp, :to_time_stamp => #to_time_stamp))
#dsn_result = #get_log_results.logs_result_obj_list
end
Here, I am trying to access #dsn_result object in "/device_troubleshootings/query_operation” method.
Please suggest me ways to access the variable.
MVC
I think you're getting confused with how Rails should work
Remember, Rails (which is just a framework for Ruby) is built on the "MVC" programming pattern. This means each time you send a request to your Rails application, it has to be handled by a single controller#action which you will then allow you to pull the relevant data from your models
The problem you have is you're trying to load multiple controller methods, and pass the same data to both. This might work in Ruby, but not Rails (Rails is stateless):
--
Model
The correct way to handle this type of setup is by creating another request for your application, which will load another controller#action, allowing you to access the data you need
As demonstrated by the MVC diagram above, each time you send a request to Rails, it's basically a new request. This means that unless you've persisted your data in the likes of a cookie, you'll need to load the data from the model.
The problem you have is you're trying to store an entire data-set in the front-end of your system. This issue is very bad, as not only is it inefficient, but it goes against the MVC pattern completely.
You'll be much better storing the bare-minimum data set you need in the front-end (ids or similar), which you will then be able send to your controller via ajax; building a new data-set from
--
Class Variables
You mentioned you tried to declare some ##class variables to no avail. The problem with this is that the class vars will only be available for an instance of a class.
As mentioned, since Rails is stateless, the class variables won't persist between requests (how can they?). I think you know this already, considering you've been trying to use cookies to store your data
The way to resolve this is to rebuild the data each time from the model (as detailed above)
Solution
The solution for you is to "go stateless"
Here's how:
Treat Method A and Method B as completely separate "ACTIONS"
When using these actions, you need to consider the smallest piece of data to pass between the two
To load Method B, you need to send a new request from your browser (as if you've never loaded Method A before)
Your method_a can be handled in the "standard" way:
#config/routes.rb
resources :your_controller do
collection do
get :method_a
get :method_b
end
end
This will mean that you can load method_a relatively simply:
#app/controllers/your_controller.rb
Class YourController < ApplicationController
def method_a
#get_log_results = LogQueryService.client.get_logs(Com::Amazon::Logqueryservice::DeviceSerialNumberQuery.new(:search_text => # search , :index => 'dms', :index_type => '_all', :from_time_stamp => #from_time_stamp, :to_time_stamp => #to_time_stamp))
#dsn_result = #get_log_results.logs_result_obj_list
end
end
As you know, the #dsn_result will not persist through to the next request.
There are two ways to resolve this (set a CONSTANT -- if you're pulling from an API, this will give you a single call -- or use a before_action to set the variable for as many actions as you need). I'll detail both for you:
#app/controllers/your_controller.rb
Class YourController < ApplicationController
before_action :set_log_data
def method_a
end
def method_b
end
private
def set_log_data
#get_log_results = LogQueryService.client.get_logs(Com::Amazon::Logqueryservice::DeviceSerialNumberQuery.new(:search_text => # search , :index => 'dms', :index_type => '_all', :from_time_stamp => #from_time_stamp, :to_time_stamp => #to_time_stamp))
#dsn_result = #get_log_results.logs_result_obj_list
end
end
This will work if you pull data from your own data-set (using the models), however, the better way to do this in your case will likely be to set a constant (considering, of course, that you don't want the data to change):
#config/initializers/dsn_result.rb
get_log_results = LogQueryService.client.get_logs(Com::Amazon::Logqueryservice::DeviceSerialNumberQuery.new(:search_text => # search , :index => 'dms', :index_type => '_all', :from_time_stamp => #from_time_stamp, :to_time_stamp => #to_time_stamp))
DSN_RESULT = get_log_results.logs_result_obj_list
In my case I solved with global variable $my_global_var
So my files look like this
routes.rb
Rails.application.routes.draw do
resources :pages
root 'pages#index'
post 'pages/test'
end
pages_controller.rb
class PagesController < ApplicationController
def firstaction
$my_global_var = "My global var"
puts $my_global_var
end
def secondaction
puts $my_global_var
end
end
index.html.erb
<%= button_to 'Test', pages_test_path, method: :post %>
I am trying to replicate the setup Ryan Bates has in this railscast on Resque, where he queues up a third party service web request and then updates his results page with results.
I am designing an application that will interact with another Rails app, not a browser, and would like to replicate analogous behavior, with key difference being that only JSON output is expected
Currently I have something like this: (my models are Lists and Tasks, a List has_many Tasks and a Task belongs_to a List.
My lists_controller.rb
def show
Resque.enqueue(TaskDataFetcher,params[:id])
# confused if I need to have a render_to below this.
end
In task_data_fetcher.rb
require "net/http"
require "uri"
class TaskDataFetcher
#queue = :tasks_queue
def self.perform(id)
list = List.new(:id => id)
url = "taskservice.com/" + id + ".json"
uri = URI.parse(url)
response = Net::HTTP.get_response(uri)
task = Task.new(:contents => response.body)
task.list = list
# how to return this to the requesting server????
end
end
In the Railscast you see that result doesn't automatically update after the Resque task finishes, he has to reload the page several times, re-making the show request. So if you want to replicate this behaviour you could do something like:
def show
list = List.find(params[:id])
if list
respond_to do |format|
format.json {render :json => list.to_json}
end
else
Resque.enqueue(TaskDataFetcher, params[:id])
render :nothing => true, :status => 202
end
end
Requerement:
So your user is requesting your service to see some tasks. And you have to fetch those from another service taskservice.com. Then i think you have to do this through database for persistency.
Suggestion:
You can have a model like TaskRequest having attributes
`id` # must ;)
`task_list` # might be xml or whatever format suits you best
`is_received` # boolean
In your show method,
You create a TaskRequest entry and render a view which will show a loading type thing and will be requesting for task via ajax. The ajax response should return the task list and the is_received. However, once you get is_received true with a content you should request again.
In parallel, your TaskDataFetcher should receive two ids. One that you are sending now and another is of TaskRequest id. So after fetching the data from the service it will store that in the TaskRequest table and will update the is_recieve to true. Setting it true will eventually turn off requesting for this data anymore.
well the whole explanation might seem a bit hazy. Just let me know if you didnt any part or you need anything else specifically.
Note: It is something like the way SO shows the code formatting while answering a question ;)
There are several stages to this, and as I am relatively new to rails I am unsure if I am approaching this in the best way.
Users follow Firms, Firms applications open and close on certain days. If a user follows a firm I would like them to automatically get an email when a) the firms application opens, b) a week before the firms applications close, c) on the day that the firms applications close.
I have tried using named scope. I have the following model method (I presume this will need a little work) setting each firms scope, depending on the date.
model firms.rb
def application_status
if open_date == Today.date
self.opening = true
else
self.opening = false
end
if ((close_day - Today.date) == 7)
self.warning = true
else
self.warning = false
end
if close_day == Today.date
self.closing = true
else
self.closing = false
end
end
I would like this method to be called on each firm once a day, so that each firm has the appropriate scope - so I have tried using the whenever gem (cron) and the following code. Running the above model method on each firm.
Schedule.rb
every 1.day do
runner "Firm.all.each do |firm|
firm.application_status
end"
end
Then for each of the scopes opening, warning, closing i have a method in the whenever schedules file, For simplicity I shall show just the opening methods. The following queries for all firms that have had the opening scope applied to them, and runs the application_open_notification method on them.
Schedule.rb
every 1.day do
runner "Firm.opening.each do |firm|
firm.application_open_notification
end"
end
This calls the following method in the Firm.rb model
def application_open_notification
self.users.each do |user|
FirmMailer.application_open(user, self).deliver
end
end
Which in turn calls the final piece of the puzzle... which should send the user an email, including the name of the firm.
def application_open(user,firm)
#firm = firm
#user = user
mail to: #user.email, subject: #firm' is now accepting applications'
end
end
Is this a viable way to approach this problem? In particular I am not very familiar with coding in the model.
Many thanks for any help that you can offer.
I'll guess that opening, warning and closing are database fields, and you have scopes like:
class Firm < ActiveRecord::Base
scope :opening, :where => { :opening => true }
# etc
end
There is a general rule for database (and, well, all storage): don't store things you can caculate, if you don't have to.
Since an application's status can be dermined from the day's date and the open_date and close_day fields, you could calculate them as needed instead of creating extra fields for them. You can do this with SQL and Active Record:
scope :opening, :where { :open_date => (Date.today .. Date.today+1) }
scope :warning, :where { :close_day => (Date.today+7 .. Date.today+8) }
scope :closing, :where { :close_day => (Date.today .. Date.today+1) }
(Note that these select time ranges. They may have to be changed depending on if you are using date or time fields.)
But there is another issue: what happens if, for some reason (computer crash, code bug etc) your scheduled program doesn't run on a particular day? You need a way of making sure notices are sent eventually even if something breaks. There are two solutions:
Write your schedule program to optionally accept a date besides today (via ARGV)
keep flags for each firm for whether each kind of notice has been sent. These will have to be stored in the databse.
Note that scopes aren't necessary. You are able to do this:
Firm.where(:open_date => (Date.today .. Date.today+1)).each do |firm|
#...
end
but the scope at least encapsulates the details of identifying the various sets of records.
Here's more of an academic question for you guys. Say I want to create a model in a ruby on rails app to track simple views information. I would like to record the user_id, the URI for the page viewed, and keep track of the number of times the user has visited a page.
Model A: One way to do this would be to create a model View with attributes user_id and page (records the uri), and then create a new entry every time a user opens a page.
Model B: A second way to do this would be to add an attribute "page_views" to the model, to track the number of times the user has accessed that page.
Pros and Cons: Model A would have more information recorded and lead to a larger db than Model B. However, Model B would require that a controller search for an existing user-page combination, and either add views to that entry, or create a new one. This leads to a smaller database, but may be worse in scale due to the need to search for existing entries.
So my question to you guys is: which is more important? Are my thoughts wrong? Am I missing something here (other performance considerations overlooked?)
NoSQL approach to tracking user activity:
model app/models/user.rb
class User < ActiveRecord::Base
include UserModules::Tracker
...
end
mixin app/models/user_modules/tracker.rb
module UserModules
module Tracker
def get
key = "user_" + self.id.to_s
arr = Resque.redis.lrange(key, 0, -1)
arr.map{|i| JSON.parse(i)}
end
def put(controller_name, action_name, details="")
key = "user_" + self.id.to_s
created = Time.now.to_formatted_s(:db)}.to_json
# silent exception handle, so you can do not run Redis localy
begin
Resque.redis.rpush key, {
:controller_name => controller_name,
:action_name => action_name,
:details => details,
:created_at => created
rescue
nil
end
end
end
end
controller app/controller/dashboard.rb
class Dashboard < ApplicationController
after_filter :track, :only => :show
# this action will be tracked
def show
end
# this action will show tracking
def logs_show
render :json => current_user.get
end
...
private
def track
details = "any details...."
current_user.put(controller_name, action_name, details)
end
end
You need to have Redis installed, I prefer to use Resque as common way to setup and initialize Redis via Resque.redis, because it will help you to browse your tracking with resque-web
On of the way to setup Redis is in my gist