I am retrieving results from NCBI's online Blast tool with 'net/http' and 'uri'. To do this I have to search through an html page to check if one of the lines is "Status=WAITING" or "Status=READY". When the Blast tool has finished the status will change to ready and results will be posted on the html page.
I have a working version to check the status and then retrieve the information that I need, but it is inefficient and is broken into two methods when I believe that there could be some way to put them into one.
def waitForBlast(rid)
get = Net::HTTP.post_form(URI.parse('http://www.ncbi.nlm.nih.gov/blast/Blast.cgi?'), {:RID => "#{rid}", :CMD => 'Get'})
get.body.each{|line| (waitForBlast(rid) if line.strip == "Status=WAITING") if line[/Status=/]}
end
def returnBlast(rid)
blast_array = Array.new
get = Net::HTTP.post_form(URI.parse('http://www.ncbi.nlm.nih.gov/blast/Blast.cgi?'), {:RID => "#{rid}", :CMD => 'Get'})
get.body.each{|line| blast_array.push(line[/<a href=#\d+>/][/\d+/]) if line[/<a href=#\d+>/]}
return blast_array
end
The first method checks the status and is my main concern because it is recursive. I believe(and correct me if I'm wrong) that designed as is takes too much computing power when all that I need is some way to recheck the results within the same method(adding in a time delay is a bonus). The second method is fine, but I would prefer if it was combined with the first somehow. Any help appreciated.
Take a look at this implementation. This is what he does:
res='http://www.ncbi.nlm.nih.gov/blast/Blast.cgi?CMD=Get&FORMAT_OBJECT=SearchInfo&RID=' + #rid
while status = open(res).read.scan(/Status=(.*?)$/).to_s=='WAITING'
#logger.debug("Status=WAITING")
sleep(3)
end
I think using the string scanner might be a bit more efficient than iterating over every line in the page, but I haven't looked at it's implementation so I may be wrong.
Related
I'm trying to get all Stripe::BalanceTransaction except those they are already in my JsonStripeEvent
What I did =>
def perform(*args)
last_recorded_txt = REDIS.get('last_recorded_stripe_txn_last')
txns = Stripe::BalanceTransaction.all(limit: 100, expand: ['data.source', 'data.source.application_fee'], ending_before: last_recorded_txt)
REDIS.set('last_recorded_stripe_txn_last', txns.data[0].id) unless txns.data.empty?
txns.auto_paging_each do |txn|
if txn.type.eql?('charge') || txn.type.eql?('payment')
begin
JsonStripeEvent.create(data: txn.to_json)
rescue StandardError => e
Rails.logger.error "Error while saving data from stripe #{e}"
REDIS.set('last_recorded_stripe_txn_last', txn.id)
break
end
end
end
end
But It doesnt get the new one from the API.
Can anyone could help me for this ? :)
Thanks
I think it's because the way auto_paging_each works is almost opposite to what you expect :)
As you can see from its source, auto_paging_each calls Stripe::ListObject#next_page, which is implemented as follows:
def next_page(params={}, opts={})
return self.class.empty_list(opts) if !has_more
last_id = data.last.id
params = filters.merge({
:starting_after => last_id,
}).merge(params)
list(params, opts)
end
It simply takes the last (already fetched) item and adds its id as the starting_after filter.
So what happens:
You fetch 100 "latest" (let's say) records, ordered by descending date (default order for BalanceTransaction API according to Stripe docs)
When you call auto_paging_each on this dataset then, it takes the last record, adds its id as the
starting_after filter and repeats the query.
The repeated query returns nothing because there are noting newer (starting later) than the set you initially fetched.
As far as there are no more newer items available, the iteration stops after the first step
What you could do here:
First of all, ensure that my hypothesis is correct :) - put the breakpoint(s) inside Stripe::ListObject and check. Then 1) rewrite your code to use starting_after traversing logic instead of ending_before - it should work fine with auto_paging_each then - or 2) rewrite your code to control the fetching order manually.
Personally, I'd vote for (2): for me slightly more verbose (probably), but straightforward and "visible" control flow is better than poorly documented magic.
I'm trying to get all connections (interactions) on a facebook page since a certain time period. I'm using the koala gem and filtering the request with "since: 1.month.ago.to_i" which seems to work fine. However, this gives me 25 results at a time. If I change the limit to 446 (the maximum it seems) that works better. But...if I use .next_page to give me the next set of results within the given time range, it instead just gives me a next set of results without obeying the time range.
For example, let's say I don't increase the limit and I have 25 results per request. I do something like:
#api.get_connections(#fan_page_id, "feed", {since: 1.month.ago.to_i})
let's assume there are 30 results for this and the first request gets me 25 (the default limit). then, if I do this:
#api.get_connections(#fan_page_id, "feed", {since: 1.month.ago.to_i}).next_page
instead of returning the last 5 results, it returns 25 more, 20 of which are not "since: 1.month.ago.to_i". I have a while loop cycling through the pages but I don't know where to stop since it just keep returning results to me no matter what as long as I keep calling .next_page.
is there a better way of doing this?
if not, what's the best way to check to make sure the post i'm looking at in the loop is still within the time range i want and to break out if not?
here's my code:
def perform(fan_page_id, pagination_options = {})
#since_date = pagination_options[:since_date] if pagination_options[:since_date]
#limit = pagination_options[:limit] if pagination_options[:limit]
#oauth = Koala::Facebook::OAuth.new
#api = Koala::Facebook::API.new #oauth.get_app_access_token
fb_page = #api.get_object(fan_page_id)
#fan_page_id = fb_page["id"]
# Collect all the users who liked, commented, or liked *and* commented on a post
process_posts(#api.get_connections(#fan_page_id, "feed", {since: #since_date})) do |post|
## do stuff based on each post
end
end
private
# Take each post from the specified feed and perform the provided
# code on each post in that feed.
#
# #param [Koala::Facebook::API::GraphCollection] feed An API response containing a page's feed
def process_posts(feed, options = {})
raise ArgumentError unless block_given?
current_feed = feed
begin
current_feed.each { |post| yield(post) }
current_feed = current_feed.next_page
end while current_feed.any?
end
current = #api.get_connections(#fan_page_id, "feed", {since: 1.month.ago.to_i})
next = current.next_page
next = next.next_page
.....
Please try these, I think they work.
My website allows users to add text to an image, and displays the result to them via an Ajax.request. Sometimes it works great, but other times the image is incomplete and javascript records the error "Image corrupt or truncated."
How can I make sure the file is completely written before sending the Ajax response back to the browser?
VIEW
new Ajax.Request('<%= url_for(:action => "update_image", :id => #user_image.id) %>?greeting=' + encodeURIComponent(elmgreeting.value), { onSuccess: document.getElementById("card_image").src=card_filename });
MODEL
def create_card
img = Magick::Image.read(self.input_image).first
# ... add the greeting to the image
img.write(self.card_filename)
self.card_width = img.columns
self.card_height = img.rows
self.card_size = img.filesize
end
CONTROLLER
def update_image
#user_image = UserImage.find(params[:id])
#user_image.greeting = params[:greeting]
#user_image.create_card
#user_image.save
render :layout => false
end
[I've also noticed the img.filesize value assigned to the card_size is often inaccurate, leading me to think it's being obtained too early, as well.)
I've tried everything I could find on stackoverflow and elsewhere, but just can't figure it out. Any help greatly appreciated.
FOUND MY PROBLEM: It was the Ajax request. It now works nicely:
new Ajax.Request('<%= url_for(:action => "update_image", :id => #user_image.id) %>?greeting=' + encodeURIComponent(elmgreeting.value), { onComplete:function(request){document.getElementById("card_image").src=card_filename}});
afaik the things you write should be already synchronous, there is probably something else that breaks things. what did your rails server log said? maybe there is an error happens and throws some errors and then your front-side tries to use it as an image source
I have this action in my controller:
def ad
#koder = #side.reklamers.pluck(:id) - [session[:log]]
#reklame = Reklamer.find(#koder.sample)
session[:log] = #reklame.id
render :text => "<span class='bannerid' data-id='#{#reklame.id}'></span><p style='margin-bottom: 7px;margin-top: 7px;font-size: 9px;text-align: center !important;'>Ad</p>#{#reklame.kode}"
end
It renders an HTML ad. This code is really fast. The problem is when I do try to example count view the response becomes VERY slow. 3s. Compared to 200ms!
#koder = #side.reklamers.pluck(:id) - [session[:log]]
#reklame = Reklamer.find(#koder.sample)
session[:log] = #reklame.id
#reklame.views += 1
#reklame.save
render :text => "<span class='bannerid' data-id='#{#reklame.id}'></span><p style='margin-bottom: 7px;margin-top: 7px;font-size: 9px;text-align: center !important;'>Ad</p>#{#reklame.kode}"
I have tried to add a resque background job with the same result..
What should I do?
Since views is really a counter maybe you should take a look at:
http://apidock.com/rails/v3.2.13/ActiveRecord/CounterCache/increment_counter
In your case it should look like this
Reklamer.increment_counter(:views, #reklame.id)
It should update only views column, don't run validations, callbacks etc. which will make it faster. But on the other hand 3s on simple save indicates that there is something else going wrong also.
First, try:
#reklame.update_attribute :views, #reklame.views + 1
See if that helps. If so, then you have some costly data validations going on that are slowing you down, but at least you know the source of the problem.
If that doesn't work, try wrapping that statement in
Thread.new { //code }
If it's still slow doing the save, then it's probably how your database is set up or something along those lines.
I am currently using MongoDB for tracking of various things in a Rails 2 app. I am using the following code to see if MongoDB is up and running and, depending upon the status, displaying a link or an "Offline" message.
This is only for admins, so it's not mission-critical, as the app will continue to run without MongoDB, but I do want to keep disabling the link in the menu when it's not running. However, I don't like the overhead of the below code (doesn't take long to run, but hope that there is a cleaner, faster way):
def verify_mongodb_status
begin
track = Track.first
#mongodb_running = true
rescue
#mongodb_running = false
logger.debug("***MongoDB not running.***")
notify_admin_about_errors("***MongoDB is not running***)
end
end
EDIT: I forgot to mention that I'm already doing a before_filter for this; the method sits in application_controller.rb.
I decided to go with action_caching as there doesn't seem to be a great way to do this. The result was quite a large speed increase from ~120ms to ~16-25ms:
def verify_mongodb_status
begin
track = Track.first
#mongodb_running = true
rescue => e
#mongodb_running = false
logger.debug("***MONGODB OFFLINE***: #{e}")
notify_admin_about_errors("MongoDB", "MongoDB error:\n#{e}", nil)
expire_action :action => :verify_mongodb_status
return
end
end
I'm adding logic now to keep from getting bombarded by emails when MongoDB goes offline (1 is enough).