My web application should have next/previous results for navigating the search timeline. I already made it by using field next_resultsfor next, and pushing refresh_url into javascript array for "previous" navigating. And it worked then (3-4) days ago. After 1-2 days api calls were not returning field next_results, omitted completely, and today that field is returning same value for all "next" calls, so you are hitting next and you are refreshing the page basically. There are also fields since_id for newer and max_id for older results in search_metadata node, so I can make queries using that values... I could also get those id's from returned tweet objects, and use them, not relying on api search_metadata completely...
So I'm asking if someone already dealt with this, what is the best way to do it, not having to check code every day what twitter-api is returning? I know there is twitter support, I think there are people on this site that has done this. I also have read docs on this.
https://dev.twitter.com/docs/working-with-timelines
https://dev.twitter.com/docs/api/1.1/get/search/tweets
Here is search_metadata node example to ilustrate.
[search_metadata] => Array
(
[completed_in] => 0.057
[max_id] => 4.1747346982858E+17
[max_id_str] => 417473469828583425
[next_results] => ?max_id=416844457594855423&q=place&result_type=mixed
[query] => place
[refresh_url] => ?since_id=417473469828583425&q=place&result_type=mixed
[count] => 15
[since_id] => 0
[since_id_str] => 0
)
Ime Ime, "what is the best way to do it", not sure but this is how I used max_id
present in next_results to fetch tweets from timeline.
Twython search API with next_results
Hope it helps.
Related
I'm confused about the friends method in the Twitter gem. I see that Twitter::REST::FriendsAndFollowers#friends uses a GET friends/list request and the cursor_from_response_with_user method.
The documentation for GET friends/list says that requests are limited to 20 users. I assume this means that 20 friends will be returned per request. But, say I am following 22 people and I use the following:
twitter_client = Twitter::REST::Client.new { [my credentials here] }
twitter_client.friends
This returns an array of all 22 friends. I didn't do anything to mess with cursors, so why am I getting all 22? On to my main question...
In my app, when someone imports their friends, I'm iterating over them and creating some other records. Something like this
twitter_client.friends.each do |friend|
SomeModel.do_what_you_need_to_with(friend)
AnotherModel.do_something_else_with(friend)
end
Let's say someone has 5001 Twitter friends. (Impressive!) I'm thinking this is going to be a problem with rate limiting, etc. What's the safest way to get all the friends, and iterate over all of them?
I've tried to figuring out the gem's documentation on cursors, but the fact that friends returned 22 results is throwing me off...
The 20 that you mention is the number of users per page returned by the API. The gem will iterate over all pages, so it will return as many users as possible within the rate limit.
As how to handle the rate limiting, it's up to you. There's an example here that will just put the process to sleep for a while and then retry. Another option is to just quit the app and run it again in a few minutes with something like cron, but it obviously depends on your app.
This is a great question. I ran into the same issue.
I can see how iterating over ALL friends is a feature. However, if you are only interested in the first 20 friends , here's how to not request any more friends:
twitter_client.friends.to_h[ :users ]
Since friends request returns 20, this may not be so useful. If you want friend_ids or follower_ids, the API will return up to 5,000. For many use cases 5,000 is plenty.
twitter_client.friends.to_h[ :ids ]
We're using the tumblr gem to get posts from our blog. We get the posts like this:
client = Tumblr::Client.new
posts = client.posts("blog_url", :type => "text", :limit => 2)["posts"] #gets a posts array
In Tumblr the images are stored in the post body, so you have to search the post body for img tags, for example, the following query gets the image from the first post:
posts = client.posts("blog_url", :type => "text", :limit => 2)["posts"][0]["body"][/<img.*/]
But what if there isn't an image… It returns nil and doesn't display an image.
I want to find the two most recent posts with images, how would I do that? Any ideas?
Make the :limit more than 2 (the documented limit is up to 20 at a time, undocumented limit seems to be 50). You can use :offset to retrieve in batches. Ignore returns of nil until you have collected two valid image posts.
As an aside you say "In Tumblr the images are stored in the post body". This is true if posts are of the type text. But there are also photo type posts. I assume you are aware of this?
The reason I ask is if you are only posting photos, or photos with brief notes. You could make photo posts with captions. You would then be able to request :type=>"photo" with :limit=>2
Alternatively - you could tag all your image blog posts with for example "image". You can then run your :limit=>2 search with :tag=>"image"
Hope that helps in some way :)
I am attempting to get all the orders from a magento instance. Once a day we grab all the orders.. (sometimes a few thousand)
Extra stuff that's more why I ask:
I'm using ruby-on-rails to grab the orders. This involves sending the soap call to the magento instance. It's easy as.
Once I have the response, I convert it into a Hash (a tree) and then pick out the increment id's of the orders and proceed to call getOrder with the increment id.
I have two problems with what's going on now, one operational, and one religious.
Grabbing the XML response to the list request takes really really long and when you tack on the work involved in converting the XML to a hash, I'm seeing a really slow processes.
The religious bit is that I just want the increment_ids so why do I have to pay for the processing/bandwidth to support a hugely bloated response.
Ok so the question...
Is there a way to set the response returned from Magento, to include only specific fields? Only the updated_at and the increment_id for instance.
If not, is there another call I'm not aware of, that can get just the increment_ids and date?
Edit
Below is an example of what I'm looking for from magento but it's for ebay. I send this xml up to ebay, and get back a really really specific bit of info about the product. It works for orders and such too. I can say "only this" and get just that. I want the same from Magento
<GetItemRequest xmlns="urn:ebay:apis:eBLBaseComponents">
<SKU>b123-332</SKU><OutputSelector>ItemId</OutputSelector>
</GetItemRequest>
I've created a rubygem that gives you your salesOrderList response in the form of a hash, and you can do what you want with the orders after you've received them back (i.e. select the fields you want including increment_id). Just run
gem install magento_api_wrapper
To do what you want to do, you would do something like this:
api = MagentoApiWrapper::Sales.new(magento_url: "yourmagentostore.com/index.php", magento_username: "soap_api_username", magento_api_key: "userkey123")
orders = api.order_list(simple_filters: [{key: "status" value: "complete"}])
orders.map {|o| [o.increment_id, o.items.first.sku] }
Rough guess, but you get the idea. You would get the array of hashes back and you can do what you want with them after that. Good luck!
I've got fullcalendar working with a small rails app (yeah) but it's sluggish because the find in my controller is finding ALL the records before it renders the calendar. I'm using a JSON approach. The field names I'm using are starts_at and ends_at. This (in the index method of the assignments_controller) works:
#assignments = Assignment.find(:all, :conditions => "starts_at IS NOT NULL")
But, as I said, it's pokey, and will only get worse as more records get added.
So this is clearly more of a rails question than a fullcalendar question: I can't figure out how to get fullcalendar to initially display the current week (when no parameters have been sent) and then accept parameters from next/previous buttons while, in either case, only looking up the relevant items from the database.
Oh - this is rails 2.x, NOT 3.
Thanks for any pointers.
Please ignore this question.
It turned out to be an issue with Date format inconsistencies between JavaScript (Epoch) and Ruby. At least that's what I think at the moment.
I'm still scratching my head, trying to figure out how exactly I "fixed" it, but it seems to be working.
I was aware of this project: http://github.com/bansalakhil/fullcalendar
but it took me ages to get the nuance of Time.at figured out.
I must say, Time is a tricky thing.
In real life as well as in code.
Thanks to everyone who gave my (misguided, as it turned out) question a glance.
I periodically fetch the latest tweets with a certain hashtag and save them locally. In order to prevent saving duplicates, I use the method below. Unfortunately, it does not seem to be working... so what's wrong with this code:
def remove_duplicates
before = #tweets.size
#tweets.delete_if {|tweet| !((Tweet.all :conditions => { :twitter_id => tweet.twitter_id}).empty?) }
duplicates = before - #tweets.size
puts "#{duplicates} duplicates found"
end
Where #tweets is an array of Tweet objects fetched from twitter. I'd appreciate any solution that works and especially one that might be more elegant...
you can validate_uniqueness_of :twitter_id in the Tweet model (where this code should be). This will cause duplicates to fail to save.
Since it sounds like you're using the Twitter search API, a better solution is to use the since_id parameter. Keep track of the last twitter status id you got from your previous query and use that as the since_id parameter on your next query.
More information is available at Twitter Search API Method: search
array.uniq!
Removes duplicate elements from self. Returns nil if no changes are made (that is, no duplicates are found).
Ok, turns out the problem was a bit of different nature: When looking closer into it, I found out that multipe Tweets were saved with the twitter_id 2147483647... This is the upper limit for integer fields :)
Changing the field to bigint solved the problem. It took me very long to figure out since MySQL did silently fail and just reverted to the maximum value as long as it could. (until I added the unique index). I quickly tried it out with postgres, which returned a nice "Integer out of range" error, which then pointed me to the real cause of the problem here.
Thanks Ben for the validation and indexing tips, as they lead to much cleaner code now!