I'm tracking all mentions of #UN with Tweepy using Twitter stream v1 API. However, I'm also getting all mentions of usernames containing #UN such as #UN_Women. I could filter them out in a post-processing step but this seems very inefficient.
Is there any way to avoid this?
This is my code:
class MyStreamListener(tweepy.StreamListener):
def on_status(self, status):
print(status.text)
myStreamListener = MyStreamListener()
myStream = tweepy.Stream(auth = api.auth, listener=myStreamListener())
myStream.filter(track=['#UN'])
Using follow instead of track should work. With follow, you supply a list of user IDs:
myStream.filter(follow=['14159148'])
I don't know if tweepy provides any further functionalities to avoid this. But what you can do here is, filter out the results while saving it to your database or csv.
Check the json response and look for entities object and in that check for user_mentions and screen_name. Save only the one with your desired screen_name
Related
I'm trying to perform a query via the Zendesk Search API that compares two fields. E.g. I want to get all tickets where created_at is equal to updated_at.
This syntax is accepted, but the result is not correct query=type:ticket created_at:(updated_at).
Are such predicates supported by the Zendesk Search API?
If not - are there other endpoints, which can provide me the desired outcome?
I can't see anything that supports this syntax in the search reference. You would have to query tickets in your preferred time range, or of all time, and assert if the creation and update time indeed match. Here is a sample code on how you do it in python using a Zendesk API wrapper called Zenpy:
from dotenv import load_dotenv
from zenpy import Zenpy
from os import environ
load_dotenv()
def main():
zenpy_client = Zenpy(
subdomain=environ["SUBDOMAIN"],
email=environ["EMAIL"],
token=environ["API_TOKEN"],
)
tickets = zenpy_client.tickets.incremental(start_time=1)
for ticket in tickets:
if ticket.created_at == ticket.updated_at:
print(ticket)
if __name__ == "__main__":
main()
The code will print the ticket id of any ticket that had no update since creation.
I have been using tweepy to scrape pieces of information from twitter like tweets with a particular keyword, user id, favorite count, retweeters id, etc. I tried to fetch user id who has retweeted certain tweets. After going through tweepy documentation I found the way to do this is.
API.get_retweets(id, *, count, trim_user)
I have gone through similar questions here ,but I could not figure out how to retrieve retweet ids?
number_of_tweets=10
tweets = []
like = []
time = []
author=[]
retweet_count=[]
tweet_id=[]
retweets_list=[]
retweeters_list=[]
for tweet in tweepy.Cursor(api.search_tweets,q='bullying',lang ='en', tweet_mode="extended").items(number_of_tweets):
tweet_id.append(tweet.id) # Instead of i._json['id']
tweets.append(tweet.full_text)
like.append(tweet.favorite_count)
time.append(tweet.created_at)
author.append(tweet.user.screen_name)
retweet_count.append(tweet.retweet_count)
retweets_list = api.get_retweets(tweet.id)
for retweet in retweets_list:
retweeters_list.append(retweet.user.id)
# print(retweet.user.id)
You should chose more explicit names for your variables (i? j?), that would help you (and the helpers here) to check the logic of your code.
Besides, what you want to achieve and what does not work is pretty unclear.
Does it work if you remove the first inner loop? (I don't understand its purpose)
for tweet in tweepy.Cursor(api.search_tweets,q='bullying',lang ='en', tweet_mode="extended").items(number_of_tweets):
tweet_id.append(tweet.id) # Instead of i._json['id']
tweets.append(tweet.full_text)
like.append(tweet.favorite_count)
time.append(tweet.created_at)
author.append(tweet.user.screen_name)
retweet_count.append(i.retweet_count)
# No need for the first inner loop
retweets_list = api.get_retweets(tweet.id)
for retweet in retweets_list:
print(retweet.id) # If you want the retweet id
print(retweet.user.id)
If not, please give more debug information (what is the problem? What do you want to do? Is there any error message? Where? What have you tried? Show some printed variables etc.).
I am learning how to use Yelp API from this yelp blog and yelp github on Rails. I was able to connect to Yelp service and got a response back, but I don't know what to do with the response that I got back.
Here is what I did, on Rails Console:
2.2.2 :057 > response = client.search('los angeles', {limit: 2})
=> #<Yelp::Response::Search:0x007fff32edc2c0 #region=#<Yelp::Response::Model::Region:0x007fff35ddf6d0 #span=#<Yelp::Response::Model::RegionSpan:0x007fff35ddf450 #latitude_delta=0.04455494999999132, #longitude_delta=0.02209966000000918>, #center=#<Yelp::Response::Model::RegionCenter:0x007fff35ddf5e0 #latitude=34.08390635, #longitude=-118.3184503>>,...
What kind of format is that? this article says that when I make API call to Yelp, it gives ruby object, but I am not sure what data type #<Yelp::Response... is. I guess I was expecting a ruby array/ json format return, like stated in the article:
`
search
response = client.search('San Francisco')
response.businesses
[< Business 1>, < Business 2 >, ...]
response.businesses[0].name
"Kim Makoi, DC"
response.businesses[0].rating
5.0
If I want to select a specific information, say display_address, or neighborhood from the return API, how can I do that? (Here is the end part of the same API request):
...#display_address=["6353 Yucca St", "Hollywood", "Los Angeles, CA 90028"], #geo_accuracy=8.0, #postal_code="90028", #country_code="US", #address=["6353 Yucca St"], #coordinate=#<Yelp::Response::Model::Coordinate:0x007fff35ddf7e8 #latitude=34.10413, #longitude=-118.32834>, #state_code="CA", #neighborhoods=["Hollywood"]>, #deals=nil, #gift_certificates=nil, #reviews=nil>]>
1.
If you were to call the API directly (say, from cURL), you'd get JSON back.
You're using the Yelp gem, though, so it's helpfully converting that JSON into a ruby object for you. If you're interested in the construction of the object, you can take a look at how the gem is doing the conversion on GitHub.
You should be able to interact with that response just like the article states, i.e. results.businesses should give you an array of businesses found.
More concretely, it looks like you can do something like:
results.businesses[0].display_address to get the display address for the first business found
results.businesses[0].neighborhoods is an array of all neighborhoods associated with that same business.
I have a Rails app on a Postgres database and I need to have a search field for the user to enter a string and look up in the database for possible address matches (within a city). In the database I have a column with full addresses.
I cannot make assumptions on the input, so I am thinking that I should first try to directly look up the address on the database somehow (using a LIKE query maybe?), and if that fails, request to a Geocoding API (i.e. Google) to return a well formatted addresses list matching the query and search those in my database.
I would appreciate any guidance on how to do this.
I don't think FTS (full text search) is what you want. You'll have to use an address API that can match addresses.
I've successfully and easily used SmartyStreets for something like this. They have a free account you can use.
http://smartystreets.com
Also if you did want to try going down the FTS route here is a Gist that explains how to do it.
https://gist.github.com/4365593
You may know it already, but postresql has a fulltext search engine integrated so it's a great time to take advantage of it. I suggest watching thats excellent railscast.
Then once implemented :
class Place < AR
def search_db_or_geokit(query)
res = db_search()
if res.empty?
res = geokit_search(query)
else
res
end
end
def geokit_search(query)
# ...
end
def db_search(query)
# ...
end
end
For the geocoding google search api there's probably a good gem out there like geokit
I'm accessing the Amazon AWS API using the ruby-aaws gem, but without going to much into details of the API or the gem, I think my problem is more of a general nature.
When I query the API I will end up with "object array", let's call it item, containing the API response.
I can easily access the data in the array, e.g. puts item.item_attributes.artist.to_s
Now the API returns attributes whose identifier are reserved words in Rails, e.g. format or binding.
So doing this:
puts item.item_attributes.format.to_s will return method not found
while
puts item.item_attributes.binding.to_s will return some object hash like #<Binding:0xb70478e4>.
I can see that there are values under that name when doing
puts item.item_attributes.to_yaml
Snippet from the resulting yaml show artist and binding:
--- !seq:Amazon::AWS::AWSArray
- !ruby/object:Amazon::AWS::AWSObject::ItemAttributes
__val__:
artist: !seq:Amazon::AWS::AWSArray
- !ruby/object:Amazon::AWS::AWSObject::Artist
__val__: Summerbirds in the Cellar
binding: !seq:Amazon::AWS::AWSArray
- !ruby/object:Amazon::AWS::AWSObject::Binding
__val__: Vinyl
This was probably a very detailed explanation with a very simple solution, but I can't seem to find the solution.
edit
Finally found it. I guess it is because it is an array of objects, duh...
puts item.item_attributes[0].binding.to_s
You may be able to access the individual attributes by using [] instead of the method name (which is probably provided using method_missing anyway).
So, item.item_attributes[:artist].to_s may return what you want. If it doesn't it would be worth trying 'artist' as the key instead.
Finally found it. I guess it is because it is an array of objects, duh...
puts item.item_attributes[0].binding.to_s