Differing search results between Twilio web search and Ruby API helper - ruby-on-rails

I'm getting different results between the searches I run on on twilio.com and the searches I run through the Ruby gem helper.
Here's a sample search:
Here's a search with the same zip code in a Rails console, returning an empty array:
> #twilio_client.account.available_phone_numbers.get('US').local.list({in_postal_code: "19428"})
=> []
These searches were conducted less than a minute apart.
Is this an issue with the REST API, the Ruby helper gem, or my search query?

Ricky from Twilio here.
The code you're using is correct to retrieve phone numbers restricted to a postal code and I can see why you would expect it to match up with the search on Twilio.com. In our search we're actually doing the lookup a bit differently. You can create an experience similar to ours by using a different set of filters to retrieve phone numbers.
#numbers = #client.available_phone_numbers.get('US').local.list(
near_lat_long: '40.6928,-73.9903',
distance: '5'
)
Since phone numbers within a postal code can be sparse we can ensure results by expanding our search to be near the location a user searches for.

Related

Google Ads API: getMetrics doesn't get results for all passed keywords

I'm on migration from the old Google AdWords API to the new Google Ads API, using PHP-SDK by Google.
This is the use case, where I'm stuck:
I feed an amount of keywords (paginating them by keyword plans a 10k) to generateHistoricalMetrics($keywordPlanResource) and collect the results.
To do so I followed instructions at https://developers.google.com/google-ads/api/docs/keyword-planning/generate-historical-metrics and, especially, https://developers.google.com/google-ads/api/docs/keyword-planning/generate-historical-metrics#mapping_to_the_ui, with using of KeywordPlanAdGroupKeywords (with a single ad group) and avoiding to pass a specific date range for now, relying on the default value.
Further I had to apply some filters on my keywords because of KEYWORD_HAS_INVALID_CHARS and KEYWORD_TEXT_TOO_LONG, but all the errors which I'm aware of are gone now.
Now, I found out, that the KeywordPlanHistoricalMetrics object does not contain any keyword id (of the form customers//keywordPlanAdGroupKeywords/) So, I have to rely on the correct ordering. This is ok as it seems, that the original ordering of keywords is preserved within the results, as here https://developers.google.com/protocol-buffers/docs/encoding#optional
But still I have the problem, that
count($keywordPlanServiceClient->generateHistoricalMetrics($keywordPlanResource)->getMetrics()) is lower then count($passedKeywords), where each of $passedKeywords where passed to
new KeywordPlanAdGroupKeyword([
'text' => $passedKeyword,
'match_type' => KeywordMatchType::EXACT
'keyword_plan_ad_group' => $planAdGroupResource
]);
Q: So I have two questions here:
Why getMetrics() does not yield the same amount of results as the amount of passed keywords?
I'm struggling with debugging at this moment: Say, I want to know which keywords are let out. Either for providing more information at this place or just to skip them, and let my customer know, that these particular keywords were not queried. How to do this, when although I have a keyword id for every passed keyword I cannot match the returned metrics to them, because the KeywordPlanHistoricalMetrics object does not contain any keyword id.
Detail: While testing I found out, that the reducing of an amount of queried keywords reduces the amount of lost keyword data:
10k of queried keywords - 4,72% loss,
5k - 2,12%,
2,5k - 0,78%,
1,25k - 0,43%,
625 - 0,3%,
500 - 0,24%,
250 - 0,03%
200 - 0,03% of lost keywords.
But I can't imagine, that keywords should be queried one by one.

Tweepy: Find all tweets in a specific language

I'd like to extract all tweets in the Arabic language in all countries.
I modified the code in this tutorial.
This is my search query.
api.search(q="*", count=tweetsPerQry, lang ['ar'],tweet_mode='extended'). I expect to find a very large number of tweets, but I only collected about 7000 tweets.
I checked the content of some of them and I noticed that they are posted in my country even I did not specify the location/Country (Can anyone explain why this happen??).
I tried to know the reason for finding a limited number of tweets, so I modified the query by replacing the lang parameter by geocode to find tweets in a city. I fetched more than 65,000 Arabic tweets. After that, I used the lang parameter with the geocode and I found a very limited number of tweets.
Can anyone help me to know why I'm not able to get a large number of tweets when I used lang parameter?
The free twitter API's are good for small projects, but keep in mind that they don't display all of the tweets. Twitter has paid API's that are much more powerful, though what you are trying to achieve should be possible. I ran the query attached bellow, it seemed to work I was able to find a considerable amount of tweets. This method also seemed to work for #ebt_dev too I think it was just the structure of your request was set out like the stream listener version not the cursor search.
# Search Query change the X of .items(X) to the amount of tweets you are looking for
for tweet in tweepy.Cursor(api.search, q='*',tweet_mode='extended', lang='de').items(9999999):
# Defining Tweets Creators Name
tweettext = str( tweet.full_text.lower().encode('ascii',errors='ignore')) #encoding to get rid of characters that may not be able to be displayed
# Defining Tweets Id
tweetid = tweet.id
# printing the text of the tweet
print('\ntweet text: '+str(tweettext))
# printing the id of the tweet
print('tweet id: '+str(tweetid))

Search results in Twitter API dont match Twitter website search results

I am using REST API to extract twitter search results in Hindi and Hebrew.
Search phrase in Hindi - "मुझे सपना आया" OR "मैंने सपना देखा की"
Twitter search results return about 100 results for this phrase - https://twitter.com/search?src=typd&q=%22%E0%A4%AE%E0%A5%88%E0%A4%82%E0%A4%A8%E0%A5%87%20%E0%A4%B8%E0%A4%AA%E0%A4%A8%E0%A4%BE%20%E0%A4%A6%E0%A5%87%E0%A4%96%E0%A4%BE%20%E0%A4%95%E0%A5%80%22%20OR%20%22%E0%A4%AE%E0%A5%81%E0%A4%9D%E0%A5%87%20%E0%A4%B8%E0%A4%AA%E0%A4%A8%E0%A4%BE%20%E0%A4%86%E0%A4%AF%E0%A4%BE%22
However Twitter explorer console API (https://dev.twitter.com/console) returns only 1 result.
Same thing happens for the Hebrew phrase as well. Website returns correct results. API only returns 1 result
Search phrase in Hebrew - "אתמול בלילה חלמתי"
Please Help!!
Thanks,
DB
Found out that for Twitter search api results only go back 1 week for exact phrase matches versus keyword matches. So changed my search criteria from exact phrases to separate words to solve the problem.
Here are the links from Twitter discussions which I found related to this topic.
https://dev.twitter.com/discussions/20596
https://dev.twitter.com/discussions/23726
When I try to console (With OAUTH) it returns more than 1 result.
If it doesn't for you try to make the count value higher.
this is my result:
http://pastebin.com/H2HVs71G
-Tim

How to paginate based on a maximum number of words per page

My internal website search engine, based on pg_search, sometimes returns so much text in its search results that Heroku cannot load the page.
The problem is, some search results are so long that I could only publish one of them per page, whereas others are so short I could easily publish 20 of them at once.
So I'd like to pagination my search results, but I'd like to limit the amount of content I publish on each page by word count, not by result count.
I've taken a look at the main pagination gems on Ruby Toolbox like will_paginate, but I can't find any that offer this function.
Does a suitable gem exist? Or is there any a straightforward way of doing this with a gem like will_paginate?
Displaying Only the first x words of a string in rails
I modified this method to get
def get_n_words_with_offset(message, first_n_words=1, offset=0)
string_arr = message.split(' ')
string_arr.count > first_n_words ? "#{string_arr[offset..(offset+first_n_words-1)].join(' ')}..." : self
end
From there you can create a pseudomodel that will "paginate" or resplit the array of models returned by word count. Have one of that model encapsulate one page, and you can use a pagination gem where one of those models is on each page.

Retrieve most retweeted tweets for a given hashtag

I'd like to retrieve the tweets for given a hashtag and sort them from the most retweeted to the less retweeted.
The closest thing I've found is using the search call and use the type tag:
E.g.: http://search.twitter.com/search.json?q=TheHashTagHere&result_type=popular
However, I'm not sure on how "popular" option works.
For instance, if it finds 100 tweets with that hashtag I believe it should show the X most retweeted tweets, and if none of those tweets have been retweeted then it should show X of them randomly (or sorted in some other way like the most recent).
Unfortunately, if follows some kind of unknown rule to identify what's popular and what not and even hashtags with thousands of tweets might return only one or two results.
I hope I made myself clear. Thanks in advance :)
PS: I'll use PHP but I think that shouldn't affect the question?
Results will sometimes contain a
result_type field into the metadata
with a value of either "recent" or
"popular". Popular results are derived
by an algorithm that Twitter computes,
and up to 3 will appear in the default
mixed mode that the Search API
operates under. Popular results
include another node in the metadata
called recent_retweets. This field
indicates how many retweets the Tweet
has had.
Source (Emphasis are mine)
Just call with result_type=popular and check the recent_retweets node to see how popular it is. result_type=popular will become the default in an upcome release so beware if you omit this parameter.
Results with popular tweets aren't ordered chronologically. *
If you would like to always have results to show, use result_type=mixed: they will have the result_type in the "metadata" section with a value of "recent", and popular results will have "popular". A small reference about result_types:
mixed: Include both popular and real time results in the response.
recent: return only the most recent results in the response
popular: return only the most popular results in the response.
If a search query has any popular results, those will be returned at the top, even if they are older than the other results. *
*[Twitter API Announcements]
This isn't a programmatic method but rather works in the browser with a chrome extension (HackyBird) :
Install the extension
Search for a phrase e.g. #Social (twitter.com/search?q=%23Social)
Click the extension to sort it (you can adjust the ratio of retweets/likes used for sorting in extension options).
P.S. It'll also sort your or any other user's timeline.

Resources