Get all tweets with specific hashtag - twitter

I've been experimenting with the Twitter API because I want to display a few lists of tweets on a special page.
Among those lists is a list with all tweets containing a specific hashtag (e.g. #test)
However I cannot find how to get that list in either XML or JSON (preferably the latter), does anyone know how? It is also fine if it can be done in TweetSharp

You can simply fetch http://search.twitter.com/search.json?q=%23test to get a list of tweets containing #test in JSON, where %23test is #test URL encoded.
I'm not familiar with TweetSharp, but I guess there must be a search command that you can use to search for #test, and then transform the resulting tweets into JSON yourself.

First install TweetSharp using github
https://github.com/danielcrenna/tweetsharp
Here is the code to do a search
TwitterService service = new TwitterService();
var tweets = service.Search("#Test", 100);
List<TwitterSearchStatus> resultList = new List<TwitterSearchStatus>(tweets.Statuses);
If you have more then one page results you can setup a loop and call each page
service.Search("#Test", i += 1, 100);

It seems like there is a change in the API since last few months. Here is the updated code:
TwitterSearchResult res = twitter.Search(new SearchOptions { Q = "xbox" });
IEnumerable<TwitterStatus> status = res.Statuses;

u access with this url for your tweet searchs. But u have to use OAuth protocols.
https://api.twitter.com/1.1/search/tweets.json?q=%40twitterapi

I struggled with the same problem. Here is my vague solution . Enjoy Programming.
It will get out of the function whenever your required number of tweets are acquired/fetched.
string maxid = "1000000000000"; // dummy value
int tweetcount = 0;
if (maxid != null)
{
var tweets_search = twitterService.Search(new SearchOptions { Q = keyword, Count = Convert.ToInt32(count) });
List<TwitterStatus> resultList = new List<TwitterStatus>(tweets_search.Statuses);
maxid = resultList.Last().IdStr;
foreach (var tweet in tweets_search.Statuses)
{
try
{
ResultSearch.Add(new KeyValuePair<String, String>(tweet.Id.ToString(), tweet.Text));
tweetcount++;
}
catch { }
}
while (maxid != null && tweetcount < Convert.ToInt32(count))
{
maxid = resultList.Last().IdStr;
tweets_search = twitterService.Search(new SearchOptions { Q = keyword, Count = Convert.ToInt32(count), MaxId = Convert.ToInt64(maxid) });
resultList = new List<TwitterStatus>(tweets_search.Statuses);
foreach (var tweet in tweets_search.Statuses)
{
try
{
ResultSearch.Add(new KeyValuePair<String, String>(tweet.Id.ToString(), tweet.Text));
tweetcount++;
}
catch { }
}
}

Related

how to get date wise email faster using gmail API in Asp.Net MVC with OAuth Token

Here I have written code for Gmail API to fetch mail with date filter
I am able to fetch MessageId and ThreadId using the First API. On the basis of MessageId, I put that messageId parameter in a List object and I have sent this parameter in foreach loop from List to the next API to fetch email body on basis of messageID. But the process is very slow for fetching messages from Gmail
public async Task<ActionResult> DisplayEmailWithFilter (string fromDate, string toDate) {
Message messageObj = new Message ();
Example exampleObj = new Example ();
List<GmailMessage> gmailMessagesList = new List<GmailMessage> ();
GmailMessage gmailMessage = new GmailMessage ();
var responseData = "";
//dateFilter string parameter Created with Date Values
string dateFilter = "in:Inbox after:" + fromDate + " before:" + toDate;
try {
// calling Gmail API to get MessageID Details by Date Filter
client.DefaultRequestHeaders.Authorization = new AuthenticationHeaderValue (scheme: "Bearer",
parameter : Session["Token"].ToString ());
HttpResponseMessage responseMessage = await client.GetAsync ("https://www.googleapis.com/gmail/v1/users/me/messages?q=" + dateFilter);
if (responseMessage.IsSuccessStatusCode) {
var data = responseMessage.Content;
}
try {
responseData = responseMessage.Content.ReadAsStringAsync ().Result;
//This Json Data Converted into List Object
var msgList = JsonConvert.DeserializeObject<Root1> (responseData);
//loop for Fetching EmailMessageData by MessageID
if (msgList.resultSizeEstimate != 0) {
foreach (var msgItem in msgList.messages) {
messageObj.id = msgItem.id;
//Calling API with MessageID Parameter to fetch Respective Message Data
HttpResponseMessage responseMessageList = await client.GetAsync ("https://www.googleapis.com/gmail/v1/users/userId/messages/id?id=" + messageObj.id.ToString () + "&userId=me&format=full");
if (responseMessageList.IsSuccessStatusCode) {
var dataNew = responseMessageList.Content;
var responseDataNew = responseMessageList.Content.ReadAsStringAsync ().Result;
//Converting json string in Object
exampleObj = JsonConvert.DeserializeObject<Example> (responseDataNew);
gmailMessage.Body = exampleObj.snippet;
//fetching Header Values comparing with string to get Data
for (int i = 1; i < exampleObj.payload.headers.Count; i++) {
if (exampleObj.payload.headers[i].name.ToString () == "Date") {
gmailMessage.RecievedDate = exampleObj.payload.headers[i].value;
}
if (exampleObj.payload.headers[i].name.ToString () == "Subject") {
gmailMessage.Subject = exampleObj.payload.headers[i].value;
}
if (exampleObj.payload.headers[i].name.ToString () == "Message-ID") {
gmailMessage.SenderEmailID = exampleObj.payload.headers[i].value;
}
if (exampleObj.payload.headers[i].name.ToString () == "From") {
gmailMessage.SenderName = exampleObj.payload.headers[i].value;
}
}
//Adding This Object Values in GmailMessgage List Object
gmailMessagesList.Add (
new GmailMessage {
Body = exampleObj.snippet,
SenderEmailID = gmailMessage.SenderEmailID,
RecievedDate = gmailMessage.RecievedDate,
SenderName = gmailMessage.SenderName,
Subject = gmailMessage.Subject,
});
}
}
}
} catch (Exception e) {
string errorMgs = e.Message.ToString ();
throw;
}
} catch (Exception e) {
string errorMgs = e.Message.ToString ();
throw;
}
return View (gmailMessagesList);
}
I can fetch Gmail email datewise but it took so much time to fetch. how can I improve my code and performance faster?
The query seems like the most you can do. If you know more information about those emails, like a specific subjects or there always come from the same sender you can try to filter that too, like you would in the Gmail interface.
Other way you would be kind of out of luck. You are limited by the files retrieved from User.messages.list.
If you need to escape from the API limitations maybe trying to retrieve the message other way would be the correct way to go. Considerate creating a small code to retrieve message by the IMAP protocol. Several questions in this topic may help you:
Reading Gmail messages using Python IMAP
Reading Gmail Email in Python
How can I get an email message's text content using Python?

Graph API: What is the correct way to interrupt Pagination

I am using this script to fetch Chats. I need 100 chats maximum but it may happen that a chat do not have 100 messages. How can I handle that case in this script?
I am using Node Package Microsoft Graph Client.
const { Client, PageIterator } = require('#microsoft/microsoft-graph-client');
async getChatList(GroupChatId) {
let messages = [];
let count = 0;
let pauseAfter = 100; // 100 messages limit
let response = await this.graphClient
.api(`/chats/${GroupChatId}/messages`)
.version('beta')
.get();
let callback = (data) => {
messages.push(data);
count++;
return count < pauseAfter;
}
let pageIterator = new PageIterator(this.graphClient, response, callback);
await pageIterator.iterate();
return messages;
}
As I answered on the GitHub issue you opened, the iterator should stop all by itself if it runs out of items to iterate before hitting your "maximum". However, I think you're hitting a bug in the specific API you're using /chats/id/messages.
The problem is that this API is returning a nextLink value in it's response even if there are no next pages. It shouldn't be, and I'm reporting that to the Teams folks. That's causing the pageIterator to try to get the next set of results, which returns 0 items and a nextLink. You're stuck in an infinite loop.
So because of this, using the pageIterator just won't work for this API. You'll need to do the iteration yourself. Here's some TypeScript code to show it:
let keepGoing: Boolean = true;
do
{
// If there are no items in the page, then stop
// iterating.
keepGoing = currentPage.value.length > 0;
// Loop through the current page
currentPage.value.forEach((message) => {
console.log(JSON.stringify(message.id));
});
// If there's a next link follow it
if (keepGoing && !isNullOrUndefined(currentPage["#odata.nextLink"]))
{
currentPage = await client
.api(currentPage["#odata.nextLink"])
.get();
}
} while (keepGoing);
You need to check with a conditional statement if the message has value or not.
The pseudo code is given below:
let callback = (data) => {
if(data != "" || data != null)
{
messages.push(data);
count++;
return count < pauseAfter;
}
else{
return;
}
}

Dynamics CRM WEB API- Select field from entity reference

I am trying to select a field from another CRM entity by using an expand with my odata query.
Query string: studios$?Select=studioid,studioname,titleid&$Expand=
Title id is a field in custom entity called title. How do I write an expand clause to expand title and select titleid?
Any help is appreciated!
first you can refer to this article from docs.microsoft explaining in detail the $expand feature, however for simplicity is followed by the relation name with targeted entity then the fields you want to select in parentheses
for ex. $expand={relation_name}($select=filed1,field2)
which might be something like this
$expand=new_new_entity1_new_entity2($select=new_name,createdon)
Notice: the values you get from expand relation will be actually navigation URL to actual data, so you don't expect [new_name,createdon] from new_entity2 to be retrieved, what you will get will be a url to OData those values and for that you might below helping function 1 will do it for you
Also you can use XRM Rest Builder that's an amazing solution when you install on the crm you will find it's button in solutions view. that solution is a tool helps you to design your OData query with nice GUI so that you don't need to write and refine your OData query which is faster and easier.
Helping function 1
`
function OdataExpand(expandUrl) {
var result = null;
var req = new XMLHttpRequest();
req.open("GET", expandUrl, false);
req.setRequestHeader("OData-MaxVersion", "4.0");
req.setRequestHeader("OData-Version", "4.0");
req.setRequestHeader("Accept", "application/json");
req.setRequestHeader("Content-Type", "application/json; charset=utf-8");
req.setRequestHeader("Prefer", "odata.include-annotations=\"*\"");
req.onreadystatechange = function () {
if (this.readyState === 4) {
req.onreadystatechange = null;
if (this.status === 200) {
var results = JSON.parse(this.response);
result = results;
} else {
console.log("OdataExpand Error : ");
console.log(this);
}
}
};
req.send();
return result;
}
`
but if you want to get all the data with one call and maybe you want to expand to more than one entity i recommend that you use Fetch XML simply design the fetch you need using advanced find tool to extract the fetchXML then pass it to Helping Function 2 that should return the full data but be aware that max fetch length in this approach is limited to 2,048 characters [Max Length for GET Request].
Helping Function 2
`
function ExecuteFetchXMLQuery(dataSet, fetchQuery) {
var encodedFetchXML = encodeURIComponent(fetchQuery);
var result;
var req = new XMLHttpRequest();
req.open("GET", Xrm.Page.context.getClientUrl() + "/api/data/v8.0/" + dataSet + "?fetchXml=" + encodedFetchXML, false);
req.setRequestHeader("OData-MaxVersion", "4.0");
req.setRequestHeader("OData-Version", "4.0");
req.setRequestHeader("Accept", "application/json");
req.setRequestHeader("Prefer", "odata.include-annotations=\"OData.Community.Display.V1.FormattedValue\"");
req.onreadystatechange = function () {
if (this.readyState === 4) {
req.onreadystatechange = null;
if (this.status === 200 || (this.status >= 200 && this.status <= 206)) {
console.log("Success Executing Fetch");
var results = JSON.parse(this.response);
result = results;
}
else {
console.log("Error Executing Fetch");
console.log(this);
console.log(this.statusText);
}
}
};
req.send();
return result;}
`
Try this:
studios?$select=studioid,studioname,titleid&$expand=titleid($select=titleid,name)
Also it’s recommended to use CRM REST Builder to compose & test CRM web api queries.
Read more
Note: Check your entity & field names, they should look like new_studio & new_studioid

Get all clientID from MCC adwords account by adwordsAPI

I want to retrieve all clientID from my MCC account. I'm using this code
AdWordsUser user = new AdWordsUser(adwordsPropertyService.getEmail(), adwordsPropertyService.getPassword(),
null, adwordsPropertyService.getUseragent(), adwordsPropertyService.getDeveloperToken(),
adwordsPropertyService.getUseSandbox());
InfoServiceInterface infoService = user.getService(AdWordsService.V201109.INFO_SERVICE);
InfoSelector selector = new InfoSelector();
selector.setApiUsageType(ApiUsageType.UNIT_COUNT_FOR_CLIENTS);
String today = new SimpleDateFormat("yyyyMMdd").format(new Date());
selector.setDateRange(new DateRange(today, today));
selector.setIncludeSubAccounts(true);
ApiUsageInfo apiUsageInfo = infoService.get(selector);
for (ApiUsageRecord record : apiUsageInfo.getApiUsageRecords()) {
......
But apiUsageInfo.getApiUsageRecords return my only some clientId.
Have you any suggests?
My Answer will be helpful for PHP Developers
I am using v201502(php), You will get all account details from ManagedCustomerService api. Please refer the following URL https://developers.google.com/adwords/api/docs/reference/v201502/ManagedCustomerService
This is the sample code i used,
function DisplayAccountTree($account, $link, $accounts, $links, $depth) {
print str_repeat('-', $depth * 2);
printf("%s, %s\n", $account->customerId, $account->name);
if (array_key_exists($account->customerId, $links)) {
foreach ($links[$account->customerId] as $childLink) {
$childAccount = $accounts[$childLink->clientCustomerId];
DisplayAccountTree($childAccount, $childLink, $accounts, $links,
$depth +1);
}
}
}
function GetAccountHierarchyExample(AdWordsUser $user) {
// Get the service, which loads the required classes.
$user->SetClientCustomerId('xxx-xxx-xxxx');
$managedCustomerService =
$user->GetService('ManagedCustomerService');
// Create selector.
$selector = new Selector();
// Specify the fields to retrieve.
$selector->fields = array('CustomerId', 'Name');
// Make the get request.
$graph = $managedCustomerService->get($selector);
// Display serviced account graph.
if (isset($graph->entries)) {
// Create map from customerId to parent and child links.
$childLinks = array();
$parentLinks = array();
if (isset($graph->links)) {
foreach ($graph->links as $link) {
$childLinks[$link->managerCustomerId][] = $link;
$parentLinks[$link->clientCustomerId][] = $link;
}
}
// Create map from customerID to account, and find root account.
$accounts = array();
$rootAccount = NULL;
foreach ($graph->entries as $account) {
$accounts[$account->customerId] = $account;
if (!array_key_exists($account->customerId, $parentLinks)) {
$rootAccount = $account;
}
}
// The root account may not be returned in the sandbox.
if (!isset($rootAccount)) {
$rootAccount = new Account();
$rootAccount->customerId = 0;
}
// Display account tree.
print "(Customer Id, Account Name)\n";
DisplayAccountTree($rootAccount, NULL, $accounts, $childLinks, 0);
} else {
print "No serviced accounts were found.\n";
}
}
GetAccountHierarchyExample($user);
SetClientCustomerId will be the parent ID of your all accounts, It will be appeared near the Sign Out button of you google AdWords account, Please see the attached image
I hope this answer will be helpful, Please add your comments below if you want any further help
If you need just the list of clientCustomerIds, try ServicedAccountService.
Here is a code example that shows how this may be done.
Next time, you might also want to consider asking the question on the official forum for AdWords API: https://groups.google.com/forum/?fromgroups#!forum/adwords-api

OData Service not returning complete response

I am reading Sharepoint list data (>20000 entries) using Odata RESTful service as detailed here -http://blogs.msdn.com/b/ericwhite/archive/2010/12/09/getting-started-using-the-odata-rest-api-to-query-a-sharepoint-list.aspx
I am able to read data but I get only the first 1000 records. I also checked that List View Throttling is set to 5000 on sharepoint server. Kindly advise.
Update:
#Turker: Your answer is spot on!! Thank you very much. I was able to get the first 2000 records in first iteration. However, I am getting the same records in each iteration of while loop. My code is as follows-
...initial code...
int skipCount =0;
while (((QueryOperationResponse)query).GetContinuation() != null)
{
//query for the next partial set of customers
query = dc.Execute<CATrackingItem>(
((QueryOperationResponse)query).GetContinuation().NextLinkUri
);
//Add the next set of customers to the full list
caList.AddRange(query.ToList());
var results = from d in caList.Skip(skipCount)
select new
{
Actionable = Actionable,
}; Created = d.Created,
foreach (var res in results)
{
structListColumns.Actionable = res.Actionable;
structListColumns.Created= res.Created;
}
skipCount = caList.Count;
}//Close of while loop
Do you see a <link rel="next"> element at the end of the feed?
For example, if you look at
http://services.odata.org/Northwind/Northwind.svc/Customers/
you will see
<link rel="next" href="http://services.odata.org/Northwind/Northwind.svc/Customers/?$skiptoken='ERNSH'" />
at the end of the feed which means the service is implementing server side paging and you need to send the
http://services.odata.org/Northwind/Northwind.svc/Customers/?$skiptoken='ERNSH'
query to get the next set of results.
I don't see anything particularly wrong with your code. You can try to dump the URLs beign requested (either from the code, or using something like fiddler) to see if the client really sends the same queries (and thus getting same responses).
In any case, here is a sample code which does work (using the sample service):
DataServiceContext ctx = new DataServiceContext(new Uri("http://services.odata.org/Northwind/Northwind.svc"));
QueryOperationResponse<Customer> response = (QueryOperationResponse<Customer>)ctx.CreateQuery<Customer>("Customers").Execute();
do
{
foreach (Customer c in response)
{
Console.WriteLine(c.CustomerID);
}
DataServiceQueryContinuation<Customer> continuation = response.GetContinuation();
if (continuation != null)
{
response = ctx.Execute(continuation);
}
else
{
response = null;
}
} while (response != null);
I had the same problem, and wanted it to be a generic solution.
So I've extended DataServiceContext with a GetAlltems methode.
public static List<T> GetAlltems<T>(this DataServiceContext context)
{
return context.GetAlltems<T>(null);
}
public static List<T> GetAlltems<T>(this DataServiceContext context, IQueryable<T> queryable)
{
List<T> allItems = new List<T>();
DataServiceQueryContinuation<T> token = null;
EntitySetAttribute attr = (EntitySetAttribute)typeof(T).GetCustomAttributes(typeof(EntitySetAttribute), false).First();
// Execute the query for all customers and get the response object.
DataServiceQuery<T> query = null;
if (queryable == null)
{
query = context.CreateQuery<T>(attr.EntitySet);
}
else
{
query = (DataServiceQuery<T>) queryable;
}
QueryOperationResponse<T> response = query.Execute() as QueryOperationResponse<T>;
// With a paged response from the service, use a do...while loop
// to enumerate the results before getting the next link.
do
{
// If nextLink is not null, then there is a new page to load.
if (token != null)
{
// Load the new page from the next link URI.
response = context.Execute<T>(token);
}
allItems.AddRange(response);
}
// Get the next link, and continue while there is a next link.
while ((token = response.GetContinuation()) != null);
return allItems;
}

Resources