LinqtoXML parsing RSS content:encoded - asp.net-mvc

I am trying to parse an RSS feed using Linq to XML like so:
XNamespace slashNamespace = "http://purl.org/rss/1.0/modules/slash/";
XDocument rssFeed = XDocument.Load(#"http://blog.daimokuchart.com/index.php/feed/");
var posts = from item in rssFeed.Descendants("item")
select new RSSData {
Title = item.Element("title").Value,
Published = DateTime.Parse(item.Element("pubDate").Value),
Url = item.Element("link").Value,
Content = item.Element("content:encoded").Value
};
However; it is having a problem with with the content:encoded item I get this error "The ':' character, hexadecimal value 0x3A, cannot be included in a name. "
How the heck to I parse this item element?

XNamespace nsContent = "http://purl.org/rss/1.0/modules/content/";
// ...
Content = item.Element(nsContent + "encoded").Value
// ...

There is a much simpler way to parse a RSS feed : the SyndicationFeed class
More details here

Hi i used Linqtoxml and successfully able to parse Rss feed try below code
public apheadlines()
{
InitializeComponent();
InitializeComponent();
WebClient downloader = new WebClient();
Uri rssurl = new Uri("http://ibnlive.in.com/ibnrss/rss/southcinema/telugunews.xml", UriKind.Absolute);
downloader.DownloadStringCompleted += new DownloadStringCompletedEventHandler(downloads);
downloader.DownloadStringAsync(rssurl);
}
private void downloads(object sender, DownloadStringCompletedEventArgs e)
{
if (e.Result == null)
{
MessageBox.Show("Error in download");
}
var Rss = from rss in XElement.Parse(e.Result).Descendants("item")
select new Data
{
Titles = rss.Element("title").Value.ToUpper(),
pubDate = rss.Element("pubDate").Value.Substring(0, 17)
};
listBox1.ItemsSource = Rss;
}

Related

how to get date wise email faster using gmail API in Asp.Net MVC with OAuth Token

Here I have written code for Gmail API to fetch mail with date filter
I am able to fetch MessageId and ThreadId using the First API. On the basis of MessageId, I put that messageId parameter in a List object and I have sent this parameter in foreach loop from List to the next API to fetch email body on basis of messageID. But the process is very slow for fetching messages from Gmail
public async Task<ActionResult> DisplayEmailWithFilter (string fromDate, string toDate) {
Message messageObj = new Message ();
Example exampleObj = new Example ();
List<GmailMessage> gmailMessagesList = new List<GmailMessage> ();
GmailMessage gmailMessage = new GmailMessage ();
var responseData = "";
//dateFilter string parameter Created with Date Values
string dateFilter = "in:Inbox after:" + fromDate + " before:" + toDate;
try {
// calling Gmail API to get MessageID Details by Date Filter
client.DefaultRequestHeaders.Authorization = new AuthenticationHeaderValue (scheme: "Bearer",
parameter : Session["Token"].ToString ());
HttpResponseMessage responseMessage = await client.GetAsync ("https://www.googleapis.com/gmail/v1/users/me/messages?q=" + dateFilter);
if (responseMessage.IsSuccessStatusCode) {
var data = responseMessage.Content;
}
try {
responseData = responseMessage.Content.ReadAsStringAsync ().Result;
//This Json Data Converted into List Object
var msgList = JsonConvert.DeserializeObject<Root1> (responseData);
//loop for Fetching EmailMessageData by MessageID
if (msgList.resultSizeEstimate != 0) {
foreach (var msgItem in msgList.messages) {
messageObj.id = msgItem.id;
//Calling API with MessageID Parameter to fetch Respective Message Data
HttpResponseMessage responseMessageList = await client.GetAsync ("https://www.googleapis.com/gmail/v1/users/userId/messages/id?id=" + messageObj.id.ToString () + "&userId=me&format=full");
if (responseMessageList.IsSuccessStatusCode) {
var dataNew = responseMessageList.Content;
var responseDataNew = responseMessageList.Content.ReadAsStringAsync ().Result;
//Converting json string in Object
exampleObj = JsonConvert.DeserializeObject<Example> (responseDataNew);
gmailMessage.Body = exampleObj.snippet;
//fetching Header Values comparing with string to get Data
for (int i = 1; i < exampleObj.payload.headers.Count; i++) {
if (exampleObj.payload.headers[i].name.ToString () == "Date") {
gmailMessage.RecievedDate = exampleObj.payload.headers[i].value;
}
if (exampleObj.payload.headers[i].name.ToString () == "Subject") {
gmailMessage.Subject = exampleObj.payload.headers[i].value;
}
if (exampleObj.payload.headers[i].name.ToString () == "Message-ID") {
gmailMessage.SenderEmailID = exampleObj.payload.headers[i].value;
}
if (exampleObj.payload.headers[i].name.ToString () == "From") {
gmailMessage.SenderName = exampleObj.payload.headers[i].value;
}
}
//Adding This Object Values in GmailMessgage List Object
gmailMessagesList.Add (
new GmailMessage {
Body = exampleObj.snippet,
SenderEmailID = gmailMessage.SenderEmailID,
RecievedDate = gmailMessage.RecievedDate,
SenderName = gmailMessage.SenderName,
Subject = gmailMessage.Subject,
});
}
}
}
} catch (Exception e) {
string errorMgs = e.Message.ToString ();
throw;
}
} catch (Exception e) {
string errorMgs = e.Message.ToString ();
throw;
}
return View (gmailMessagesList);
}
I can fetch Gmail email datewise but it took so much time to fetch. how can I improve my code and performance faster?
The query seems like the most you can do. If you know more information about those emails, like a specific subjects or there always come from the same sender you can try to filter that too, like you would in the Gmail interface.
Other way you would be kind of out of luck. You are limited by the files retrieved from User.messages.list.
If you need to escape from the API limitations maybe trying to retrieve the message other way would be the correct way to go. Considerate creating a small code to retrieve message by the IMAP protocol. Several questions in this topic may help you:
Reading Gmail messages using Python IMAP
Reading Gmail Email in Python
How can I get an email message's text content using Python?

How to Upload a Profile photo in base64 format For Community Users Using ConnectApi.UserProfiles.setPhoto

1Am Uploading Profile Photo for Community Users in base64 format By Using ConnectApi.UserProfiles.setPhoto Method. But am getting "ConnectApi.ConnectApiException: The file you uploaded doesn't appear to be a valid image" This error, Help me to Fix this issue.
Hi you can try the below method:
public PageReference upload() {
Blob b;
document.AuthorId = UserInfo.getUserId();
document.FolderId = UserInfo.getUserId(); // put it in running user's folder
try {
document.type = 'jpg';
document.IsPublic = true;
insert document;
// ImageId = '06990000001HnuB';
b = document.Body;
//ConnectApi.ChatterUsers newPhoto = new ConnectApi.ChatterUsers();
} catch (DMLException e) {
ApexPages.addMessage(new ApexPages.message(ApexPages.severity.ERROR, 'Error uploading file'));
return null;
} finally {
document.body = null; // clears the viewstate
document = new Document();
}
ApexPages.addMessage(new ApexPages.message(ApexPages.severity.INFO, 'File uploaded successfully : ' + b));
String communityId = null;
String userId = UserInfo.getUserId();
//ID fileId = ImageId;
// Set photo
ConnectApi.Photo photo = ConnectApi.ChatterUsers.setPhoto(communityId, userId, new ConnectApi.BinaryInput(b, 'image/jpg', 'userImage.jpg'));
return null;
}
I was getting the same error, there isn't too much detail so I'm not too sure about your problem. I solved the problem using the code below. Modify as needed.
public static Boolean updateUserProfilePic(String userProfilePicString, String userId, String fileType){
Boolean updateSuccessful = true;
System.debug('-------------' + userProfilePicString.length());
try{
Blob blobImage = EncodingUtil.base64Decode(userProfilePicString);
ConnectApi.BinaryInput fileUpload = new ConnectApi.BinaryInput(blobImage, 'image/jpg', 'userImage.jpg');
ConnectApi.Photo photoProfile = ConnectApi.UserProfiles.setPhoto(null, userId, fileUpload);
}
catch(Exception exc){
updateSuccessful = false;
}
return updateSuccessful;
}

Trouble exporting to excel from mvc action

I created a simple action to download some content as excel file:
public FileResult ExportToExcel()
{
string filename = "list.xlsx";
string contentType = "application/vnd.openxmlformats-officedocument.spreadsheetml.sheet";
List<string[]> list = new List<string[]>();
list.Add(new[] { "col1", "col2", "cols3" });
list.Add(new[] { "col4", "col5", "cols6" });
list.Add(new[] { "col7", "col8", "cols9" });
StringWriter sw = new StringWriter();
sw.WriteLine("ID,Date,Description");
foreach (string[] item in list)
{
sw.WriteLine("{0},{1},{2}", item[0], item[1], item[2]);
}
byte[] fileContents = Encoding.UTF8.GetBytes(sw.ToString());
return this.File(fileContents, contentType, filename);
}
I have 2 issues with it:
1. The file is downloaded but I cannot open it and am getting a warning:
Excel cannot open the file ... because the file format or file extension is not valid. Verify that the file has not been corrupted and that the file extension matches the format of the file.
When I use old excel format:
string filename = "List.xls";
string contentType = "application/vnd.ms-excel";
I am able to open the file but after 3 different warnings about file being corrupted etc.
Btw I compared saving and tried to write file as pdf
string filename = "List.pdf";
string contentType = "application/pdf";
And I still couldn't open the file - it said format is not valid etc.
2. The contents appear in the file in the second example however the commas are not recognised as column separators and all data in a row is written as one column.
What separator to use for excel format or how to write data to file to have it in a table excel format?
Ideal solution for me would be just return exported view (strongly typed) but I didn't find out how to do it so far.
--- EDIT: Working solution ---
public FileResult ExportToExcel()
{
string filename = "List.xlsx";
string contentType = "application/vnd.openxmlformats-officedocument.spreadsheetml.sheet";
List<string[]> titles = new List<string[]>() { new[] { "a", "be", "ce" } };
List<string[]> list = new List<string[]>
{
new[] { "col1", "col2", "cols3" },
new[] { "col4", "col5", "cols6" },
new[] { "col7", "col8", "cols9" },
new[] { "col10", "col11", "cols12" }
};
XLWorkbook wb = new XLWorkbook();
XLTables xt = new XLTables();
var ws = wb.Worksheets.Add("List");
ws.Cell(1, 1).InsertData(titles);
ws.Cell(2, 1).InsertData(list);
ws.Columns().AdjustToContents();
var stream = new MemoryStream();
wb.SaveAs(stream);
stream.Seek(0, SeekOrigin.Begin);
wb.Dispose();
return this.File(stream, contentType, filename);
}
The reason why it is not being correctly rendered is because you cannot just return the mime type and expect the framework to figure out the rest.
I would go with a nuget package called closedXML which will allow you to create an excel file in memory and stream it back to the client.
it comes with a full documentation (here) for more information.
Using this package you can do something like
XLWorkbook wb = new XLWorkbook();
XLTables xt = new XLTables();
var ws = wb.Worksheets.Add("Sheet 1");
var firstCell = ws.Cell(1, 1);
var lastCell = ws.Cell(3, list.Count);
var table = ws.Range(firstCell.Address, lastCell.Address).AsTable();
table.Cell(2, 1).InsertData(list);
table.CreateTable();
ws.Columns().AdjustToContents();
using(var stream = new MemoryStream())
{
wb.SaveAs(stream);
stream.Seek(0, SeekOrigin.Begin);
wb.Dispose();
return File(stream , contentType, filename);
}

how to screen scrape and put valid href link that will go to that link

I have scraped a webpage but I want the link to have valid link and will jump to that linkpage when clicked.
ex scraped data: day 1 - Go to my Page - status
I want the Go to my page to jump to whatever link is in its href.
ex. actual html I got
<td>Go to my Page</td>
I need it to be like this:
<td>Go to my Page</td>
here's my code for scraping:
public string ScreenScrape()
{
string url = "http://somewebsite.com/tab/form/index.php";
string strResult = "";
WebResponse objResponse;
WebRequest objRequest = System.Net.HttpWebRequest.Create(url);
objResponse = objRequest.GetResponse();
using (StreamReader sr = new StreamReader(objResponse.GetResponseStream()))
{
strResult = sr.ReadToEnd();
// Close and clean up the StreamReader
sr.Close();
}
var webGet = new HtmlAgilityPack.HtmlWeb();
var doc = webGet.Load(url);
foreach (HtmlNode link in doc.DocumentNode.SelectNodes("//a[#href]"))
{
HtmlAttribute att = link.Attributes["href"];
att.Value = "http://somewebsite.com/tab/form/"+att.Value;
}
return strResult;
}
Here's my attempt to change the link and remove the javascript string but could not figure out how to get to the right index . Also, once I am able to change that, how do I replace each href in the strResult(above) to the new href?
foreach (HtmlNode link in doc.DocumentNode.SelectNodes("//a[#href]"))
{
HtmlAttribute att = link.Attributes["href"];
att.Value = "http://somewebsite.com/tab/form/" + ....
}
Can anyone pls help me? thanks
nevermind I got it, but I know the html url parsing is not the best way (if you have suggestion on how to parse it better, pls do so). right now, the only goal is to change the href link so here it goes.
public string ScreenScrape()
{
string url = "http://somewebsite.com/tab/form/index.php";
string strResult = "";
WebResponse objResponse;
WebRequest objRequest = System.Net.HttpWebRequest.Create(url);
objResponse = objRequest.GetResponse();
using (StreamReader sr = new StreamReader(objResponse.GetResponseStream()))
{
strResult = sr.ReadToEnd();
// Close and clean up the StreamReader
sr.Close();
}
var webGet = new HtmlAgilityPack.HtmlWeb();
var doc = webGet.Load(url);
foreach (HtmlNode link in doc.DocumentNode.SelectNodes("//a[#href]"))
{
string removeString ="javascript:jsFormAuth('";
string removeEnd = "');";
HtmlAttribute att = link.Attributes["href"];
String strUrl = HttpContext.Current.Request.Url.AbsoluteUri.Replace(att.XPath, "(");
string sub1 = att.Value.Replace(removeString,"");
string sub2 = sub1.Replace(removeEnd,"");
att.Value = "http://somewebsite.com/tab/form/" + sub2;
}
return doc.DocumentNode.InnerHtml;
}

Get all tweets with specific hashtag

I've been experimenting with the Twitter API because I want to display a few lists of tweets on a special page.
Among those lists is a list with all tweets containing a specific hashtag (e.g. #test)
However I cannot find how to get that list in either XML or JSON (preferably the latter), does anyone know how? It is also fine if it can be done in TweetSharp
You can simply fetch http://search.twitter.com/search.json?q=%23test to get a list of tweets containing #test in JSON, where %23test is #test URL encoded.
I'm not familiar with TweetSharp, but I guess there must be a search command that you can use to search for #test, and then transform the resulting tweets into JSON yourself.
First install TweetSharp using github
https://github.com/danielcrenna/tweetsharp
Here is the code to do a search
TwitterService service = new TwitterService();
var tweets = service.Search("#Test", 100);
List<TwitterSearchStatus> resultList = new List<TwitterSearchStatus>(tweets.Statuses);
If you have more then one page results you can setup a loop and call each page
service.Search("#Test", i += 1, 100);
It seems like there is a change in the API since last few months. Here is the updated code:
TwitterSearchResult res = twitter.Search(new SearchOptions { Q = "xbox" });
IEnumerable<TwitterStatus> status = res.Statuses;
u access with this url for your tweet searchs. But u have to use OAuth protocols.
https://api.twitter.com/1.1/search/tweets.json?q=%40twitterapi
I struggled with the same problem. Here is my vague solution . Enjoy Programming.
It will get out of the function whenever your required number of tweets are acquired/fetched.
string maxid = "1000000000000"; // dummy value
int tweetcount = 0;
if (maxid != null)
{
var tweets_search = twitterService.Search(new SearchOptions { Q = keyword, Count = Convert.ToInt32(count) });
List<TwitterStatus> resultList = new List<TwitterStatus>(tweets_search.Statuses);
maxid = resultList.Last().IdStr;
foreach (var tweet in tweets_search.Statuses)
{
try
{
ResultSearch.Add(new KeyValuePair<String, String>(tweet.Id.ToString(), tweet.Text));
tweetcount++;
}
catch { }
}
while (maxid != null && tweetcount < Convert.ToInt32(count))
{
maxid = resultList.Last().IdStr;
tweets_search = twitterService.Search(new SearchOptions { Q = keyword, Count = Convert.ToInt32(count), MaxId = Convert.ToInt64(maxid) });
resultList = new List<TwitterStatus>(tweets_search.Statuses);
foreach (var tweet in tweets_search.Statuses)
{
try
{
ResultSearch.Add(new KeyValuePair<String, String>(tweet.Id.ToString(), tweet.Text));
tweetcount++;
}
catch { }
}
}

Resources