When executing a twitter query i get a 403 error, the error message is below, however my other queries work perfectly and are executed prior to this one, can anyone spot what may be wrong here:
TWITTER EXCEPTION: TwitterException{exceptionCode=[f3acd3ed-00581fa3], statusCode=403, retryAfter=0, rateLimitStatus=null, version=2.1.5-SNAPSHOT(build: d372a51b9b419cbd73d416474f4a855f3e889507)}
this occurs when i execute a search from my app, im not overdoing the limits as i can execute my other searches perfectly its just this one, any help would be appreciated, the code is listed below. im using a combination of Twitter4j and Processing with controlP5 to handle the input like the search.
void setup(){
...
cp5.addTextfield("SEARCH")
.setPosition(30,20)
.setSize(100,20)
.setFocus(true)
.setColor(color(255,0,0))
.setGroup(g2)
;
}
public void SEARCH(String theText) {
qm.srch = true;
qm.theText = theText;
qm.userSearch();
qm.srch = false;
// automatically receives results from controller input
println("a textfield event for controller 'input' : "+theText);
}
void userSearch() {
try {
if (srch) {
ConfigurationBuilder cb9 = new ConfigurationBuilder();
cb9.setOAuthConsumerKey("XXXXXXXXXXXXXXXXXXXXXX");
cb9.setOAuthConsumerSecret("XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXx");
cb9.setOAuthAccessToken("XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX");
cb9.setOAuthAccessTokenSecret("XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX");
println("Connected");
Twitter twitter = new TwitterFactory(cb9.build()).getInstance();
Query srchh = new Query(theText2);
srchh.setRpp(5);
QueryResult srchhRes = twitter.search(srchh);
ArrayList srchhTwe = (ArrayList) srchhRes.getTweets();
for (int i = 0; i < srchhTwe.size(); i++) {
Tweet t = (Tweet) srchhTwe.get(i);
String user = t.getFromUser();
GeoLocation l = t.getGeoLocation();
String locNam = t.getLocation();
String msg = t.getText();
wholeTweetsL.add(msg);
println("\nMessage: " + msg);
println("\nLocation: " + locNam);
}
}
}
catch(TwitterException e) {
println("TWITTER EXCEPTION: " + e);
}
}
From twitter at https://dev.twitter.com/docs/error-codes-responses
403 Forbidden The request is understood, but it has been refused or access is not allowed. An accompanying error message will explain why. This code is used when requests are being denied due to update limits.
Related
When writing a message to the Azure Service Bus (using Microsoft.Azure.ServiceBus standard library, not the .Net Framework version) it works fine. However, when switching networks to a network that blocks that traffic and running it again I would expect an error being raised by SendAsync yet no error is thrown, therefor the function considers the send successful even though it is not.
Am I missing some logic to make sure that errors do get raised and trapped, it seems to be inline with all the examples I have seen.
I have tried this possible solution ..
Trouble catching exception on Azure Service Bus SendAsync method
.ContinueWith(t =>
{
Console.WriteLine(t.Status + "," + t.IsFaulted + "," + t.Exception.InnerException);
}, TaskContinuationOptions.OnlyOnFaulted);
.. and at no point does ContinueWith get hit.
[HttpPost]
[Consumes("application/json")]
[Produces("application/json")]
public ActionResult<Boolean> Post(Contract<T> contract)
{
Task.Run(() => SendMessage(contract));
// Other stuff
}
private async Task<ActionResult<Boolean>> SendMessage(Contract<T> contract)
{
JObject json = JObject.FromObject(contract);
Message message = new Message();
message.MessageId = Guid.NewGuid().ToString();
message.ContentType = ObjectType;
message.PartitionKey = ObjectType;
message.Body = Encoding.UTF8.GetBytes(JsonConvert.SerializeObject(contract));
foreach (KeyValuePair<String, String> route in DataRouting)
{
JToken jToken = json.SelectToken(route.Value);
if (jToken != null)
{
message.UserProperties[route.Key] = jToken.Value<String>();
}
else
{
String routeError = $"Could not find routing information in request for: {route.Key} in {route.Value}";
Logger.LogError(routeError);
return new UnprocessableEntityObjectResult(routeError);
}
}
// Send the message
try
{
await topicClient.SendAsync(message);
}
catch(Exception ex)
{
return new UnprocessableEntityObjectResult($"'Could not transmit message to service bus - {ex.Message}'");
}
return new OkObjectResult(true);
}
I expect that the error trap would be hit if the SendAsync fails to send the message. However it essentially fire and forgets, the message send is blocked by the firewall but is never reported to the caller by throwing an error.
Ok, found the answer, but I will leave this out there in case anyone else does this to themselves. It was down to my general muppetry when putting the MVC Controller together. Set async on the Post action and configure the await on the send. Obvious really but I missed it.
public virtual async Task<ActionResult<Boolean>> Post(Contract<T> contract){}
...
// Send the message
try
{
await topicClient.SendAsync(message).ConfigureAwait(false);
return new OkObjectResult(true); // Success if we got here
}
catch(Exception ex)
{
return new UnprocessableEntityObjectResult($"'Could not transmit message to service bus - {ex.Message}'");
}
I created a custom attribute, inheriting from HandleErrorAttribute:
public class CustomHandleErrorAttribute : HandleErrorAttribute
{
public override void OnException(ExceptionContext filterContext)
{
try
{
Utility.LogAndNotifyOfError(filterContext.Exception, null, true);
}
catch(Exception ex)
{
filterContext.Exception = ex;
}
}
}
, and then registered with:
filters.Add(new CustomHandleErrorAttribute());
This has always worked as intended. However a common problem with my log method is that it uses a custom event log source when writing to the event log, which the app pool account typically doesn't have the permissions to create. Creating the event log source is a simple powershell script, however I wanted to actually include that tidbit in the error:
try
{
log.WriteEntry(error, EventLogEntryType.Error);
}
catch(SecurityException ex1)
{
throw new ErrorHandlerException($"The event log could not be written to due to a SecurityExcption. The likely issue is that the '{eventLogSource}' does not already exist. Please run the following powershell command:\r\n"
+ $"New - EventLog - LogName Application - Source {eventLogSource}", ex1);
}
The problem is that the catch in the OnException is never hit. When debugging, the custom error I throw from LogAndNotifyOfError instead triggers a second call to OnException, and the detail of my ErrorHandlerException is never seen. I want the asp.net error page that comes up to be with my custom error detail rather than the SecurityException that was originally raised.
You can even see the surrounding try in the displayed error:
Edit: Entire log method listed:
public static void LogAndNotifyOfError(Exception ex, String extraInfo, Boolean sendEmail)
{
//if the error handler itself faulted...
if (ex is ErrorHandlerException)
return;
string eventLogName = "Application";
string eventLogSource = "MySourceName";
String error = ex.ToString();
if (error.Length > 28000)
error.Substring(0, 28000);//event log is limited to 32k
error += "\r\n\r\nAdditional Information: \r\n"
+ "Machine Name: " + Environment.MachineName + "\r\n"
+ "Logged in user:" + App.CurrentSecurityContext.CurrentUser?.UserId + "\r\n"
+ extraInfo + "\r\n";
EventLog log = new EventLog(eventLogName);
log.Source = eventLogSource;
try
{
log.WriteEntry(error, EventLogEntryType.Error);
}
catch(SecurityException ex1)
{//this doesn't work - for some reason, OnError still reports the original error.
throw new ErrorHandlerException($"The event log could not be written to due to a SecurityExcption. The likely issue is that the '{eventLogSource}' does not already exist. Please run the following powershell command:\r\n"
+ $"New - EventLog - LogName Application - Source {eventLogSource}", ex1);
}
//if the email-to field has been set...
if (!String.IsNullOrEmpty(App.Config.General.ErrorHandlerSendToAddresses) && sendEmail)
{
//...then send the email
MailMessage email = new MailMessage();
email.To.Add(App.Config.General.ErrorHandlerSendToAddresses);
email.IsBodyHtml = false;
email.Subject = String.Format("Error in {0}", eventLogSource);
email.Body = email.Subject + "\r\n\r\n"
//+ "Note: This error may be occuring continuously, but this email is only sent once per hour, per url, in order to avoid filling your mailbox. Please check the event log for reoccurances and variations of this error.\r\n\r\n"
+ "The error description is as follows: \r\n\r\n"
+ error + "\r\n\r\n";
SmtpClient smtp = new SmtpClient();
smtp.Send(email);
}
}
I figured it out (sort of). It would appear that when the newly throw exception has an inner exception, it is only displaying that inner exception. It does not matter what the type is on the outer or inner exception.
There seems to be many different tutorials and examples out there to allow for tweets to be pulled into Processing from one specific user.
And yet I'm still having problems getting any code to work. I have managed to get tweets by searching with hashtags, so the twitter4j library (latest) is working within Processing (also latest software). I still a complete coding novice...
I've found the following code to do exactly what I need, but unfortunately it isn't complete, where I'm assuming you need to declare your Consumer Keys and Access tokens... But I've no idea how to do this with this code. Is this something that someone is able to provide and explain?
Essentially, I need the full sketch... Any help would be greatly appreciated!
Code from elsewhere:
final Twitter twitter = new TwitterFactory().getInstance();
twitter.setOAuthConsumer(CONSUMER_KEY, CONSUMER_KEY_SECRET);
AccessToken accessToken = new AccessToken(TWITTER_TOKEN,
TWITTER_TOKEN_SECRET);
twitter.setOAuthAccessToken(accessToken);
try {
Status status = twitter.showStatus(Long.parseLong(tweetID));
if (status == null) { //
// don't know if needed - T4J docs are very bad
} else {
System.out.println("#" + status.getUser().getScreenName()
+ " - " + status.getText());
}
} catch (TwitterException e) {
System.err.print("Failed to search tweets: " + e.getMessage());
// e.printStackTrace();
// DON'T KNOW IF THIS IS THROWN WHEN ID IS INVALID
}
EDIT: This is how I've added the consumer/access keys - is this right?
twitter.setOAuthConsumer("MyConsumerKey", "MyConsumerSecret");
AccessToken accessToken = new AccessToken("MyAccessToken", "MyAccessTokenSecret");
twitter.setOAuthAccessToken(accessToken);
EDIT2: This is what I have now to get the User's tweets. But produced the error: 'cannot convert from ResponseList to Status'
String user="USER ID";
final Twitter twitter = new TwitterFactory().getInstance();
twitter.setOAuthConsumer("MY CONSUMER KEY", "MY CONSUMER KEY SECRET");
AccessToken accessToken = new AccessToken("MY TWITTER TOKEN", "MY TWITTER TOKEN SECRET");
twitter.setOAuthAccessToken(accessToken);
try {
Status status = twitter.getUserTimeline(user);
if (status == null) { //
// don't know if needed - T4J docs are very bad
} else {
System.out.println("#" + status.getUser().getScreenName()
+ " - " + status.getText());
}
} catch (TwitterException e) {
System.err.print("Failed to search tweets: " + e.getMessage());
// e.printStackTrace();
// DON'T KNOW IF THIS IS THROWN WHEN ID IS INVALID
}
you need first to create a twitter app, to do so go to https://apps.twitter.com/, from there you can get all the infos needed to get your credentials.
in your code just replace the 'CONSUMER_KEY','CONSUMER_KEY_SECRET','Access Token','Access Token Secret' by the credentials.
ConfigurationBuilder cb = new ConfigurationBuilder();
cb.setDebugEnabled(true)
.setOAuthConsumerKey("XXXXX")
.setOAuthConsumerSecret("XXXXXX")
.setOAuthAccessToken("XXXX-XXXXX")
.setOAuthAccessTokenSecret("XXXXXXX");
hope this help!
I have a list of tweets with information about the user who tweeted them that I am using for an undergrad research project. To build a social network graph of these tweets I need to grab their friend and follower lists. I have tried using the GET Follower IDs call through the twitter4j platform. My authentication is Oauth with Read, write, and direct messages. I get a 400 response code with no further error code. I also get the following exception code
exceptionCode=[92c30ec6-19bed99c 70a5018f-1e1c55ac 70a5018f-1e1c55aa], statusCode=-1, message=null, code=-1, retryAfter=-1, rateLimitStatus=null, version=3.0.3}
This tells me that I'm not authenticated to make this request which from what I have read is because the people are not followers of mine. Is there a way I can request this information without having this relationship with the user?
here is my code
public static void main (String[] args){
ConfigurationBuilder cb = new ConfigurationBuilder();
cb.setDebugEnabled(true)
.setOAuthConsumerKey("something")
.setOAuthConsumerSecret("something else")
.setOAuthAccessToken("another thing")
.setOAuthAccessTokenSecret("a secret thing")
.setUseSSL(true)
.setUserStreamRepliesAllEnabled(true);
Twitter twitter = new TwitterFactory(cb.build()).getInstance();
long cursor = -1;
IDs ids = null;
String[] users = new String[16717];
BufferedReader br = null;
try {//getting user screen names
String sCurrentLine;
br = new BufferedReader(new FileReader("users.txt"));
int i = 0;
while ((sCurrentLine = br.readLine()) != null) {
users[i]=sCurrentLine;
i++;
}
} catch (IOException e) {
e.printStackTrace();
} finally {
try {
if (br != null)br.close();
} catch (IOException ex) {
ex.printStackTrace();
}
}
for(int i=0;i<users.length;i++){
System.out.println("==================="+users[i]+"===================");
do {
try {
ids = twitter.getFollowersIDs(users[i], cursor);
for (long id : ids.getIDs()) {
System.out.println(id);
User user = twitter.showUser(id);
System.out.println(user.getName());
}
} catch (TwitterException e) {
e.printStackTrace();
}
} while ((cursor = ids.getNextCursor()) != 0);
}
}
One can obtain the list of followers IDs for any public twitter user using this API from twitter. I don't use twitter4j but it should work fine.
Main thing to be conscious of, outside of authentication, is that twitter allows fetching maximum 5000 IDs in one call and rate limits aggressively (15 calls per app/user token) so your application has to be designed and built to honor those considerations/limitations with appropriate tokens/sleeps etc.
For e.g. if you use the application token and a given user has 100K followers, twitter will start returning rate_limit_exceeded errors after fetching 75K (5K * 15) followers IDs.
In an extremely embarrassing turn of events it turns out that the reason the request could not be made is that the usernames I was searching with had an extra space. So I trimmed each name and it works now.
I'm currently trying the Neo4j 2.0.0 M3 and see some strange behaviour. In my unit tests, everything works as expected (using an newImpermanentDatabase) but in the real thing, I do not get results from the graphDatabaseService.findNodesByLabelAndProperty.
Here is the code in question:
ResourceIterator<Node> iterator = graphDB
.findNodesByLabelAndProperty(Labels.User, "EMAIL_ADDRESS", emailAddress)
.iterator();
try {
if (iterator.hasNext()) { // => returns false**
return iterator.next();
}
} finally {
iterator.close();
}
return null;
This returns no results. However, when running the following code, I see my node is there (The MATCH!!!!!!!!! is printed) and I also have an index setup via the schema (although that if I read the API, this seems not necessary but is important for performance):
ResourceIterator<Node> iterator1 = GlobalGraphOperations.at(graphDB).getAllNodesWithLabel(Labels.User).iterator();
while (iterator1.hasNext()) {
Node result = iterator1.next();
UserDao.printoutNode(emailAddress, result);
}
And UserDao.printoutNode
public static void printoutNode(String emailAddress, Node next) {
System.out.print(next);
ResourceIterator<Label> iterator1 = next.getLabels().iterator();
System.out.print("(");
while (iterator1.hasNext()) {
System.out.print(iterator1.next().name());
}
System.out.print("): ");
for(String key : next.getPropertyKeys()) {
System.out.print(key + ": " + next.getProperty(key).toString() + "; ");
if(emailAddress.equals( next.getProperty(key).toString())) {
System.out.print("MATCH!!!!!!!!!");
}
}
System.out.println();
}
I already debugged through the code and what I already found out is that I pass via the InternalAbstractGraphDatabase.map2Nodes to a DelegatingIndexProxy.getDelegate and end up in IndexReader.Empty class which returns the IteratorUtil.EMPTY_ITERATOR thus getting false for iterator.hasNext()
Any idea's what I am doing wrong?
Found it:
I only included neo4j-kernel:2.0.0-M03 in the classpath. The moment I added neo4j-cypher:2.0.0-M03 all was working well.
Hope this answer helps save some time for other users.
#Neo4j: would be nice if an exception would be thrown instead of just returning nothing.
#Ricardo: I wanted to but I was not allowed yet as my reputation wasn't good enough as a new SO user.