How can I visualise tweets on a map in Processing? - twitter

I'm trying to use the twitter API to search for a keyword and get the location of that particular tweet to then visualise onto a map.
I've successfully created my map using unfolding maps and tilemill, I'm just struggling with the twitter part. Using the twitter 4J library, I've tried the following code but I'm not able to get the location as coordinates. Also, once I've got the coordinates I dont know how to go about visualising them on my map.
import twitter4j.*;
import twitter4j.api.*;
import twitter4j.auth.*;
import twitter4j.conf.*;
import twitter4j.json.*;
import twitter4j.management.*;
import twitter4j.util.*;
import twitter4j.util.function.*;
ConfigurationBuilder cb = new ConfigurationBuilder();
Twitter twitterInstance;
Query queryForTwitter;
void setup() {
cb.setOAuthConsumerKey("**********");
cb.setOAuthConsumerSecret("*******");
cb.setOAuthAccessToken("*********");
cb.setOAuthAccessTokenSecret("*********");
twitterInstance = new TwitterFactory( cb.build()).getInstance();
queryForTwitter = new Query("#nature");
size(640, 440);
background(0);
FetchAndDrawTweets();
}
void FetchAndDrawTweets() {
try {
QueryResult result = twitterInstance.search(queryForTwitter);
ArrayList tweets = (ArrayList) result.getTweets();
for (int i=0; i<tweets.size(); i++) {
Status t = (Status) tweets.get(i);
String user = (t.getUser()).getLocation();
text(user + ":" , 20,15+i*15);
}
}
catch(TwitterException te) {
println("Couldn't connect:" + te);
}
}here
I'm quite new to processing, so please be patient with me as I don't even know if I'm going about this the right way.
Any help would be gratefully appreciated!!

Nice work on separating your problem into smaller sub-problems. It's a good idea to separate the "get tweets and locations" step from the "show stuff on map" step. Good work.
As for getting the location of a tweet, there are two places you should look:
The Twitter4J JavaDocs: This is a list of every class, function, and variable available to you from the Twitter4J library. This should be your first stop. Do you see anything that looks useful in here? Specifically, the Status class has a getGeoLocation() function that looks pretty promising.
The Twitter API: this is the underlying JavaScript API that the Twitter4J library is built on. Check out this documentation for more details on what's going on underneath the Twitter4J library. Specifically, this page says the the geo object is deprecated and that you should use the coordinates field instead.
So the first thing I would try is the getGeoLocation() function. But note that not every tweet will have a location, since users can disable location tracking. Also note that if the underlying JavaScript library no longer provides the geo object, and if the Twitter4J library is using that to populate its getGeoLocation() function, then you won't be able to get at the location through Twitter4J. I haven't tested that at all though.

Related

Need to remove numbers with a javascript code step in Zapier

I am not a developer but have used Google search and trial and error test scenarios with Zapier for the last few days and have given up on figuring this out myself. I need help!
I'm using the Run JavaScript code step in Zapier and provided the following details to Input Data.
It says: What input data should we provide to your code (as strings) via an object set to a variable named inputData?
I'm using "street" with a street address example "1402 Spring Garden Rd"
What is the code to use that regardless of the street address provided all the numbers and first space are removed so that the results is "Spring Garden Rd"
Thank you in advance!
var street = inputData;
var streetNoNumbers = inputData.replace(/[0-9]/g, '');
return streetNoNumbers
The error message I'm getting is
TypeError: inputData.replace is not a function
I've learned that strings are immutable and a new string can be made from manipulating another string but doing this in zapier seems to require a function and creating another var with the calculation generates a ... is not a function.
I've tried to write a function but can't get the output or return to show the proper results either.
I can do the following successfully,
var street = inputData
return street
1402 Spring Garden Road
I want to include the code that manipulates street to produce the following:
Spring Garden Road
David here, from the Zapier Platform team. Great question!
The key understanding you're missing is that inputData is a js object with a street property. Before your code is run, we set it up like so:
const inputData = {street: '1402 Spring Garden Rd'}
Since inputData is an object, it doesn't have a replace method (the error you're seeing). Instead, perform your operation on .street and return that.
Try the following:
// need to return an object, not just a string
return {streetNoNumbers: inputData.replace(/[0-9]/g, '')}
If you want to learn more, I recommend our simple examples: https://zapier.com/help/code/#simple-email-extraction

How do I fire a list var from a DTM direct call?

I am trying to fire values into a list var in Adobe Analytics from a DTM direct call but can't seem to get any values to appear.
In my custom code in the direct call rule I have
cTS = _satellite.getVar('conversionTypeShown');
s.list1 = cTS;
and the Data Element conversionTypeShown is getting information from the digitalData layer on the page (which is updated just before the direct call)
if ((digitalData.searchResults !== undefined) && (digitalData.searchResults !== ""))
{
return digitalData.otherJobsType + digitalData.searchResults;
}
I know that these values are being populated correctly because I am firing an eVar with the same data in it (within the same rule) which is coming through OK into Adobe Analytics. But I am not getting any values for the list var?
Does a direct call not allo me to use custom code in this manner?
Any help would be gratefully received.
Owen.
Many thanks to Owen. I didn't find this hint in the Adobe documentation.
Finally my code looks like this and works.
s.linkTrackVars="list1,list2";
s.list1=_satellite.getVar("FieldsSubmitted");
s.list2=_satellite.getVar("FieldsAborted");

Retrieving data via POST request

I am having trouble obtaining data programmatically from a particular webpage.
http://www.uschess.org/msa/thin2.php allows one to search for US Chess ratings by name and state.
Submitting a POST request, I can get to the equivalent of http://www.uschess.org/msa/thin2.php?memln=nakamura&memfn=hikaru but still requires one to clicking the "Search" button to get useful data. What is the best way to get to that results page?
import urllib.request
import urllib.parse
data = {'memfn':'hikaru', 'memln':'nakamura'}
url = r'http://www.uschess.org/msa/thin2.php'
s = urllib.parse.urlopen(url, bytes(urllib.parse.urlencode(data),'UTF-8'))
s.read()
Thanks!
This one works:
#!/usr/bin/env python
import urllib
data = {'memfn':'hikaru', 'memln':'nakamura', 'mode':'Search'}
url = r'http://www.uschess.org/msa/thin2.php'
s = urllib.urlopen(url, bytes(urllib.urlencode(data)))
print s.read()
Basically you need to submit hidden parameter mode with value Search to imitate the button press.
Note: I rewrote it for python 2.x, sorry, but I didn't have python3 handy.

How to get user's geolocation?

On many sites I saw printed out my current city where I am (eg "Hello to Berlin."). How they do that? What everything is needed for that?
I guess the main part is here javascript, but what everything I need for implementing something like this to my own app? (or is there some gem for Rails?)
Also, I would like to ask for one thing yet - I am interesting in the list of states (usually in select box), where user select his state (let's say Germany), according to the state value are in another select displayed all regions in Germany and after choosing a region are displayed respective cities in the selected region.
Is possible anywhere to obtain this huge database of states/cities/regions? Would be interesting to have something similar in our app, but I don't know, where those lists get...
You need a browser which supports the geolocation api to obtain the location of the user (however, you need the user's consent (an example here) to do so (most newer browsers support that feature, including IE9+ and most mobile OS'es browsers, including Windows Phone 7.5+).
all you have to do then is use JavaScript to obtain the location:
if (window.navigator.geolocation) {
var failure, success;
success = function(position) {
console.log(position);
};
failure = function(message) {
alert('Cannot retrieve location!');
};
navigator.geolocation.getCurrentPosition(success, failure, {
maximumAge: Infinity,
timeout: 5000
});
}
The positionobject will hold latitude and longitude of the user's position (however this can be highly inaccurate in less densely populated areas on desktop browsers, as they do not have a GPS device built in). To explain further: Here in Leipzig in get an accuracy of about 300 meters on a desktop computer - i get an accuracy of about 30 meters with my cell phone's GPS device.
You can then go on and use the coordinates with the Google Maps API (see here for reverse geocoding) to lookup the location of the user. There are a few gems for Rails, if you want. I never felt the need to use them, but some people seem to like them.
As for a list of countries/cities, we used the data obtainable from Geonames once in a project, but we needed to convert it for our needs first.
Internet Service Providers buy up big chunks of IP addresses, so what you're most likely seeing is a backtrace your IP to a known ISP. They have a database with ISP's and their location in the world, so they can try to see where you're from. You could try to use a site like http://www.ipaddresslocation.org/ to do your work. If you look around, there is bound to be a site that lets you enter an IP and get a location, so you just send a POST request to that site with your visitor's IP and scrape the location from the response.
Alternatively you could try to look for an ISP database that has location and what chunks of the IP range they have been allocated. You could probably find one for money, but a free one might be harder to find.
Alternatively, check out this free database http://www.maxmind.com/app/geolite
I've found getCurrentPosition() to often be inaccurate since it doesn't spend a lot of time waiting on the GPS to acquire a good accuracy. I wrote a small piece of JavaScript that mimics getCurrentPosition() but actually uses watch position and monitors the results coming back until they are better accuracy.
Here's how it looks:
navigator.geolocation.getAccurateCurrentPosition(onSuccess, onError, {desiredAccuracy:20, maxWait:15000});
Code is here - https://github.com/gwilson/getAccurateCurrentPosition
Correc syntax would be :
navigator.geolocation.getCurrentPosition(successCallBack, failureCallBack);
Use :
navigator.geolocation.getCurrentPosition(
function(position){
var latitude = position.coords.latitude;
var longitude = position.coords.longitude;
console.log("Latitude : "+latitude+" Longitude : "+longitude);
},
function(){
alert("Geo Location not supported");
}
);
If you prefer to use ES6 and promises here is another version
function getPositionPromised() {
function successCb(cb) {
return position => cb(position);
}
function errorCb(cb) {
return () => cb('Could not retrieve geolocation');
}
return new Promise((resolve, reject) => {
if (window.navigator.geolocation) {
navigator.geolocation.getCurrentPosition(successCb(resolve), errorCb(reject));
} else {
return reject('No geolocation support');
}
})
}
And you can use it like this:
getPositionPromised()
.then(position => {/*do something with position*/})
.catch(() => {/*something went wrong*/})
Here is an another api to find out the location in PHP,
http://ipinfodb.com/ip_location_api.php
I have been using geoip.maxmind.com for quite a while and it works 100%. It can be accessed via HTTP requests.

How to receive updates to the Twitter social graph?

I just got some crazy ideas for analyzing the Twitter social graph (i.e., representing follow-relations as the edges of a graph). Interestingly, the Twitter API provides methods for creating the graph. It is possible to read out a static snapshot of the social graph, whereas Twitter is a very dynamic network. It would be great if one could dynamically update the graph. So my question is: Is there any way to get notified by Twitter when anyone starts or stops to follow anyone?
I believe that the documentation you linked to would definitely mention that.
I'm quite certain that you need to do your own follower-list checking, and compare results on a regular basis.
I do this if someone follows me or not and how many followers they have and i generate this chart
public function existsFriendship($username,$friend)
{
try
{
if ($this->twitter->existsFriendship($username, $friend))
return true;
}
catch(Exception $e)
{
$this->debug($e->getMessage());
}
}
for the chart generation i use pchart.
in smarty template the code looks like this;
include("pChart/pData.class");
include("pChart/pChart.class"); ![alt text][1]
// Initialise the graph
$Test = new pChart(700,230);
$Test->setFontProperties("Fonts/tahoma.ttf",13);
$Test->setGraphArea(40,30,680,200);
$Test->drawGraphArea(252,252,252,TRUE);
$Test->drawScale($DataSet->GetData(),$DataSet->GetDataDescription(),SCALE_NORMAL,150,150,150,TRUE,0,2);
$Test->drawGrid(4,TRUE,230,230,230,70);
// Draw the line graph
$Test->drawLineGraph($DataSet->GetData(),$DataSet->GetDataDescription());
$Test->drawPlotGraph($DataSet->GetData(),$DataSet->GetDataDescription(),3,2,255,255,255);
// Finish the graph
$Test->setFontProperties("Fonts/tahoma.ttf",12);
$Test->drawLegend(45,35,$DataSet->GetDataDescription(),255,255,255);
$Test->setFontProperties("Fonts/tahoma.ttf",12);
$Test->drawTitle(60,22,"Twitter Graph",50,50,50,585);
$example = $Test->Render("templates/example1.png");
$smarty->assign("example",$example);
$smarty->display('index.tpl');
finaly the result
alt text http://img691.imageshack.us/img691/6749/example1k.png

Resources