On many sites I saw printed out my current city where I am (eg "Hello to Berlin."). How they do that? What everything is needed for that?
I guess the main part is here javascript, but what everything I need for implementing something like this to my own app? (or is there some gem for Rails?)
Also, I would like to ask for one thing yet - I am interesting in the list of states (usually in select box), where user select his state (let's say Germany), according to the state value are in another select displayed all regions in Germany and after choosing a region are displayed respective cities in the selected region.
Is possible anywhere to obtain this huge database of states/cities/regions? Would be interesting to have something similar in our app, but I don't know, where those lists get...
You need a browser which supports the geolocation api to obtain the location of the user (however, you need the user's consent (an example here) to do so (most newer browsers support that feature, including IE9+ and most mobile OS'es browsers, including Windows Phone 7.5+).
all you have to do then is use JavaScript to obtain the location:
if (window.navigator.geolocation) {
var failure, success;
success = function(position) {
console.log(position);
};
failure = function(message) {
alert('Cannot retrieve location!');
};
navigator.geolocation.getCurrentPosition(success, failure, {
maximumAge: Infinity,
timeout: 5000
});
}
The positionobject will hold latitude and longitude of the user's position (however this can be highly inaccurate in less densely populated areas on desktop browsers, as they do not have a GPS device built in). To explain further: Here in Leipzig in get an accuracy of about 300 meters on a desktop computer - i get an accuracy of about 30 meters with my cell phone's GPS device.
You can then go on and use the coordinates with the Google Maps API (see here for reverse geocoding) to lookup the location of the user. There are a few gems for Rails, if you want. I never felt the need to use them, but some people seem to like them.
As for a list of countries/cities, we used the data obtainable from Geonames once in a project, but we needed to convert it for our needs first.
Internet Service Providers buy up big chunks of IP addresses, so what you're most likely seeing is a backtrace your IP to a known ISP. They have a database with ISP's and their location in the world, so they can try to see where you're from. You could try to use a site like http://www.ipaddresslocation.org/ to do your work. If you look around, there is bound to be a site that lets you enter an IP and get a location, so you just send a POST request to that site with your visitor's IP and scrape the location from the response.
Alternatively you could try to look for an ISP database that has location and what chunks of the IP range they have been allocated. You could probably find one for money, but a free one might be harder to find.
Alternatively, check out this free database http://www.maxmind.com/app/geolite
I've found getCurrentPosition() to often be inaccurate since it doesn't spend a lot of time waiting on the GPS to acquire a good accuracy. I wrote a small piece of JavaScript that mimics getCurrentPosition() but actually uses watch position and monitors the results coming back until they are better accuracy.
Here's how it looks:
navigator.geolocation.getAccurateCurrentPosition(onSuccess, onError, {desiredAccuracy:20, maxWait:15000});
Code is here - https://github.com/gwilson/getAccurateCurrentPosition
Correc syntax would be :
navigator.geolocation.getCurrentPosition(successCallBack, failureCallBack);
Use :
navigator.geolocation.getCurrentPosition(
function(position){
var latitude = position.coords.latitude;
var longitude = position.coords.longitude;
console.log("Latitude : "+latitude+" Longitude : "+longitude);
},
function(){
alert("Geo Location not supported");
}
);
If you prefer to use ES6 and promises here is another version
function getPositionPromised() {
function successCb(cb) {
return position => cb(position);
}
function errorCb(cb) {
return () => cb('Could not retrieve geolocation');
}
return new Promise((resolve, reject) => {
if (window.navigator.geolocation) {
navigator.geolocation.getCurrentPosition(successCb(resolve), errorCb(reject));
} else {
return reject('No geolocation support');
}
})
}
And you can use it like this:
getPositionPromised()
.then(position => {/*do something with position*/})
.catch(() => {/*something went wrong*/})
Here is an another api to find out the location in PHP,
http://ipinfodb.com/ip_location_api.php
I have been using geoip.maxmind.com for quite a while and it works 100%. It can be accessed via HTTP requests.
Related
I just want to fetch all my liked videos ~25k items. as far as my research goes this is not possible via the YouTube v3 API.
I have already found multiple issues (issue, issue) on the same problem, though some claim to have fixed it, but it only works for them as they don't have < 5000 items in their liked video list.
playlistItems list API endpoint with playlist id set to "liked videos" (LL) has a limit of 5000.
videos list API endpoint has a limit of 1000.
Unfortunately those endpoints don't provide me with parameters that I could use to paginate the requests myself (e.g. give me all the liked videos between date x and y), so I'm forced to take the provided order (which I can't get past 5k entries).
Is there any possibility I can fetch all my likes via the API?
more thoughts to the reply from #Yarin_007
if there are deleted videos in the timeline they appear as "Liked https://...url" , the script doesnt like that format and fails as the underlying elements dont have the same structure as existing videos
can be easily fixed with a try catch
function collector(all_cards) {
var liked_videos = {};
all_cards.forEach(card => {
try {
// ignore Dislikes
if (card.innerText.split("\n")[1].startsWith("Liked")) {
....
}
}
catch {
console.log("error, prolly deleted video")
}
})
return liked_videos;
}
to scroll down to the bottom of the page ive used this simple script, no need to spin up something big
var millisecondsToWait = 1000;
setInterval(function() {
window.scrollTo(0, document.body.scrollHeight);
console.log("scrolling")
}, millisecondsToWait);
when more ppl want to retrive this kind of data, one could think about building a proper script that is more convenient to use. If you check the network requests you can find the desired data in the response of requests called batchexecute. One could copy the authentification of one of them provide them to a script that queries those endpoints and prepares the data like the other script i currently manually inject.
Hmm. perhaps Google Takeout?
I have verified the youtube data contains a csv called "liked videos.csv". The header is Video Id,Time Added, and the rows are
dQw4w9WgXcQ,2022-12-18 23:42:19 UTC
prvXCuEA1lw,2022-12-24 13:22:13 UTC
for example.
So you would need to retrieve video metadata per video ID. Not too bad though.
Note: the export could take a while, especially with 25k videos. (select only YouTube data)
I also had an idea that involves scraping the actual liked videos page (which would save you 25k HTTP Requests). But I'm unsure if it breaks with more than 5000 songs. (also, emulating the POST requests on that page may prove quite difficult, albeit not impossible. (they fetch /browse?key=..., and have some kind of obfuscated / encrypted base64 strings in the request-body, among other parameters)
EDIT:
Look. There's probably a normal way to get a complete dump of all you google data. (i mean, other than takeout. Email them? idk.)
anyway, the following is the other idea...
Follow this deep link to your liked videos history.
Scroll to the bottom... maybe with selenium, maybe with autoit, maybe put something on the "end" key of your keyboard until you reach your first liked video.
Hit f12 and run this in the developer console
// https://www.youtube.com/watch?v=eZPXmCIQW5M
// https://myactivity.google.com/page?utm_source=my-activity&hl=en&page=youtube_likes
// go over all "cards" in the activity webpage. (after scrolling down to the absolute bottom of it)
// create a dictionary - the key is the Video ID, the value is a list of the video's properties
function collector(all_cards) {
var liked_videos = {};
all_cards.forEach(card => {
// ignore Dislikes
if (card.innerText.split("\n")[1].startsWith("Liked")) {
// horrible parsing. your mileage may vary. I Tried to avoid using any gibberish class names.
let a_links = card.querySelectorAll("a")
let details = a_links[0];
let url = details.href.split("?v=")[1]
let video_length = a_links[3].innerText;
let time = a_links[2].parentElement.innerText.split(" • ")[0];
let title = details.innerText;
let date = card.closest("[data-date]").getAttribute("data-date")
liked_videos[url] = [title,video_length, date, time];
// console.log(title, video_length, date, time, url);
}
})
return liked_videos;
}
// https://stackoverflow.com/questions/57709550/how-to-download-text-from-javascript-variable-on-all-browsers
function download(filename, text, type = "text/plain") {
// Create an invisible A element
const a = document.createElement("a");
a.style.display = "none";
document.body.appendChild(a);
// Set the HREF to a Blob representation of the data to be downloaded
a.href = window.URL.createObjectURL(
new Blob([text], { type })
);
// Use download attribute to set set desired file name
a.setAttribute("download", filename);
// Trigger the download by simulating click
a.click();
// Cleanup
window.URL.revokeObjectURL(a.href);
document.body.removeChild(a);
}
function main() {
// gather relevant elements
var all_cards = document.querySelectorAll("div[aria-label='Card showing an activity from YouTube']")
var liked_videos = collector(all_cards)
// download json
download("liked_videos.json", JSON.stringify(liked_videos))
}
main()
Basically it gathers all the liked videos' details and creates a key: video_ID - Value: [title,video_length, date, time] object for each liked video.
It then automatically downloads the json as a file.
I have an iPhone app, where I want to show how many people are currently viewing an item as such:
I'm doing that by running this transaction when people enter a view (Rubymotion code below, but functions exactly like the Firebase iOS SDK):
listing_reference[:listings][self.id][:viewing_amount].transaction do |data|
data.value = data.value.to_i + 1
FTransactionResult.successWithValue(data)
end
And when they exit the view:
listing_reference[:listings][self.id][:viewing_amount].transaction do |data|
data.value = data.value.to_i + -
FTransactionResult.successWithValue(data)
end
It works fine most of the time, but sometimes things go wrong. The app crashes, people loose connectivity or similar things.
I've been looking at "onDisconnect" to solve this - https://firebase.google.com/docs/reference/ios/firebasedatabase/interface_f_i_r_database_reference#method-detail - but from what I can see, there's no "inDisconnectRunTransaction".
How can I make sure that the viewing amount on the listing gets decremented no matter what?
A Firebase Database transaction runs as a compare-and-set operation: given the current value of a node, your code specifies the new value. This requires at least one round-trip between the client and server, which means that it is inherently unsuitable for onDisconnect() operations.
The onDisconnect() handler is instead a simple set() operation: you specify when you attach the handler, what write operation you want to happen when the servers detects that the client has disconnected (either cleanly or as in your problem case involuntarily).
The solution is (as is often the case with NoSQL databases) to use a data model that deals with the situation gracefully. In your case it seems most natural to not store the count of viewers, but instead the uid of each viewer:
itemViewers
$itemId
uid_1: true
uid_2: true
uid_3: true
Now you can get the number of viewers with a simple value listener:
ref.child('itemViewers').child(itemId).on('value', function(snapshot) {
console.log(snapshot.numChildren());
});
And use the following onDisconnect() to clean up:
ref.child('itemViewers').child(itemId).child(authData.uid).remove();
Both code snippets are in JavaScript syntax, because I only noticed you're using Swift after typing them.
I want to examine http requests in an extension for firefox. To begin figuring out how to do what I want to do I figured I'd just log everything and see what comes up:
webRequest.onResponseStarted.addListener(
(stuff) => {console.log(stuff);},
{urls: [/^.*$/]}
);
The domain is insignificant, and I know the regex works, verified in the console. When running this code I get no logging. When I take out the filter parameter I get every request:
webRequest.onResponseStarted.addListener(
(stuff) => {console.log(stuff);}
);
Cool, I'm probably doing something wrong, but I can't see what.
Another approach is to manually filter on my own:
var webRequest = Components.utils.import("resource://gre/modules/WebRequest.jsm", {});
var makeRequest = function(type) {
webRequest[type].addListener(
(stuff) => {
console.log(!stuff.url.match(/google.com.*/));
if(!stuff.url.match(/google.com.*/))
return;
console.log(type);
console.log(stuff);
}
);
}
makeRequest("onBeforeRequest");
makeRequest("onBeforeSentHeaders");
makeRequest("onSendHeaders");
makeRequest("onHeadersReceived");
makeRequest("onResponseStarted");
makeRequest("onCompleted");
With the console.log above the if, I can see the regex returning true when I want it to and the code making it past the if. When I remove the console.log above the if the if no longer gets executed.
My question is then, how do I get the filtering parameter to work or if that is indeed broken, how can I get the code past the if to be executed? Obviously, this is a fire hose, and to begin searching for a solution I will need to reduce the data.
Thanks
urls must be a string or an array of match patterns. Regular expressions are not supported.
WebRequest.jsm uses resource://gre/modules/MatchPattern.jsm. Someone might get confused with the util/match-pattern add-on sdk api, which does support regular expressions.
I am looking to write a small firefox add-on that detects when files that were downloaded are (or have been) deleted locally and removes the corresponding entry in the firefox download list.
Can anybody point me to the relevant api to manipulate the download list? I cannot seem to find it.
The relevant API is PlacesUtils which abstracts the complexity of the Places database.
If your code runs in the context of a chrome window then you get a PlacesUtils glabal variable for free. Otherwise (bootstrapped, Add-on SDK, whatever) you have to import PlacesUtils.jsm.
Cu.import("resource://gre/modules/PlacesUtils.jsm");
As far as Places is concerned, downloaded files are nothing more than a special kind of visited pages, annotated accordingly. It's a matter of just one line of code to get an array of all downloaded files.
var results = PlacesUtils.annotations.getAnnotationsWithName("downloads/destinationFileURI");
Since we asked for the destinationFileURI annotation, each element of the resultarray holds the download location in the annotationValue property as a file: URI spec string.
With that you can check if the file actually exists
function getFileFromURIspec(fileurispec){
// if Services is not available in your context Cu.import("resource://gre/modules/Services.jsm");
var filehandler = Services.io.getProtocolHandler("file").QueryInterface(Ci.nsIFileProtocolHandler);
try{
return filehandler.getFileFromURLSpec(fileurispec);
}
catch(e){
return null;
}
}
getFileFromURIspec will return an instance of nsIFile, or null if the spec is invalid which shouldn't happen in this case but a sanity check never hurts. With that you can call the exists() method and if it returns false then the associated page entry in Places is eligible for removal. We can tell which is that page by its uri, which conveniently is also a property of each element of the results.
PlacesUtils.bhistory.removePage(result.uri);
To sum it up
var results = PlacesUtils.annotations.getAnnotationsWithName("downloads/destinationFileURI");
results.forEach(function(result){
var file = getFileFromURIspec(result.annotationValue);
if(!file){
// I don't know how you should treat this edge case
// ask the user, just log, remove, some combination?
}
else if(!file.exists()){
PlacesUtils.bhistory.removePage(result.uri);
}
});
I just got some crazy ideas for analyzing the Twitter social graph (i.e., representing follow-relations as the edges of a graph). Interestingly, the Twitter API provides methods for creating the graph. It is possible to read out a static snapshot of the social graph, whereas Twitter is a very dynamic network. It would be great if one could dynamically update the graph. So my question is: Is there any way to get notified by Twitter when anyone starts or stops to follow anyone?
I believe that the documentation you linked to would definitely mention that.
I'm quite certain that you need to do your own follower-list checking, and compare results on a regular basis.
I do this if someone follows me or not and how many followers they have and i generate this chart
public function existsFriendship($username,$friend)
{
try
{
if ($this->twitter->existsFriendship($username, $friend))
return true;
}
catch(Exception $e)
{
$this->debug($e->getMessage());
}
}
for the chart generation i use pchart.
in smarty template the code looks like this;
include("pChart/pData.class");
include("pChart/pChart.class"); ![alt text][1]
// Initialise the graph
$Test = new pChart(700,230);
$Test->setFontProperties("Fonts/tahoma.ttf",13);
$Test->setGraphArea(40,30,680,200);
$Test->drawGraphArea(252,252,252,TRUE);
$Test->drawScale($DataSet->GetData(),$DataSet->GetDataDescription(),SCALE_NORMAL,150,150,150,TRUE,0,2);
$Test->drawGrid(4,TRUE,230,230,230,70);
// Draw the line graph
$Test->drawLineGraph($DataSet->GetData(),$DataSet->GetDataDescription());
$Test->drawPlotGraph($DataSet->GetData(),$DataSet->GetDataDescription(),3,2,255,255,255);
// Finish the graph
$Test->setFontProperties("Fonts/tahoma.ttf",12);
$Test->drawLegend(45,35,$DataSet->GetDataDescription(),255,255,255);
$Test->setFontProperties("Fonts/tahoma.ttf",12);
$Test->drawTitle(60,22,"Twitter Graph",50,50,50,585);
$example = $Test->Render("templates/example1.png");
$smarty->assign("example",$example);
$smarty->display('index.tpl');
finaly the result
alt text http://img691.imageshack.us/img691/6749/example1k.png