I'm not really sure how to frame the question (maybe the reason why I could not find the answer, so please let me know if it's duplicated question) but I'm not able to get the exact address for a website that I want to scrape. The website address is "https://www.nysar.com/find-a-realtor/". And I want to only scrape information in Albany, NY. And when I search by the address with Albany, NY, I get new search results but the URL is still the same....
Is there any way that I can find out the detailed URL address?
Thank you in advance!
That page is dynamically loaded using javascript, so in order to see the source of your search result, you have to go into the Developer tools of your browser and look for it under Network/XHR. There's a lot of information about this, including here. It's not a simple concept so you should study up on it before attempting to scrape this type of sites.
In any event, in this particular case, the data is extracted from here.
Related
I seem to not being lucky searching for an understandable anwer in this forum, so I decided to make my own question. I apoplogyze for any existing post that I could have missed.
Briefly, I want to know statistics from certain pages that I can address by setting a filter accounting the URL. The problem is that I can also found some visits that were made while administering the site (joomla) which show some queries.
I would like to get results from pages under, let's say, /index.php/certain_group/
(e.g.
/index.php/certain_group/this-page,
/index.php/certain/group/another-page)*
but not those like
/index.php/certain/group/another-page?view=form&layout=edit&a_id=89&return=aHR0cCUzQSUyR...bla bla
I have tried lost of combinations in http://www.analyticsmarket.com/freetools/regex-tester
I am being able to find only thos that I do not want, I mean, if I use "/index.php/group/.\?.$"
I get
/index.php/certain/group/another-page?view=form&layout=edit&a_id=89&return=aHR0cCUzQSUyR...bla bla
Any clue?
Thanks in advance
I have used custom url builder to build custom url and submitted to different directories.
URL is
?utm_source=directory&utm_medium=referral&utm_campaign=DirectoryReferrals
Now i am stuck with reporting coz it shows source as "directory" thats what i need it to show but full referral part is showing as not set.
How can i get full url of the referral source? Need help as soon as possible.
Is there any way to get the url of the referral?
I think you may be misunderstanding the usage of the utm_source parameter in the context of custom campaigns. It wouldn't be used to get the source URL that you used when tagging your campaign, but rather it should show the source where the campaign is originating from, for example advertiser, site (eg. "google"), publication (eg. "newsletter", "billboard"). More information on campaigns can be found here: https://support.google.com/analytics/answer/1033863. Also some good info on why you are getting "not set": https://support.google.com/analytics/answer/2820717?hl=en.
If you are trying to see where traffic is coming from, eg. www.google.com, www.facebook.com, then you wouldn't use custom campaigns for that.
Hope this helps.
I agree with Nyuen, you might have misunderstood the concept of UTM tracking code.
I would suggest that if you want to know that from which referral source you got the traffic then you can write it under UTM_SOURCE.
Suppose if you have promoted your post in the Facebook and Google, then in the UTM_SOURCE you can write Facebook there.
Example:
http://www.grazitti.com/resources/webinars/5-must-have-features-for-your-service-cloud-implementation.html?utm_source=Twitter&utm_medium=Webinar_Tweet&utm_term=Service_Cloud&utm_campaign=Social
Hope this Works :)
You always able to find your refferal traffic under > Acquisition > referral even you tagged link with utm.
If it's not help, please upload some screenshot.
Some website like DIGG or NGOOR allow user to input an article or website URL then it will fetch that website title and description, display and images on its page. I searched around internet and also stackoverflow but cannot find any solutions that is the same as these websites. i planned to create this function using Java and Jquery, so i wonder if someone could help to give me some hints on this?
Moreover, in discussions related to this issue, some idea discussed on if we need to load all web page to read meta data tags or just load only necessary information? what's your ideas?
I am sorry if this post is an old question but i could not find any suitable data
Thank you very much.
We have a site that serves up the same content but on country specific domains - so a potential duplicate content issue.
After doing some research, we went with Google's recommendation of using country specific domains instead of www.domain.com/country-directory/
However, when you search from another country, the correct domain does not appear. We have a person in Australia and every time they search google, the .com.au domain doesn't show up.
We have both country domains setup in Google's Webmaster's tools and both have country specific sitemap.xml files which Webmaster tools has no issue with seeing - in fact, there are no errors of any kind (crawl errors etc) as far as Webmaster tools is concerned.
Does anyone know what we might be doing wrong?
Make sure your australian friend is using http://google.com.au. You can conduct the search yourself on their to check if it is working or not, you do not need someone in another country to do so.
Basically I want to know how many people have tweeted a link to a url, but since there are dozens of link shortener out there I don't see any way to do this without having access to all of their url maps. I found a previous question here but it was over a year old and didn't have any new answers.
So #1, does anyone know of a service/API for doing this?
And #2, can anyone think of a way to accomplish this task other than submitting the long url in question to all the popular link shortening sites?
ps- I'm also open to comments about why this is impossible or impractical.
You could perform a Google search (or the equivalent via API) for any pages that link to your page. This is done with the link: keyword. So if you're trying to figure out how many people link to www.example.com (regardless of whether it's through a link shortner URL), then you would just do a Google search for link:www.example.com.
e.g.: http://www.google.com/search?q=link:www.example.com
Note that this will only find pages that have been indexed, so pages that haven't been crawled, or pages that get crawled infrequently, will not show up in the results until a later date (if at all).
Since all sites have different algorithms for shortening the URLs, and these are different sites that most likely do not share their data with each other, how can you hope to find all of them in a single or small number of queries?
All you can do is brute-force it, and even then this might not be any good if a site is content to create a new value for the same long-form URL (especially if you send a different long-form URL that maps to the same place, like http://www.stackoverflow.com/ rather than http://stackoverflow.com/).
In order to really get this to work, there would have to be a site that ALREADY automatically collects all of this information from every site, which the URL shortening sites voluntarily call. And even if you wrote such a site, that doesn't account for the URL-shortening sites already out there who already have data!
In short, I do not see how this is remotely possible, unless I'm wrong about there being such a database somewhere out there.
So months after asking this question I came across a solution to a similar question, that is how to tell how many times a link has been shared on facebook. The solution, via a simple new API call:
http://graph.facebook.com/http://stackoverflow.com
returns the following json data:
{
"id": "http://stackoverflow.com",
"shares": 1627
}