Just wondering if there's any way to search all web pages which link to some specific url? For example, all web pages containing link to example.com? Thanks
You probably might want to explore the Google Search API which allows you to use Google search results in your programs.
Related
I want to create a custom google search for my site. My requirements are as follows:
I need a text box for search and two radio buttons(one for searching my site and other for searching web).
Depending on the radio button selected the results should be displayed.
I have a separate search results page ('/search').
I know how to create a simple google custom search. But this requirement of providing an option to search the site or web is a little tricky for me.
Please help
I was able to solve this issue.
I first created a search engine in https://cse.google.com/cse/ for my site and configured it to search the web in Basics tab-> Sites to Search option.
With this the search engine would search the entire web.
The code that I placed on the search results page had something this:
"gcse:searchresults-only as_sitesearch='site to search'"
This is a part of the code that google gives you.
Here the parameter 'as_sitesearch' will control which site is to be searched. When the radio button of Searching the web was selected I did not pass any value to 'as_sitesearch' and when the radio button of Searching my site was selected I passed the site url to 'as_sitesearch'.
This worked for me
If you search for Twitter on Google, you will see the following:
How can I make our website show up as this (with section sub-headers) in the Google search results?
i came to know that google does it at its own when required. We can use the webmasters tool to remove the sitelinks when the search is done.
See Google's article on site links
https://support.google.com/webmasters/answer/47334?hl=en
This can be done on the Google Webmaster Tools page as described in Google's Sitelinks article.
Demote a sitelink URL:
On the Webmaster Tools Home page, click the site you want.
Under Search Appearance, click Sitelinks.
In the For this search result box, complete the URL for which you don't want a specific sitelink URL to appear. (How to find the right URL.)
In the Demote this sitelink URL box, complete the URL of the sitelink you want to demote.
I'm currently parsing twitter feeds through my rails app and wondering how it would be possible to follow links in the tweets and scrape the external content (for example, if the tweet contains a link to an article on TechCrunch, follow that link to the article and scrape the title and body content of the article). Flipboard for iPad is a perfect example of this.
I'm familiar with screen scraping using Nokogirl / Mechanize, but trying to figure out how to accomplish this in a way it could work regardless of where the link in the tweet is linking to (whether it be to TechCrunch, or Flickr, etc...).
Any thoughts / insights would be much appreciated!
Many of the major content providers provide a oembed endpoints. Take a look at the oembed_links gem. From the readme:
It allows you to easily parse text and
query configured providers for embedding information on the links
inside the text. A sample configuration file for configuring the
library has been included (oembed_links_example.yml), though you
may also configure the library programmatically (see rdocs).
If you use oembed_links in conjuction with http://oohembed.com/ you'll have dozens of content providers handled for you. You can easily write a custom provider to handle the rest.
Sorry for the bad title and description, but I was wondering if there is anyway I could search/list products from other sites (say Express, American Eagle), from a web app I create, even if the site doesn't have an API.
Thanks
Sure. How do you think Google and every other search engine does it? They just spider the sites and index the contents. The devil, of course, is in the details. But it's certainly possible to do.
I don't think so. Unless you want only to fetch some data from a certain HTML page, then you need to use some regular expressions. But searching the database is not possible if you don't have the ability to connect to it directly or via some APIs.
I want to create a search webpage which should display the Google results page as well as results from our intranet webpage. Can I design it using Google Custom Search Engine?
Not unless you expose your intranet to the public Internet for Google to index, which is probably not something you want to do.
They have services to index intranet content as well, but they might be a big costly.