I make a Q&A website like Stack Overflow, and now I want to add it searching capability.
I want going to use the Google Search API but one of my friends said that it's better to have your own searching engine because Google doesn't index all pages (specially recently added pages) and also it has negative effect on the site ranking.
My question is that these statements is true or not and what is the best way for searching in this site?
Depending on your database full text search capabilities may be available.
For example check MySQL : https://dev.mysql.com/doc/refman/5.0/en/fulltext-search.html
Alternative solution is to use a dedicated search index like Lucene: https://lucene.apache.org/core/
I'm try to access the Office365 (SharePoint) search service of my tenant, e.g.:
https://myTenant.sharepoint.com/_vti_bin/search.asmx
from iOS using the ADALiOS library (https://github.com/AzureADSamples/NativeClient-iOS).
The samples all seem quite complex, setting up another application, integrating Web API, etc.
In the end, I think I just have to authenticate once, get the token and send it which each request.
Is there any sample available that shows how to do this (seemingly) simple task?
Or has anybody done this before, maybe even without ADALiOS?
Any hint is appreciated!
Eau
I use REST web service for the purpose.
The official guide is too obscure. Below you can find samples that cover simple and advanced queries. They will make things clear:
full-text and property "title"
_https://MYTENANT.sharepoint.com/_api/search/query?QueryText='part title:document'&selectproperties='Title,Author,Size,Path,Description,Write,LastModifiedTime,HitHighlightedSummary,ServerRedirectedPreviewURL,FileExtension,UniqueId,DisplayAuthor,UrlZone,Culture'&rowlimit=20&rowsperpage=20
full-text and property "author" and "title"
_https://MYTENANT.sharepoint.com/_api/search/query?QueryText='part title:document author:kapil'&selectproperties='Title,Author,Size,Path,Description,Write,LastModifiedTime,HitHighlightedSummary,ServerRedirectedPreviewURL,FileExtension,UniqueId,DisplayAuthor,UrlZone,Culture'&rowlimit=20&rowsperpage=20
full-text and size limitation
_https://MYTENANT.sharepoint.com/_api/search/query?QueryText='part title:document size>60000'&selectproperties='Title,Author,Size,Path,Description,Write,LastModifiedTime,HitHighlightedSummary,ServerRedirectedPreviewURL,FileExtension,UniqueId,DisplayAuthor,UrlZone,Culture'&rowlimit=20&rowsperpage=20
full-text and last modified time
_https://MYTENANT.sharepoint.com/_api/search/query?QueryText='part title:document LastModifiedTime>2014-08-04T04:09:00.0000000Z'&selectproperties='Title,Author,Size,Path,Description,Write,LastModifiedTime,HitHighlightedSummary,ServerRedirectedPreviewURL,FileExtension,UniqueId,DisplayAuthor,UrlZone,Culture'&rowlimit=20&rowsperpage=20
full-text and last modified date
_https://MYTENANT.sharepoint.com/_api/search/query?QueryText='part title:document LastModifiedTime>2014-08-04'&selectproperties='Title,Author,Size,Path,Description,Write,LastModifiedTime,HitHighlightedSummary,ServerRedirectedPreviewURL,FileExtension,UniqueId,DisplayAuthor,UrlZone,Culture'&rowlimit=20&rowsperpage=20
I am currently developing an application in Rails, which requires to check whether a website has been listed in Google, Bing, Yahoo, Yelp and Yellow Pages. From my research the best is to check site: domain.com on Google and Bing and look for results and check in Yahoo directory for the domain.
Is there any other way to do it? I mean some code snippet to check on domain's home page or using their API or something like that. Also how to check on Yelp and Yellow pages.
You can use mechanize and write web-style drivers
Google: do a search on your domain with this on the search term
site:checkmeout360.com
https://www.google.com/search?q=site%3A<SITE_NAME>.com
Try to see how yelp, yahoo, bing and yellow pages do indexing. Then you can use mechanize to automate the searching process for you, you can use mechanize to do the search like above with google, then write asserts (check if stuff you are looking for is on the search result)
Search engines don't appreciate automated queries that are sent their way.
Here is what Google has to say about it:
Google's Terms of Service do not allow the sending of automated queries of any sort to our system without express permission in advance from Google. Sending automated queries consumes resources and includes using any software (such as WebPosition Gold) to send automated queries to Google to determine how a website or webpage ranks in Google search results for various queries. In addition to rank checking, other types of automated access to Google without permission are also a violation of our Webmaster Guidelines and Terms of Service.
Previously these folks promised a release of their implementation of Bing search for their site at the following article: http://www.globalnerdy.com/2009/06/29/learnhub-powered-by-rails-searches-with-bing/
Is anyone familiar with a Ruby or Rails lib that would facilitate site search with Bing? Google just hasn't been a good match so far with their site search and a search with MS Bing, surprisingly, seems to be a much better solution.
Otherwise, an example of how to accomplish this, even without a lib and directly using the API, would be much appreciated.
While not a custom site search per se, you should be able to use RBing for accessing Bings search API. There's an introductory tutorial over at http://9astronauts.com/code/ruby/rbing/
To make it work like a site search, simply append a site:example.com to your queries and it will only return results from that domain. For example:
bing = RBing.new("YOURAPPID")
query = "something interesting"
results = bing.web("#{query} site:stackoverflow.com")
puts results.web.results[0].title
=> "javascript - How to illuminate a browser window/tab when something ..."
Most of the traffic from my site is sent by search engines. I am trying to track the keywords used on the search (EG to have real-time stats on the 'hottest' searches).
What are my options? I thought of:
Parsing the HTTP_REFERER (using a list regexps for major search engines)
Importing google analytics data (using google analytics api) - however this is not realtime
Are there any gems/libraries that offer support for this?
I would write a middleware for rack to do this. http://www.hokstad.com/slidingstats is an example of one that tracks referrers. It is pretty easy to hack and not complicated so you can probably use this as a launchpad for what you want to do.