Advanced site search with google - url

I try to find out which URLs exists for a specific domain and a specific domain-path in the google index. The urls have the following schema:
https://example.org/path1/<keyword>/path2/
the following google search works fine:
site:https://example.org/path1/*/path2/
but it delivers more than 40.000 findings. So I'll try to search for
https://example.org/path1/a*/path2/
but there where no results found (what can't be). Whats wrong? Any chance to deliver only Findings where Site-URL contains keywords starting with an "a"?
Thank you,
Jan

You can try the following
https://example.org/path1/*a
This will search for all the URL's which starts with https://example.org/path1/ which also contains the keyword a
You can refine your search by specifying multiple keywords:
https://example.org/path1/*a*/path2/
This will search for the same as in the 1st example but will conatin the /path2/ part of the URL as well. However this will match URL's if the keyword a is either before or after the 2nd path /path2/

Related

twitter: search hashtags in a twitter list

i'm trying to use 'Twitter Search Widget' here searching an #hashtag in a 'Twitter List', but i can't fix the exact query. Someone did it before me?
Thanks in advance, sorry for my poor english.
Francesco
There doesn't seem to be a way to directly search twitter lists using the widget. If you look at the "operators" link on this page:
https://twitter.com/#!/search-home
You can use "from:userid OR from:userid2 #hashtag" to search for #hashtag tweets from specific users, and hashtags work fine - so you could manually build a search for a list if you wanted.
You can see what operators search can take by looking at the advanced search page here:
https://twitter.com/#!/search-advanced
If i understand correctly, you're trying to search for tweets containing a specific #hashtag, authored uniquely by certain list members?
If this is the case, say for example your looking for the hashtag #COVID19 relayed only by #Twitter's list of official Twitter accounts, you could do it with the following Twitter search query:
#COVID19 list:84839422
Try it...
The structure is quite simple...
[#hashtag] list:[list_id]
You can find list IDs by looking at the URL line in the browser when opening lits on Twitter. For example, here is the URL of #Twitter's official accounts liss:
https://twitter.com/i/lists/84839422
The list ID are the digits after the last /.

limit results for Ask.com per URL parameter

I'm looking for an URL parameter to set the results per page on Ask (ask.com) search engine? For Google and Bing I found a parameter, but for Ask.com I get only some results for a Web Search API, but that's not what I'm looking for. Does anyone know, if there is a parameter like num=30 or so to display X results, like:
http://de.ask.com/web?q=something&qsrc=0&o=312&l=dir&num=30
Ok, found out, that this is not possible. So I have to grab next page on my own.

Can't parse new google urls - HTTP_REFERER doesn't contain parameters anymore

It seems a little odd to my, but although everybody knows about the new google search urls (see Google using # instead of search? in URL. Why?) no one has a problem with the HTTP_REFERER.
I'm using the referrer to parse the google string for the searchquery (&q= ) but as this is all in a hash-tag it wont be sent to the server and all i get is "http://www.google.de/".
So do you know a way of getting the query the user searched for, befor landing on my site?
Due to late-2011 Google security changes, this is no longer possible when the search was performed by a signed-in Google user. See:
http://googleblog.blogspot.com/2011/10/making-search-more-secure.html
http://analytics.blogspot.com/2011/10/making-search-more-secure-accessing.html
Since there are multiple q's in the query string you have to match the "q" parameter globally and take the last one:
/[?|&|#]q=([^&|^#]+)/ig
Get rid of "site:" searches (there are others, but I haven't done them)
/[\+|?|&]?site:([^&|^#])+/g, '');
Then parse the results.
/[\w^'\(\)\{\}]+|"[^"]+"/g
This has been working well for me.

Twitter API - Display all tweets with a certain hashtag?

How would I go about displaying tweets that contain a certain hashtag using the Twitter API? Thanks
I'd also like to know if there is a way to get all tweets from a certain hashtag in a separate file, also the ones that don't show up in your feed anymore. I suppose that's what the earlier question was about, too.
This answer was written in 2010. The API it uses has since been retired. It is kept for historical interest only.
Search for it.
Make sure include_entities is set to true to get hashtag results. See Tweet Entities
Returns 5 mixed results with Twitter.com user IDs plus entities for the term "blue angels":
GET http://search.twitter.com/search.json?q=blue%20angels&rpp=5&include_entities=true&with_twitter_user_id=true&result_type=mixed
UPDATE for v1.1:
Rather than giving q="search_string" give it q="hashtag" in URL encoded form to return results with HASHTAG ONLY. So your query would become:
GET https://api.twitter.com/1.1/search/tweets.json?q=%23freebandnames
%23 is URL encoded form of #. Try the link out in your browser and it should work.
You can optimize the query by adding since_id and max_id parameters detailed here. Hope this helps !
Note: Search API is now a OAUTH authenticated call, so please include your access_tokens to the above call
Updated
Twitter Search doc link:
https://developer.twitter.com/en/docs/tweets/search/api-reference/get-search-tweets.html
The answer here worked better for me as it isolates the search on the hashtag, not just returning results that contain the search string. In the answer above you would still need to parse the JSON response to see if the entities.hashtags array is not empty.

How does a website highlight search terms you used in the search engine?

I've seen some websites highlight the search engine keywords you used, to reach the page. (such as the keywords you typed in the Google search listing)
How does it know what keywords you typed in the search engine? Does it examine the referrer HTTP header or something? Any available scripts that can do this? It might be server-side or JavaScript, I'm not sure.
This can be done either server-side or client-side. The search keywords are determined by looking at the HTTP Referer (sic) header. In JavaScript you can look at document.referrer.
Once you have the referrer, you check to see if it's a search engine results page you know about, and then parse out the search terms.
For example, Google's search results have URLs that look like this:
http://www.google.com/search?hl=en&q=programming+questions
The q query parameter is the search query, so you'd want to pull that out and un-URL-escape it, resulting in:
programming questions
Then you can search for the terms on your page and highlight them as necessary. If you're doing this server side-you'd modify the HTML before sending it to the client. If you're doing it client-side you'd manipulate the DOM.
There are existing libraries that can do this for you, like this one.
Realizing this is probably too late to make any difference...
Please, I beg you -- find out how to accomplish this and then never do it. As a web user, I find it intensely annoying (and distracting) when I come across a site that does this automatically. Most of the time it just ends up highlighting every other word on the page. If I need assistance finding a certain word within a page, my browser has a much more appropriate "find" function built right in, which I can use or not use at will, rather than having to reload the whole page to get it to go away when I don't want it (which is the vast majority of the time).
Basically, you...
Examine document.referrer.
Have a list of domains to GET param that contains the search terms.
var searchEnginesToGetParam = {
'google.com' : 'q',
'bing.com' : 'q'
}
Extract the appropriate GET param, and decodeURIComponent() it.
Parse the text nodes where you want to highlight the terms (see Replacing text with JavaScript).
You're done!

Resources