I am trying to scrape the results from a Quora search query using ImportXML.
The URL is of this form: https://www.quora.com/search?q=scrape%20Quora&time=year
I've tried using ImportXML, and can't get anything to work. As an example, I inspected the questions, and found they were inside a div with a class name of 'q-text puppeteer_test_question_title'. So I tried to import like this, but I just get #N/A:
importxml("https://www.quora.com/search?q=scrape%20Quora&time=year","//div[#class='q-text puppeteer_test_question_title']")
This is clearly not working: is there a fix or just not possible (and why)?
Thank you.
Quora (as of now) runs on JavaScript and google sheets import formulae do not support the scrapping of JS elements:
You can try to fetch the first 3 responses this way (quickly written, could be improved)
function myFunction() {
var options = {
'muteHttpExceptions': true,
'followRedirects': false
};
var url = 'https://www.quora.com/search?q=scrape%20Quora&time=year'
var jsonStrings = UrlFetchApp.fetch(url,options).getContentText().split('window.ansFrontendGlobals.data.inlineQueryResults.results["')
jsonStrings.forEach((jsonString,i) => {
if (i > 0) {
console.log(jsonString.split('"] = ')[1].split('\n')[0])
}
})
}
and then parse the complex json inside. However, other answers are transmitted by quora when scrolling down by ajax asynchronous request.
Related
In Sapper, AFAIK from documentation. The only way to access URL params are through preload() function, from which params are available inside params object.
The thing is that I want to access these params ouside of preload() function. From an eagle eye view of documentation. I don't / can't see the solution to my problem / requirement.
I have tried setting a property for url param inside data(). But it seems preload() has no access to data whether getting wise or setting wise. It is not meant for those things.
<script>
import { stores } from "#sapper/app";
const { page } = stores();
const { slug } = $page.params;
</script>
https://sapper.svelte.dev/docs/#Stores
If you are using v3 Svelte and latest alpha of Sapper, import page which is now provided as a store.
import { page } from '#sapper/app';
const {slug} = $page.params;
I'm trying to pull dynamic data (form) to a google sheet.
I can't seem to find the right function.
I'm running this:
function name(){
return $('input[type="text"]').val();
}
I tried this:
function fullname(){
return $('#form-field-1-1').val();
}
No success for now.
I've attached the elements below.
Thank you [https://i.stack.imgur.com/Tt5Gk.png]
Your best choise is to use a Variable "DOM element" and capture it by ID. but if you prefer a custom JS this will do:
function(){
return document.getElementById('form-field-1-1').innerHTML;
}
Hope it helps.
I am building a simple blog page where I wish to use markdown as the text format.
I have a working page when running in Dartium but when I compile to js the markdown does not come out properly formatted. It's missing paragraphs only I think but headers and lists are working fine.
I'm displaying the blog post in a polymer element and reading in a simple file from the server. I have made a simple sample without polymer which seems to work fine but I haven't tried it on the production server.
The basic code is outlined below, any tips or a better way of doing this? I will eventually move the posts to a db as text but I'm open to suggestions for other ways of presenting blog posts with some simple formatting, thanks.
getPostsFromServer(){
String path = 'post1.md';
HttpRequest req = new HttpRequest();
req
..open('GET', path)
..onLoadEnd.listen((e) => printPost(req))
..send('');
}
void printPost(HttpRequest req){
var postdiv = $['article'];
if(req.status == 200){
var postText = req.responseText;
print(postText);
postdiv.innerHtml = markdownToHtml(postText);
}
else{
postdiv.innerHtml = 'Failed to load newsletter, sorry.';
}
}
I'm looking to do a Play Scala project where I'll do especially drag & drop.
Is jQuery UI the best and the simplest way to do that? and How could one do this? could someone give me some examples or pointers?
In Play 1.0 there is a good example on how to use jQuery UI (http://www.playframework.com/modules/jqueryui-1.0/home) but I don't found examples or documentation on Play2.0 Scala !
first of all you need to define an endpoint for your data in the controller.
Simething like this:
def autocompleteSearch(a: String) = Action {
request =>
val data = List("apple", "apple iphone", "apple ipad", "microsoft windows", "microsot office");
val filteredData = data.filter(_.startsWith(a))
Ok(Json.toJson(filteredData))
}
Then you need to add this endpoint to your routes:
GET /some-ajax controllers.Application.autocompleteSearch
Then in your .scala.html file you can write a script that will do a get you you, example:
var updateTime = function() {
$.get("#routes.Application.timeUpdate()", function(data) {
display.html(data)
})
return false
}
When you have the var you just pass it to the jQueryUI.
My examples are copied from different things and they don't really fit together :) but I hope you can grasp the general idea.
Sorry the answer is not more specific, but I'm no JavaScript master and had to fight with the jQuery once, and this is what I have left from the struggle.
Cheers!
I want to create a simple bookmarklet, that grabs the URL of the current webpage "location.ref" and saves it in a Google Spreadsheet. After it saves it, I want to stay on the current webpage.
The only way I know of writing to Google Spreadsheet is using Google App Script. So I wrote a simple script that does just that:
function doGet(request) {
var ss = SpreadsheetApp.openByUrl( "https://docs.google.com/spreadsheet/ccc?key=<MY-SPREADSHEET-ID>");
var sheet = ss.getSheets()[0];
var headers = ["Timestamp", "url"];
var nextRow = sheet.getLastRow();
var cell = sheet.getRange('a1');
var col = 0;
for (i in headers){
if (headers[i] == "Timestamp"){
val = new Date();
} else {
val = request.parameter[headers[i]];
}
cell.offset(nextRow, col).setValue(val);
col++;
}
return ContentService.createTextOutput(request.parameter.url)
.setMimeType(ContentService.MimeType.TEXT);
}
I published this as a webapp. I wrote the bookmarklet:
<a href="javascript:(
function(){
alert(window.open('https://script.google.com/macros/s/<MYWEBAPP>/exec?url='+encodeURIComponent(location.href), '_self'));
}
)();">
BOOKMARK
</a>
So far so good. It actually works when I click on the bookmarklet, it does grab the URL of the current webpage and save it in my spreadsheet. But then, the webapp returns a text response and the bookmarklet displays the text causing me to move away from my current website.
Is there a way to ignore the response? GAS webapp script requires me to use doGet() that has to return something. Is there a way to not return anything from GAS script? Alternatively, is there a way i could use some other call to replace window.open to invoke the webapp that would allow me to store the response in a variable and ignore it?
I know it's been over a year but I was trying to do exactly this. It took me a while to figure out, but this works. The 1 second delay was necessary to let the script finish loading.
javascript:(function(){
my_window=window.open('https://script.google.com/macros/s/<MYWEBAPP>/exec?url='+encodeURIComponent(location.href)+'&title='+encodeURIComponent(document.title));
(window.setTimeout(function(){my_window.close();},1000));
void(0);
})();
Instead of using window.open you may consider sending a HTTP GET request using XMLHttpRequest.
Refer here on its usage.
Change _self to something else, e.g. bookmarker and it will open in a new window or tab. If you use it on many pages, they will all reuse the same tab if it keeps the same name.