I am writing a short scrapper using Google Spreadsheets using Xpatch and IMPORTXML
on that page, I am trying to get in B3 and following all the titles of articles (class 'library-document-summary') and in C3 and follow all the URLS of said articles
however, I am getting nowhere as the returns of my XPATH are always empty. Could someone with knowledge in this area help?
B2= https://resources.norrag.org/categories/591,595
=IMPORTXML(B2,"//div//a[#class='library-document-summary']/text()")
I don't think the IMPORTXML function supports XPaths that select text nodes. But I think if your XPath selects the a elements themselves, then their text content will be imported. e.g.
//div[#id='article_search_results']//a
... and for the links:
//div[#id='article_search_results']//a/#href
Related
I want to get the price from mercari, a japanese online shop.
For example, in this link, I like to get 1,488.
https://jp.mercari.com/item/m78226870756
when I copy the xpath of
<span class="number">
I get
//*[#id="item-info"]/section[1]/section[1]/div[1]/mer-price//span[2]
Now, using google sheet importxml
=IMPORTXML("https://jp.mercari.com/item/m78226870756","//*[#id='item-info']/section[1]/section[1]/div[1]/mer-price//span[2]")
I receive a
#N/A Imported content is empty.
I would really like to know how to get the price.
I am not familiar with this at all.
Any other way other than google sheet is also welcome.
you are getting #N/A error due to importxml (or any other import) formula does not support the scrapping of JavaScript elements. you can test this always by disabling JS for a given site and what's left can be usually imported into google sheets
I'm having a tough time pulling info in on Google Sheets using the ImportXML function. I want to pull in the price of a crypto coin so that I have a real-time feed. The link that I'm hoping to pull from is:
https://www.dextools.io/app/uniswap/pair-explorer/0x40f0e70a7d565985b967bcdb0ba5801994fc2e80
I've tried out a lot of different formulas and keep getting an #N/A or an error. Some of the ones I've tried:
Copy XPATH fully:
=IMPORTXML("https://www.dextools.io/app/uniswap/pair-explorer/0x40f0e70a7d565985b967bcdb0ba5801994fc2e80","/html/body/app-root/div[3]/div/main/app-uniswap/div/app-pairexplorer/app-layout/div/div/div[2]/div[2]/ul/li[2]/span")
Shortened XPATH (also tried deleting the second backslash before 'li' but that didn't work):
=IMPORTXML("https://www.dextools.io/app/uniswap/pair-explorer/0x40f0e70a7d565985b967bcdb0ba5801994fc2e80","//li[2]/span")
Include class:
=IMPORTXML("https://www.dextools.io/app/uniswap/pair-explorer/0x40f0e70a7d565985b967bcdb0ba5801994fc2e80","//li[2]/span[#class='ng-tns-c93-2 ng-star-inserted']")
Does anyone have thoughts? Thanks!
upon disabling JavaScript the site is empty = can't be scraped by Google Sheets by any import formula.
To avoid the problem above, consider using a proper API service that gives you easy access to the data.
For instance you could get Zero price in USD using
=IMPORTDATA("https://cryptoprices.cc/ZERO/")
If you need it in comparison to ETH you could try doing it by hand
=IMPORTDATA("https://cryptoprices.cc/ZERO/")/=IMPORTDATA("https://cryptoprices.cc/ETH/")
Or use a more advanced API such as CoinGecko's
https://www.coingecko.com/en/api
Im trying to scrape this website (https://kamadan.gwtoolbox.com/) with google sheets for material costs for a game that I play. There are two tables; "Common Materials" and "Rare Materials" in a drop down in the top right corner. I am trying to pull the values for both as the prices update. I copied the full Xpath and used the function below in an empty cell on a sheet.
=importxml("https://kamadan.gwtoolbox.com/","/html/body/div[2]/div[1]/div/div[2]/table/tbody")
This returns a #N/A error saying it is returning an empty value.
I also tried it with the regular xpath...
=importxml("https://kamadan.gwtoolbox.com/","//*[#id='trader-overlay-items']")
Which just returns a blank cell. I have also tried both methods using the inspect function through chrome on the ancestors and children they return either of the two errors above.
Sorry if this is a really easy one. I am not familiar at all with Xpaths or html. I mostly dabble in VBA in excel.
Answer:
IMPORTXML can not retrieve data which is populated by a script, and so using this formula to retrieve data from this table is not possible to do.
More Information:
As you've already mentioned, you can attempt to get the data directly from the table using:
=IMPORTXML("https://kamadan.gwtoolbox.com/","//table[#id='trader-overlay-items']")
Which just gets a blank cell.
I went a step further and tried to reverse-engineer this by calling IMPORTXML on the HTML elements on the page in steps:
=IMPORTXML("https://kamadan.gwtoolbox.com/","html")
=IMPORTXML("https://kamadan.gwtoolbox.com/","html/body")
=IMPORTXML("https://kamadan.gwtoolbox.com/","html/body/div[1]")
=IMPORTXML("https://kamadan.gwtoolbox.com/","html/body/div[1]/div[0]")
...
html/body/div[1]/div[0] is the first path which gives no imported content, and we can see from importing html/body that the full body does not contain the imformation and only a template of it - in cell B1 we have references to 'Common materials' and 'Rare materials':
And in D1 we start to see JavaScript and JSON objects which are not called by IMPORTXML and so the results of which can not be retrieved:
As you can see if you disable JavaScript on the site, almost nothing is actually rendered and so can't be obtained using IMPORTXML:
References:
IMPORTXML - Docs Editors Help
I would like to have some help to get the data beside ROE from this link using importxml / xpath. http://fundamentus.com.br/detalhes.php?papel=TAEE11 ... so in this case the ROE data is 20,8% . I would like to get this value using importxml / xpath.
How to do that? I've tried some formulas but.. not able to get the details from the website.
You can do this with a combination of importxml, match and index:
=index(IMPORTXML("http://fundamentus.com.br/detalhes.php?papel=TAEE11","//*[#class='txt']"),match("roe",IMPORTXML("http://fundamentus.com.br/detalhes.php?papel=TAEE11","//*[#class='txt']"),0)+1,0)
Basically what is happening is that by point to the class # txt it stacks all the labels above the data fields, so you can consistently search for a label such as ROE, and just increase the index by 1 to retrieve the corresponding value.
I'm having some problems with ImportXML in my Google Spreadsheet. I currently have two sheets, each with their own ImportXML, retireving (basically) the same data - the server providing the data has updated their feed service to require the use of a user-specific "key" in the URL to track who is retrieving what. Prior to this change, my ImportXML worked just fine. They are about to turn off the non-key feeds, and my spreadsheets are about to break.
In the first (working) sheet, this is the feed.
I can import the data sucessfully by using the following syntax in cell A1:
=importXML(ʺhttp://atilla.hinttech.nl/fseconomy/xml?id=18649&key=M3LRG43T&query=GroupLogByMonth&month=10ʺ,ʺ//GroupLogByMonthʺ)
In the new (non-working) sheet, the URL to the feed (including my user-specific "keys") is here.
I am unable to create a working importXML on this sheet. None of my attempted Xpath queries worked, except "*"; but that resulted in all elements being lumped into a single cell.
I have shared my spreadsheet file (link is in the comments below - I am unable to post more than 2 links) with each of these sheets so that the above examples can be seen and played with. Any advise on the non-working sheet would be wonderful.
In the new XML feed there is no tag "GroupLogByMonth". This might explain why your Xpath query won't return anything when you look for that.
Did the format of the XML change too, next to the new URL?