New to APIs in Google sheets, but I feel like I'm 95% of the way to where I'm trying to go.
I'm trying to pull crypto data into my spreadsheet to do a simple 24 hour price comparison and gauge whether the price has gone up or down, maybe use some conditional code to change the background to green or red. Simple enough. Most of the sites that offer APIs have given me various errors, though, so coinbase pro (and weirdly the deprecated gdax) have been most reliable (although I haven't ruled out that it started breaking because I'm now putting in too many call requests).
Found this as a way to get the current price of ETH, for instance:
=VALUE(SUBSTITUTE(SUBSTITUTE(INDEX(IMPORTDATA("https://api.gdax.com/products/ETH-USD/ticker"),0,2), "price:",""), """", ""))
Works like a charm. So I changed the request to target different info, specifically the 24hr stats listed on the API doc, and the first value in that section, "open" for opening price (this ensures I get the price exactly 24hrs earlier). But I got a weird parsing error using the request, which is here:
=VALUE(SUBSTITUTE(SUBSTITUTE(INDEX(IMPORTDATA("https://api.pro.coinbase.com/products/ETH-USD/stats"),0,1), "open:",""), """", ""))
I've figured out the issue, but not the solution. Google Sheets says I am pulling in text. Because the "open" (opening price) value is the first listed in the JSON code, it is pulling in the code bracket from the nested HTML/JSON code. For instance, it says I can't parse "{open" into a number. And I get the same problem in reverse when I pull the last value listed in the stats section, which is "volume_30day:"
=VALUE(SUBSTITUTE(SUBSTITUTE(INDEX(IMPORTDATA("https://api.pro.coinbase.com/products/ETH-USD/stats"),0,6), "volume_30day:",""), """", ""))
This returns an error saying "volume_30day: #}" can't be parsed, so now it is the closing bracket of the JSON code. So I can't use "open" the first item in the API 24hr stats section, or Volume_30day, which is the sixth item on that list, but items 2-5 work just fine. Seems super weird to me, but I've tested it and it is seems to be what's going on.
There must be something stupid I need to tweak here, but I don't know what it is.
Answer 1:
About =VALUE(SUBSTITUTE(SUBSTITUTE(INDEX(IMPORTDATA("https://api.pro.coinbase.com/products/ETH-USD/stats"),0,1), "open:",""), """", ""))
When I checked =SUBSTITUTE(SUBSTITUTE(INDEX(IMPORTDATA("https://api.pro.coinbase.com/products/ETH-USD/stats"),0,1), "open:",""), """", ""), the value is {open:617. I think that when by this, when VALUE is used for the value, the error occurs.
In order to retrieve the values you expect, I would like to propose to use REGEXREPLACE instead of SUBSTITUTE. The modified formula is as follows.
=VALUE(REGEXREPLACE(INDEX(IMPORTDATA("https://api.pro.coinbase.com/products/ETH-USD/stats"),0,1), "open|""|{|}|:",""))
In this modified formula, open|""|{|}|: is used as the regex. These are replaced with "".
In this case, I think that =VALUE(REGEXEXTRACT(INDEX(IMPORTDATA("https://api.pro.coinbase.com/products/ETH-USD/stats"),0,1), "\d+")) can be also used. But when I thought about your 2nd question, I thought that above formula might be useful.
Result:
Answer 2:
About =VALUE(SUBSTITUTE(SUBSTITUTE(INDEX(IMPORTDATA("https://api.pro.coinbase.com/products/ETH-USD/stats"),0,6), "volume_30day:",""), """", ""))
When I checked =SUBSTITUTE(SUBSTITUTE(INDEX(IMPORTDATA("https://api.pro.coinbase.com/products/ETH-USD/stats"),0,6), "volume_30day:",""), """", ""), the value is 7101445.64098932}. I think that when by this, when VALUE is used for the value, the error occurs.
In order to retrieve the values you expect, I would like to propose to use REGEXREPLACE instead of SUBSTITUTE. The modified formula is as follows.
=VALUE(REGEXREPLACE(INDEX(IMPORTDATA("https://api.pro.coinbase.com/products/ETH-USD/stats"),0,6), "volume_30day|""|{|}|:",""))
In this modified formula, volume_30day|""|{|}|: is used as the regex. These are replaced with "".
In this regex, it can use by replacing open of open|""|{|}|: to volume_30day at above regex.
Result:
Other pattern 1:
As other pattern using the built-in formula, how about the following modified formulas?
=VALUE(TEXTJOIN("",TRUE,ARRAYFORMULA(IFERROR(VALUE(REGEXREPLACE(IMPORTDATA("https://api.pro.coinbase.com/products/ETH-USD/stats"), "open|""|{|}|:","")),""))))
=VALUE(TEXTJOIN("",TRUE,ARRAYFORMULA(IFERROR(VALUE(REGEXREPLACE(IMPORTDATA("https://api.pro.coinbase.com/products/ETH-USD/stats"), "volume_30day|""|{|}|:","")),""))))
In these formulas, the values can be retrieved by replacing KEY of KEY|""|{|}|: of the regex.
Other pattern 2:
The returned value from https://api.pro.coinbase.com/products/ETH-USD/stats is the JSON value. So in this case, when the custom function created by Google Apps Script can be also used.
The Google Apps Script is as follows.
const SAMPLE = (url, key) => JSON.parse(UrlFetchApp.fetch(url).getContentText())[key] || "no value";
When you use this script, please copy and paste the above script to the script editor of Spreadsheet and save it. And please put the custom function like =SAMPLE("https://api.pro.coinbase.com/products/ETH-USD/stats","open") and =SAMPLE("https://api.gdax.com/products/ETH-USD/ticker","price") to a cell. By this, the value can be retrieved.
References:
REGEXREPLACE
Custom Functions in Google Sheets
[![enter image description here][4]][4][![enter image description here][5]][5]I have a PDF that has tabular data that runs over 50+ pages, i want to extract this table into an excel file using Automation Anywhere. (i am using community version of AA 11.3). I watched videos of the PDF integration command but haven't had any success trying this for tabular data.
Requesting assistance.
Thanks.
I am afraid that your case will be quite challenging... and the main reason for that are the values that contains multiple lines. You can still achieve what you need, and with good performance, but the code itself will not be pretty. You will also be facing challanges with Automation Anywhere, since it does not really provide the right tools to do such a thing and you may need to resort to scripting (VBScripts) or Metabots.
Solution 1
This one will try to use purely text extraction and Regular expressions. Mainly standard functionality, nothing too "dirty".
First you need to realise how do the exported data look like. You can see that you can export to Plain or Structured.
The Plain one is not useful at all as the data is all over the place, without any clear pattern.
The Structured one is much better as the data structure resembles the data from the original document. From looking at the data you can make these observations:
Each row contains 5 columns
All columns are always filled (at least in the visible sample set)
The last two columns can serve as a pattern "anchor" (identifier), because they contain a clear pattern (a number followed by minimum of two spaces followed by a dollar sign and another number)
Rows with data are separated by a blank row
The text columns may contain a multiline value, which will duplicate the rows (this one thing makes it especially tricky)
First wou need to ensure that the Structured data contain only the table, nothing else. You can probably use the Before-After string command for that.
Then you need to check if you can reliably identify the character width of every column. You can try this for yourself if you copy the text into Excel, use the Text to Columns with the Fixed Width option and try to play around with the sliders
The you need to try to find a way how to reliably identify each row and prepare it for the Split command in AA. For that you need to have a delimiter. But since each data row can actually consists of multiple text rows, you need to create a delimiter of your own. I used the Replace function with Regular Expression option and replace a specific pattern for a delimiter (pipe). See here.
Now that you have added a custom delimiter, you can use the Split command to add each row into a list and loop through it.
Because each data row may consists of several rows, you will need to use Split again, this time use the [ENTER] as delimiter. Now you need to loop through each of the text line of a single data line and use the Substring function to extract data based on column width and concatenate them to a single value that you store somewhere else.
All in all, a painful process.
Solution 2
This may not be applicable, but it's worth a try - open the PDF in Microsoft Word. It will give you a warning, ignore it. Word will attempt to open the document and, if you're lucky, it will recognise your table as a table. If it works, it will make the data extraction much easier an you will be able to use Macros/VBA or even simple Copy&Paste. I tried it on a random PDF of my own and it works quite well.
I need a way to extract the Salesforce record ID from a URL using Zapier Push. How can I find the first 3 characters in a string that match the start of the Id like 006 and then return a set number of characters after that?
The url is formatted as such:
https://useindio.lightning.force.com/lightning/r/Opportunity/006f400000AiVufAAF/view
David here, from the Zapier Platform team. Good question!
Whenever you want to extract data from a string and you know the exact format the string will be in, Regular Expressions are the answer.
Assuming you want to grab anything after 006 (and you know it'll always be there), you could use the regex 006(\w{15}) (more info), which will find the 15 characters after that. If you know the surrounding url will always be the same, you could easily grab the whole ID by anchoring via Opportunity and view: \/Opportunity\/(.*)\/view (more info).
Either way, there's info about setting up a formatter zap here, or you could do it in code (JS Example, Python Example).
Let me know if you've got any other questions!
I want to make custom slots that accepts any and all entries as long as those entries follow a certain regex pattern, eg:any number of alphabets or numbers but without a space in between. Can anyone tell me if there is a way in amazon lex to achieve it?
Also, if I want to take a certain type of data, say, email ids, but want to give the user option to give any number of email ids (more than one), what is the way to do that.
I am new to Amazon Lex and any suggestions would be appreciated.
Make a slot in Lex console in your intent but do not tick as required, and give any type as slot type.
Now in lambda code, first set the slot to null and then parse the inputText using regex and assign the correct value to the slot.
This way both of your problems will be addressed.
Hope it helps. Let us know if you run in any problems.
I'm trying to blast an 8-mer (string of length 8) against the NCBI database. However, whenever I use qblast, it comes up empty with respect to matches. This is my code:
from Bio.Blast.NCBIWWW import qblast
import Bio.Blast.NCBIXML as parser
a = qblast('blastp','nr','GGMPSGCS')
b = parser.read(a)
print b.alignments`
Whenever I do this, it just prints the empty list []. Why is this happening? Can anyone shine a light on it?
I can get a match using the NCBI online BLAST tool, and I can even get a match if I use a longer kmer like "SSRVQDGMGLYTARRVR". It just happens that all the 8-mers I search come up empty.
From the FAQ at http://biopython.org/DIST/docs/tutorial/Tutorial.html
Why doesn’t Bio.Blast.NCBIWWW.qblast() give the same results as the NCBI BLAST website?
You need to specify the same options – the NCBI often adjust the default settings on the >website, and they do not match the QBLAST defaults anymore. Check things like the gap >penalties and expectation threshold.
Check that qblast is using the same defaults, if not sure, make them explicit. I wouldn't be surprised if it's doing some sort of "read to short" filtering step.
As in this answer, you have to fine-tuning the qblast to override the defaults. The WWW frontend of NCBI-BLAST automatically adjusts your parameters to match the short (8 bp) sequences, but if you do it through Biopython API you have to do it manually.