Okay, so right now I have an Active Choice Parameter, let's call it 'SKU' that is a list of SKUs a user may select from.
I have a Active Choices Reactive Parameter that is referencing the value of SKU and using that value to query files through the GitHub API to generate a list of titles.
When I run my script with a real, but hardcoded, value for the SKU it works.
When I run my script trying to call in the SKU parameter so that it is reactive to what the user originally picks -- using ${SKU} -- it goes to the fallback which I have set to just generate "error":
Groovy script (with hardcoding):
token = "value of token"
def command = """
curl -H 'Authorization: token ${token}' -H "Accept: application/vnd.github.VERSION.raw" https://api.github.com/repos/org/SKU/path/SKU-File.xml
"""
process = [ 'bash', '-c', command].execute()
parser = new XmlSlurper()
def rootNode = parser.parseText(process.text)
def count = rootNode.children().size() - 1
titles = []
for (i in 1..count) {
title = rootNode.children()[i].title
titles += title
}
return titles
Note: This works -- with the exception that instead of switching the list of titles depending on the SKU you select, it is always the same list from the hardcoded SKU.
Groovy script (without hardcoding):
token = "value of token"
def command = """
curl -H 'Authorization: token ${token}' -H "Accept: application/vnd.github.VERSION.raw" https://api.github.com/repos/org/${SKU}/path/${SKU}-File.xml
"""
process = [ 'bash', '-c', command].execute()
parser = new XmlSlurper()
def rootNode = parser.parseText(process.text)
def count = rootNode.children().size() - 1
titles = []
for (i in 1..count) {
title = rootNode.children()[i].title.toString()
titles += title
}
return titles
Note: This fails and defaults to the Fallback script.
Another important note: This is working when I try the script console, however, since the script console is not able to reference the other parameters from my project, I have to harcode in a SKU = "SKU" variable, so not sure it's as equivalent as I'd like it to be.
Fallback script:
return ["error"]
Note also, I have another Active Choice Reactive Parameter also pointing to the SKU and it is working just fine. But in that I'm not calling the SKU in a curl command. I can't quite figure out what's going wrong here, or if I'm referencing something inappropriately.
I've tried converting the ${SKU} parameter to a string as a variable before passing it to the command variable and that didn't change the outcome.
So here's the answer:
Not all of my SKUs were accurately pulling titles based on the same criteria as the SKU I was hardcoding.
It's a bit strange that an error on one SKU would pull apart the entire ability to get an accurate list -- e.g. if SKU1 had an error, toggling to SKU2 gave an error even if SKU2 was working while hardcoded. I've adjusted my script to work for both SKU1 and SKU2 and now it is working as intended.
This is a 'gotcha' to look out for!
Related
I generate an object with parameters. It's a mix of custom ones that I create (no $) or the SDK that I have integrated generated a bunch with $:
{$campaign_id: 471764, $campaign_name: TPUSA, $channel: email, $first_match: false, $first_match_link: false, $guaranteed_match: true, $link_id: 504455, $link_name: TPUSA Demo, $reinstall: false, $suspicious: false, $token: tpusademo, creatorID: timfong888, referralID: tpusa-demo}
I am able to access the params without $ but get one with it:
socialdata.creator.value = globalReferralData.linkParams['creatorID'];
socialdata.campaign.value = globalReferralData.linkParams['$campaign_id'];
Is there something I need to do to access the values that have $ without an error (I get an error for $campaign_id.
You can either escape the $ by putting \ before the $ like: '\$campaign_id'.
Or specify you want a raw string where every character should be read as they are written by putting an r before the string definition like: r'$campaign_id'
I'm trying to make templates for my dahboards, and I have problems when it comes to referring to measuerment names.
My variables:
$space = SHOW MEASUREMENTS
Then I would like a variable that contains only values from a specific $space, which is actually a MEASUREMENT:
$app = SHOW TAG VALUES WITH KEY = "Application" WHERE MEASUREMENT =~ /^$space$/
Here I get a message: Template variables could not be initialized: error parsing query: found MEASUREMENT, expected identifier, string, number, bool at line 1, char 48
In the official example it is like this, though it refers to another tag:
$datacenter = SHOW TAG VALUES WITH KEY = "datacenter"
$host = SHOW TAG VALUES WITH KEY = "hostname" WHERE "datacenter" =~ /^$datacenter$/
I cannot find any info how to refer to MEASUREMENTS which would work. WHERE, WITH, etc.. Maybe is it not possible at all?
I found only this in the official tutorial, but this is for keys, not values.
SHOW TAG KEYS [FROM <measurement_name>]
I actually figured it out:
SHOW TAG VALUES FROM /^$space$/ WITH KEY = "Application"
When I try regexner it works as expected with the following settings and data;
props.setProperty("annotators", "tokenize, cleanxml, ssplit, pos, lemma, regexner");
Bachelor of Laws DEGREE
Bachelor of (Arts|Laws|Science|Engineering|Divinity) DEGREE
What I would like to do is that using TokenRegex. For example
Bachelor of Laws DEGREE
Bachelor of ([{tag:NNS}] [{tag:NNP}]) DEGREE
I read that to do this, I should use TokensregexNERAnnotator.
I tried to use it as follows, but it did not work.
Pipeline.addAnnotator(new TokensRegexNERAnnotator("expressions.txt", true));
Or I tried setting annotator in another way,
props.setProperty("annotators", "tokenize, cleanxml, ssplit, pos, lemma, tokenregexner");
props.setProperty("customAnnotatorClass.tokenregexner", "edu.stanford.nlp.pipeline.TokensRegexNERAnnotator");
I tried to different TokenRegex formats but either annotator could not find the expression or I got SyntaxException.
What is the proper way to use TokenRegex (query on tokens with tags) on NER data file ?
BTW I just see a comment in TokensRegexNERAnnotator.java file. Not sure if it is related pos tags does not work with RegexNerAnnotator.
if (entry.tokensRegex != null) {
// TODO: posTagPatterns...
pattern = TokenSequencePattern.compile(env, entry.tokensRegex);
}
First you need to make a TokensRegex rule file (sample_degree.rules). Here is an example:
ner = { type: "CLASS", value: "edu.stanford.nlp.ling.CoreAnnotations$NamedEntityTagAnnotation" }
{ pattern: (/Bachelor/ /of/ [{tag:NNP}]), action: Annotate($0, ner, "DEGREE") }
To explain the rule a bit, the pattern field is specifying what type of pattern to match. The action field is saying to annotate every token in the overall match (that is what $0 represents), annotate the ner field (note that we specified ner = ... in the rule file as well, and the third parameter is saying set the field to the String "DEGREE").
Then make this .props file (degree_example.props) for the command:
customAnnotatorClass.tokensregex = edu.stanford.nlp.pipeline.TokensRegexAnnotator
tokensregex.rules = sample_degree.rules
annotators = tokenize,ssplit,pos,lemma,ner,tokensregex
Then run this command:
java -Xmx8g edu.stanford.nlp.pipeline.StanfordCoreNLP -props degree_example.props -file sample-degree-sentence.txt -outputFormat text
You should see that the three tokens you wanted tagged as "DEGREE" will be tagged.
I think I will push a change to the code to make tokensregex link to the TokensRegexAnnotator so you won't have to specify it as a custom annotator.
But for now you need to add that line in the .props file.
This example should help in implementing this. Here are some more resources if you want to learn more:
http://nlp.stanford.edu/software/tokensregex.shtml#TokensRegexRules
http://nlp.stanford.edu/nlp/javadoc/javanlp/edu/stanford/nlp/ling/tokensregex/SequenceMatchRules.html
http://nlp.stanford.edu/nlp/javadoc/javanlp/edu/stanford/nlp/ling/tokensregex/types/Expressions.html
I can't find any reference to an API that enables Rest API clients to export an existing Google Sheet to a csv file.
https://developers.google.com/sheets/
I believe there should be a way to export them.
The following URL gives you the CSV of a Google spreadsheet per sheet. The sheet must be accessible by the public, by anyone with the link (unlisted).
The parameters you need to provide are:
sheet ID (that is simply the ID in the URL of a Google Spreadsheet https://docs.google.com/spreadsheets/d/{{ID}}/edit)
sheet name (that is simply the name of the sheet as given by the user)
https://docs.google.com/spreadsheets/d/{{ID}}/gviz/tq?tqx=out:csv&sheet={{sheet_name}}
With that URL you can run a GET-request to fetch the CSV.
Or paste it in your browser address bar.
You can use the Drive API to do this today -- see https://developers.google.com/drive/v3/web/manage-downloads#downloading_google_documents, however that will limit you to the first sheet of the document. The Sheets API doesn't expose exporting as CSV today, but may offer it in the future.
Nobody's mentioned gspread yet, so here's how I did it:
#open sheet
sheet = gc.open_by_key(sheet_id)
#select worksheet
worksheet = sheet.get_worksheet(0)
#download values into a dataframe
df = pd.DataFrame(worksheet.get_all_records())
#save dataframe as a csv, using the spreadsheet name
filename = sheet.title + '.csv'
df.to_csv(filename, index=False)
Firstly you should make document accessible for anyone. Then you get url. From this url you should extract long id composed from big and small letters and numbers. Then use this script.
#!/bin/bash
long_id="id_assigned_to_your_document"
g_id="number_assigned_to_card_in_google_sheet"
wget --output-document=temp.csv "https://docs.google.com/spreadsheets/d/$long_id/export?gid=$g_id&format=csv&id=$long_id"
If you use only one card in document, their number is: g_id="0"
The problem you will probably have is connected with strange spaces in obtained file. I use this second script to process it
#!/bin/bash
#Delete all lines beginning with a # from a file
#http://stackoverflow.com/questions/8206280/delete-all-lines-beginning-with-a-from-a-file
sed '/^#/ d' temp.csv |
# reomve spaces
# http://stackoverflow.com/questions/9953448/how-to-remove-all-white-spaces-from-a-given-text-file
tr -d "[:blank:]" |
# regexp "1,2" into 1.2
# http://www.funtoo.org/Sed_by_Example,_Part_2
sed 's/\"\([−]\?[0-9]*\),\([0-9]*\)\"/\1.\2/g' > out.csv
Update
As Sam mentioned, api is better solution. There is now great documentation on address:
https://developers.google.com/sheets/quickstart/php
With example that generate output having CSV structure.
If you don't have easy access to or familiarity with PHP, here's a very barebones Google Apps Script Web App that once deployed and the caller permission accepted, should allow clients with an appropriately scoped access token or api key to export an existing Google Sheet to a csv file. It takes a Google Sheets spreadsheet id and sheet name (and optional download filename) as query parameters, and returns the corresponding theoretically RFC 4180 compliant CSV file.
Further instructions on deploying an Apps Script project as a web app are here: https://developers.google.com/apps-script/guides/web#deploying_a_script_as_a_web_app.
You can deploy it and test it out easily in the browser just by visiting the "Current web app URL" (as provided when you publish as web app from the script editor), and accepting the consent screen, or even just visit the one that I deployed (configured to execute as the accessing user, and unverified/scary consent) at the example URL.
The tricky part (as usual) is getting the OAuth token or API key set up, but if you're already calling the Google Sheets V4 API, you've probably already got that dialed in. I used CURL to make sure that it behaved as a REST api, but the technique I used to get an OAuth token there is both a distraction and frankly a little scary to include here since it's really easy to mess up. If you don't already have a way to get one, that's probably a good topic for a separate SO question in any case.
One related (and big!) caveat: I'm not 100% sure how the consent and verification interact with a pure Rest client (i.e. how that works if you DON'T visit this in the browser first...), and/or whether this script would need to be in the same GCP project as the other code that uses the Sheets API. If there's interest, and/or it doesn't work right out of the box, please let me know and I'll happily dig deeper and follow up.
// Example URL, assuming:
// "Current web app URL": https://script.google.com/a/tillerhq.com/macros/s/AKfycbyZlWAW6bpCpnFoPjbdjznDomFRbTNluG4siCBMgOy2qU2AGoA/exec
// spreadsheetId: 1xNDWJXOekpBBV2hPseQwCRR8Qs4LcLOcSLDadVqDA0E
// sheet name: Sheet1
// (optional) filename: mycsv.csv
//
// https://script.google.com/a/tillerhq.com/macros/s/AKfycbyZlWAW6bpCpnFoPjbdjznDomFRbTNluG4siCBMgOy2qU2AGoA/exec?spreadsheetid=1xNDWJXOekpBBV2hPseQwCRR8Qs4LcLOcSLDadVqDA0E&sheetname=Sheet1&filename=mycsv.csv?spreadsheetid=1xNDWJXOekpBBV2hPseQwCRR8Qs4LcLOcSLDadVqDA0E&sheetname=Sheet1&filename=mycsv.csv
//
var REQUIRED_PARAMS = [
'spreadsheetid', // example: "1xNDWJXOekpBBV2hPseQwCRR8Qs4LcLOcSLDadVqDA0E"
'sheetname' // Case-sensitive; example: "Sheet1"
];
// Returns an RFC 4180 compliant CSV for the specified sheet in the specified spreadsheet
function doGet(e) {
REQUIRED_PARAMS.forEach(function(requiredParam) {
if (!e.parameters[requiredParam]) throw new Error('Missing required parameter ' + requiredParam);
});
var spreadsheet = SpreadsheetApp.openById(e.parameters.spreadsheetid);
var sheet = spreadsheet.getSheetByName(e.parameters.sheetname);
if (!sheet) throw new Error("Could not find sheet " + e.parameters.sheetname + " in spreadsheet " + e.parameters.spreadsheetid);
var filename = e.parameters.filename || (spreadsheet.getName() + "_" + e.parameters.sheetname + ".csv");
var numRows = sheet.getLastRow();
var numColumns = sheet.getLastColumn();
var values = sheet.getSheetValues(1, 1, numRows, numColumns);
function quote(s) {
s = s.toString();
if ((s.indexOf("\r") == -1)
&& (s.indexOf("\n") == -1)
&& (s.indexOf(",") == -1)
&& (s.indexOf("\"") == -1)) return s;
// Fields containing line breaks (CRLF)*, double quotes, and commas should be enclosed in double-quotes;
// anything other than that we already returned, so if we get here -- escape it and quote it.
// *That's what the text of the RFC says, but the ABNF (...and Excel) treat EITHER CR or LF as requiring quotes.
// Replace any double quote with a double double quote, and wrap the whole thing in quotes
return "\"" + s.replace(/"/g, '""') + "\"";
};
var csv = values.map(function(row) {
return row.map(quote).join();
}).join("\r\n") + "\r\n";
return ContentService
.createTextOutput(csv)
.setMimeType(ContentService.MimeType.CSV)
.downloadAsFile(filename);
}
I need to read a file where the content is like below :
Computer Location = afp.local/EANG
Description = RED_TXT
Device Name = EANG04W
Domain Name = afp.local
Full Name = Admintech
Hardware Monitoring Type = ASIC2
Last Blocked Application Scan Date = 1420558125
Last Custom Definition Scan Date = 1348087114
Last Hardware Scan Date = 1420533869
Last Policy Sync Date = 1420533623
Last Software Scan Date = 1420533924
Last Update Scan Date = 1420558125
Last Vulnerability Scan Date = 1420558125
LDAP Location = **CN=EANG04W**,OU=EANG,DC=afp,DC=local
Login Name = ADMINTECH
Main Board OEM Name = Dell Inc.
Number of Files = 384091
Primary Owner = **CN= LOUHICHI anoir**,OU=EANG,DC=afp,DC=localenter code here
I need to replace CN=$value by CN=Compagny where $value is what is retrived after CN= and before ,.
Ok, so you really should have updated your question an not posted the code in a comment, because it's really hard to read. Here's what I think you intended:
$file = 'D:\sources\scripts\2.txt'
$content = Get-Content $file | foreach ($line in $content) {
if ($line.Contains('CN=')) {
$variable = $line.Split(',').Split('=')[2]
$variable1 = $variable -replace $variable, "Compagny"
} Set-Content -path $file
}
That deffinately has some syntax errors. The first line is great, you define the path. Then things go wrong... Your call to Get-Content is fine, that will get the contents of the file, and send them down the pipe.
You pipe that directly into a ForEach loop, but it's the wrong kind. What you really want there is a ForEach-Object loop (which can be confusing, because it can be shortened to just ForEach when used in a pipeline like this). The ForEach-Object loop does not declare an internal variable (such as ($line in $content)) and instead the scriptblock uses the automatic variable $_. So your loop needs to become something like:
Get-Content $file | ForEach { <do stuff> } | Set-Content
Next let's look inside that loop. You use an If statement to see if the line contains "CN=", understandable, and functional. If it does you then split the line on commas, and then again on equals, selecting the second record. Hm, you create an array of strings anytime you split one, and you have split a string twice, but only specify which record of the array you want to work with for the second split. That could be a problem. Anyway, you assign that substring to $variable, and proceed to replace that whole thing with "company" and store that output to $variable1. So there's a couple issues here. Once you split the string on the commas you have the following array of strings:
"LDAP Location = **CN=EANG04W**"
"OU=EANG"
"DC=afp"
"DC=local"
That's an array with 4 string objects. So then you try to split at least one of those (because you don't specify which one) on the equals sign. You now have an array with 4 array objects, where each of those has 2 string objects:
("LDAP Location", "**CN", "EANG04W**")
("OU", "EANG")
("DC","afp")
("DC","local")
You do specify the third record at this point (arrays in PowerShell start at record 0, so [2] specifies the third record). But you didn't specify which record in the first array so it's just going to throw errors. Let's say that you actually selected what you really wanted though, and I'm guessing that would be "EANG04W". (by the way, that would be $_.Split(",")[0].Split("=")[1]). You then assign that to $Variable, and proceed to replace all of it with "Company", so after PowerShell expands the variable it would look like this:
$variable1 = "EANG04W" -replace "EANG04W", "company"
Ok, you just successfully assigned "company" to a variable. And your If statement ends there. You never output anything from inside your If statement, so Set-Content has nothing to set. Also, it would set that nothing for each and every line that is piped to the ForEach statement, re-writing the file each time, but fortunately for you the script didn't work so it didn't erase your file. Plus, since you were trying to pipe to Set-Content, there was no output at the end of the pipeline, you have assigned absolutely nothing to $content.
So let's try and fix it, shall we? First line? Works great! No change. Now, we aren't saving anything in a variable, we just want to update a file's content, so there's no need to have $Content = there. We'll just move on then, shall we? We pipe the Get-Content into a ForEach loop, just like you tried to do. Once inside the ForEach loop, we're going to do things a bit differently though. The -replace method performs a RegEx match. We can use that to our advantage here. We will replace the text you are interested in for each line, and if it's not found, no replacement will be made, and pass each line on down the pipeline. That will look something like this for the inside of the ForEach:
$_ -replace "(<=CN\=).*?(?=,)", "Company"
The breakdown of that RegEx match can be seen here: https://regex101.com/r/gH6hP2/1
But, let's just say that it looks for text that has 'CN=' immediately before it, and goes up to the first comma following it. In your example, that includes the two trailing asterisks, but it doesn't touch the leading ones. Is that what you intended? That would make the last line of your example file:
Primary Owner = **CN=Company,OU=EANG,DC=afp,DC=localenter code here
Well, if that is as intended, then we have a winner. Now we close out the ForEach loop, and pipe the output to Set-Content and we're all set! Personally, I would highly suggest outputting to a new file, in case you need to reference the original file for some reason later, so that's what I'm going to do.
$file = 'D:\sources\scripts\2.txt'
$newfile = Join-Path (split-path $file) -ChildPath ('Updated-'+(split-path $file -Leaf))
Get-Content $file | ForEach{$_ -replace "(?<=CN\=).*?(?=,)", "Company"} | Set-Content $newfile
Ok, that's it. That code will produce D:\sources\scripts\Updated-2.txt with the following content:
Computer Location = afp.local/EANG
Description = RED_TXT
Device Name = EANG04W
Domain Name = afp.local
Full Name = Admintech
Hardware Monitoring Type = ASIC2
Last Blocked Application Scan Date = 1420558125
Last Custom Definition Scan Date = 1348087114
Last Hardware Scan Date = 1420533869
Last Policy Sync Date = 1420533623
Last Software Scan Date = 1420533924
Last Update Scan Date = 1420558125
Last Vulnerability Scan Date = 1420558125
LDAP Location = **CN=Company,OU=EANG,DC=afp,DC=local
Login Name = ADMINTECH
Main Board OEM Name = Dell Inc.
Number of Files = 384091
Primary Owner = **CN=Company,OU=EANG,DC=afp,DC=localenter code here