Adding numbers to variables in WebHarvest - webharvest

I should start off by saying that I'm very new to javascript.
I need to feed a bunch of urls to webharvest based on a number. It's a long story why, but my url structure looks kind of like this:
http://www.example.com/foo/bar?page=0. ?page= increases by 25 each step. So the next page will be http://www.example.com/foo/bar?page=25 then http://www.example.com/foo/bar?page=50 and so forth. There's a maximum, which I can set by another variable, call it ${maxpages}.
So what I need to do is modify a variable to feed into the otherwise fully predictable url, so that the variable has 25 added to it each time. I'm thinking of doing a while loop, like this:
<var-def name="pageNo">0</var-def>
<while condition="${pageNo} < ${maxpages}">
<body>
<html-to-xml><http url="${url}?${pageNo}"/></html-to-xml>
<var-def name="pageNo">
<var name="pageNo">[this is where I want to add 25]</var>
</var-def>
</body>
</while>
So I'm really unsure of the syntax here.
My questions are:
How can I check that my variable pageNo is less than maxpages in the while condition?
Can you add integers to variables in webharvest? How?

I think I'm developing a habit of answering myself.
After an hour of trial and error, I have this:
<var-def name="commentCount">432</var-def>
<var-def name="pageNo">0</var-def>
<while condition="${pageNo.toInt() < commentCount.toInt()}">
<html-to-xml>
<http url="http://www.example.com/foo/bar?${pageNo}"/>
</html-to-xml>
<var-def name="pageNo"><template>${pageNo.toInt() + 25}</template></var-def>
</while>

Related

In custom Instrument, how can I include the duration in a value

I have a custom Instrument with a os-signpost-interval-schema that captures a "state" string. I would like the final plot value to be <state>: <duration>, but I don't know how to get the duration into the string.
My working schema is the following, which just stores the state itself in the column:
<os-signpost-interval-schema>
<id>state-interval</id>
<title>State Interval</title>
<subsystem>"..."</subsystem>
<category>"..."</category>
<name>"state"</name>
<start-pattern>
<message>?state</message>
</start-pattern>
<column>
<mnemonic>state</mnemonic>
<title>State</title>
<type>string</type>
<expression>?state</expression>
</column>
</os-signpost-interval-schema>
I would like to change the expression in the column to (str-cat ?state ": " ?duration), but that fails with:
Variable '?duration' must appear in a pattern element to be used in a later expression.
I don't see any way to compute this later in the graph, lane, or plot. I've also tried explicitly creating a <duration-column>, but that doesn't seem to change anything.
The rest of the pieces include the table:
<create-table>
<id>state-table</id>
<schema-ref>state-interval</schema-ref>
</create-table>
And the lane, which I would like to display as <state>: <duration> rather than just the duration:
<lane>
<title>State</title>
<table-ref>state-table</table-ref>
<plot>
<value-from>state</value-from>
</plot>
</lane>
This turns out to be impossible. Apple does not expose duration as a variable. It can be solved by writing a custom modeler, though this adds a lot of complexity.

Xquery to map the value of a specific name attribute

I am trying to create an xquery in jdeveloper . I am stuck at a small portion of it. It would be great if I get some suggestions.
Below is the part I am stuck at
The request is:
`<variables>
<variable name="StartTime" value="01:00:00"/>
<variable name= "EndTime" value="05:00:00"/>
The response I want to map is a single element with two values looks like below:
<ns2:time ns2:startTime="01:00:00" ns2:endTime="05:00:00"/>
Below is the xquery I tried. But I get only the start time at both places. I want some way by which I can correctly assign the values looking at the name value in the request.
if (fn:data($Prefereces/ns1:variables/ns1:variable/#name="StartTime")or fn:data($Prefereces/ns1:variables/ns1:variable/#name="EndTime")) then
( <ns2:time ns2:startTime="{fn:data($Prefereces/ns1:variables/ns1:variable/#value)}" ns2:endTime="{fn:data($Prefereces/ns1:variables/ns1:variable/#value)}">
</ns2:time>)
else
()
Thanks in advance.
You can use this :
<ns2:time ns2:startTime="{fn:data($Prefereces/variables/variable[#name = 'StartTime']/#value)}" ns2:endTime="{fn:data($Prefereces/variables/variable[#name = 'EndTime']/#value)}"/>
Note that the ns2 prefix has to be defined beforehand.
XQuery's predicates are specified using brackets ([condition]), which were missing from your tries.

Transform portion of ETL using Scriptella?

I am trying out Scriptella to see if it will meet my needs. So far, it seems like a great tool. I've spent several hours studying sample scripts, searching forums, and trying to get the hang of nested queries/scripts.
This is an example of my ETL file, slightly cleaned up for brevity. Lines beginning with # are added and not part of the actual ETL file. I am trying to insert/retrieve IDs and then pass them on to later script blocks. The most promising way to do this appears to be using global variables but I'm getting null when trying to retrieve the values. Later, I will be adding code in the scripts blocks that parse and significantly transform fields before adding them into the DB.
There are no errors. I'm just not getting the OS ID and Category IDs that I'd expect. Thank you in advance.
<!DOCTYPE etl SYSTEM "http://scriptella.javaforge.com/dtd/etl.dtd">
<etl>
<connection id="in" driver="csv" url="mycsvfile.csv"/>
<connection id="dest" url="jdbc:mysql://localhost:3306/pvm3" user="user" password="password"/>
<connection id="js" driver="script"/>
<query connection-id="in">
<!-- all columns are selected, notably: OPERATINGSYSTEM, CATEGORY, QID, TITLE -->
<query connection-id="dest">
#Check to see if the OS already exists, and get the ID if it does
select max(os_id) as os_id, count(*) as os_cnt from etl_os where os = ?OPERATINGSYSTEM;
#If it doesnt exist then add it and get the auto_increment value
<script if="os_cnt==0">
insert into etl_os(os) values(?OPERATINGSYSTEM);
<query connection-id="dest">
select last_insert_id() as os_id;
#Store in global so it can be accessed in later script blocks
<script connection-id="js">
etl.globals.put('os_id', os_id);
</script>
</query>
</script>
#Same style select/insert as above for category_id (excluded for brevity)
#See if KB record exists by qid, if not then add it with the OS ID and category ID we got earlier
<query connection-id="dest">
select max(qid) as existing_qid, count(*) as kb_cnt from etl_qids where qid = ?QID
<script if="kb_cnt==0">
insert into etl_qids(qid, category_id, os_id) values (?QID, ?{etl.globals.get('category_id')}, ?{etl.globals.get('os_id')});
</script>
</query>
</query>
</query>
</etl>
Found out how to do it. Essentially, just nest queries to modify the data before passing it to a script. The below is a quick type-up of the solution. I did not understand at first that queries could be immediately nested to transform the row before passing it for processing. My impression was also that only scripts could manipulate the data.
(Query)Raw data -> (Query)manipulate data -> (Script)write new data.
.. in is a CSV file ..
.. js is a driver="script" block ..
<query connection-id="in">
<query connection-id="js">
//transform data as needed here
if (BASE_TYPE == '-') BASE_TYPE = '0';
if (SECONDARY_TYPE == '-') SECONDARY_TYPE = '0';
SIZES = SIZES.toLowerCase();
query.next(); //call nested scripts
<script connection-id="db">
INSERT IGNORE INTO sizes(size) VALUES (?SIZE);
INSERT IGNORE INTO test(base_type,secondary_type) VALUES (?BASE_TYPE, ?SECONDARY_TYPE);
</script>
</query>
</query>

docusign - adding tabs to the document using API: It only adds signhere tab and ignores all other

I am trying to add dateSingedTab, fullNameTab,signHereTab to my document. When I send the document, docusign is only adding the signHereTab and ignoring other tabs. Here is my xml. Can you please help.
<envelopeDefinition xmlns="http://www.docusign.com/restapi">
<emailSubject>API Call for adding signature request to document and sending</emailSubject>
<status>sent</status>
<documents>
<document>
<documentId>1</documentId>
<name>documentName</name>
</document>
</documents>
<recipients>
<signers>
<signer>
<recipientId>1</recipientId>
<email>custEmail</email>
<name>recipientNameCust</name>
<routingOrder>1</routingOrder>
<tabs>
<dateSignedTabs>
<dateSignedTab>
<anchorString>SIGNED on behalf of the Customer</anchorString>
<anchorXOffset>100</anchorXOffset>
<anchorYOffset>300</anchorYOffset>
<anchorIgnoreIfNotPresent>false</anchorIgnoreIfNotPresent>
<anchorUnits>Pixels</anchorUnits>
<documentId>1</documentId>
<recipientId>1</recipientId>
<tabLabel>Date Signed</tabLabel>
<name>Date Signed</name>
</dateSignedTab>
</dateSignedTabs>
<fullNameTabs>
<fullNameTab>
<anchorString>SIGNED on behalf of the Customer</anchorString>
<anchorXOffset>100</anchorXOffset>
<anchorYOffset>100</anchorYOffset>
<anchorIgnoreIfNotPresent>false</anchorIgnoreIfNotPresent>
<anchorUnits>Pixels</anchorUnits>
<documentId>1</documentId>
<recipientId>1</recipientId>
<tabLabel>Full Name</tabLabel>
<name>Full Name</name>
</fullNameTab>
</fullNameTabs>
<signHereTabs>
<signHere>
<anchorString>SIGNED on behalf of the Customer</anchorString>
<anchorXOffset>0</anchorXOffset>
<anchorYOffset>50</anchorYOffset>
<anchorIgnoreIfNotPresent>false</anchorIgnoreIfNotPresent>
<anchorUnits>Pixels</anchorUnits>
<documentId>1</documentId>
<recipientId>1</recipientId>
<tabLabel>Sign Here</tabLabel>
<name>Sign Here</name>
</signHere>
</signHereTabs>
</tabs>
</signer>
</signers>
</recipients></envelopeDefinition>
I got the tags wrong in above xml. I corrected them and they work fine. dateSignedTab is not correct, it should be dateSigned. Also fullNameTab is not correct, it should be fullName.
Please ignore this post.
You need to drop the tabs part of the inside tab names. For instance, for dateSigned tabs the <xml> nodes need to look like this:
<dateSignedTabs>
<dateSigned>
...
</dateSigned>
</dateSignedTabs>
Same thing with your other tabs... the full name tabs need to be setup like this:
<fullNameTabs>
<fullName>
...
</fullName>
</fullNameTabs>
Give that a try and it should work.

Grails: out vs return

An strange problem occourse on grails 1.2.4 on my machine only....
We are using an custom taglib which can be accessed from services via gspTagLibraryLookup-bean from AppContext.
On my local machine
<my:span value="abc" title="${my.write(text:'123')}"/>
writes:
123<span title="">abc</span> <!-- what i see -->
<span title="123">abc</span> <!-- what my collegue see -->
my:write is defied as:
def write = {out << attrs.text}
But: If i use return instead of out, the html generate what my collegue see.
Anyone know the difference?
Argument value for the title attribute is evaluated before passing it to <my:span>. So if you define write as {out << attrs.text}, and use it in <my:span>, the write function will write to out before span function does, and return nothing - so 123 will be written to output before <span>, and the title attribute will be empty.
If you define write as {return attrs.text}, its evaluation won't write anything to out, and return 123 which will be inserted as value of title attribute.
Not sure why the first definition works on your collegue's machine.

Resources