I have two databases with the same structure and I want to compare records between databases. The records in second database are copied from the first database, but the copying process sometime doesn't work and in the first database in one table I have more records than in the same table in the second database. So I want to know which records from the first database doesn't exists in the second database. I have tried with something like that:
<etl>
<connection id="db1" driver="auto"
url="jdbc:mysql://localhost:3306/db" user="user"
password="xxx"
classpath="C:/mysql-connector-java-5.1.20.jar" />
<connection id="db2" driver="auto"
url="jdbc:mysql://localhost:3307/db" user="user"
password="xxx"
classpath="C:/mysql-connector-java-5.1.20.jar" />
<connection id="text" driver="text" />
<query connection-id="db1">
SELECT * FROM table;
<query connection-id="db2">
SELECT * FROM table WHERE id = '$id';
<script connection-id="text">
sometext, $rownum
</script>
</query>
</query>
</etl>
The problem is, when the result of the query against db2 is empty the script is not executed.
How to solve this problem?
Regards,
Jacek
You can use count to check the actual number of records. In this case resultset will always return one row. Example:
<query connection-id="db1">
SELECT * FROM table;
<query connection-id="db2">
SELECT count(id) as CNT FROM table WHERE id = ?id;
<!-- The script is executed ONLY IF number of results is zero -->
<script connection-id="text" if="CNT==0">
No matching record for id $id
</script>
</query>
</query>
Probably he doesn't need the if condition, because his problem is the script wasn't executed ;)
Related
#i hope it can return some log, then continue insert
select *
from seerid.buddyIds
where id REGEXP
<foreach collection="idSplits" item="item" index="index" open="'" close="'" separator="|">
#{item}
</foreach>
</select>```
I have two databases in postgresql. I want to copy data from database to another based on a condition.I used scriptella but what i want is to copy rows when a column is not empty.But i always have the empty ones which are copied here what i did , i want to copy based on condition on a special column .
Here is the file
<!DOCTYPE etl SYSTEM "http://scriptella.javaforge.com/dtd/etl.dtd">
<etl>
<description>
test script
</description>
<connection id="in" driver="postgresql" url="jdbc:postgresql://localhost:5432/testMonoprix" user="postgres" password="maher" >
</connection>
<connection id="out" driver="postgresql" url="jdbc:postgresql://localhost:5432/testMonoprix2" user="postgres" password="maher">
</connection>
<query connection-id="in" >
SELECT * FROM public.param_type;
<script connection-id="out" if=" parent_param_type_id != null ">
INSERT INTO public.param_type VALUES (?1, ?2,?3,?4,?5,?6) ;
</script>
</query>
</etl>
How would the file be in order to copy non empty ones ,
Thanks
I have an XML file that looks like this:
<XML>
<Table name='test'>
<Row>
<Field name='key'>1000</Field>
<Field name='text'>Test</Field>
</Row>
</Table>
</XML>
id like to parse this xml and use it within an insert statement:
<query connection-id="in">
/XML/Table/Row
<script connection-id="out">
INSERT INTO X t (
t.entitykey,
t.text
)
VALUES
(
????????
);
</script>
</query>
How do I access a specific Field-Tag from within the insert statement using XPATH?
We prefer to have one XSD that takes all table layouts into account and not to maintain n xsd for each table hence the Field[#name] design.
Thanks
Matthias
Xpath driver exposes a variable called node which provides a context for executing xpath expressions over a currently returned node. You can use the following expression to get the value of a particular field:
<script connection-id="out">
INSERT INTO X t (t.entitykey, t.text)
VALUES ( ?{node.getString("./Field[#name = 'text']")} );
</script>
I am having a problem in date format conversion while I am exporting data from MySQL to CSV.
I am using scriptelaa.1.1, the latest one I guess.
Here is my etl.properties file:
driver=mysql
url=jdbc:mysql://localhost:3306/<my_DB_name>
user=<user_name>
password=<password>
classpath=/path/to/mysql-connector-java-5.1.19.jar;
here is my etl.xml file:
<!DOCTYPE etl SYSTEM "http://scriptella.javaforge.com/dtd/etl.dtd">
<etl>
<description>Scriptella ETL File Template.</description>
<properties>
<include href="/path/to/etl.properties"/> <!--Load from external properties file-->
</properties>
<!-- Connection declarations -->
<connection id="in" driver="${driver}" url="${url}" user="${user}" password="${password}" classpath="$classpath">
</connection>
<connection id="out" driver="csv" url="report.csv">
#Use empty quote to turn off quoting
quote=
null_string=\\N
format.dob.type=date
format.dob.pattern=yyyy-MM-dd HH:mm:ss
</connection>
<query connection-id="in">
SELECT * FROM test;
<script connection-id="out">
$1,$2,$3,$4
</script>
</query>
</etl>
dob is my column name in MySQL table, it is datetime type column there.
Now when I export the data from MySQL the time comes in the format yyyy-MM-dd HH:mm:ss.S
But I want yyyy-MM-dd HH:mm:ss, so I have used
format.dob.type=date
format.dob.pattern=yyyy-MM-dd HH:mm:ss
As it is suggested scriptella.1.1 has the feature and to use it, in the following link:
http://scriptella.javaforge.com/reference/index.html
But it is not working.
Can anyone help me out.
Thanks. :)
Formatting rules are applied to the variable name, therefore exactly the same variable name should be used for both format description and the expansion placeholder. In your cause, try using $dob instead of the column number.
YQL returns the number of records retrieved in its XML output:
<query xmlns:yahoo="http://www.yahooapis.com/v1/base.rng"
yahoo:count="2" yahoo:created="2012-08-24T14:02:32Z" yahoo:lang="en-US">
<diagnostics>
But I've been experimenting with my own custom Open Data Tables, at least ones which employ an execute block containing JavaScript to create the response, and no matter how I create the response the count field is always set to 1 when I make a query using the table.
I've also dug around in the documentation and can't seem to find anything addressing this.
Is this by design? Is it a bug? Have I missed something obvious?
This is commonly caused by only returning one result, which shouldn't really come as a surprise. The most usual cause of this, from my own experience, is forgetting to specify a suitable itemPath for the <select>.
Take the following examples, and see how the response.object structure and itemPath combine to give the query results.
Without an itemPath
<select itemPath="" produces="XML">
<execute>
<![CDATA[
response.object = <letters>
<letter>A</letter>
<letter>B</letter>
<letter>C</letter>
</letters>
]]>
</execute>
</select>
Produces a query result similar to:
<query xmlns:yahoo="http://www.yahooapis.com/v1/base.rng"
yahoo:count="1" yahoo:created="…" yahoo:lang="en-US">
<results>
<letters>
<letter>A</letter>
<letter>B</letter>
<letter>C</letter>
</letters>
</results>
</query>
With itemPath="letters"
<select itemPath="letters" produces="XML">
…
</select>
Produces a query result identical to the previous result.
<query xmlns:yahoo="http://www.yahooapis.com/v1/base.rng"
yahoo:count="1" yahoo:created="…" yahoo:lang="en-US">
<results>
<letters>
<letter>A</letter>
<letter>B</letter>
<letter>C</letter>
</letters>
</results>
</query>
With itemPath="letters.letter"
<select itemPath="letters.letter" produces="XML">
…
</select>
Note that now, the path now specifies a collection of letter items. This produces a query result similar to:
<query xmlns:yahoo="http://www.yahooapis.com/v1/base.rng"
yahoo:count="3" yahoo:created="…" yahoo:lang="en-US">
<results>
<letter>A</letter>
<letter>B</letter>
<letter>C</letter>
</results>
</query>