I have two databases in postgresql. I want to copy data from database to another based on a condition.I used scriptella but what i want is to copy rows when a column is not empty.But i always have the empty ones which are copied here what i did , i want to copy based on condition on a special column .
Here is the file
<!DOCTYPE etl SYSTEM "http://scriptella.javaforge.com/dtd/etl.dtd">
<etl>
<description>
test script
</description>
<connection id="in" driver="postgresql" url="jdbc:postgresql://localhost:5432/testMonoprix" user="postgres" password="maher" >
</connection>
<connection id="out" driver="postgresql" url="jdbc:postgresql://localhost:5432/testMonoprix2" user="postgres" password="maher">
</connection>
<query connection-id="in" >
SELECT * FROM public.param_type;
<script connection-id="out" if=" parent_param_type_id != null ">
INSERT INTO public.param_type VALUES (?1, ?2,?3,?4,?5,?6) ;
</script>
</query>
</etl>
How would the file be in order to copy non empty ones ,
Thanks
Related
I have taken over this ColdFusion project and found that I need a value out of a database field that includes HTML. The field data looks like this (without the new lines):
<wddxPacket version="1.0">
<header />
<data>
<struct>
<var name="en">
<string>3 Nights' Lodging</string>
</var>
<var name="sp">
<string>3 Noches alojamiento</string>
</var>
</struct>
</data>
</wddxPacket>
I am wanting to use this data but I only need the text between the:
<var name='en'><string>3 Nights' Lodging</string></var>
I used a function that ColdFusion has to remove HTML:
#REReplaceNoCase(pkg.title, "<[^><]*>", '', 'ALL')#
But when I use that, I get something like this:
3 Nights' Lodging3 Noches alojamiento
All I want is:
3 Nights' Lodging
Examining the beginning of the string, ie <wddxPacket ...> it is actually WDDX.
If you do a search for ColdFusion + WDDX you will find the documentation for CFWDDX. It is a built in tag which supports conversions of WDDX strings to CFML objects (and vice versa) for easier manipulation. In your case use action="wddx2cfml" to convert the string back into a CF structure.
<cfwddx action="wddx2cfml" input="#text#" output="result">
<cfdump var="#result#" label="Raw object">
Then use the key #result.en# to grab the string you want.
I have an XML file that looks like this:
<XML>
<Table name='test'>
<Row>
<Field name='key'>1000</Field>
<Field name='text'>Test</Field>
</Row>
</Table>
</XML>
id like to parse this xml and use it within an insert statement:
<query connection-id="in">
/XML/Table/Row
<script connection-id="out">
INSERT INTO X t (
t.entitykey,
t.text
)
VALUES
(
????????
);
</script>
</query>
How do I access a specific Field-Tag from within the insert statement using XPATH?
We prefer to have one XSD that takes all table layouts into account and not to maintain n xsd for each table hence the Field[#name] design.
Thanks
Matthias
Xpath driver exposes a variable called node which provides a context for executing xpath expressions over a currently returned node. You can use the following expression to get the value of a particular field:
<script connection-id="out">
INSERT INTO X t (t.entitykey, t.text)
VALUES ( ?{node.getString("./Field[#name = 'text']")} );
</script>
I am having a problem in date format conversion while I am exporting data from MySQL to CSV.
I am using scriptelaa.1.1, the latest one I guess.
Here is my etl.properties file:
driver=mysql
url=jdbc:mysql://localhost:3306/<my_DB_name>
user=<user_name>
password=<password>
classpath=/path/to/mysql-connector-java-5.1.19.jar;
here is my etl.xml file:
<!DOCTYPE etl SYSTEM "http://scriptella.javaforge.com/dtd/etl.dtd">
<etl>
<description>Scriptella ETL File Template.</description>
<properties>
<include href="/path/to/etl.properties"/> <!--Load from external properties file-->
</properties>
<!-- Connection declarations -->
<connection id="in" driver="${driver}" url="${url}" user="${user}" password="${password}" classpath="$classpath">
</connection>
<connection id="out" driver="csv" url="report.csv">
#Use empty quote to turn off quoting
quote=
null_string=\\N
format.dob.type=date
format.dob.pattern=yyyy-MM-dd HH:mm:ss
</connection>
<query connection-id="in">
SELECT * FROM test;
<script connection-id="out">
$1,$2,$3,$4
</script>
</query>
</etl>
dob is my column name in MySQL table, it is datetime type column there.
Now when I export the data from MySQL the time comes in the format yyyy-MM-dd HH:mm:ss.S
But I want yyyy-MM-dd HH:mm:ss, so I have used
format.dob.type=date
format.dob.pattern=yyyy-MM-dd HH:mm:ss
As it is suggested scriptella.1.1 has the feature and to use it, in the following link:
http://scriptella.javaforge.com/reference/index.html
But it is not working.
Can anyone help me out.
Thanks. :)
Formatting rules are applied to the variable name, therefore exactly the same variable name should be used for both format description and the expansion placeholder. In your cause, try using $dob instead of the column number.
I have two databases with the same structure and I want to compare records between databases. The records in second database are copied from the first database, but the copying process sometime doesn't work and in the first database in one table I have more records than in the same table in the second database. So I want to know which records from the first database doesn't exists in the second database. I have tried with something like that:
<etl>
<connection id="db1" driver="auto"
url="jdbc:mysql://localhost:3306/db" user="user"
password="xxx"
classpath="C:/mysql-connector-java-5.1.20.jar" />
<connection id="db2" driver="auto"
url="jdbc:mysql://localhost:3307/db" user="user"
password="xxx"
classpath="C:/mysql-connector-java-5.1.20.jar" />
<connection id="text" driver="text" />
<query connection-id="db1">
SELECT * FROM table;
<query connection-id="db2">
SELECT * FROM table WHERE id = '$id';
<script connection-id="text">
sometext, $rownum
</script>
</query>
</query>
</etl>
The problem is, when the result of the query against db2 is empty the script is not executed.
How to solve this problem?
Regards,
Jacek
You can use count to check the actual number of records. In this case resultset will always return one row. Example:
<query connection-id="db1">
SELECT * FROM table;
<query connection-id="db2">
SELECT count(id) as CNT FROM table WHERE id = ?id;
<!-- The script is executed ONLY IF number of results is zero -->
<script connection-id="text" if="CNT==0">
No matching record for id $id
</script>
</query>
</query>
Probably he doesn't need the if condition, because his problem is the script wasn't executed ;)
Hello I wasn't quite sure how to title this question, but I'll explain why I'm trying to do.
First of all, I have a folder that contains SQL scripts in a specific format, which is updateXtoY.sql, where X and Y are integers. What I need is to know which Y is the highest number. (basically, to know which is the latest script)
So if I have in my folder "scripts/" 3 files:
update3to5.sql
update2to5.sql
update1to6.sql
the result I need is to have assign a property 'latest.version' the value of 6.
From that point I can easily run the script. So the problem I have is 3-fold:
1- How to load the file names into a data structure.
2- How to iterate over the data structure.
3- How to evaluate each file name so that I can extract the "Y" part of the file and get the highest value. (I'm reading on regex right now)
I'm new to ANT and I'm not sure if this is possible and/or feasible.
Thanks for any suggestions.
The first part of the task - getting the filenames into a 'structure' is best done using a FileSet - say for SQL scripts in a directory called scripts:
<fileset dir="scripts" includes="*.sql" id="versions" />
That creates an Ant resource collection that can be referred to using the id versions.
The collection knows about your SQL script files.
Using (as you suggest) a regexp mapper, we can convert the set of files into a collection of strings, holding just the version parts from the filenames:
<mappedresources id="versions">
<fileset dir="scripts" includes="*.sql" />
<regexpmapper from="update.*to(.*).sql" to="\1" />
</mappedresources>
In this example versions now holds a 'list', which would be "5,5,6" for your example files.
It gets trickier now, because you probably need to carry out a numeric sort on what is a list of strings - to avoid 10 sorting as 'less' than 9. Ant ships with an embbeded Javascript interpreter, so you could use that to find the maximum. Another option would be to make use of the numeric sorting capability that ant-contrib has to offer.
Here's a Javascript 'max finder':
<scriptdef name="numeric_max" language="javascript">
<attribute name="property" />
<attribute name="resources_id" />
<![CDATA[
var iter = project.getReference(
attributes.get( "resources_id" )
).iterator( );
var max_n = 0.0;
while ( iter.hasNext() )
{
var n = parseFloat( iter.next() );
if ( n > max_n ) max_n = n;
}
project.setProperty( attributes.get( "property" ), max_n );
]]>
</scriptdef>
That defines a new Ant XML entity - numeric_max - that looks like a task, and can be used to find the numeric maximum of a collection of strings.
It's not perfect - there's no validation of the strings, and I've used floats rather than ints.
Combining that with the mappedresources above:
<mappedresources id="versions">
<fileset dir="scripts" includes="*.sql" />
<regexpmapper from="update.*to(.*).sql" to="\1" />
</mappedresources>
<numeric_max property="latest.version" resources_id="versions" />
<echo message="Latest SQL script version: ${latest.version}." />
When I run that with your three files I get:
[echo] Latest SQL script version: 6.