Liquibase doesn't commit changeSet with Ant - ant

I've already tried liquibase 3.0 beta with Spring( btw It didn't work with 2.0 ). It went well.
Now I'm testing it with Ant with the same database and changelog, after rollbacked all the changes and dropped table DATABASECHANGELOG and DATABASECHANGELOGLOCK.
The problem is liquibase logged all the changesets in table DATABASECHANGELOG , which means there isn't any error in my config , but It didn't commit the changes to the database.
Here is my changelog.xml file:
<?xml version="1.0" encoding="UTF-8"?>
<databaseChangeLog xmlns="http://www.liquibase.org/xml/ns/dbchangelog" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.liquibase.org/xml/ns/dbchangelog http://www.liquibase.org/xml/ns/dbchangelog/dbchangelog-3.0.xsd">
<preConditions>
<dbms type="mysql"/>
</preConditions>
<changeSet id="1" author="bob" runAlways="true">
<createTable tableName="department">
<column name="id" type="int">
<constraints primaryKey="true" nullable="false"/>
</column>
<column name="name" type="varchar(50)">
<constraints nullable="false"/>
</column>
<column name="active" type="boolean" defaultValueBoolean="true"/>
</createTable>
</changeSet>
<changeSet id="2" author="roger" runAlways="true">
<comment>test add column</comment>
<addColumn tableName="department">
<column name="header" type="varchar(8)"/>
</addColumn>
</changeSet>
<changeSet id="3" author="gabriel" runAlways="true">
<createTable tableName="records">
<column name="id" type="int" autoIncrement="true">
<constraints primaryKey="true" nullable="false"/>
</column>
<column name="title" type="varchar(50)"/>
</createTable>
</changeSet>
<changeSet id="4" author="gabriel" context="test" runAlways="true">
<insert tableName="records">
<column name="title" value="Liquibase 0.8 Released"/>
</insert>
<insert tableName="records">
<column name="title" value="Liquibase 0.9 Released"/>
</insert>
</changeSet>
<changeSet id="6" author="nvoxland" runAlways="true">
<comment>test update eam_company</comment>
<sql>update eam_company set companyName='haha' where companyId=15</sql>
</changeSet>
</databaseChangeLog>
Ant build.xml file:
<?xml version="1.0" encoding="utf-8"?>
<project name="ncpsys_v2" default="all">
<!-- define time stamp -->
<tstamp>
<format property="current.time" pattern="yyyy_MM_dd_HH_mm_ss"/>
</tstamp>
<!-- define varibles and path -->
<property file="liquibaseconf.properties"/>
<path id="ant.classpath">
<fileset dir="${ant.home}">
<include name="**/*.jar"/>
</fileset>
</path>
<target name="sync-database">
<fail unless="db.changelog.file">db.changelog.file not set</fail>
<fail unless="database.url">database.url not set</fail>
<fail unless="database.username">database.username not set</fail>
<fail unless="database.password">database.password not set</fail>
<taskdef resource="liquibasetasks.properties">
<classpath refid="ant.classpath"/>
</taskdef>
<changeLogSync changeLogFile="${db.changelog.file}" driver="${database.driver}" url="${database.url}" username="${database.username}" password="${database.password}" promptOnNonLocalDatabase="true" defaultSchemaName="root" classpathref="ant.classpath">
</changeLogSync>
</target>
<target name="all" depends="sync-database"/>
</project>
liquibasetasks.properties file
local.dir=.
#gwt.home=E:/work/libaray/gwt-2.3.0
jdk.home.1.6=C:/Program Files/Java/jdk1.6.0_10
ant_home=D:/software/apache-ant-1.8.3
tomcat.home=C:/Program Files/Apache Software Foundation/apache-tomcat-6.0.36
#liquibase config
database.driver=com.mysql.jdbc.Driver
database.url=jdbc:mysql://localhost:3306/ncpsys_v2?useUnicode=true&characterEncoding=UTF-8
database.username=root
database.password=1234
db.changelog.file=changelog.xml
And I run the Ant task, liquibase created table DATABASECHANGELOG and DATABASECHANGELOGLOCK for me, and inserted logs in DATABASECHANGELOG .
my build log:
sync-database:
[changeLogSync] INFO 13-4-2 下午1:22:liquibase: Successfully acquired change log lock
[changeLogSync] INFO 13-4-2 下午1:22:liquibase: Creating database history table with name: `ncpsys_v2`.`DATABASECHANGELOG`
[changeLogSync] INFO 13-4-2 下午1:22:liquibase: Reading from `ncpsys_v2`.`DATABASECHANGELOG`
[changeLogSync] INFO 13-4-2 下午1:22:liquibase: Reading from `ncpsys_v2`.`DATABASECHANGELOG`
[changeLogSync] INFO 13-4-2 下午1:22:liquibase: Successfully released change log lock
all:
BUILD SUCCESSFUL
Total time: 2 seconds
but the change didn't happened in the database. I didn't see the table RECORDS and DEPARTMENT been created and inserted with data. The update sql didn't apply neither.
Is there anything I did wrong ? Or is there a bug that haven't been fixed under version 3.0beta1 (oh .. I also try beta2, didn't work, and I got a chinese garbled character problem )
I posted this question on liquibase forum and got no answer, so I come to you guys.
Please, help me! I got all confused.

You're running the changeLogSync ANT task. Described as follows:
Marks all change sets as ran against the database. Useful when you have manually updated your database.
This explains why nothing was committed to your database. I think you should have used the updateDatabase task instead

Related

ColdFusion TestBox Ant error

My goal is to run TestBox scripts on Jenkins. But using the Ant script from
https://testbox.ortusbooks.com/content/running_tests/ant_runner.html
as a template, I get this error
BUILD FAILED
C:\public\data\trunk\AutomatedTesting\Box_Unit_Tests\build.xml:38: The reference to entity "bundles" must end with the ';' delimiter.
with this script:
<?xml version="1.0" encoding="UTF-8"?>
<project name="testbox-ant-runner" default="init" basedir=".">
<!-- THE URL TO THE RUNNER, PLEASE CHANGE ACCORDINGLY-->
<property name="basedir" value="C:\public\data\trunk\AutomatedTesting\Box_Unit_Tests" />
<property name="url.runner" value="C:\public\data\ColdBox\testbox\test-harness\runner.cfm?"/>
<!-- FILL OUT THE BUNDLES TO TEST, CAN BE A LIST OF CFC PATHS -->
<property name="test.bundles" value="http://localhost/application/testing/TestBox/Hello.cfc?method=runRemote" />
<!-- FILL OUT THE DIRECTORY MAPPING TO TEST -->
<property name="test.directory" value="test.specs" />
<!-- FILL OUT IF YOU WANT THE DIRECTORY RUNNER TO RECURSE OR NOT -->
<property name="test.recurse" value="true" />
<!-- FILL OUT THE LABELS YOU WANT TO APPLY TO THE TESTS -->
<property name="test.labels" value="" />
<!-- FILL OUT THE TEST REPORTER YOU WANT, AVAILABLE REPORTERS ARE: ANTJunit, Codexwiki, console, dot, doc, json, junit, min, raw, simple, tap, text, xml -->
<property name="test.reporter" value="simple" />
<!-- FILL OUT WHERE REPORTING RESULTS ARE STORED -->
<property name="report.dir" value="${basedir}\results" />
<property name="junitreport.dir" value="${report.dir}\junitreport" />
<target name="init" description="Init the tests">
<mkdir dir="${junitreport.dir}" />
<tstamp prefix="start">
<format property="TODAY" pattern="MM-dd-YYYY hh:mm:ss aa"/>
</tstamp>
<concat destfile="${report.dir}\Latestrun.log">Tests ran at ${start.TODAY}</concat>
</target>
<target name="run">
<get dest="${report.dir}/results.html"
src="${url.runner}&bundles=${test.bundles}&reporter=${test.reporter}"
verbose="true"/>
<-- Create fancy junit reports -->
<junitreport todir="${junitreport.dir}">
<fileset dir="${report.dir}">
<include name="TEST-*.xml"/>
</fileset>
<report format="frames" todir="${junitreport.dir}">
<param name="TITLE" expression="My Awesome TestBox Results"/>
</report>
</junitreport>
</target>
</project>
Any thoughts?

Liquibase gives different results from Ant and the command line

When I run Liquibase from the command line it populates the FILENAME column of DATABASECHANGELOG with the relative paths to the changelog files, as you'd hope. But when I run exactly the same changelog from exactly the same directory, using Ant, it populates the column with the absolute paths.
Among other things this means that the Ant and the command-line versions are not interoperable for me.
But I can't find anyone else having reported this problem, so I'm sure it's something I'm doing; something that I haven't set up correctly. I've seen some suggestions that the root directory of the changelog needs to be on the classpath, so I've included it in the Ant classpath, but it doesn't make any difference.
Here's my Ant build file:
<project name="Database Build" default="build" basedir="." xmlns:liquibase="antlib:liquibase.integration.ant">
<path id="liquibase.lib.path">
<fileset dir="liquibase/lib">
<include name="**/*.jar" />
</fileset>
<fileset dir="liquibase">
<include name="liquibase.jar" />
</fileset>
</path>
<path id="driver.classpath">
<filelist files="${classpath}" />
</path>
<path id="main.classpath">
<pathelement location="." />
<path refid="driver.classpath" />
</path>
<taskdef
resource="liquibase/integration/ant/antlib.xml"
uri="antlib:liquibase.integration.ant">
<classpath refid="liquibase.lib.path" />
</taskdef>
<liquibase:database
id="main-schema"
driver="${driver}"
url="${url}"
user="${username}"
password="${password}"
defaultSchemaName="${defaultSchemaName}"
/>
<target
name="build"
description="Builds the database based on values set in the properties file">
<echo message="Building DB..." />
<liquibase:updateDatabase
databaseref="main-schema"
changelogfile="${changeLogFile}"
classpathref="main.classpath"
logLevel="debug"
>
<!-- Here we're effectively passing the values set as Ant properties in as Liquibase parameters -->
<liquibase:changeLogParameters>
<liquibase:changeLogParameter name="main.schema" value="${defaultSchemaName}" />
<liquibase:changeLogParameter name="tablespace.data" value="${tablespace.data}" />
<liquibase:changeLogParameter name="tablespace.index" value="${tablespace.index}" />
<liquibase:changeLogParameter name="tablespace.long" value="${tablespace.long}" />
</liquibase:changeLogParameters>
</liquibase:updateDatabase>
</target>
<target
name="createSchema"
description="Create a schema on the database"
>
<echo>${toString:main.classpath}</echo>
<sql
driver="${driver}"
classpathref="main.classpath"
url="${url}"
userid="${username}"
password="${password}"
expandProperties="true"
>
<transaction>
CREATE SCHEMA ${defaultSchemaName};
</transaction>
</sql>
</target>
<target
name="createOracleUsers"
description="Create a user in Oracle"
>
<sql
rdbms="oracle"
print="true"
driver="${driver}"
classpathref="main.classpath"
url="${url}"
userid="${username}"
password="${password}"
expandProperties="true"
>
<transaction>
CREATE USER ${defaultSchemaName} IDENTIFIED BY ${defaultSchemaName}
default tablespace ${tablespace.data}
temporary tablespace TEMP quota unlimited on ${tablespace.data}
quota unlimited on ${tablespace.index};
GRANT create session, alter session, create sequence,
create table, create view to ${defaultSchemaName};
</transaction>
</sql>
</target>
</project>
Edited to add some changelog files.
Here's the root changelog file:
<?xml version="1.0" encoding="UTF-8"?>
<databaseChangeLog
xmlns="http://www.liquibase.org/xml/ns/dbchangelog"
xmlns:ext="http://www.liquibase.org/xml/ns/dbchangelog-ext"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://www.liquibase.org/xml/ns/dbchangelog http://www.liquibase.org/xml/ns/dbchangelog/dbchangelog-3.3.xsd
http://www.liquibase.org/xml/ns/dbchangelog-ext
http://www.liquibase.org/xml/ns/dbchangelog/dbchangelog-ext.xsd">
<include file="changes/sequences/sequences.xml" relativeToChangelogFile="true" />
<include file="changes/baseobjects/db-511.xml" relativeToChangelogFile="true" />
<include file="changes/data/data-511.xml" relativeToChangelogFile="true" />
</databaseChangeLog>
The first of the included ones starts like this:
<?xml version="1.0" encoding="UTF-8"?>
<databaseChangeLog
xmlns="http://www.liquibase.org/xml/ns/dbchangelog"
xmlns:ext="http://www.liquibase.org/xml/ns/dbchangelog-ext"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://www.liquibase.org/xml/ns/dbchangelog http://www.liquibase.org/xml/ns/dbchangelog/dbchangelog-3.3.xsd
http://www.liquibase.org/xml/ns/dbchangelog-ext
http://www.liquibase.org/xml/ns/dbchangelog/dbchangelog-ext.xsd">
<changeSet dbms="oracle,db2,db2i" author="mccallim (generated)" id="1419011907193-1">
<createSequence schemaName="${main.schema}" cacheSize="100" cycle="false" incrementBy="1" minValue="1" sequenceName="SEQ_ALLOWEDCURRENCIES" startValue="1"/>
</changeSet>
...
</databaseChangeLog>
It has lots of other sequences. The next one does the tables, indexes and views, and is pretty much as you'd expect.
It looks like a bug. I created https://liquibase.jira.com/browse/CORE-2290 to track the fix for 3.3.3

ant > map property names into a new set of properties

I'd like to "map" a bunch of ant properties, based on a prefix (sounds simple enough).
I have a solution, but it's not elegant (having to write out to a properties file, then read it back in!)
Question: Is there a quicker/more generic/simpler/out-of-the-box/straight-forward way of doing the below "load-propertyset" within ANT? (... than the example I've provided below)
(Roughly analogous to the Groovy > ConfigSlurper > Special Environment Configuration behaviour.)
For example:
<?xml version="1.0" encoding="UTF-8"?>
<project name="Config">
<!-- Section 1. (These will be loaded from a property file...) -->
<property name="a.yyy" value="foo" />
<property name="a.zzz" value="cat" />
<property name="b.xxx" value="bar" />
<property name="b.zzz" value="dog" />
<macrodef name="load-propertyset">
<attribute name="prefix" />
<attribute name="outfile" default="123" />
<attribute name="propset" default="123" />
<sequential>
<propertyset id="#{propset}">
<propertyref prefix="#{prefix}" />
<globmapper from="#{prefix}.*" to="*" />
</propertyset>
<echo level="debug">Created propertyset - '#{propset}' from prefix='#{prefix}'</echo>
<tempfile property="#{outfile}" suffix=".properties" deleteonexit="true" />
<echo level="debug">Writing propset to temp file - '${#{outfile}}'</echo>
<echoproperties destfile="${#{outfile}}">
<propertyset refid="#{propset}"/>
</echoproperties>
<echo level="debug">Reading props from temp file - '${#{outfile}}'</echo>
<property file="${#{outfile}}" />
<delete file="${#{outfile}}" />
</sequential>
</macrodef>
<load-propertyset prefix="a" />
<load-propertyset prefix="b" />
<echo>>>> Using variables xxx=${xxx} yyy=${yyy} zzz=${zzz}</echo>
</project>
I'm sure I'm missing something simple, for instance:
Can I reference properties within a propertyset? (e.g. ${myprops.yyy} ?)
I'd like to avoid something like ${${filter}.hostname}.
The third-party Ant-Contrib has an <antcallback> task that takes the <antcall> task and adds the ability to return properties to the caller:
<?xml version="1.0" encoding="UTF-8"?>
<project name="Config" default="run">
<taskdef resource="net/sf/antcontrib/antlib.xml" />
<!-- Section 1. (These will be loaded from a property file...) -->
<property name="a.yyy" value="foo" />
<property name="a.zzz" value="cat" />
<property name="b.xxx" value="bar" />
<property name="b.zzz" value="dog" />
<macrodef name="load-propertyset">
<attribute name="prefix" />
<attribute name="return" />
<sequential>
<antcallback target="-empty-target" return="#{return}">
<propertyset>
<propertyref prefix="#{prefix}" />
<globmapper from="#{prefix}.*" to="*" />
</propertyset>
</antcallback>
</sequential>
</macrodef>
<target name="-empty-target"/>
<target name="run">
<property name="properties-to-return" value="xxx,yyy,zzz"/>
<load-propertyset prefix="a" return="${properties-to-return}"/>
<load-propertyset prefix="b" return="${properties-to-return}"/>
<echo>>>> Using variables xxx=${xxx} yyy=${yyy} zzz=${zzz}</echo>
</target>
</project>
The properties-to-return concept is a bit of a maintenance burden. Luckily, <antcallback> doesn't fail when asked to return a property that wasn't set. Only the properties-to-return property needs to be modified when you want a new mapped property.

Configure Sonar for Delphi with sub-modules

I can get Delphi analysis working on single projects, so the plugin works.
I can get Submodules analysis set up as per my previous question but no delphi analysis is performed.
refer to Configuration of build.xml file for Sonar modules with Ant for reference.
If I include the (I believe) required delphi configuration in the master build file it only works if I include the sonar.sources line.
BUT, when I include that line it is attempting to analyse all delphi code and is ignorning the submodules.
How (assuming it can be done) do I configure this to work or do I have to run each project completely separately?
Master Build File
<?xml version="1.0" encoding="UTF-8"?>
<project name = "EXO" default = "sonar" basedir = "." xmlns:sonar="antlib:org.sonar.ant">
<echo>Root Project</echo>
<taskdef uri="antlib:org.sonar.ant" resource="org/sonar/ant/antlib.xml">
<classpath path="c:/ANT/lib" />
</taskdef>
<property name="sonar.host.url" value="http://localhost:9000" />
<property name="sonar.modules" value="exonet6000/build.xml,CRM/build.xml" />
<target name="sonar">
<sonar:sonar key="EXO.key" version="0.1">
<property key="sonar.sources" value="." />
<property key="sonar.language" value="delph" />
</sonar:sonar>
</target>
</project>
Sub Project Build File
<?xml version="1.0" encoding="UTF-8"?>
<project name="CRM" default="all" basedir=".">
<echo>CRM Module</echo>
<property name="sonar.language" value="delph" />
<property name="sonar.projectKey" value="EXO:CRM" />
<property name="sonar.sources" value="." />
<target name="all" />
</project>

Configuration of build.xml file for Sonar modules with Ant

I am setting up a Sonar project (using the delphi plugin), for simplicity sake assume there are two modules I want to report on.
Each module is in it's own sub-folder and each has it's own build.xml file.
At this point I can successfully run the sonar tasks and generate reports for each module as an independent project.
My problem is with configuring the "master" build.xml file.
The module build.xml file looks like this:
<?xml version="1.0"?>
<project name = "CRM" default = "sonar" basedir = ".">
<!-- Add the Sonar task -->
<taskdef uri="antlib:org.sonar.ant" resource="org/sonar/ant/antlib.xml">
<classpath path="c:/ANT/lib" />
</taskdef>
<target name="sonar">
<property name="sonar.projectKey" value="EXO:CRM" />
<property name="sonar.host.url" value="http://localhost:9000" />
<sonar:sonar workDir="." key="CRM.key" version="0.1" xmlns:sonar="antlib:org.sonar.ant">
<property key="sonar.sources" value="." /> <!-- project sources directories (required) -->
<property key="sonar.language" value="delph" /> <!-- project language -->
<property key="sonar.delphi.codecoverage.excluded" value=".\tests" /> <!-- code coverage excluded directories -->
<property key="sonar.importSources" value="true" /> <!-- should we show sources or not? -->
<property key="sonar.delphi.sources.excluded" value="" /> <!-- excluded directories -->
<property key="sonar.delphi.sources.include" value=".\includes" /> <!-- include directories, "," separated -->
<property key="sonar.delphi.sources.include.extend" value="true" /> <!-- should we extend includes in files? -->
</sonar:sonar>
</target>
</project>
The "Master" build.xml looks like this:
<?xml version="1.0"?>
<project name = "EXO" default = "sonar" basedir = ".">
<taskdef uri="antlib:org.sonar.ant" resource="org/sonar/ant/antlib.xml">
<classpath path="c:/ANT/lib" />
</taskdef>
<target name="sonar">
<property name="sonar.modules" value="exonet6000/build.xml,CRM/build.xml" />
<sonar:sonar workDir="." key="EXO.key" version="0.1" xmlns:sonar="antlib:org.sonar.ant">
<!-- project sources directories (required) -->
<property key="sonar.sources" value="." />
<property key="sonar.language" value="delph" />
<property key="sonar.importSources" value="true" />
<property key="sonar.delphi.sources.excluded" value="" />
<property key="sonar.delphi.sources.include" value=".\includes" />
<property key="sonar.delphi.sources.include.extend" value="true" />
</sonar:sonar>
</target>
</project>
It is always scanning all sources (required value) i.e. it is not respecting my modules.
The only way I can get this to work currently is by limiting the source code like this
<property key="sonar.sources" value="./crm/,./exonet6000/" />
I'm sure I must be mis-configuring something obvious here.
EDIT: I have now what I believe is a more consistent set of files based on examples here https://github.com/SonarSource/sonar-examples/tree/master/projects/languages/java/java-ant-modules
Master build file:
<?xml version="1.0" encoding="UTF-8"?>
<project name = "EXO" default = "sonar" basedir = "." xmlns:sonar="antlib:org.sonar.ant">
<echo>Root Project</echo>
<taskdef uri="antlib:org.sonar.ant" resource="org/sonar/ant/antlib.xml">
<classpath path="c:/ANT/lib" />
</taskdef>
<property name="sonar.host.url" value="http://localhost:9000" />
<property name="sonar.modules" value="exonet6000/build.xml,CRM/build.xml" />
<target name="sonar">
<sonar:sonar key="EXO.key" version="0.1">
</sonar:sonar>
</target>
</project>
and one of the submodule files
<?xml version="1.0" encoding="UTF-8"?>
<project name="CRM" default="all" basedir=".">
<echo>CRM Module</echo>
<property name="sonar.language" value="delph" />
<property name="sonar.projectKey" value="EXO:CRM" />
<property name="sonar.sources" value="." />
<target name="all" />
</project>
At this point the sonar process is completing successfully BUT no actual anlysis is being done. A key point is that I am not seeing the echo of the submodule so I suspect these build tasks are not actually running.
If you look at this sample project using Ant and multimodules, I'd say that you should not specify any property inside the tag in your master build.xml file, and let the submodules specify those properties.
I've got a project with many submodules. My sonar target looks like this:
<target name="sonar" depends="build.dependencies" description="collect code metrics">
<property name="m1" value="a/build.xml,b/build.xml,c/build.xml,d/build.xml"/>
... more properties defining modules...
<property name="sonar.modules" value="${m1},${m2},${m3},${m4},${m5},${m6}..."/>
<property name="sonar.projectName" value="MyProject"/>
<sonar:sonar key="my:project" version="${version}" xmlns:sonar="antlib:org.sonar.ant">
</sonar:sonar>
</target>
No property definitions in the sonar:sonar task at all, and it is working as I want it to.
Sources are defined in the submodule build.xml files. (Actually, in a base-build.xml that all our build.xml files include... but that's an Ant thing not directly related to Sonar.)

Resources