Why would MST not include files with different content - ant

Question

Trying to make a multilingual installer - the process is working in general, but seems to be failing for localised files that are to be installed.
Each of the localised installers works fine as a standalone and installs the localised files like the eula.pdf.
I'm hoping that I'm just missing a flag on the transform generation step (or maybe using the wrong tool?)
Process being to start by having a bunch of installers 1 for each language.
To build each installer we are 'lighting' using -b "folder" and -loc "folder" option to specify each language which includes some alternate file content such as the licence.pdf.
The files to be localised have a common source name
<File Id='License.pdf' Name='eula.pdf' Source='License(EULA).pdf' KeyPath='yes'/>
<WixVariable Id="WixUILicenseRtf" Value="License.rtf" />
Folders for example being
en-US/License(EULA).pdf
en-US/License.rtf
en-US/Product.wxl
fr-FR/License(EULA).pdf
fr-FR/License.rtf
fr-FR/Product.wxl
There are also some files such as binary.dll and binary.exe which are not localised and are the same for all msi - don't expect to see them in MST.
Following the next step in the process is creating an MST diff between a base language (english) and each of the other languages. Using Windows\v7.1\Bin\MsiTran.exe from the Window SDK
The MST seem a bit small for the change in content.
Merging all the MST files into a single installer using the Windows\v7.1\Samples\sysmgmt\msi\scripts\wisubstg.vbs
Installing in a language other than english shows the whole installer UI including rtf version of the licence as being localised, but the eula.pdf on disk is always the base english.
Using the Ant-dotnet tasks to run the build (in case it makes a difference)
The msi build task
<wix mode="light" target="${outlocation}\${lang.folder}\my.msi" wixHome="${wixhome}">
<lightArg line="-b "${location}""/> <!-- provide the location of the signable binaries -->
<lightArg line="-b "${msiwixsource}\Localisation\${lang.folder}""/> <!-- provide the location of the localisation -->
<lightArg line="-sice:ICE57"/>
<lightArg line="-cultures:${lang.culture}"/>
<lightArg line="-loc "${msiwixsource}\Localisation\${lang.folder}\Product.wxl""/>
<lightArg line="-ext "${wixhome}\WixUtilExtension.dll""/>
<lightArg line="-ext "${wixhome}\WixUIExtension.dll""/>
<lightArg line="-ext "${wixhome}\WixFirewallExtension.dll""/>
<lightArg line="-ext "${wixhome}\WixNetFxExtension.dll""/>
<sources dir="${msiwixobjects}">
<include name="*.wixobj"/>
</sources>
<moresources dir="${msiwixsource}\Localisation\${lang.folder}">
<include name="*"/>
</moresources>
<moresources dir="${location}">
<include name="binary.dll,binary.exe"/>
</moresources>
</wix>
The transform task
<exec executable="${windowsSDKBin}">
<arg value="-g"/>
<arg value="${outlocation}\en-US\my.msi"/>
<arg value="${outlocation}\${lang.folder}\my.msi"/>
<arg value="${outlocation}\${lang.folder}\my.mst"/>
</exec>
The repack task
<exec executable="cscript">
<arg value="${windowsSDKMsi}"/>
<arg value="${outlocation}\my.msi"/>
<arg value="${outlocation}\${lang.folder}\my.mst"/>
<arg value="${lang.id}"/>
</exec>
Product/#Language="!(loc.Lang)" specified in Product.wxl
for English base we listed all 1033,1028,1029,1031,1036,1040,1041,1043,1045,1046,2052,3082 for other languages just the specific such as 1036 for fr-FR

Answer 1

Ok, so after a couple of days of searching I found an article that describes the issue.
How To Localize the Setup GUI and Deliver Localized Files Using a MST.
The issue being that MST doesn't include '_Streams' table elements which is where the CAB files are stored.
You could go several ways from that -
1) make multiple file & component entries for the different languages with conditional inclusion based on language or culture
This means changing your wxs if you want to change which languages are supported.
<Component Directory="INSTALLDIR" Id='EULADoc' Guid='***'>
<Condition><![CDATA[(ProductLanguage = "1033")]]></Condition>
<File Id='License.pdf' Name='!(loc.EULA)' Source='en-US\License(EULA).pdf' KeyPath='yes'/>
</Component>
<Component Directory="INSTALLDIR" Id='EULADoc' Guid='***'>
<Condition><![CDATA[(ProductLanguage = "1046")]]></Condition>
<File Id='License.pdf' Name='!(loc.EULA)' Source='pt-BR\License(EULA).pdf' KeyPath='yes'/>
</Component>
... add a new block each time you add a language, changes to files have to be duplicated for every block.
Which in turn means recompiling instead of just relinking the wxl. You still need to repack the mst files.
2) manually adjust the localised MSI
As per the How To Localize the Setup GUI and Deliver Localized Files Using a MST.
3) automated split the localised items into a seperate CAB with a localised name
Rather than recompile this is then relinking and extra process for managing the CAB files.
Extra process to -
* extract the CAB file(s) along with generating the mst,
* then when merging the mst also add the cab files.
Add an extra media which has a localised name - I'm using the lang id as it is convenient
<Media Id="2" Cabinet="localised_!(loc.LANG).cab" EmbedCab="yes" />
Change the localised files/components to come from the new media
<Component Directory="INSTALLDIR" Id='EULADoc' Guid='***' DiskId='2'>
<File Id='License.pdf' Name='!(loc.EULA)' Source='License(EULA).pdf' KeyPath='yes'/>
</Component>
When generating the transforms using MSITran.exe also export the localised CAB files using MsiDb.Exe
When merging the transforms into the base MSI using WiSubStg.vbs also add the CAB files to the _Streams using WiStram.vbs
I've gone with option 3
I hope somebody else finds this useful.
Note:
I don't write alot of ANT so feel free to offer suggestions
we build an unsigned for testing and signed version for release so need to adjust the input/output locations to find binaries and msi.
and the ant build looks something like the following.
<taskdef resource="net/sf/antcontrib/antcontrib.properties" classpathref="buildjars"/>
<taskdef name="for" classname="net.sf.antcontrib.logic.ForTask" classpathref="buildjars"/>
<taskdef name="wix" classname="org.apache.ant.dotnet.wix.WixTask" classpathref="buildjars"/>
<property name="wixhome" value="C:\Program Files (x86)\Windows Installer XML v3.5\bin"/>
<property name="binarieslocation" value="build"/>
<property name="msiwixsource" value="install"/>
<property name="msiwixobjects" value="${msiwixsource}\obj\x64\Release"/>
<!-- <dirname property="windowsSDK" file="C:\Program Files\Microsoft SDKs\Windows\v7.1\"/> Ant doesn't like path segments with . -->
<property name="windowsSDKBin" location="C:\Program Files\Microsoft SDKs\Windows\v7.1\Bin\MsiTran.exe"/>
<property name="windowsSDKCab" location="C:\Program Files\Microsoft SDKs\Windows\v7.1\Bin\MsiDb.Exe"/>
<property name="windowsSDKMsiStorage" location="C:\Program Files\Microsoft SDKs\Windows\v7.1\Samples\sysmgmt\msi\scripts\WiSubStg.vbs"/>
<property name="windowsSDKMsiStream" location="C:\Program Files\Microsoft SDKs\Windows\v7.1\Samples\sysmgmt\msi\scripts\WiStream.vbs"/>
<property name="windowsSDKMsiLanguages" location="C:\Program Files\Microsoft SDKs\Windows\v7.1\Samples\sysmgmt\msi\scripts\WiLangId.vbs"/>
<property name="cultureBase" value="en-US:en-US:1033"/>
<property name="cultures" value="pt-BR:pt-BR:1046,cs-CZ:en-US:1029"/>
<target name="installer">
<echo>Building ${culture} msi using binaries in "${location}"...</echo>
<propertyregex property="lang.folder" input="${culture}" regexp="(.*):(.*):(.*)" select="\1"/>
<propertyregex property="lang.culture" input="${culture}" regexp="(.*):(.*):(.*)" select="\2"/>
<wix mode="light" target="${outlocation}\${lang.folder}\my.msi" wixHome="${wixhome}">
<lightArg line="-b "${location}""/> <!-- provide the location of the signable binaries -->
<lightArg line="-b "${msiwixsource}\Localisation\${lang.folder}""/> <!-- provide the location of the localisation -->
<lightArg line="-sice:ICE57"/>
<lightArg line="-cultures:${lang.culture}"/>
<lightArg line="-loc "${msiwixsource}\Localisation\${lang.folder}\Product.wxl""/>
<lightArg line="-ext "${wixhome}\WixUtilExtension.dll""/>
<lightArg line="-ext "${wixhome}\WixUIExtension.dll""/>
<sources dir="${msiwixobjects}">
<include name="*.wixobj"/>
</sources>
<moresources dir="${msiwixsource}\Localisation\${lang.folder}">
<include name="*"/>
</moresources>
<moresources dir="${location}">
<include name="binary.dll,binary.exe"/>
</moresources>
</wix>
</target>
<target name="checkTransform" depends="installer">
<uptodate property="transform.notRequired" targetfile="${outlocation}\${culture}\my.mst" >
<srcfiles dir= "${outlocation}\en-US\" includes="my.msi"/>
<srcfiles dir= "${outlocation}\${lang.culture}\" includes="my.msi"/>
</uptodate>
</target>
<target name="transform" depends="checkTransform" unless="transform.notRequired">
<echo>Building ${culture} mst ...</echo>
<propertyregex property="lang.folder" input="${culture}" regexp="(.*):(.*):(.*)" select="\1"/>
<propertyregex property="lang.id" input="${culture}" regexp="(.*):(.*):(.*)" select="\3"/>
<!-- Generate the mst 'diff' file - this does not include exporting the language specific cab file to match -->
<exec executable="${windowsSDKBin}">
<arg value="-g"/>
<arg value="${outlocation}\en-US\my.msi"/>
<arg value="${outlocation}\${lang.folder}\my.msi"/>
<arg value="${outlocation}\${lang.folder}\my.mst"/>
</exec>
<!-- Exporting the language specific cab file to match -->
<delete file="localised_${lang.id}.cab" />
<delete file="${outlocation}\${lang.folder}\localised_${lang.id}.cab" />
<exec executable="${windowsSDKCab}">
<arg value="-d"/>
<arg value="${outlocation}\${lang.folder}\my.msi"/>
<arg value="-x"/>
<arg value="localised_${lang.id}.cab"/>
</exec>
<move file="localised_${lang.id}.cab" tofile="${outlocation}\${lang.folder}\localised_${lang.id}.cab"/>
</target>
<!-- Target to build MSIs using unsigned binaries -->
<target name="MSIs">
<!-- we always need english as it is the base multilingual -->
<antcall target="installer">
<param name="culture" value="${cultureBase}" />
<param name="location" value="${location}" />
<param name="outlocation" value="${outlocation}" />
</antcall>
<!-- build the different cultures and make transforms
parallel="true" sometimes fails-->
<for list="${cultures}" param="culture" >
<sequential>
<antcall target="transform">
<param name="culture" value="#{culture}" />
<param name="location" value="${location}" />
<param name="outlocation" value="${outlocation}" />
</antcall>
</sequential>
</for>
</target>
<!-- depends="transform" -->
<target name="transformRepack" >
<echo>Packing ${culture} mst into multilingual installer...</echo>
<propertyregex property="lang.folder" input="${culture}" regexp="(.*):(.*):(.*)" select="\1"/>
<propertyregex property="lang.id" input="${culture}" regexp="(.*):(.*):(.*)" select="\3"/>
<exec executable="cscript">
<arg value="${windowsSDKMsiStream}"/>
<arg value="${outlocation}\my.msi"/>
<arg value="${outlocation}\${lang.folder}\localised_${lang.id}.cab"/>
<arg value="localised_${lang.id}.cab"/>
</exec>
<exec executable="cscript">
<arg value="${windowsSDKMsiStorage}"/>
<arg value="${outlocation}\my.msi"/>
<arg value="${outlocation}\${lang.folder}\my.mst"/>
<arg value="${lang.id}"/>
</exec>
</target>
<target name="MSIMulti" depends="MSIs">
<echo>Making multilingual installer...</echo>
<copy file="${outlocation}\en-US\my.msi" todir="${outlocation}"/>
<for list="${cultures}" param="culture">
<sequential>
<antcall target="transformRepack">
<param name="culture" value="#{culture}" />
<param name="location" value="${location}" />
<param name="outlocation" value="${outlocation}" />
</antcall>
</sequential>
</for>
<!-- report what was packed -->
<propertyregex property="lang.ids" input="${cultures}" regexp="([^:,]*):([^:]*):([^,]*)" replace="\3" global="true" defaultvalue="failed"/>
<echo/>
<echo>Multilingual installer should include transform language ids </echo>
<echo>${lang.ids}</echo>
<echo>Multilingual installer reports transform language ids...</echo>
<exec executable="cscript">
<arg value="${windowsSDKMsiStorage}"/>
<arg value="${outlocation}\my.msi"/>
</exec>
<exec executable="cscript">
<arg value="${windowsSDKMsiLanguages}"/>
<arg value="${outlocation}\my.msi"/>
<arg value="Package"/>
<arg value="1033,${lang.ids}"/>
</exec>
</target>
<target name="UnsignedMSIMulti" >
<antcall target="MSIMulti">
<param name="location" value="${binarieslocation}" />
<param name="outlocation" value="${binarieslocation}" />
</antcall>
</target>