WSO2 ESB: How to download a file over HTTP? - vfs

Question

I want to download an XML file from a Web site using WSO2 ESB. Assume that the URL is static for the sake of simplicity.
I have tried VFS both as a proxy service and an individual sequence without success, and couldn't find any relevant material on the Internet.
Here is the sequence XML:
<sequence xmlns="http://ws.apache.org/ns/synapse" name="download">
<in>
<log level="headers">
<property name="?" value="[download] in: started"/>
</log>
<property name="OUT_ONLY" value="true" scope="default" type="STRING"/>
<property name="transport.vfs.ContentType" value="text/xml" scope="transport" type="STRING"/>
<property name="transport.vfs.FileURI" value="http://static.nvd.nist.gov/feeds/xml/cve/nvdcve-2.0-2013.xml" scope="transport" type="STRING"/>
<log level="headers">
<property name="?" value="[download] in: sending over vfs"/>
</log>
<send>
<endpoint>
<address uri="http://static.nvd.nist.gov/feeds/xml/cve/nvdcve-2.0-2013.xml"/>
</endpoint>
</send>
<log level="headers">
<property name="?" value="[download] in: ended"/>
</log>
</in>
<out>
<log level="headers">
<property name="?" value="[download] out: started"/>
</log>
<send/>
<log level="headers">
<property name="?" value="[download] out: ended"/>
</log>
</out>
</sequence>
So, how to download a large file over HTTP with WSO2 ESB?

Answer 1

I have a similar problem, and by the moment I've solved it with an API and this code:
<api xmlns="http://ws.apache.org/ns/synapse" name="EcoRest" context="/services/ecorest">
<resource methods="GET" uri-template="/{idFile}">
<inSequence>
<send>
<endpoint>
<http method="get" uri-template="http://slx00012001:8888/repositoryfiles/{uri.var.idFile}">
<suspendOnFailure>
<errorCodes>101500,101510,101001,101000</errorCodes>
<initialDuration>10</initialDuration>
<progressionFactor>1.0</progressionFactor>
<maximumDuration>10</maximumDuration>
</suspendOnFailure>
</http>
</endpoint>
</send>
<property name="enableSwA" value="true" scope="axis2"></property>
</inSequence>
<faultSequence>
<log level="custom">
<property name="FAULT GETTING FILE" expression="get-property('uri.var.idFile')"></property>
</log>
<payloadFactory media-type="json">
<format> {"descError":"$1", "error": "true"} </format>
<args>
<arg evaluator="json" expression="$.descError"></arg>
</args>
</payloadFactory>
<property name="HTTP_SC" value="500" scope="axis2" type="STRING"></property>
<respond></respond>
</faultSequence>
</resource>
</api>
This API works perfectly and make a throught-pass with the files. But if I send an id_file doesn't exist, the backend service return an http 400 with a json-message, the process go into the faultsequence, but the payloadFactory in the faultsequence doesn't work, and I don't know why :(

Answer 2

Have a go with this configuration:
<proxy xmlns="http://ws.apache.org/ns/synapse" name="download" transports="https,http,vfs" statistics="disable" trace="disable" startOnLoad="true">
<target>
<inSequence>
<property name="OUT_ONLY" value="true" scope="default" type="STRING"/>
<property name="ClientApiNonBlocking" value="true" scope="axis2" action="remove"/>
<send>
<endpoint>
<address uri="vfs:file://home/secondary/file.xml"/>
</endpoint>
</send>
</inSequence>
</target>
<parameter name="transport.vfs.Streaming">true</parameter>
<parameter name="transport.vfs.FileURI">http://static.nvd.nist.gov/feeds/xml/cve/nvdcve-2.0-2013.xml</parameter>
<parameter name="transport.vfs.Locking">disable</parameter>
<description></description>
</proxy>
You can initiate the download using:
curl -v http://localhost:8280/services/download
You can find out more about the VFS Transport specific parameters here: http://docs.wso2.org/wiki/display/ESB460/VFS+Transport. You can also find another VFS example here: http://docs.wso2.org/wiki/pages/viewpage.action?pageId=16846489. Hope this helps.