WSO2 ESB. Transfer file with VFS - vfs

I need to check each file in directory "/home/user/from/" with com.wso2esb.command.MyCommand and if the result == true, replace file to "home/user/to/". I have the following proxy:
<?xml version="1.0" encoding="UTF-8"?>
<proxy xmlns="http://ws.apache.org/ns/synapse"
name="fromto"
transports="vfs"
statistics="enable"
trace="enable"
startOnLoad="true">
<target>
<inSequence>
<pojoCommand name="com.wso2esb.command.MyCommand">
<axis2ns47:property xmlns:axis2ns47="http://ws.apache.org/ns/synapse"
name="fileURL"
expression="$trp:FILE_PATH"
action="ReadMessage"/>
<axis2ns48:property xmlns:axis2ns48="http://ws.apache.org/ns/synapse"
name="result"
context-name="com.fx.FileIsOBI"
action="UpdateContext"/>
</pojoCommand>
<switch source="$ctx:com.fx.FileIsOBI">
<case regex="^true$">
<log level="custom">
<property name="OUTPUT"
expression="fn:concat('===555===', $ctx:com.fx.FileIsOBI)"/>
</log>
<property name="OUT_ONLY" value="true" scope="default"/>
<property xmlns:ns="http://org.apache.synapse/xsd"
xmlns:ns3="http://org.apache.synapse/xsd"
name="transport.vfs.ReplyFileName"
expression="$trp:FILE_NAME"
scope="transport"/>
<send>
<endpoint>
<address uri="vfs:file:////home/user/to"/>
</endpoint>
</send>
</case>
</switch>
</inSequence>
</target>
<parameter name="transport.PollInterval">10</parameter>
<parameter name="transport.vfs.ActionAfterProcess">DELETE</parameter>
<parameter name="transport.vfs.FileURI">vfs:file:////home/user/from/</parameter>
<parameter name="transport.vfs.FileNamePattern">.*\.txt</parameter>
<parameter name="transport.vfs.ContentType">application/octet-stream</parameter>
<parameter name="transport.vfs.ActionAfterFailure">DELETE</parameter>
<description/>
</proxy>
I can see the string "===555=== true" in the log file, it means that com.fx.FileIsOBI == true. But my file wasn't moved to "/home/user/to/". I think the problem is in
<property name="OUT_ONLY" value="true" scope="default"/>
<property xmlns:ns="http://org.apache.synapse/xsd"
xmlns:ns3="http://org.apache.synapse/xsd"
name="transport.vfs.ReplyFileName"
expression="$trp:FILE_NAME"
scope="transport"/>
<send>
<endpoint>
<address uri="vfs:file:////home/user/to"/>
</endpoint>
</send>
What's wrong?

Please check if VFS transport is enabled,
https://docs.wso2.com/display/ESB490/VFS+Transport
Following works fine for me
<?xml version="1.0" encoding="UTF-8"?>
<proxy xmlns="http://ws.apache.org/ns/synapse"
name="fromto"
statistics="enable"
trace="enable"
startOnLoad="true">
<target>
<inSequence>
<log level="custom">
<property name="OUTPUT" value="===555==="/>
</log>
<property name="OUT_ONLY" value="true" scope="default"/>
<property xmlns:ns="http://org.apache.synapse/xsd"
xmlns:ns3="http://org.apache.synapse/xsd"
name="transport.vfs.ReplyFileName"
expression="$trp:FILE_NAME"
scope="transport"/>
<send>
<endpoint>
<address uri="vfs:file:///Users/lahiru/work/wso2/esb/9/to"/>
</endpoint>
</send>
</inSequence>
</target>
<parameter name="transport.vfs.ActionAfterProcess">DELETE</parameter>
<parameter name="transport.PollInterval">10</parameter>
<parameter name="transport.vfs.FileURI">vfs:file:///Users/lahiru/work/wso2/esb/9/from</parameter>
<parameter name="transport.vfs.FileNamePattern">.*\.txt</parameter>
<parameter name="transport.vfs.ContentType">application/octet-stream</parameter>
<parameter name="transport.vfs.ActionAfterFailure">DELETE</parameter>
<description/>
</proxy>

I think there is an issue in your file path. Could you change it as below.
<parameter name="transport.vfs.FileURI">vfs:file:///home/user/from/</parameter>
<send>
<endpoint>
<address uri="vfs:file:///home/user/to"/>
</endpoint>
</send>
More over since you are processing an xml file change the content type as follows.
<parameter name="transport.vfs.FileNamePattern">.*\.xml</parameter>
<parameter name="transport.vfs.ContentType">text/xml</parameter>
Do these suggested changes and let us know your feedback.

Related

wso2ei - 6.1.1 - Axis2Sender Unexpected error during sending message out

When I try to send a soap request to my web service from wso2ei I got this error:
If I call direct the proxy or web service, everything is fine, seams is something with send mediator, but I cant figure out! Even if I change the code to call direct the web service by endpoint the error is the same.
[2018-05-28 14:39:34,150] [EI-Core] ERROR - Axis2Sender Unexpected error during sending message out
org.apache.axis2.AxisFault: The system cannot infer the transport information from the /helloworld URL.
at org.apache.axis2.description.ClientUtils.inferOutTransport(ClientUtils.java:81)
at org.apache.synapse.core.axis2.DynamicAxisOperation$DynamicOperationClient.executeImpl(DynamicAxisOperation.java:115)
at org.apache.axis2.client.OperationClient.execute(OperationClient.java:149)
at org.apache.synapse.core.axis2.Axis2FlexibleMEPClient.send(Axis2FlexibleMEPClient.java:595)
at org.apache.synapse.core.axis2.Axis2Sender.sendOn(Axis2Sender.java:83)
at org.apache.synapse.core.axis2.Axis2SynapseEnvironment.send(Axis2SynapseEnvironment.java:548)
at org.apache.synapse.endpoints.AbstractEndpoint.send(AbstractEndpoint.java:382)
at org.apache.synapse.endpoints.WSDLEndpoint.send(WSDLEndpoint.java:75)
at org.apache.synapse.mediators.builtin.SendMediator.mediate(SendMediator.java:121)
at org.apache.synapse.mediators.AbstractListMediator.mediate(AbstractListMediator.java:97)
at org.apache.synapse.mediators.AbstractListMediator.mediate(AbstractListMediator.java:59)
at org.apache.synapse.mediators.base.SequenceMediator.mediate(SequenceMediator.java:158)
at org.apache.synapse.rest.Resource.process(Resource.java:343)
at org.apache.synapse.rest.API.process(API.java:399)
at org.apache.synapse.rest.RESTRequestHandler.apiProcess(RESTRequestHandler.java:123)
at org.apache.synapse.rest.RESTRequestHandler.dispatchToAPI(RESTRequestHandler.java:101)
at org.apache.synapse.rest.RESTRequestHandler.process(RESTRequestHandler.java:69)
at org.apache.synapse.core.axis2.Axis2SynapseEnvironment.injectMessage(Axis2SynapseEnvironment.java:304)
at org.apache.synapse.core.axis2.SynapseMessageReceiver.receive(SynapseMessageReceiver.java:78)
at org.apache.axis2.engine.AxisEngine.receive(AxisEngine.java:180)
at org.apache.synapse.transport.passthru.ServerWorker.processNonEntityEnclosingRESTHandler(ServerWorker.java:326)
at org.apache.synapse.transport.passthru.ServerWorker.processEntityEnclosingRequest(ServerWorker.java:372)
at org.apache.synapse.transport.passthru.ServerWorker.run(ServerWorker.java:151)
at org.apache.axis2.transport.base.threads.NativeWorkerPool$1.run(NativeWorkerPool.java:172)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Here is my code:
<?xml version="1.0" encoding="UTF-8"?>
<api context="/helloworld" name="HelloWorldAPI" xmlns="http://ws.apache.org/ns/synapse">
<resource methods="POST">
<inSequence>
<property description="createName" expression="json-eval($.name)" name="name" scope="default" type="STRING"/>
<log description="requestLog" level="custom">
<property expression="fn:concat("User name: ",get-property("name"))" name="message"/>
</log>
<payloadFactory description="convertToSoap" media-type="xml">
<format>
<hel:GreetingRequest xmlns:hel="http://qualicorp.com.br/ws/HelloWorldWS/">
<hel:name>$1</hel:name>
</hel:GreetingRequest>
</format>
<args>
<arg evaluator="xml" expression="get-property("name")"/>
</args>
</payloadFactory>
<property name="ContentType" value="application/xml" scope="axis2"/>
<property name="HTTP_METHOD" value="POST" scope="axis2" />
<log description="requestSoapLog">
<property name="message" value="Sending request to Proxy"/>
</log>
<send>
<endpoint>
<wsdl port="GreetingProxyHttpSoap11Endpoint" service="GreetingProxy" uri="http://localhost:8280/services/GreetingProxy?wsdl"/>
</endpoint>
</send>
</inSequence>
<outSequence>
<property name="ContentType" value="application/json" scope="axis2"/>
<log description="responseLog"/>
<send/>
</outSequence>
<faultSequence>
<log description="ErrorLog"/>
</faultSequence>
</resource>
</api>
<?xml version="1.0" encoding="UTF-8"?>
<proxy name="GreetingProxy" startOnLoad="true" transports="https,http" xmlns="http://ws.apache.org/ns/synapse">
<target>
<endpoint name="endpoint_urn_uuid_1FF78996948B9A0B6E14565751214774-1218279313">
<wsdl port="GreetingPortSoap11" service="GreetingPortService" uri="http://localhost:8080/HelloWorldWS/ws/greeting.wsdl"/>
</endpoint>
<inSequence>
<log description="requestLog">
<property name="message" value="Sending request"/>
</log>
</inSequence>
<outSequence>
<log description="responseLog">
<property name="message" value=""Returning web service""/>
</log>
<send/>
</outSequence>
<faultSequence>
<log description="ErrorLog"/>
</faultSequence>
</target>
</proxy>
I just found some interesting behavior. If I change wsdl-endpoint by an address-endpoint the API works. I added a payloadFactory to create a restResponse and some logs.
What I still dont know is why wsdl-endpoint doesnt work or what I should change to make it works.
<?xml version="1.0" encoding="UTF-8"?>
<api context="/helloworld" name="HelloWorldAPI" xmlns="http://ws.apache.org/ns/synapse">
<resource methods="POST">
<inSequence>
<property description="createName" expression="json-eval($.name)" name="name" scope="default" type="STRING"/>
<log description="requestLog" level="custom">
<property expression="fn:concat("Nome do usuario recebido: ",get-property("name"))" name="message"/>
</log>
<payloadFactory description="convertToSoap" media-type="xml">
<format>
<hel:GreetingRequest xmlns:hel="http://qualicorp.com.br/ws/HelloWorldWS/">
<hel:name>$1</hel:name>
</hel:GreetingRequest>
</format>
<args>
<arg evaluator="xml" expression="get-property("name")"/>
</args>
</payloadFactory>
<property name="ContentType" scope="axis2" type="STRING" value="application/xml"/>
<property name="HTTP_METHOD" scope="axis2" type="STRING" value="POST"/>
<log description="requestSoapLog">
<property name="message" value="Enviando requisicao ao Proxy"/>
</log>
<send>
<endpoint>
<address uri="http://localhost:8280/services/GreetingProxy?wsdl"/>
</endpoint>
</send>
</inSequence>
<outSequence>
<payloadFactory description="convertToSoap" media-type="json">
<format>
{
greeting: "$1"
}
</format>
<args>
<arg evaluator="xml" expression="//ns2:greeting/ns2:message/text()" xmlns:ns2="http://qualicorp.com.br/ws/HelloWorldWS/"/>
</args>
</payloadFactory>
<property name="ContentType" scope="axis2" type="STRING" value="application/json"/>
<log description="responseLog"/>
<send/>
</outSequence>
<faultSequence>
<log description="ErrorLog"/>
</faultSequence>
</resource>
</api>
<?xml version="1.0" encoding="UTF-8"?>
<proxy name="GreetingProxy" startOnLoad="true" transports="https http" xmlns="http://ws.apache.org/ns/synapse">
<target>
<endpoint name="endpoint_urn_uuid_1FF78996948B9A0B6E14565751214774-1218279313">
<wsdl port="GreetingPortSoap11" service="GreetingPortService" uri="http://localhost:8080/HelloWorldWS/ws/greeting.wsdl"/>
</endpoint>
<inSequence>
<log description="requestLog">
<property name="message" value="Recebendo requisicao"/>
</log>
</inSequence>
<outSequence>
<log description="responseLog">
<property name="message" value=""Retorno do web service""/>
</log>
<log description="responseXmlLog" level="custom">
<property expression="//ns2:greeting" name="message" xmlns:ns2="http://qualicorp.com.br/ws/HelloWorldWS/"/>
</log>
<send/>
</outSequence>
<faultSequence>
<log description="ErrorLog"/>
</faultSequence>
</target>
</proxy>
I think I´ve found something. I can use wsdl-endpoint tag if I send it direct to tomcat´s web server, if I access my web service through a wso2´s proxy, address-endpoint must be the choice.
I dont have any explanation why, but that´s wso2 behavior so far.

wso2esb vfs receiver for text/plain file, but got binary

/Users/lixiaoguang/Desktop/Screen Shot 2018-02-24 at 16.22.13.png[log before smooks][1]
<proxy name="LocalVFStextToXML" startOnLoad="true" transports="vfs" xmlns="http://ws.apache.org/ns/synapse">
<target>
<inSequence>
<log level="full"/>
<smooks config-key="smooks_test">
<input type="text"/>
<output type="xml"/>
</smooks>
<log level="full"/>
</inSequence>
<outSequence/>
<faultSequence/>
</target>
<parameter name="transport.vfs.Streaming">true</parameter>
<parameter name="transport.PollInterval">2</parameter>
<parameter name="transport.vfs.FileURI">file:///home/wso2/fileintxt</parameter>
<parameter name="transport.vfs.ContentType">text/plain</parameter>
<parameter name="transport.vfs.ActionAfterProcess">MOVE</parameter>
<parameter name="transport.vfs.ActionAfterFailure">DELETE</parameter>
<parameter name="transport.vfs.FileNamePattern">.*.txt</parameter>
<parameter name="transport.vfs.MoveAfterProcess">file:///home/wso2/filebackup</parameter></proxy>
I want to got the text/plain with wso2 vfs transport, is there any issue with the code or some configuration I have to check?thanks~

NLog not writing to Database Target

Having trouble getting the database target for NLog to work. I have 2 targets: FILE and DB.. here is my NLog.config:
<?xml version="1.0" encoding="utf-8" ?>
<nlog xmlns="http://www.nlog-project.org/schemas/NLog.xsd"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
autoReload="true"
throwExceptions="false">
<variable name="appName" value="KoreCMS" />
<targets async="true">
<target xsi:type="File"
name="default"
layout="${longdate} - ${level:uppercase=true}: ${message}${onexception:${newline}EXCEPTION\: ${exception:format=ToString}}"
fileName="${basedir}\App_Data\Log\Debug.log"
keepFileOpen="false"
archiveFileName="${basedir}\App_Data\Log\Debug_${shortdate}.{##}.log"
archiveNumbering="Sequence"
archiveEvery="Day"
maxArchiveFiles="30"/>
<target xsi:type="Database"
name="database"
keepConnection="true"
useTransactions="true"
dbProvider="System.Data.SqlClient"
connectionStringName="DefaultConnection"
commandText="INSERT INTO Kore_Log(EventDateTime, EventLevel, UserName, MachineName, EventMessage, ErrorSource, ErrorClass, ErrorMethod, ErrorMessage, InnerErrorMessage) VALUES (#EventDateTime, #EventLevel, #UserName, #MachineName, #EventMessage, #ErrorSource, #ErrorClass, #ErrorMethod, #ErrorMessage, #InnerErrorMessage)">
<parameter name="#EventDateTime" layout="${date:s}" />
<parameter name="#EventLevel" layout="${level}" />
<parameter name="#UserName" layout="${aspnet-user-identity}" />
<parameter name="#MachineName" layout="${machinename}" />
<parameter name="#EventMessage" layout="${message}" />
<parameter name="#ErrorSource" layout="${event-context:item=error-source}" />
<parameter name="#ErrorClass" layout="${event-context:item=error-class}" />
<parameter name="#ErrorMethod" layout="${event-context:item=error-method}" />
<parameter name="#ErrorMessage" layout="${event-context:item=error-message}" />
<parameter name="#InnerErrorMessage" layout="${event-context:item=inner-error-message}" />
</target>
</targets>
<rules>
<logger name="*" writeTo="default" minlevel="Info" />
<logger name="*" writeTo="database" minlevel="Info" />
</rules>
</nlog>
And here is the DB design:
And here is the connection string:
<connectionStrings>
<add name="DefaultConnection" connectionString="Server=.\SQLExpress2012;Initial Catalog=MyDatabaseName;Integrated Security=True;Persist Security Info=True;MultipleActiveResultSets=True"
providerName="System.Data.SqlClient" />
</connectionStrings>
What am I missing?
Alright, I figured it out... When I opened up SQL Profiler, I could see the INSERT command was indeed being sent, so I attempted to run the same command myself manually and discovered that I was being an idiot; basically, my Id column is setup as a uniqueidentifier and so of course it could not be auto incremented. The solution was to change the commandText in the Nlog.config to the following:
commandText="INSERT INTO Kore_Log(Id, EventDateTime, EventLevel, UserName, MachineName, EventMessage, ErrorSource, ErrorClass, ErrorMethod, ErrorMessage, InnerErrorMessage) VALUES (NEWID(), #EventDateTime, #EventLevel, #UserName, #MachineName, #EventMessage, #ErrorSource, #ErrorClass, #ErrorMethod, #ErrorMessage, #InnerErrorMessage)">
I got into same problem, reason was dbProvider value and connectionStringName key in nlog.config. I got it from here
<target name="database" xsi:type="Database" connectionString="${configsetting:name=sqlconnection.connectionString}"
dbProvider="Microsoft.Data.SqlClient.SqlConnection, Microsoft.Data.SqlClient">
Also,
connectionString should be used in place of connectionStringName
if Microsoft.Data.SqlClient is not added in dependency, it need to be added either using nuget package manager or manually.

WSO2 ESB: How to download a file over HTTP?

I want to download an XML file from a Web site using WSO2 ESB. Assume that the URL is static for the sake of simplicity.
I have tried VFS both as a proxy service and an individual sequence without success, and couldn't find any relevant material on the Internet.
Here is the sequence XML:
<sequence xmlns="http://ws.apache.org/ns/synapse" name="download">
<in>
<log level="headers">
<property name="?" value="[download] in: started"/>
</log>
<property name="OUT_ONLY" value="true" scope="default" type="STRING"/>
<property name="transport.vfs.ContentType" value="text/xml" scope="transport" type="STRING"/>
<property name="transport.vfs.FileURI" value="http://static.nvd.nist.gov/feeds/xml/cve/nvdcve-2.0-2013.xml" scope="transport" type="STRING"/>
<log level="headers">
<property name="?" value="[download] in: sending over vfs"/>
</log>
<send>
<endpoint>
<address uri="http://static.nvd.nist.gov/feeds/xml/cve/nvdcve-2.0-2013.xml"/>
</endpoint>
</send>
<log level="headers">
<property name="?" value="[download] in: ended"/>
</log>
</in>
<out>
<log level="headers">
<property name="?" value="[download] out: started"/>
</log>
<send/>
<log level="headers">
<property name="?" value="[download] out: ended"/>
</log>
</out>
</sequence>
So, how to download a large file over HTTP with WSO2 ESB?
I have a similar problem, and by the moment I've solved it with an API and this code:
<api xmlns="http://ws.apache.org/ns/synapse" name="EcoRest" context="/services/ecorest">
<resource methods="GET" uri-template="/{idFile}">
<inSequence>
<send>
<endpoint>
<http method="get" uri-template="http://slx00012001:8888/repositoryfiles/{uri.var.idFile}">
<suspendOnFailure>
<errorCodes>101500,101510,101001,101000</errorCodes>
<initialDuration>10</initialDuration>
<progressionFactor>1.0</progressionFactor>
<maximumDuration>10</maximumDuration>
</suspendOnFailure>
</http>
</endpoint>
</send>
<property name="enableSwA" value="true" scope="axis2"></property>
</inSequence>
<faultSequence>
<log level="custom">
<property name="FAULT GETTING FILE" expression="get-property('uri.var.idFile')"></property>
</log>
<payloadFactory media-type="json">
<format> {"descError":"$1", "error": "true"} </format>
<args>
<arg evaluator="json" expression="$.descError"></arg>
</args>
</payloadFactory>
<property name="HTTP_SC" value="500" scope="axis2" type="STRING"></property>
<respond></respond>
</faultSequence>
</resource>
</api>
This API works perfectly and make a throught-pass with the files. But if I send an id_file doesn't exist, the backend service return an http 400 with a json-message, the process go into the faultsequence, but the payloadFactory in the faultsequence doesn't work, and I don't know why :(
Have a go with this configuration:
<proxy xmlns="http://ws.apache.org/ns/synapse" name="download" transports="https,http,vfs" statistics="disable" trace="disable" startOnLoad="true">
<target>
<inSequence>
<property name="OUT_ONLY" value="true" scope="default" type="STRING"/>
<property name="ClientApiNonBlocking" value="true" scope="axis2" action="remove"/>
<send>
<endpoint>
<address uri="vfs:file://home/secondary/file.xml"/>
</endpoint>
</send>
</inSequence>
</target>
<parameter name="transport.vfs.Streaming">true</parameter>
<parameter name="transport.vfs.FileURI">http://static.nvd.nist.gov/feeds/xml/cve/nvdcve-2.0-2013.xml</parameter>
<parameter name="transport.vfs.Locking">disable</parameter>
<description></description>
</proxy>
You can initiate the download using:
curl -v http://localhost:8280/services/download
You can find out more about the VFS Transport specific parameters here: http://docs.wso2.org/wiki/display/ESB460/VFS+Transport. You can also find another VFS example here: http://docs.wso2.org/wiki/pages/viewpage.action?pageId=16846489. Hope this helps.

How to append response message to a text file?

I need write the response message to a "response.xml" file. And the response should be appended to this file, I set parameter "Append" true, but no use, is completely overwritten every time. Anyone can help me? Thanks and best regards.
<proxy xmlns="http://ws.apache.org/ns/synapse" name="EndpointTest" transports="https,http" statistics="disable" trace="disable" startOnLoad="true">
<target>
<inSequence>
<log level="full" />
</inSequence>
<outSequence>
<log level="full" />
<send>
<endpoint>
<address uri="vfs:file:///usr/local/wso2esb-4.0.3/Log" format="pox" />
<property name="OUT_ONLY" value="true" />
<property name="transport.vfs.Append" value="true" />
<property name="transport.vfs.ReplyFileName" value="response.xml" />
</endpoint>
</send>
<send />
</outSequence>
<endpoint>
<address uri="http://172.21.13.151:18989/aaadc/services/receiveMsg" format="pox" />
</endpoint>
</target>
</proxy>
Remove the append= true parameter from service level and use with endpoint url
eg:
vfs:file:///usr/local/wso2esb-4.0.3/Log?transport.vfs.Append=true

Resources