I'm new to Java. Working on QuikcFixJ.
I'm trying to load custom.xml file using datadictionary and later parsing msg string from this datadictionary. Not sure what is wrong with datadictionary , its not throwing any error whether I pass correct fix.xml file path or incorrect one.
Later when I pass msg string to Message object it just parse initial 3 tags irrespective of whatever xml file I mention in datadictionary.
Please give me some pointers to resolve it.
Adding code snippet:
public Message createMsg(String type, String message)
{
message = "35=" + type + "\001" + message;
message = "8=FIX4.2\0019=" + message.length() + "\001"+message ;
String msgCharStr = message.replaceAll("\\|", "\001");
int checksum = MessageUtils.checksum(msgCharStr);
msgCharStr = msgCharStr +"\00110=" + (1 +
MessageUtils.checksum(msgCharStr)) + "\001" ;
quickfix.Message msg = new Message();
msg.fromString(msgCharStr, null,true);
}
This code works fine for regular neworder or quoterequest msgs.
But, when I pass quoterequest msg with custom repeating group which I loaded via UseDictionary config settings, it gives undesired output msg.
e.g: Lets say if I pass input, "35=R|146=1|...|453=2|448=XX|447=G|452=1|448=YY|447=D|452=2|......"
parser gives output as :
"35=R|146=1|...|448=XX|447=G|452=1|453=2|448=YY|447=D|452=2|......"
----Code re-build steps for new custom FIX42.xml ---
Copied quickfixj2.3.1.zip (url: https://github.com/quickfix-j/quickfixj/releases )
Updated xml at path quickfixj-messages/quickfixj-messages-fix42/src/main/resources/FIX42.xml
run mvn build command.
Maven re-build jar for below:
quickfixj-all
quickfixj-core/
quickfixj-messages/
quickfixj-codegenerator/
quickfixj-dictgenerator/
Tried using all above jar files in my RobotFramework with an assumption that it will now refer the modified QuoteRequest msg which has repeating group support.
To my surprise, its output was exactly same. No changes.
After quickfixj rebuild for customised xml file.
Final version of code snippet:
public Message createMsg(String type, String message)
{
message = "35=" + type + "\001" + message;
message = "8=FIX4.2\0019=" + message.length() + "\001"+message ;
String msgCharStr = message.replaceAll("\\|", "\001");
int checksum = MessageUtils.checksum(msgCharStr);
msgCharStr = msgCharStr +"\00110=" + (1 +
MessageUtils.checksum(msgCharStr)) + "\001" ;
DataDictioanry dd = DataDictionary("FIX42.xml");
quickfix.Message msg = new Message();
msg.fromString(msgCharStr, dd ,true);
}
Related
As per in this FileHelpers 3.1 example, you can automatically detect a CSV file format using the FileHelpers.Detection.SmartFormatDetector class.
But the example goes no further. How do you use this information to dynamically parse a CSV file? It must have something to do with the DelimitedFileEngine but I cannot see how.
Update:
I figured out a possible way but had to resort to using reflection (which does not feel right). Is there another/better way? Maybe using System.Dynamic? Anyway, here is the code I have so far, it ain't pretty but it works:
// follows on from smart detector example
FileHelpers.Detection.RecordFormatInfo lDetectedFormat = formats[0];
Type lDetectedClass = lDetectedFormat.ClassBuilderAsDelimited.CreateRecordClass();
List<FieldInfo> lFieldInfoList = new List<FieldInfo>(lDetectedFormat.ClassBuilderAsDelimited.FieldCount);
foreach (FileHelpers.Dynamic.DelimitedFieldBuilder lField in lDetectedFormat.ClassBuilderAsDelimited.Fields)
lFieldInfoList.Add(lDetectedClass.GetField(lField.FieldName));
FileHelperAsyncEngine lFileEngine = new FileHelperAsyncEngine(lDetectedClass);
int lRecNo = 0;
lFileEngine.BeginReadFile(cReadingsFile);
try
{
while (true)
{
object lRec = lFileEngine.ReadNext();
if (lRec == null)
break;
Trace.WriteLine("Record " + lRecNo);
lFieldInfoList.ForEach(f => Trace.WriteLine(" " + f.Name + " = " + f.GetValue(lRec)));
lRecNo++;
}
}
finally
{
lFileEngine.Close();
}
As I use the SmartFormatDetector to determine the exact format of the incoming Delimited files you can use following appoach:
private DelimitedClassBuilder GetFormat(string file)
{
var detector = new FileHelpers.Detection.SmartFormatDetector();
var format = detector.DetectFileFormat(file);
return format.First().ClassBuilderAsDelimited;
}
private List<T> ConvertFile2Objects<T>(string file, out DelimitedFileEngine engine)
{
var format = GetSeperator(file); // Get Here your FormatInfo
engine = new DelimitedFileEngine(typeof(T)); //define your DelimitdFileEngine
//set some Properties of the engine with what you need
engine.ErrorMode = ErrorMode.SaveAndContinue; //optional
engine.Options.Delimiter = format.Delimiter;
engine.Options.IgnoreFirstLines = format.IgnoreFirstLines;
engine.Options.IgnoreLastLines = format.IgnoreLastLines;
//process
var ret = engine.ReadFileAsList(file);
this.errorCount = engine.ErrorManager.ErrorCount;
var err = engine.ErrorManager.Errors;
engine.ErrorManager.SaveErrors("errors.out");
//return records do here what you need
return ret.Cast<T>().ToList();
}
This is an approach I use in a project, where I only know that I have to process Delimited files of multiple types.
Attention:
I noticed that with the files I recieved the SmartFormatDetector has a problem with tab delimiter. Maybe this should be considered.
Disclaimer: This code is not perfected but in a usable state. Modification and/or refactoring is adviced.
I have the following scenario:
I have thousands of text files with the below format.The column names are written in separate lines where as the row values are delimited by Pipe(|).
START-OF-FILE
PROGRAMNAME=getdata
DATEFORMAT=yyyymmdd
#Some Text
#Some Text
#Some Text
#Some Text
#Some Text
START-OF-FIELDS
Field1
Field2
Field3
------
FieldN
END-OF-FIELDS
TIMESTARTED=Tue May 12 16:04:42 JST 2015
START-OF-DATA
Field1Value|Field2value|Field3Value|...|Field N Value
Field1Value|Field2value|Field3Value|...|Field N Value
------|...........|----|-------
END-OF-DATA
DATARECORDS=30747
TIMEFINISHED=Tue May 12 16:11:53 JST 2015
END-OF-FILE
Now I have a corresponding SQL Server table, where I can easily load the data as destination.
Since I am new to SSIS, having trouble as to how to write the Script Component so that I can filter the Source Text files and easily load into sql server table.
Thanks in advance!
There are a few ways to do it. If the format of the files are constant, there are some useful properties of the flat file connection manager editor. For example, you can add a new flat file connection into the connection managers. There are some properties such as "Rows to skip" for the above file, you could set this to 18. Then it would start at the columns line with the "|".
Another property of the flat file connection manager that may be useful is that if you open the flat file connection manager, and then click on columns in the side menu, you can set the column delimter to the pipe "|"
But if the format of the file will change, e.g. variable number of header rows, you can use a script task to remove any non-piped rows. e.g. the header and footer.
For example, you can add a method such as file.readalllines and then edit or remove the lines as needed then save the file.
Info about that method is here:
https://msdn.microsoft.com/en-us/library/s2tte0y1%28v=vs.110%29.aspx
e.g. to remove last line in script task
string[] lines = File.ReadAllLines( "input.txt" );
StringBuilder sb = new StringBuilder();
int count = lines.Length - 1; // all except last line
for (int i = 0; i < count; i++)
{
sb.AppendLine(lines[i]);
}
File.WriteAllText("output.txt", sb.ToString());
USE Below VB Script in your SSIS SCript Component Task as source
enter code here
Imports System
Imports System.Data
Imports System.Math
Imports System.IO
Imports Microsoft.SqlServer.Dts.Runtime
Imports Microsoft.SqlServer.Dts.Pipeline.Wrapper
Imports Microsoft.SqlServer.Dts.Runtime.Wrapper
<Microsoft.SqlServer.Dts.Pipeline.SSISScriptComponentEntryPointAttribute()> _
<CLSCompliant(False)> _
Public Class ScriptMain
Inherits UserComponent
'Private strSourceDirectory As String
'Private strSourceFileName As String
Private strSourceSystem As String
Private strSourceSubSystem As String
Private dtBusinessDate As Date
Public Overrides Sub PreExecute()
MyBase.PreExecute()
'
' Add your code here for preprocessing or remove if not needed
''
End Sub
Public Overrides Sub PostExecute()
MyBase.PostExecute()
'
' Add your code here for postprocessing or remove if not needed
' You can set read/write variables here, for example:
Dim strSourceDirectory As String = Me.Variables.GLOBALSourceDirectory.ToString()
Dim strSourceFileName As String = Me.Variables.GLOBALSourceFileName.ToString()
'Dim strSourceSystem As String = Me.Variables.GLOBALSourceSystem.ToString()
'Dim strSourceSubSystem As String = Me.Variables.GLOBALSourceSubSystem.ToString()
'Dim dtBusinessDate As Date = Me.Variables.GLOBALBusinessDate.Date
End Sub
Public Overrides Sub CreateNewOutputRows()
'
' Add rows by calling the AddRow method on the member variable named "<Output Name>Buffer".
' For example, call MyOutputBuffer.AddRow() if your output was named "MyOutput".
'
Dim sr As System.IO.StreamReader
Dim strSourceDirectory As String = Me.Variables.GLOBALSourceDirectory.ToString()
Dim strSourceFileName As String = Me.Variables.GLOBALSourceFileName.ToString()
'Dim strSourceSystem As String = Me.Variables.GLOBALSourceSystem.ToString()
'Dim strSourceSubSystem As String = Me.Variables.GLOBALSourceSubSystem.ToString()
'Dim dtBusinessDate As Date = Me.Variables.GLOBALBusinessDate.Date
'sr = New System.IO.StreamReader("C:\QRM_SourceFiles\BBG_BONDS_OUTPUT_YYYYMMDD.txt")
sr = New System.IO.StreamReader(strSourceDirectory & strSourceFileName)
Dim lineIndex As Integer = 0
While (Not sr.EndOfStream)
Dim line As String = sr.ReadLine()
If (lineIndex <> 0) Then 'remove header row
Dim columnArray As String() = line.Split(Convert.ToChar("|"))
If (columnArray.Length > 1) Then
Output0Buffer.AddRow()
Output0Buffer.Col0 = columnArray(0).ToString()
Output0Buffer.Col3 = columnArray(3).ToString()
Output0Buffer.Col4 = columnArray(4).ToString()
Output0Buffer.Col5 = columnArray(5).ToString()
Output0Buffer.Col6 = columnArray(6).ToString()
Output0Buffer.Col7 = columnArray(7).ToString()
Output0Buffer.Col8 = columnArray(8).ToString()
Output0Buffer.Col9 = columnArray(9).ToString()
Output0Buffer.Col10 = columnArray(10).ToString()
Output0Buffer.Col11 = columnArray(11).ToString()
Output0Buffer.Col12 = columnArray(12).ToString()
Output0Buffer.Col13 = columnArray(13).ToString()
Output0Buffer.Col14 = columnArray(14).ToString()
Output0Buffer.Col15 = columnArray(15).ToString()
Output0Buffer.Col16 = columnArray(16).ToString()
Output0Buffer.Col17 = columnArray(17).ToString()
Output0Buffer.Col18 = columnArray(18).ToString()
Output0Buffer.Col19 = columnArray(19).ToString()
Output0Buffer.Col20 = columnArray(20).ToString()
Output0Buffer.Col21 = columnArray(21).ToString()
Output0Buffer.Col22 = columnArray(22).ToString()
Output0Buffer.Col23 = columnArray(23).ToString()
Output0Buffer.Col24 = columnArray(24).ToString()
End If
End If
lineIndex = lineIndex + 1
End While
sr.Close()
End Sub
End Class
Code End
I'm implementing a methond in my application that uses the Jessp parser class in order to open a file and getting the deftemplates and deffacts inside of it. The problem is that when trying to obtain the result into a object variable, it asks on the constructor for a JessTokenStream. I tried to pass a JessToken, but then it complains about the type, that it should be e8. Searched through the Jess documentation but didn't found an explanation for the arguments, only the syntax of the constructor.
Anyone can help?.
Thanks in advance!!!
The class JessTokenStream is not public, so you can't actually call those parseXXX() methods. They are public for historical reasons but aren't actually usable by clients. They should actually be removed from the public interface.
Instead, use the two-argument form of parseExpression(), and then test the returned object to determine its type. Then you can do what you want with the returned object:
Rete engine = ...
Jesp jesp = ...
Object o = jesp.parseExpression(engine.getGlobalContext(), false);
if (o instanceof Deffacts) {
Deffacts d = (Deffacts) o;
for (int i = 0; i<d.getNFacts(); ++i) {
Fact f = d.getFact(i);
Deftemplate t = f.getDeftemplate();
System.out.println("Fact name is " + f.getName();
System.out.println("Fact name is " + f.getName();
for (String name: t.getSlotNames())
System.out.println("Slot " + name + " contains " + f.getSlotValue(name));
}
}
This one has kept me stumped for a couple of days now.
It's my first dabble with CLR & UDF ...
I have created a user defined function that takes a multiline String as input, scans it and replaces a certain line in the string with an alternative if found. If it is not found, it simply appends the desired line at the end. (See code)
The problem, it seems, comes when the final String (or Stringbuilder) is converted to an SqlString or SqlChars. The converted, returned String always contains the Nul character as every second character (viewing via console output, they are displayed as spaces).
I'm probably missing something fundamental on UDF and/or CLR.
Please Help!!
Code (I leave in the commented Stringbuilder which was my initial attempt... changed to normal String in a desperate attempt to find the issue):
[Microsoft.SqlServer.Server.SqlFunction]
[return: SqlFacet(MaxSize = -1, IsFixedLength = false)]
//public static SqlString udf_OmaChangeJob(String omaIn, SqlInt32 jobNumber) {
public static SqlChars udf_OmaChangeJob(String omaIn, SqlInt32 jobNumber) {
if (omaIn == null || omaIn.ToString().Length <= 0) return new SqlChars("");
String[] lines = Regex.Split(omaIn.ToString(), "\r\n");
Regex JobTag = new Regex(#"^JOB=.+$");
//StringBuilder buffer = new StringBuilder();
String buffer = String.Empty;
bool matched = false;
foreach (var line in lines) {
if (!JobTag.IsMatch(line))
//buffer.AppendLine(line);
buffer += line + "\r\n";
else {
//buffer.AppendLine("JOB=" + jobNumber);
buffer += ("JOB=" + jobNumber + "\r\n");
matched = true;
}
}
if (!matched) //buffer.AppendLine("JOB=" + jobNumber);
buffer += ("JOB=" + jobNumber) + "\r\n";
//return new SqlString(buffer.ToString().Replace("\0",String.Empty)) + "blablabla";
// buffer = buffer.Replace("\0", "|");
return new SqlChars(buffer + "\r\nTheEnd");
}
I know in my experiences, the omaIn parameter should be of type SqlString and when you go to collect its value/process it, set a local variable:
string omaString = omaIn != SqlString.Null ? omaIn.Value : string.empty;
Then when you return on any code path, to rewrap the string in C#, you'd need to set
return omaString == string.empty ? new SqlString.Null : new SqlString(omaString);
I have had some fun wrestling matches learning the intricate hand-off between local and outbound types, especially with CLR TVFs.
Hope that can help!
I am struggling with a JQL query.
We have a custom field called 'Build Reported' which is a text field. It has values like '4.7.323H', '5.1.123L', '3.1.456E', etc.
I need to write a simple query that will give me all issues reported after the user-specified version.
JQL function prototype: searchIssues('Build Integrated', '>', '4.7.323B')
To do this, I am firing a JQL Query that gives me the Build Reported for all the issues, I then iterate through each issue and perform a char-by-char comparison to determine if the Build Reported version of the current issue is greater than the one specified by the user. This seems to take too long to execute since I have to retrieve all the issues from jira database.
Is there a faster way to achieve this? Here is what I have so far:
// Get all the arguments
java.util.List args = operand.getArgs();
CustomField cf = customFieldManager.getCustomFieldObjectByName((String)args.get(0));
Long cfID = cf.getIdAsLong();
String operator = (String)args.get(1);
String userVersion = (String)args.get(2);
String jiraVersion = "";
java.util.List issues;
Iterator issuesIterator;
Issue issue;
issues = getAllIssues(user, interestedInVersion, cfID);
issuesIterator = issues.iterator();
// Iterate over all the issues
while(issuesIterator.hasNext())
{
issue = (Issue)issuesIterator.next();
// Get the Build reported value
jiraVersion = (String)issue.getCustomFieldValue(cf);
if(jiraVersion != null &&
!jiraVersion.equals(""))
{
// Compare user-specified version to the one retrieved from database
if(compareVersions(jiraVersion, userVersion, operator))
{
// Add the issue to the result set
literals.add(new QueryLiteral(operand, issue.getId()));
}
}
}
// cfID is the ID for the custom field Build Reported
private java.util.List getAllIssues(User user, Long cfID) throws SearchException, ParseException
{
JqlQueryBuilder builder = JqlQueryBuilder.newBuilder();
builder.where().project("SDEV").and().customField(cfID).isNotEmpty();
Query query = builder.buildQuery();
SearchResults results = searchService.search(user, query, PagerFilter.getUnlimitedFilter());
return results.getIssues();
}
Please note that I do not have any other filters that I could use for the JQL Query Builder to help me reduce the size of the result set.
I found an alternative to the issue I described in my question. Instead of using JQL, I ended up firing a direct SELECT and this turned out to be way quicker. The function below is a part of the JQL Plugin. Here is the code:
This is what I ended up doing:
public java.util.List getValues(#NotNull QueryCreationContext queryCreationContext, #NotNull FunctionOperand operand, #NotNull TerminalClause terminalClause)
{
try
{
// User
User user = queryCreationContext.getUser();
// Args
java.util.List args = operand.getArgs();
CustomField cf = customFieldManager.getCustomFieldObjectByName((String)args.get(0));
Long cfID = cf.getIdAsLong();
String operator = (String)args.get(1);
String userVersion = (String)args.get(2);
// Locals
java.util.List literals = new java.util.LinkedList();
MutableIssue issue = null;
String issueId = "";
String jiraVersion = "";
// DB
Connection conn = null;
String url = "jdbc:jtds:sqlserver://*****:*****/jiradb";
String driver = "net.sourceforge.jtds.jdbc.Driver";
String userName = "*******";
String password = "*******";
String sqlQuery = null;
Statement statement = null;
ResultSet resultSet = null;
Class.forName(driver).newInstance();
conn = DriverManager.getConnection(url, userName, password);
// Get all the issues that has the custom field set
sqlQuery = " SELECT t2.id AS IssueId, t1.stringvalue AS JiraVersion " + "\n" +
" FROM jiradb.jiraschema.customfieldvalue t1 " + "\n" +
" INNER JOIN jiradb.jiraschema.jiraissue t2 " + "\n" +
" ON t1.issue = t2.id " + "\n" +
" WHERE t1.customfield = " + Long.toString(cfID) + " " + "\n" +
" AND t1.stringvalue IS NOT NULL " + "\n" +
" AND t1.stringvalue != '' " + "\n" +
" AND t2.pkey LIKE 'SDEV-%' ";
// Iterate over the result set
statement = conn.createStatement();
resultSet = statement.executeQuery(sqlQuery);
while (resultSet.next())
{
issueId = resultSet.getString("IssueId");
jiraVersion = resultSet.getString("JiraVersion");
// Compare the version from jira with the user provided version
// This is my own function that does char-by-char comparison
if(compareVersions(jiraVersion, userVersion, operator))
{
// Get the issue object to add to the result
issue = ComponentManager.getInstance().getIssueManager().getIssueObject(Long.parseLong(issueId));
// Add the issue to the result
literals.add(new QueryLiteral(operand, issue.getId()));
}
}
// Return all the matching issues here
return literals;
}
catch(Exception e)
{
// Exception handling
}
return null;
}
And this is how the plugin is used:
issue in searchIssues('Build Reported', '>', '5.1.104');