How to use EOBI simple binary encoding - binary-data

I am trying to use the sbeTool with the Eurex codecs:
JAVA -Dsbe.target.namespace=eobiV81 -classpath "../sbe-tool-1.7.0.jar;../agrona-0.9.6.jar" uk.co.real_logic.sbe.SbeTool eobi81.xml
This eobi.xml file looks slightly different from the sample provided and the tool execution fails (it succeeds on the the car.xml example file):
Exception in thread "main" java.lang.NullPointerException
at uk.co.real_logic.sbe.xml.XmlSchemaParser.getAttributeValue(XmlSchemaParser.java:221)
at uk.co.real_logic.sbe.xml.MessageSchema.<init>(MessageSchema.java:47)
at uk.co.real_logic.sbe.xml.XmlSchemaParser.parse(XmlSchemaParser.java:105)
at uk.co.real_logic.sbe.SbeTool.parseSchema(SbeTool.java:274)
at uk.co.real_logic.sbe.SbeTool.main(SbeTool.java:199)
Can anyone help me find a way to get the xml compiling? I believe maybe the eobi.xsd file should be useful, but not sure how.
Thanks

Eurex EOBI is not compliant with the SBE 1.0 standard. You can see this if schema validation is turned on for the SBE Tool. To parse Eurex messages you will need a different codec.

Related

Neo4j to Gephi import : Failed to invoke procedure Invalid UTF-8

I tried to import my data from Neo4j into Gephi but it doesn't work.
I have the following result in Neo4j :
Failed to invoke procedure apoc.gephi.add: Caused by: com.fasterxml.jackson.core.JsonParseException: Invalid UTF-8 start byte 0xfb at [Source: (apoc.export.util.CountingInputStream); line: 1, column: 136]
As previously mentioned, it looks like neo4j is not exporting using UTF-8, so that, I would check how neo4j is generating the output.
Another possibility is that, when neo4j writing the output, something went slightly wrong.
I got this very same problem in the past when concurrently managing content in a file.
A thread crashed and closed "not correctly enough" the file. I mean, when reviewing the file, everything looks pretty normal, but some characters have been introduced which are not UTF-8. A tool like Atom can help you.
Best

H2O POJO causing javac java.lang.IllegalArgumentException

I have a distributed random forest POJO model using the default model setting except for:
ntrees = 150
max_depth = 50
min_rows = 5
Here are the full settings:
buildModel 'drf', {"model_id":"drf-335270ee-8970-4855-b521-c4fb4ca184f5","training_frame":"frame_0.750","validation_frame":"frame_0.250","nfolds":0,"response_column":"DENIAL","ignored_columns":["tx_match_date"],"ignore_const_cols":true,"ntrees":"150","max_depth":"50","min_rows":"5","nbins":20,"seed":-1,"mtries":-1,"sample_rate":0.6320000290870667,"score_each_iteration":true,"score_tree_interval":0,"balance_classes":false,"nbins_top_level":1024,"nbins_cats":1024,"r2_stopping":1.7976931348623157e+308,"stopping_rounds":0,"stopping_metric":"AUTO","stopping_tolerance":0.001,"max_runtime_secs":0,"checkpoint":"","col_sample_rate_per_tree":1,"min_split_improvement":0.00001,"histogram_type":"AUTO","categorical_encoding":"AUTO","build_tree_one_node":false,"sample_rate_per_class":[],"binomial_double_trees":true,"col_sample_rate_change_per_level":1,"calibrate_model":false}
When I try to compile the pojo with:
$javac -cp "h2o-genmodel.jar" -J-Xmx2g -J-XX:MaxPermSize=128m drf_335270ee_8970_4855_b521_c4fb4ca184f5.java
I get the following error.
An exception has occurred in the compiler (1.8.0_131). Please file a bug against the Java compiler via the Java bug reporting page (http://bugreport.java.com) after checking the Bug Database (http://bugs.java.com) for duplicates. Include your program and the following diagnostic in your report. Thank you.
java.lang.IllegalArgumentException
at java.nio.ByteBuffer.allocate(ByteBuffer.java:334)
at com.sun.tools.javac.util.BaseFileManager$ByteBufferCache.get(BaseFileManager.java:325)
at com.sun.tools.javac.util.BaseFileManager.makeByteBuffer(BaseFileManager.java:294)
at com.sun.tools.javac.file.RegularFileObject.getCharContent(RegularFileObject.java:114)
at com.sun.tools.javac.file.RegularFileObject.getCharContent(RegularFileObject.java:53)
at com.sun.tools.javac.main.JavaCompiler.readSource(JavaCompiler.java:602)
at com.sun.tools.javac.main.JavaCompiler.parse(JavaCompiler.java:665)
at com.sun.tools.javac.main.JavaCompiler.parseFiles(JavaCompiler.java:950)
at com.sun.tools.javac.main.JavaCompiler.compile(JavaCompiler.java:857)
at com.sun.tools.javac.main.Main.compile(Main.java:523)
at com.sun.tools.javac.main.Main.compile(Main.java:381)
at com.sun.tools.javac.main.Main.compile(Main.java:370)
at com.sun.tools.javac.main.Main.compile(Main.java:361)
at com.sun.tools.javac.Main.compile(Main.java:56)
at com.sun.tools.javac.Main.main(Main.java:42)
I don't get this error when replacing the DRF model with a deep learning pojo that I have also downloaded from h2o's Flow UI, so I'm thinking it is likely related to the drf_335270ee_8970_4855_b521_c4fb4ca184f5.java file (note that the POJO was too big to preview in H2O's Flow UI). What could be going on here?
Thanks
Instead of trying to compile an H2O random forest POJO, you can download and use a MOJO instead in almost exactly the same way without needing the compile step.
See:
http://docs.h2o.ai/h2o/latest-stable/h2o-genmodel/javadoc/index.html

HL7 parser to parse v2.7 messages in java

I'm looking for a HL7 parser that would parse v2.7 messages. I have tried Hapi but it has support only upto v2.6.
Can you anyone please provide any suggestions in parsing v2.7 messages?
Additionally to allowing unknown versions (like nradov pointed out), you need to inject the right model class factory, e.g. GenericModelClassFactory, into the parser or you might end up with an exception:
ca.uhn.hl7v2.HL7Exception: No map found for version null. Only the following are available: [V22, V25, V21, V23, V24]
So the complete solution is
use the GenericModelClassFactory
allow unknown versions
and it looks like this:
final ModelClassFactory modelClassFactory = new GenericModelClassFactory();
final PipeParser parser = new PipeParser(modelClassFactory);
parser.getParserConfiguration()
.setAllowUnknownVersions(true);
final Message message = parser.parse(message);
Like nradov said, you can use HAPI to parse V2.7. But you'll need to call this to prevent the "2.7 is not recognized" Exception:
parser.getParserConfiguration().setAllowUnknownVersions(true);
You can still use HAPI to parse HL7 V2.7. It just doesn't have convenient methods to easily access the new fields that were added after V2.6.

Get warnings when programmatically parsing Dart file with analyzer_experimental

I am using analyzer_experimental to parse a Dart file into a CompilationUnit:
import 'package:analyzer_experimental/analyzer.dart';
var unit;
try {
unit = parseDartFile(path);
} on AnalyzerErrorGroup catch(e){
print(e);
}
The above code will catch any parsing errors encountered.
I am also interested in seeing any warnings associated with the file (e.g. 'Undefined name "foo"'). I know that the experimental_analyzer library has the capability to generate these warnings when running from the command line but it does not seem to be possible to get the warnings programmatically, without directly referencing classes in the src folder (which seems like a bad idea).
Is there any way to achieve this?
It's likely this package was very incomplete at the time.
There's now an analyzer package on pub and also a (work-in-progress) STDIN/STDOUT Analyzer Service aimed to help making tooling support easier for IDE extension authors.

XML parsing in Ruby

I am using a REXML Ruby parser to parse an XML file. But on a 64 bit AIX box with 64 bit Ruby, I am getting the following error:
REXML::ParseException: #<REXML::ParseException: #<RegexpError: Stack overflow in
regexp matcher:
/^<((?>(?:[\w:][\-\w\d.]*:)?[\w:][\-\w\d.]*))\s*((?>\s+(?:[\w:][\-\w\d.]*:)?[\w:][\-\w\d.]*\s*=\s*(["']).*?\3)*)\s*(\/)?>/mu>
The call for the same is something like this:
REXML::Document.new(File.open(actual_file_name, "r"))
Does anyone have an idea regarding how to solve this issue?
I've had several issues for REXML, it doesn't seem to be the most mature library. Usually I use Nokogiri for Ruby XML parsing stuff, it should be faster and more stable than REXML. After installing it with sudo gem install nokogiri, you can use something like this to get a DOM instance:
doc = Nokogiri.XML(File.open(actual_file_name, 'rb'))
# => #<Nokogiri::XML::Document:0xf1de34 name="document" [...] >
The documentation on the official webpage is also much better than that of REXML, IMHO.
I almost immediately found the answer.
The first thing I did was to search in the ruby source code for the error being thrown.
I found that regex.h was responsible for this.
In regex.h, the code flow is something like this:
/* Maximum number of duplicates an interval can allow. */
#ifndef RE_DUP_MAX
#define RE_DUP_MAX ((1 << 15) - 1)
#endif
Now the problem here is RE_DUP_MAX. On AIX box, the same constant has been defined somewhere in /usr/include.
I searched for it and found in
/usr/include/NLregexp.h
/usr/include/sys/limits.h
/usr/include/unistd.h
I am not sure which of the three is being used(most probably NLregexp.h).
In these headers, the value of RE_DUP_MAX has been set to 255! So there is a cap placed on the number of repetitions of a regex!
In short, the reason is the compilation taking the system defined value than that we define in regex.h!
This also answers my question which i had asked recently:
Regex limit in ruby 64 bit aix compilation
I was not able to answer it immediately as i need to have min of 100 reputation :D :D
Cheers!

Resources