Understanding mahout classification output - mahout

I have trained mahout model for three categories Category_A,Category_B,Category_C using 20newsGroupExample , Now i want to classify my documents using this model. Can somebody help me to understand output i am getting from this model.
Here is my output
{0:-2813549.8786637094,1:-2651723.736745838,2:-2710651.7525975127}
According to output category of document is 1, But expected category is 2. Am i going right or something is missing in my code ?
public class NaiveBayesClassifierExample {
public static void loadClassifier(String strModelPath, Vector v)
throws IOException {
Configuration conf = new Configuration();
NaiveBayesModel model = NaiveBayesModel.materialize(new Path(strModelPath), conf);
AbstractNaiveBayesClassifier classifier = new StandardNaiveBayesClassifier(model);
Vector st = classifier.classifyFull(v);
System.out.println(st.asFormatString());
System.out.println(st.maxValueIndex());
st.asFormatString();
}
public static Vector createVect() throws IOException {
FeatureVectorEncoder encoder = new StaticWordValueEncoder("text");
Analyzer analyzer = new StandardAnalyzer(Version.LUCENE_36);
String inputData=readData();
StringReader in = new StringReader(inputData);
TokenStream ts = analyzer.tokenStream("body", in);
CharTermAttribute termAtt = ts.addAttribute(CharTermAttribute.class);
Vector v1 = new RandomAccessSparseVector(100000);
while (ts.incrementToken()) {
char[] termBuffer = termAtt.buffer();
int termLen = termAtt.length();
String w = new String(termBuffer, 0, termLen);
encoder.addToVector(w, 1.0, v1);
}
v1.normalize();
return v1;
}
private static String readData() {
// TODO Auto-generated method stub
BufferedReader reader=null;
String line, results = "";
try{
reader = new BufferedReader(new FileReader("c:\\inputFile.txt"));
while( ( line = reader.readLine() ) != null)
{
results += line;
}
reader.close();
}
catch(Exception ex)
{
ex.printStackTrace();
}
return results;
}
public static void main(String[] args) throws IOException {
Vector v = createVect();
String mp = "E:\\Final_Model\\model";
loadClassifier(mp, v);
}
}

Related

Error executing: Java.Lang.Class[] parameterType = new Java.Lang.Class[] { Java.Lang.Class.FromType(typeof(sbyte[])) }; in Xamarin.Android

I'm trying to compile an android vector image in Xamarin Android based on a Java Sample but The system shows the following error:
{System.ArgumentException: type Parameter name: Type is not derived from a java type. at Java.Lang.Class.FromType (System.Type type) [0x00012] in the following part of the code:
Java.Lang.Class[] parameterType = new Java.Lang.Class[] { Java.Lang.Class.FromType(typeof(sbyte[])) };
Basically
private Drawable GetVectorDrawable(Context context, byte[] binXml)
{
try
{
// Get the binary XML parser (XmlBlock.Parser) and use it to create the drawable
// This is the equivalent of what AssetManager#getXml() does
// get the Class instance using forName method
var xmlBlock = Java.Lang.Class.ForName("android.content.res.XmlBlock");
System.Diagnostics.Debug.WriteLine(typeof(sbyte[]));
Java.Lang.Class[] parameterType = new Java.Lang.Class[] { Java.Lang.Class.FromType(typeof(sbyte[])) };
var xmlBlockConstr = xmlBlock.GetConstructor(parameterType);
//var xmlBlockConstr = xmlBlock.GetConstructor(Class.FromType(typeof(sbyte[])));
var xmlParserNew = xmlBlock.GetDeclaredMethod("newParser");
xmlBlockConstr.Accessible = true;
xmlParserNew.Accessible = true;
var parser = xmlParserNew.Invoke(xmlBlockConstr.NewInstance(binXml)) as XmlPullParser;
//System.Xml.XmlReader reader = parser.ToString();
//if (Build.VERSION.SdkInt >= (BuildVersionCodes)24)
//{
//ByteArray
//Drawable.CreateFromXml(context.Resources, reader);
//}
}
catch (System.Exception e)
{
}
return null;
}
This is the constructor I try to get:
public XmlBlock(byte[] data) {
mAssets = null;
mNative = nativeCreate(data, 0, data.length);
mStrings = new StringBlock(nativeGetStringBlock(mNative), false);
}
And this is the Java code I tried to traduce:
/**
* Create a vector drawable from a binary XML byte array.
* #param context Any context.
* #param binXml Byte array containing the binary XML.
* #return The vector drawable or null it couldn't be created.
*/
public static Drawable getVectorDrawable(#NonNull Context context, #NonNull byte[] binXml) {
try {
// Get the binary XML parser (XmlBlock.Parser) and use it to create the drawable
// This is the equivalent of what AssetManager#getXml() does
#SuppressLint("PrivateApi")
Class<?> xmlBlock = Class.forName("android.content.res.XmlBlock");
Constructor xmlBlockConstr = xmlBlock.getConstructor(byte[].class);
Method xmlParserNew = xmlBlock.getDeclaredMethod("newParser");
xmlBlockConstr.setAccessible(true);
xmlParserNew.setAccessible(true);
XmlPullParser parser = (XmlPullParser) xmlParserNew.invoke(
xmlBlockConstr.newInstance((Object) binXml));
if (Build.VERSION.SDK_INT >= 24) {
return Drawable.createFromXml(context.getResources(), parser);
} else {
// Before API 24, vector drawables aren't rendered correctly without compat lib
final AttributeSet attrs = Xml.asAttributeSet(parser);
int type = parser.next();
while (type != XmlPullParser.START_TAG) {
type = parser.next();
}
return VectorDrawableCompat.createFromXmlInner(context.getResources(), parser, attrs, null);
}
} catch (Exception e) {
Log.e(TAG, "Vector creation failed", e);
}
return null;
}
from this url: Given byte-array of VectorDrawable, how can I create an instance of VectorDrawable from it?
Can you help me?

Sentiment Analysis with OpenNLP

I found this description of implementing a Sentiment Analysis task with OpenNLP. In my case I am using the newest OPenNLP-version, i.e., version 1.8.0. In the following example, they use a Maximum Entropy Model. I am using the same input.txt (tweets.txt)
http://technobium.com/sentiment-analysis-using-opennlp-document-categorizer/
public class StartSentiment {
public static DoccatModel model = null;
public static String[] analyzedTexts = {"I hate Mondays!"/*, "Electricity outage, this is a nightmare"/*, "I love it"*/};
public static void main(String[] args) throws IOException {
// begin of sentiment analysis
trainModel();
for(int i=0; i<analyzedTexts.length;i++){
classifyNewText(analyzedTexts[i]);
}
}
private static String readFile(String pathname) throws IOException {
File file = new File(pathname);
StringBuilder fileContents = new StringBuilder((int)file.length());
Scanner scanner = new Scanner(file);
String lineSeparator = System.getProperty("line.separator");
try {
while(scanner.hasNextLine()) {
fileContents.append(scanner.nextLine() + lineSeparator);
}
return fileContents.toString();
} finally {
scanner.close();
}
}
public static void trainModel() {
MarkableFileInputStreamFactory dataIn = null;
try {
dataIn = new MarkableFileInputStreamFactory(
new File("bin/text.txt"));
ObjectStream<String> lineStream = null;
lineStream = new PlainTextByLineStream(dataIn, StandardCharsets.UTF_8);
ObjectStream<DocumentSample> sampleStream = new DocumentSampleStream(lineStream);
TrainingParameters tp = new TrainingParameters();
tp.put(TrainingParameters.CUTOFF_PARAM, "2");
tp.put(TrainingParameters.ITERATIONS_PARAM, "30");
DoccatFactory df = new DoccatFactory();
model = DocumentCategorizerME.train("en", sampleStream, tp, df);
} catch (Exception e) {
e.printStackTrace();
} finally {
if (dataIn != null) {
try {
} catch (Exception e2) {
e2.printStackTrace();
}
}
}
}
public static void classifyNewText(String text){
DocumentCategorizerME myCategorizer = new DocumentCategorizerME(model);
double[] outcomes = myCategorizer.categorize(new String[]{text});
String category = myCategorizer.getBestCategory(outcomes);
if (category.equalsIgnoreCase("1")){
System.out.print("The text is positive");
} else {
System.out.print("The text is negative");
}
}
}
In my case no matter what input String I am using, I am only getting a positive estimation of the input string. Any idea what could be the reason?
Thanks

Flume Custom Interceptor is not Working

I want to change IP address in data by its mask IP. This is done in the "backup" part of my Flume agent (see below).
In this configuration there are 2 channels: the first channel dumps data to HBase, while the second one is used for backup:
a1.sources = r1 r2
a1.channels = channel1 Backup_channel
a1.sinks = FSink
a1.sources.r1.handler = com.flume.handler.JSONHandler
a1.sources.r1.type = avro
a1.sources.r1.bind = x.x.x.x
a1.sources.r1.port = 10008
a1.sources.r2.handler = com.flume.handler.JSONHandler
a1.sources.r2.type = avro
a1.sources.r2.bind = x.x.x.x
a1.sources.r2.port = 10009
a1.sources.r2.interceptors = i1
a1.sources.r2.interceptors.i1.type = com.flume.interceptor.DcInterceptor
a1.channels.channel1.type = file
a1.channels.channel1.checkpointDir = /root/flume/channels/Livechannel/checkpoint
a1.channels.channel1.dataDirs = /root/flume/channels/Livechannel/data
a1.sinks.FSink.type = hbase
a1.sinks.FSink.table = Temp_Test
a1.sinks.FSink.batchSize = 300
a1.sinks.FSink.columnFamily = T
a1.sinks.FSink.serializer = com.flume.sink.TestTP
a1.sources.r1.channels = channel1
a1.sources.r2.channels = Backup_channel
a1.channels.Backup_channel.type = file
a1.channels.Backup_channel.checkpointDir = /data/disk/flume/backup/checkpoint
a1.channels.Backup_channel.dataDirs = /data/disk/flume/backup/data
a1.sinks.FSink.channel = channel1
Following is my custom Java Interceptor Code. It implements the interception method, which get an IP address from the body, calculates its IP mask and then adds it to the body. But somehow it's not working:
public class DcInterceptor implements Interceptor {
private byte[] jsonTestBeans;
private final Type listType = new TypeToken < List < TestBeans >> () {}.getType();
#Override
public void close() {
// TODO Auto-generated method stub
}
#Override
public void initialize() {
// TODO Auto-generated method stub
new Logger();
}
#Override
public Event intercept(Event event) {
// TODO Auto-generated method stub
List < Row > actions = new ArrayList < Row > ();
this.jsonTestBeans = event.getBody();
Logger.logger.debug("In Interceptor");
System.out.println("In Interceptor");
Gson _Gson = new Gson();
String jsonstr = "";
try {
jsonstr = new String(jsonTestBeans, "UTF-8");
} catch (Exception e) {
// TODO: handle exception
Logger.logger.error(e.getMessage() + "In Interceptor");
jsonstr = new String(jsonTestBeans);
}
List < TestBeans > TestBeanss = _Gson.fromJson(jsonstr, listType);
System.out.println("Json String :" + jsonstr);
List < String > gTouch = new ArrayList < String > ();
for (TestBeans TestBeans: TestBeanss) {
String str = TestBeans.getIp();
Logger.logger.debug("IP : " + str);
String st = (str.substring(0, str.lastIndexOf(".") + 1) + "x");
Logger.logger.debug("Mask IP : " + st);
TestBeans.setRemoteIp(st);
}
event.setBody(_Gson.toJson(TestBeanss).getBytes());
Logger.logger.debug("Interceptor Ends");
return event;
}
#Override
public List < Event > intercept(List < Event > events) {
// TODO Auto-generated method stub
System.out.println("In List Interceptor");
Logger.logger.debug("In List Interceptor");
for (Event event: events) {
intercept(event);
}
return events;
}
public static class CounterInterceptorBuilder implements Interceptor.Builder {
private Context ctx;
#Override
public Interceptor build() {
Logger.logger.debug("In Interceptor Build");
System.out.println("In Build Interceptor");
return new DcInterceptor();
}
#Override
public void configure(Context context) {
this.ctx = context;
}
}
At least, I can see:
The configuration lines regarding your interceptor refer to an agent called ECircleTp_Test, while the rest of the configuration refer to a1.
You have configured com.flume.interceptor.DcInterceptor2, but the interceptor class you have developed is called DcInterceptor (without the final 2).
You have configured com.flume.interceptor.DcInterceptor2 as the fully qualified class name of your custom interceptor. Nevertheless, the code of the interceptor does not declare any package for the DcInterceptor(2) class.

URL Read CodeName One

I'm relatively new to Codename One. I'm trying to read an URL and save the content on a String. I tried:
private String lectura = "";
private String escritura = "";
/*-------------------------------------------------------
* Methods
*-------------------------------------------------------
*/
public Bulbs(int i, char rtype){
type = rtype;
number = i;
status = readCNO(type, number);
}
public String giveStatus(){
status = readCNO(type, number);
return status;
}
public void turnBulbOn(){
writeCNO('B', number, 1);
}
public void turnBulbOff(){
writeCNO('B', number, 0);
}
public String readCNO(char type, int number){
ConnectionRequest r = new ConnectionRequest();
r.setUrl("http://192.168.1.3/arduino/R!" + type + "/" + Integer.toString(number));
r.setPost(false);
r.addResponseListener(new ActionListener()
{
public void actionPerformed(ActionEvent ev)
{
try
{
NetworkEvent event = (NetworkEvent) ev;
byte[] data= (byte[]) event.getMetaData();
String decodedData = new String(data,"UTF-8");
System.out.println(decodedData);
lectura = decodedData;
} catch (Exception ex)
{
ex.printStackTrace();
lectura = "NoBulb";
}
}
});
NetworkManager.getInstance().addToQueue(r);
return lectura;
}
public String writeCNO(char type, int number, int action){
ConnectionRequest r2 = new ConnectionRequest();
r2.setUrl("http://192.168.1.3/arduino/R!" + type + "/" + Integer.toString(number) + "/"+ action);
r2.setPost(false);
r2.addResponseListener(new ActionListener()
{
public void actionPerformed(ActionEvent ev)
{
try
{
NetworkEvent event = (NetworkEvent) ev;
byte[] data= (byte[]) event.getMetaData();
String decodedData = new String(data,"UTF-8");
System.out.println(decodedData);
escritura = decodedData;
} catch (Exception ex)
{
ex.printStackTrace();
escritura = "NoBulb";
}
}
});
NetworkManager.getInstance().addToQueue(r2);
return escritura;
}
However when I run it, the Console displays a bunch of errors like:
Duplicate entry in the queue: com.codename1.io.ConnectionRequest: com.codename1.io.ConnectionRequest#22b3c488
Help very appreciated!
David.
You are adding the exact same URL to the queue twice which Codename One detects as a probable mistake. If this is intentional just invoke setDuplicateSupported(true) on both connection requests.

Using streamreader to read line containing this "//"?

Read a Text file having any line starts from "//" omit this line and moved to next line.
The Input text file having some seprate partitions. Find line by line process and this mark.
If you are using .Net 3.5 you can use LINQ with a IEnumerable wrapped around a Stream Reader. This cool part if then you can just use a where statement to file statmens or better yet use a select with a regular expression to just trim the comment and leave data on the same line.
//.Net 3.5
static class Program
{
static void Main(string[] args)
{
var clean = from line in args[0].ReadAsLines()
let trimmed = line.Trim()
where !trimmed.StartsWith("//")
select line;
}
static IEnumerable<string> ReadAsLines(this string filename)
{
using (var reader = new StreamReader(filename))
while (!reader.EndOfStream)
yield return reader.ReadLine();
}
}
...
//.Net 2.0
static class Program
{
static void Main(string[] args)
{
var clean = FilteredLines(args[0]);
}
static IEnumerable<string> FilteredLines(string filename)
{
foreach (var line in ReadAsLines(filename))
if (line.TrimStart().StartsWith("//"))
yield return line;
}
static IEnumerable<string> ReadAsLines(string filename)
{
using (var reader = new StreamReader(filename))
while (!reader.EndOfStream)
yield return reader.ReadLine();
}
}
I'm not sure what you exactly need but, if you just want to filter out // lines from some text in a stream... just remember to close the stream after using it.
public string FilterComments(System.IO.Stream stream)
{
var data = new System.Text.StringBuilder();
using (var reader = new System.IO.StreamReader(stream))
{
var line = string.Empty;
while (!reader.EndOfStream)
{
line = reader.ReadLine();
if (!line.TrimStart(' ').StartsWith("//"))
{
data.Append(line);
}
}
}
return data.ToString();
}
Class SplLineIgnorStrmReader:StreamReader // derived class from StreamReader
SplLineIgnorStrmReader ConverterDefFileReadStream = null;
{
//created the Obj for this Class.
Obj = new SplLineIgnorStrmReader(strFile, Encoding.default);
}
public override string ReadLine()
{
string strLineText = "", strTemp;
while (!EndOfStream)
{
strLineText = base.ReadLine();
strLineText = strLineText.TrimStart(' ');
strLineText = strLineText.TrimEnd(' ');
strTemp = strLineText.Substring(0, 2);
if (strTemp == "//")
continue;
break;
}
return strLineText;
This is if u want to read the Text file and omit any comments from that file(here exclude "//" comment).

Resources