Ml.NET - Pedicting department based on certain properties - machine-learning

Question

I'm trying to predict what department should recieve a new ticket whenever a customer writes a support ticket. A support ticket will always consist of:
Message: the actual message of the ticket
Subject: the subject of the ticket
Header: and optional header to further describe the issue
Until now, the assignment of a ticket is manual, but I'm trying to create a model that can predict the department this ticket should be assigned to. My training data contains these three variables as well as the department name
Support
Development
...
My training data consists of 169k rows, and there is no issue training the model at all. I'm getting some pretty neat metrics. The issue comes when I try to predict from some kind of ticket. Instead of getting a department name, I'm getting a decimal (2.5xx). I'm not sure where this is going wrong. I've taken inspiration from the following guide from Microsoft: https://learn.microsoft.com/en-us/dotnet/machine-learning/tutorials/github-issue-classification
Loading training data
var dataView = mlContext.Data.LoadFromTextFile<Message>(dataPath, hasHeader: true, allowQuoting: true);
Building and training the model
var pipeline = mlContext.Transforms.Conversion.MapValueToKey(inputColumnName: "Name", outputColumnName: "Label") // Department name
.Append(mlContext.Transforms.Text.FeaturizeText(inputColumnName: "Message", outputColumnName: "MessageFeaturized"))
.Append(mlContext.Transforms.Text.FeaturizeText(inputColumnName: "Subject", outputColumnName: "SubjectFeaturized"))
.Append(mlContext.Transforms.Text.FeaturizeText(inputColumnName: "Header", outputColumnName: "HeaderFeaturized"))
.Append(mlContext.Transforms.Concatenate(
"Features",
"MessageFeaturized",
"SubjectFeaturized",
"HeaderFeaturized"))
.AppendCacheCheckpoint(mlContext)
.Append(mlContext.MulticlassClassification.Trainers.SdcaMaximumEntropy("Label", "Features"))
.Append(mlContext.Transforms.Conversion.MapKeyToValue("PredictedLabel"));
var model = pipeline.Fit(dataView);
Predicting department name
var predictionEngine = mlContext.Model.CreatePredictionEngine<Message, PredictedDepartment>(model);
var message = new Message()
{
Message = #"...", // removed for brevity
Subject = "Other questions",
Header = "Internet connection"
};
var prediction = predictionEngine.Predict(message);
Console.WriteLine($"Prediction result: {prediction.Name}"); // Department name
Result
Prediction result: 2.581
Models used
class Message
{
[LoadColumn(2)]
public string Message { get; set; }
[LoadColumn(0)]
public string Subject { get; set; }
[LoadColumn(1)]
public string Header { get; set; }
[LoadColumn(3)]
public string Name { get; set; } // Department name
}
class PredictedDepartment
{
[ColumnName("PredictedLabel")]
public string Name { get; set; } // Department name
}
Could someone explain me what's happening, and why I'm not getting some string value back, and instead getting some decimal?

Answer 1

I found my issue.
It has nothing to do with the my pipeline or the way I train my model. Nor is it an issue with the MapKeyToValue method in my pipeline. I simply had an issue with my dataset.
My CSV was comma seperated and used quoting for strings, as my Message property was multiline. Instead of trying to support this, I changed my export from my data to remove line breaks, and not quote strings. I also changed to a tab seperator instead of a comma seperator.
The model now seems to work, and I'm getting quite a good prediction.