How to parse using Grok from Java.. Is there any example available.? - parsing

Question

I have seen Grok being very strong and lethal in parsing the log data. I wanted to use Grok for log parsing in our application, which is in java.. How can i connect/work with Grok from Java.?

Answer 1

Try downloading java-grok from GitHub: https://github.com/NFLabs/java-grok
You can test patterns using the Grok Debugger: http://grokdebug.herokuapp.com/

Answer 2

Check out this Java library
https://github.com/aicer/grok
You can include it in your project as a maven dependency
<dependency>
<groupId>org.aicer.grok</groupId>
<artifactId>grok</artifactId>
<version>0.9.0</version>
</dependency>
It comes with pre-defined patterns and you can also add yours.
The named patterns are extracted and the results are available in a map with the groups names as the keys and the retrieved values are mapped to these keys.
final GrokDictionary dictionary = new GrokDictionary();
// Load the built-in dictionaries
dictionary.addBuiltInDictionaries();
// Add custom pattern
dictionary.addDictionary(new File(patternDirectoryOrFilePath));
// Resolve all expressions loaded
dictionary.bind();
This next examples adds string patterns directly to the dictionary without using a file
final GrokDictionary dictionary = new GrokDictionary();
// Load the built-in dictionaries
dictionary.addBuiltInDictionaries();
// Add custom pattern directly
dictionary.addDictionary(new StringReader("DOMAINTLD [a-zA-Z]+"));
dictionary.addDictionary(new StringReader("EMAIL %{NOTSPACE}#%{WORD}\.%{DOMAINTLD}"));
// Resolve all expressions loaded
dictionary.bind();
Here is a complete example of how to use the library
public final class GrokStage {
private static final void displayResults(final Map<String, String> results) {
if (results != null) {
for(Map.Entry<String, String> entry : results.entrySet()) {
System.out.println(entry.getKey() + "=" + entry.getValue());
}
}
}
public static void main(String[] args) {
final String rawDataLine1 = "1234567 - israel.ekpo#massivelogdata.net cc55ZZ35 1789 Hello Grok";
final String rawDataLine2 = "98AA541 - israel-ekpo#israelekpo.com mmddgg22 8800 Hello Grok";
final String rawDataLine3 = "55BB778 - ekpo.israel#example.net secret123 4439 Valid Data Stream";
final String expression = "%{EMAIL:username} %{USERNAME:password} %{INT:yearOfBirth}";
final GrokDictionary dictionary = new GrokDictionary();
// Load the built-in dictionaries
dictionary.addBuiltInDictionaries();
// Resolve all expressions loaded
dictionary.bind();
// Take a look at how many expressions have been loaded
System.out.println("Dictionary Size: " + dictionary.getDictionarySize());
Grok compiledPattern = dictionary.compileExpression(expression);
displayResults(compiledPattern.extractNamedGroups(rawDataLine1));
displayResults(compiledPattern.extractNamedGroups(rawDataLine2));
displayResults(compiledPattern.extractNamedGroups(rawDataLine3));
}
}