I am using serilog in asp.net MVC (.net framework 4.6).
The jsonformatter allows to emit json logs in the text file using rollingfile. How can the output format be controlled for this so that we can change the json template and also add delimiters so that when the file is produced it has [ at the top and ] at the end after writting all the logs, plus comma , after ever log?
Also how to add classname and methodname in output json logs. I tried using enrich property but it is creating another object called properties. Rather than creating another object can it create a simple property like this:
{"Timestamp":"2022-10-27T16:25:44.8339002+05:30","Level":"Information","MessageTemplate":"Starting the program....","ClassName":"Program","MethodName":"Main"}
And for multiple such logs the file looks like below:
[{"Timestamp":"2022-10-27T16:25:44.8339002+05:30","Level":"Information","MessageTemplate":"Starting the program....","ClassName":"Program","MethodName":"Main"},
{"Timestamp":"2022-10-27T16:25:44.8339002+05:30","Level":"Information","MessageTemplate":"Starting the program....","ClassName":"Program","MethodName":"Main"},
{"Timestamp":"2022-10-27T16:25:44.8339002+05:30","Level":"Information","MessageTemplate":"Starting the program....","ClassName":"Program","MethodName":"Main"}]
var log = new LoggerConfiguration()
.MinimumLevel.Debug()
.WriteTo.RollingFile(new JsonFormatter(), LogFileNameWithPath, fileSizeLimitBytes: 5242880, retainedFileCountLimit: null)
.CreateLogger();
Related
We are trying to load test our infrastructure of logstash/elastic. Since the actual logs are generated by a software that uses hardware, we are unable to simulate it at scale.
I am wondering if we can store the logs using file sink and later write a program that reads the log files and send data through the actual sink. Since, we are trying different setup, it would be great if we can swap different sinks for testing. Say http sink and elastic sink.
I thought of reading the json file one line at a time and then invoking Write method on the Logger. However I am not sure how to get the properties array from the json. Also, it would be great to hear if there are better alternatives in Serilog world for my needs.
Example parsing
var events= File.ReadAllLines(#"C:\20210520.json")
.Select(line => JsonConvert.DeserializeObject<dynamic>(line));
foreach (var o in objects)
{
DateTime timeStamp = o.Timestamp;
LogEventLevel level = o.Level;
string messageTemplate = o.MessageTemplate;
string exception = o.Exception;
var properties = (o.Properties as JObject);
List<object> parameters = new List<object>();
foreach (var property in properties)
{
if(messageTemplate.Contains(property.Key))
parameters.Add(property.Value.ToString());
}
logInstance.Write(level, messageTemplate, parameters.ToArray());
count++;
}
Example Json Event written to the file
{"Timestamp":"2021-05-20T13:15:49.5565372+10:00","Level":"Information","MessageTemplate":"Text dialog with {Title} and {Message} displayed, user selected {Selected}","Properties":{"Title":"Unload Device from Test","Message":"Please unload the tested device from test jig","Selected":"Methods.Option","SinkRepository":null,"SourceRepository":null,"TX":"TX2937-002 ","Host":"Host1","Session":"Host1-2021.05.20 13.12.44","Seq":87321,"ThreadId":3}}
UPDATE
Though this works for simple events,
it is not able to handle Context properties (there is a work around though using ForContext),
also it forces all the properties to be of type string and
not to mention that destucturing (#property) is not handled properly
If you can change the JSON format to Serilog.Formatting.Compact's CLEF format, then you can use Serilog.Formatting.Compact.Reader for this.
In the source app:
// dotnet add package Serilog.Formatting.Compact
Log.Logger = new LoggerConfiguration()
.WriteTo.File(new CompactJsonFormatter(), "./logs/myapp.clef")
.CreateLogger();
In the load tester:
// dotnet add package Serilog.Formatting.Compact.Reader
using (var target = new LoggerConfiguration()
.MinimumLevel.Verbose()
.WriteTo.Console()
.CreateLogger())
{
using (var file = File.OpenText("./logs/myapp.clef"))
{
var reader = new LogEventReader(file);
while (reader.TryRead(out var evt))
target.Write(evt);
}
}
Be aware though that load testing results won't be accurate for many sinks if you use repeated timestamps. You should consider re-mapping the events you read in to use current timestamps.
E.g. once you've loaded up evt:
var current = new LogEvent(DateTimeOffset.Now,
evt.Level,
evt.Exception,
evt.MessageTemplate,
evt.Properties);
target.Write(current);
I want to read the error messages from messages file but I am unable to. What mistake am I making?
The code where I want to read the string from messages file is
Future { Ok(Json.toJson(JsonResultError(messagesApi("error.incorrectBodyType")(langs.availables(0))))) }
The messages file error.incorrectBodyType=Incorrect body type. Body type must be JSON
The messagesApi("error.incorrectBodyType") should return Incorrect body type. Body type must be JSON but it returns error.incorrectBodyType.
If I remove double quotes in messagesApi(error.incorrectBodyType) then the code doesn't compile
Update
I added a couple of debug prints and notice that the keys I am using in MessagesApi are not defined. I don't know why though as I have created them in messages file.
println("langs array"+langs.availables)
println("app.title"+messagesApi.isDefinedAt("app.title")(langs.availables(0)))
println("error"+messagesApi.isDefinedAt("error.incorrectBodyType")(langs.availables(0)))
prints
langs arrayList(Lang(en_GB))
app.titlefalse
errorfalse
Update 2
I might have found the issue but I don't know how to resolve it. Basically, I am running my test case without an instance of the Application. I am mocking messagesApi by calling stubMessagesApi() defined in Helpers.stubControllerComponents, If I run the same code using an Application eg class UserControllerFunctionalSpec extends PlaySpec with OneAppPerSuiteWithComponents then app.title and error are defined. It seems without an instance of Application, MessagesApi is not using the messages file.
I was able to solve the issue by creating a new instance of MessagesApi using DefaultMessagesApi
val messagesApi = new DefaultMessagesApi( //takes map of maps. the first is the language file, the 2nd is the map between message title and description
Map("en" -> //the language file
Map("error.incorrectBodyType" -> "Incorrect body type. Body type must be JSON") //map between message title and description
)
)
val controller = new UserController(mockUserRepository,mockControllerComponents,mockSilhouette,messagesApi,stubLangs())
In my Grails 3 application.yml, I'm defining a list of maps as follows:
tvoxx:
cfpApis:
-
url: http://cfp.devoxx.be/api/conferences
youtubeChannelId: UCCBVCTuk6uJrN3iFV_3vurg
-
url: http://cfp.devoxx.fr/api/conferences
-
url: http://cfp.devoxx.ma/api/conferences
youtubeChannelId: UC6vfGtsJr5RoBQBcHg24XQw
-
url: http://cfp.devoxx.co.uk/api/conferences
-
url: http://cfp.devoxx.pl/api/conferences
But when I try to load this config in my service using the following code, apiConfig is null:
def apiConfig = grailsApplication.config.getProperty("tvoxx.cfpApis")
I don't get any error when the application starts and my YAML code parses correctly on http://yaml-online-parser.appspot.com/ so I don't know what's wrong.
Just to confirm what we discussed on Slack.
Using grailsApplication.config.getProperty("tvoxx.cfpApis"), Grails will try to find value of type String and because your value is a Map null will be returned.
You have to explicitly tell what type you expect, using:
grailsApplication.config.getProperty("tvoxx.cfpApis", Map)
Other way is to use getAt() method, where object is returned, so you can use
grailsApplication.config.tvoxx.cfpApis to get the value.
First one may be better for .java and #CompileStatic but for standard .groovy class latter has easier syntax. Just watch out for keys which does not exist, because it will return empty ConfigObject instead of null, and for example ?.toString() method will result in 'ConfigObject#123123 instead of null
We are using Tapestry 5.4-beta-4. My problem is:
I need to keep files with locale data in an external location and under different file name then tapestry usual app.properties or pageName_locale.properties. Those files pool messages that should be then used on all pages as required (so no tapestry usual one_page-one_message_file). The files are retrieved and loaded into tapestry during application startup. Currently i am doing it like this:
#Contribute(ComponentMessagesSource.class)
public void contributeComponentMessagesSource(OrderedConfiguration<Resource> configuration, List<String> localeFiles, List<String> languages) {
for(String language: languages){
for(String fileName : localeFiles){
String localeFileName = fileName + "_" + language + ".properties";
Resource resource = new Resource(localeFileName );
configuration.add(localeFileName, resource, "before:AppCatalog");
}
}
}
The above code works in that the message object injected into pages is populated with all the messages. Unfortunatly these are only the messages that are in the default ( first on the tapestry.supported-locales list) locale. This never changes.
We want the locale to be set to the browser locale, send to the service in the header. This works for those messages passed to tapestry in the traditional way (through app.properties) but not for those set in the above code. Actually, if the browser language changes, the Messages object changes too but only those keys that were in the app.properties are assigned new values. Keys that were from external files always have the default values.
My guess is that tapestry doesn't know which keys from Messages object it should refresh (the keys from external files ale not beeing linked to any page).
Is there some way that this could be solved with us keeping the current file structure?
I think the problem is that you add the language (locale) to the file name that you contribute to ComponentMessagesSource.
For example if you contribute
example_de.properties
Tapestry tries to load
example_de_<locale>.properties
If that file does not exist, it will fall back to the original file (i.e. example_de.properties).
Instead you should contribute
example.properties
and Tapestry will add the language to the file name automatically (see MessagesSourceImpl.findBundleProperties() for actual implementation).
#Contribute(ComponentMessagesSource.class)
public void contributeComponentMessagesSource(OrderedConfiguration<Resource> configuration, List<String> localeFiles, List<String> languages) {
for(String language: languages){
for(String fileName : localeFiles){
String localeFileName = fileName + ".properties";
Resource resource = new Resource(localeFileName );
configuration.add(localeFileName, resource, "before:AppCatalog");
}
}
}
How will i be able to
set parameters from java class
access it from Birt report
please elaborate i am new to this.
Use IRunAndRenderTask class in your java program to add parameters.
eg:
IRunAndRenderTask task = engine.createRunAndRenderTask(design);
// pass necessary parameters
task.setParameterValue("ordParam", (new Integer(10101))); //static parameter
task.setParameterValue("value", Integer.parseInt(value)); // pass dynamic parameter
task.validateParameters();
Also add a new parameter in Data Explorer Report parameter with name same as that you have given in your program. like here "value".
And Atlast,Drag your parameter in your report Design. and its done.
Welcome user.
To print Birt Report in Excel format Use EXCELRenderOption class.
EXCELRenderOption xlsOptions = new EXCELRenderOption(options);
xlsOptions.setOutputFormat("xls");
response.setHeader( "Content-Disposition", "attachment; filename="+downloadFileName);
xlsOptions.setImageHandler(new HTMLServerImageHandler());
xlsOptions.setOutputStream(response.getOutputStream());
//xlsOptions.setOption(IRenderOption.EMITTER_ID,"org.uguess.birt.report.engine.emitter.xls");
xlsOptions.setOption(IRenderOption.EMITTER_ID, "org.eclipse.birt.report.engine.emitter.prototype.excel");
iRenderTask.setRenderOption(xlsOptions);