Not able to send log4net logs to application insights - docker

I have dotnetcore service which is running in Kubernetes. For logging, I have used the log4net. I am trying to send the logs of service to application-insights but the logs are not showing there.
I have added the below configuration in my service for application insights.
Runtime version: netcoreapp3.1 version-2.31.0.1
Hosting environment: Azure-cluster
app-service.csproj
<ItemGroup>
<PackageReference Include="Microsoft.ApplicationInsights.AspNetCore" Version="2.16.0" />
<PackageReference Include="Microsoft.ApplicationInsights.Log4NetAppender" Version="2.16.0" />
...
<ItemGroup>
log4net.config
...
<appender name="aiAppender" type="Microsoft.ApplicationInsights.Log4NetAppender.ApplicationInsightsAppender, Microsoft.ApplicationInsights.Log4NetAppender">
<InstrumentationKey name="AppInsightsKey" value="abcdefgh-abcd-abcd-abcd-abcdefghijkl" />
<layout type="log4net.Layout.PatternLayout">
<conversionPattern value="%message%newline"/>
</layout>
</appender>
startup.cs
public Startup(IConfiguration configuration)
{
Configuration = configuration;
Logger.ConfigureLog4Net("./logs/app.log", Configuration)
...
}
public void ConfigureServices(IServiceCollection services)
{
// The following line enables Application Insights telemetry collection.
services.AddApplicationInsightsTelemetry();
services.AddSingleton<ITelemetryInitializer, CloudRoleNameTelemetryInitializer>();
// This code adds other services for the application.
services.AddMvc();
...
}
CloudRoleNameTelemetryInitializer.cs
using System.Net;
using Microsoft.ApplicationInsights;
using Microsoft.ApplicationInsights.Channel;
using Microsoft.ApplicationInsights.Extensibility;
public class CloudRoleNameTelemetryInitializer : ITelemetryInitializer
{
public void Initialize(ITelemetry telemetry)
{
// set custom role name here
telemetry.Context.Cloud.RoleName = "app-service";
}
}
I have also added the APPINSIGHTS_INSTRUMENTATIONKEY environment variable for service. I am able to see the request information for service but not logs. Only 2 logs are showing in the log section of application-insights but it is not from the application(log4net).
No XML encryptor configured. Key {XXX-XXX-XXX-XXX-XXX} may be persisted to storage in unencrypted form.
Storing keys in a directory '/root/.aspnet/DataProtection-Keys' that may not be persisted outside of the container. Protected data will be unavailable when the container is destroyed.
Below are some links I have explored.
https://learn.microsoft.com/en-us/azure/azure-monitor/app/asp-net-trace-logs
https://medium.com/#muralimohan.mothupally/application-insights-integration-with-log4net-e1bef68fe3c8
https://blog.ehn.nu/2014/11/using-log4net-for-application-insights/
https://jan-v.nl/post/using-application-insights-in-your-log4net-application/
But still, logs are not adding in the application insights. Is there anything I am missing here? Also is it possible to add Console.WriteLine logs to application-insights as like in nodejs app we can send it by enabling the application-insights module's configuration? Can you please help me?
Thanks...

Thanks for Peter Drier's answer, I thinks below code he provided is useful to you.
For more details, you can refer the orgin post.
Log4Net and Application Insights-no data is coming through
<appender name="aiAppender" type="Microsoft.ApplicationInsights.Log4NetAppender.ApplicationInsightsAppender, Microsoft.ApplicationInsights.Log4NetAppender">
<layout type="log4net.Layout.PatternLayout">
<conversionPattern value="%message%newline" />
</layout>
<threshold value="INFO" />
<InstrumentationKey value="12345678-7946-1234-1234-8b330fbe1234" />
</appender>

Related

How to remove swagger production .net core 2.1

I have swagger working on multiple microservices.
When deploying to Azure we need to remove all together the option of swagger due to security best practices.
Working with .net core 2.1
Looking for example of definitions.
First, what "security best practices"? There's nothing wrong with having your API documentation in production. That's actually kind of the whole point: clients should be able to look at the documentation so that they can properly use your API. If these microservices aren't exposed to be used by external clients, then it's even less of an issue, because no one outside can get to the service or the documentation, anyways. If the services are exposed, then they should also be requiring requests to be authorized, and the documentation itself can be locked down via the same mechanism.
Regardless, if you insist on removing this in production, your best bet is to never add it there in the first place. In other words, wrap all your Swagger setup in Startup.cs with if (env.IsDevelopment()) or if you want it available in things like a staging environment: if (!env.IsProduction()).
If you're coming at this from .net core 3.1:
Assuming that the Startup class' constructor copies the injected IConfiguration to a local field called configuration, then you can setup the Configure method like so:
public void configure(IApplicationBuilder app, IWebHostEnvironment env)
{
var applicationName = configuration.GetValue<string>("ApplicationName") ?? "MyApi";
var basePath = configuration.GetValue<string>("BasePath");
if (!string.IsNullOrEmpty(basePath))
app.UsePathBase(basePath);
if (!env.IsProduction())
{
app.UseSwagger();
app.UseSwaggerUI(c =>
{
c.SwaggerEndpoint($"{basePath}/swagger/v1/swagger.json",
$"{applicationName} {ReflectionUtils.GetAssemblyVersion<Program>()}");
});
}
}
First option, disabling it in Configure-method like:
public void Configure(IApplicationBuilder app, IWebHostEnvironment env)
{
// read from config
var config = (IConfiguration)app.ApplicationServices.GetService(typeof(IConfiguration));
enableSwagger = bool.Parse(config.GetValue<string>("EnableSwagger") ?? "false");
if (enableSwagger /*|| env.IsDevelopment()*/)
{
app.UseSwagger(o =>
{
o.RouteTemplate = "swagger/{documentName}/swagger.json";
});
// Enable middleware to serve swagger-ui (HTML, JS, CSS, etc.),
// specifying the Swagger JSON endpoint.
app.UseSwaggerUI(c =>
{
c.SwaggerEndpoint("/swagger/v1/swagger.json", "My API V1");
});
}
}
or use:
if (env.IsDevelopment())
{
...
}
Second option is by completely removing all references and usings in code files by
#if USE_SWAGGER
using Microsoft.OpenApi.Models;
...
#endif
#if USE_SWAGGER
<some method>
...
#endif
add in .csproj:
<Choose>
<When Condition="$(DefineConstants.Contains('USE_SWAGGER'))">
<ItemGroup Condition="'$(TargetFramework)' == 'netstandard3.1'">
<PackageReference Include="Swashbuckle.AspNetCore.Swagger" Version="5.6.3" />
<PackageReference Include="Swashbuckle.AspNetCore.SwaggerGen" Version="5.6.3" />
<PackageReference Include="Swashbuckle.AspNetCore.SwaggerUI" Version="5.6.3" />
</ItemGroup>
</When>
<Otherwise />
</Choose>
or
<Choose>
<When Condition="'$(Configuration)' == 'Debug'">
<ItemGroup Condition="'$(TargetFramework)' == 'netstandard3.1'">
<PackageReference Include="Swashbuckle.AspNetCore.Swagger" Version="5.6.3" />
<PackageReference Include="Swashbuckle.AspNetCore.SwaggerGen" Version="5.6.3" />
<PackageReference Include="Swashbuckle.AspNetCore.SwaggerUI" Version="5.6.3" />
</ItemGroup>
</When>
<Otherwise />
</Choose>
Play around and you get profound!
You could also use [ApiExplorerSettings(IgnoreApi = true)] in the controller classes you don't want to show in the documentation. This is an excludent action, meaning every class will be shown, except the ones marked with this attribute. (this is good when you want to make only a set of your APIs docs visible to a third-party consumer);

Log4Net can't locate logfile anywhere

I have followed many different guides on how to configure the log4net, it is up and running but i can't find a log file anywhere ...
This is how my configuration look like:
Web.Config
<section name="log4net" type="log4net.Config.Log4NetConfigurationSectionHandler, log4net" />
<log4net debug="true">
<appender name="RollingLogFileAppender" type="log4net.Appender.RollingFileAppender">
<file value="C:\\temp\\Log.txt" />
<appendToFile value="true" />
<rollingStyle value="Size" />
<maxSizeRollBackups value="10" />
<maximumFileSize value="10MB" />
<staticLogFileName value="true" />
<layout type="log4net.Layout.PatternLayout">
<conversionPattern value="%-5p %d %5rms %-22.22c{1} %-18.18M - %m%n" />
</layout>
</appender>
<root>
<level value="DEBUG" />
<appender-ref ref="RollingLogFileAppender" />
</root>
</log4net>
Global.asax:
XmlConfigurator.Configure(new FileInfo(Server.MapPath("~/Web.config")));
//XmlConfigurator.Configure();
StartUp.cs
//[assembly: XmlConfigurator(ConfigFile = "Web.config", Watch = true)]
[assembly: XmlConfigurator(Watch = true)]
Declaration
readonly log4net.ILog logger = log4net.LogManager.GetLogger(System.Reflection.MethodBase.GetCurrentMethod().DeclaringType);
Logging
BasicConfigurator.Configure();
logger.Info("Info logging ...");
logger.Error("Homepage loading test logging ...");
Where my file value is: <file value="C:\\temp\\Log.txt" />
I have tried several paths, and commented out what above but no success.
What am i doing wrong?
UPDATE:
As suggested by John H i have tried configuring and calling the logger in the Application_Start method and tried several alternative configs with it but with no luck. Here are 2 screenshots of some debugging info:
Main properties:
Below are the Logger properties:
What am i doing wrong?
OK so i got it to work following this tutorial: log4net-guide-dotnet-logging
I have created a log4net.config file with content as showed in tutorial.
used [assembly: XmlConfigurator(ConfigFile = "log4net.config", Watch = true)]
Called it like this:
ILog logger = LogManager.GetLogger(MethodBase.GetCurrentMethod().DeclaringType);
logger.Info("Application started.");
file is created and content logged as well.
I am gonna compare the config files content and see if the difference is in there and go gradually comparing everything till i have found what caused it not to work.
Thank you for helping me!
Kind regards
From your screenshots, we can see that your logger is not being initialised with your configuration, because IsDebug is false. One thing I notice from your screenshot, is you're trying to pass the path to Web.config directly to the Configure() method. I realise that may be an attempt to solve the problem, so you may have already tried my next suggestion, but calling Configure() in the manner you currently have won't work because Web.config is not published to your bin\debug folder. It will called Web.projectname.config. Calling
XmlConfigurator.Configure()
with no parameters, will automatically resolve the correct configuration file in your output directory. I'm guessing you've tried that, but if that still doesn't work, try this as well:
using log4net;
protected void Application_Start(object sender, EventArgs e)
{
// Initialising configuration before requesting a logger.
XmlConfigurator.Configure();
// Requesting a logger only after the configuration has been initialised.
var logger = LogManager.GetLogger(typeof(Global));
logger.Info("Application started.");
}
I'm not sure it will make any difference, but your configuration looks fine to me.
But by inspecting the IsDebug property on the logger, you'll at least be able to tell if the configuration has even been read.
Edit: One other thing, make sure the application will have the permissions to write to the file. From the documentation:
The RollingFileAppender extends the FileAppender and has the same behavior when opening the log file. The appender will first try to open the file for writing when ActivateOptions() is called. This will typically be during configuration. If the file cannot be opened for writing the appender will attempt to open the file again each time a message is logged to the appender. If the file cannot be opened for writing when a message is logged then the message will be discarded by this appender.

Intercept elmah logging ASP.NET MVC [duplicate]

This question already has an answer here:
elmah error handling - store in database
(1 answer)
Closed 6 years ago.
I would like to try to intercept and do some other processing ( export a database ) when elmah raises an error.Is there a way to do that?
ELMAH exposes two events in order to do this: ErrorLog_Filtering and ErrorLog_Logged. ErrorLog_Filtering is called just before logging to the configured error log and ErrorLog_Logged is called just after. You will find documentation about error filtering on the ELMAH site. ErrorLog_Logged isn't really documented, but you can see an example of it in this article: Logging to multiple ELMAH logs.
With that said, you probably don't want to execute any long running tasks as part of ErrorLog_Logged and ErrorLog_Filtering. It will slow down your system. I'm not sure on what you are trying to achieve here by exporting a database on every error?
Yes there is way around
1.Lets Create a Project Name ElmahMvc
2.Install nuget package nuget
Install-Package Elmah.MVC
3. Raise an Exception .There you go access error log by your local url/elmah
http://localhost:20351/elmah
The saving database and Emailing is bit of configuration to take ....
Lets look at it
1.Download ELMAH script form official site
ELMAH
2.add a database named log and run the script
3.add a connection string in web.config
<connectionStrings>
<add name="elmah" connectionString="Data Source=.;Initial Catalog=log;Persist Security Info=True;User ID=sa;Password=pass" providerName="System.Data.SqlClient" />
</connectionStrings>
4.Add a elmah element in web.config
<elmah>
<errorLog type="Elmah.SqlErrorLog, Elmah" connectionStringName="elmah" />
<!--For Addtional Mail Config .Remove it if not need for mail on every error-->
<errorMail from="waltoncrm#waltonbd.com"
to="mrahman.cse32#waltonbd.com"
subject="Application Exception"
async="false"
smtpPort="25"
smtpServer="YourServer"
userName="uName"
password="pass">
</errorMail>
</elmah>
and mail additional
under System.WebServer
<system.net>
<mailSettings>
<smtp deliveryMethod ="Network">
<network host="smtp.gmail.com" port="587" userName="yourgmailEmailAddress" password="yourGmailEmailPassword" />
</smtp>
</mailSettings>
</system.net>
Download my sample project and Elmah script with Visual Studio 2013
akash365.com
Note:if you donot want to crash your program or use it in asp.net web form Static [WebMethod]
This seems perfect
try
{
//some web method code
}
catch(Exception ex)
{
Elmah.ErrorSignal.FromCurrentContext().Raise(ex);
}

Log4net logging not working - asp.net mvc

My log4net configuration is this,
<?xml version="1.0" encoding="utf-8" ?>
<log4netConfiguration>
<configSections>
<section name="log4net" type="log4net.Config.Log4NetConfigurationSectionHandler, log4net" />
</configSections>
<appSettings>
<add key="log4net.Config" value="log4net.config" />
</appSettings>
<log4net>
<appender name="RollingFileAppender" type="log4net.Appender.RollingFileAppender">
<file type="log4net.Util.PatternString" value="C:\my_logs/my_web_logs/my_log_%date{ddMMyyyy}.log" />
<appendToFile value="true" />
<rollingStyle value="Size" />
<maxSizeRollBackups value="5" />
<maximumFileSize value="30MB" />
<datePattern value="yyyyMMdd" />
<layout type="log4net.Layout.PatternLayout">
<conversionPattern value="%4t %d{ABSOLUTE} %-5p %m%n"/>
<!--<conversionPattern value="%date [%thread] %-5level %logger [%property{NDC}] – %message%newline" />-->
</layout>
</appender>
<root>
<level value="INFO" />
<appender-ref ref="RollingFileAppender" />
</root>
</log4net>
</log4netConfiguration>
I have a Logger helper class as,
public static class Logger
{
private static readonly log4net.ILog log = log4net.LogManager.GetLogger(System.Reflection.MethodBase.GetCurrentMethod().DeclaringType);
public static log4net.ILog Log
{
get { return log; }
}
}
In my assembly info, I have this entry,
// Configure log4net using the .config file
[assembly: log4net.Config.XmlConfigurator(ConfigFile = "log4net.config", Watch = true)]
log4net.config is the config file added to the web project.
In the code I log using the logger class,
Logger.Log.Info("User visits Sign In Page.");
Logging has been working when I set up the above setting. But suddenly logging has stopped working.
But when I created a new asp.net mvc website with above settings, logging works for that.
I tried with IIS Express and Local IIS. In both cases logging works for the test application I have created.
I cannot figure out why it's not logging? How can I diagnose this? What are the possible issues?
Solved by myself, reason was "for some reason" log4net configuration was not loaded from assembly info. Still I do not know why that happens.
I tried so many fixes proposed by different posts. Finally fixed the issue.
Solution mentioned in this post helped me to solve the issue.
I have added following configuration,
<!--These settings load the log4net configuration.-->
<add key="log4net.Config" value="log4net.config"/>
<add key="log4net.Config.Watch" value="True"/>
It starts logging!
Then I removed following line from assembly info,
// Configure log4net using the .config file
[assembly: log4net.Config.XmlConfigurator(ConfigFile = "log4net.config", Watch = true)]
The reason is that Log4Net tries to load the config from the assembly that first uses LogManager.GetLogger(). If it's one of your class libraries it will simply ignore the attribute in all other assemblies.
The easiest way to fix it is to invoke LogManager in your start file (like Program.cs or Global.asax):
var logger = LogManager.GetLogger(typeof(Program));
logger.Info("Application started.");
//rest of app init.
Doing that will get you the expected behavior with the assembly attribute.
I found log4net won't load the webconfig configuration unless you call log4net.Config.XmlConfigurator.Configure(); on start up.
protected void Application_Start()
{
// your other codes
log4net.Config.XmlConfigurator.Configure(); // must have this line
Logger = log4net.LogManager.GetLogger(typeof(MvcApplication));
}
I had a similar issue: I would not get any logging output, when I ran my assembly from IIS or IISExpress.
However, none of the answers above worked for me.
In my case the solution was to specify the path to the config file as an absolute path. It turned out that IISExpress does not set the current directory to the bin folder and log4net would not find the config file, so I had to use this workaround:
string codeBase = Assembly.GetExecutingAssembly().CodeBase;
// remove file:// part from uri
UriBuilder ub = new UriBuilder(codeBase);
string path = Uri.UnescapeDataString(ub.Path);
var fi = new FileInfo(Path.Combine(Path.GetDirectoryName(path), "Logging.config.xml"));
if (fi.Exists)
{
log4net.Config.XmlConfigurator.ConfigureAndWatch(fi);
var logger = log4net.LogManager.GetLogger(typeof(WebApiApplication));
logger.Info("Application started.");
}
else
{
throw new FileNotFoundException("log4net config file not found", fi.FullName);
}

Log4Net only logging when browsing from localhost

I've used Log4Net in multiple applications for a while. It has been working fine, but recently I noticed that the applications were suddenly not logging anymore. Turns out this issue is the same for all my applications, and they all suddenly stopped logging some months ago. The strange thing is that the logging works when I access the applications directly on the server (http://localhost/myApp), while nothing is logged when I access the application from another PC. My first thought was that it must be related to file/folder permissions, but allowing "Everyone" (Windows user group) full access to the log folder did not help.
They are all ASP.Net MVC 4 applications running on IIS7 (Windows 2008 R2 Enterprise OS), and the application pool is using "ApplicationPoolIdentity". Log4Net version is 1.2.10.0 and I am using a custom CompositeRollingFileAppender. I thought it may have been something wrong with the custom appender, but the problem remained the same when I tried switching to the standard RollingFileAppender. I've seen the issue on multiple servers.
Has anyone seen something similar? Please share your thoughts, as I cannot see why there should be any difference in accessing the applications locally or remotly.
Here is the log4net section in one of my applications' web.config:
<log4net>
<appender name="RollingFileAppender" type="[mynamespace].CompositeRollingFileAppender">
<file value="Logs/ApplicationLog.log" />
<appendToFile value="true" />
<rollingStyle value="Composite" />
<maxSizeRollBackups value="20" />
<maximumFileSize value="10MB" />
<datePattern value="_yyyy-MM-dd" />
<staticLogFileName value="true" />
<preserveLogFileNameExtension value="true" />
<layout type="log4net.Layout.PatternLayout,log4net">
<param name="ConversionPattern" value="%utcdate;%property{ErrorCode};%property{Severity};%property{ErrorName};%property{Module};%m%n" />
</layout>
</appender>
<root>
<priority value="ALL" />
<appender-ref ref="RollingFileAppender" />
<level value="Warn" />
</root>
<logger name="NHibernate">
<level value="OFF" />
</logger>
<logger name="NHibernate.SQL">
<level value="OFF" />
</logger>
</log4net>
Turns out the issue was not directly related to log4net, but to the way ASP.Net MVC3 and newer handles exceptions by default. Some months ago we updated our applications from MVC2 to MVC4, and because of this code which is executed by default from global.asax.cs.Application_Start(), it "bypassed" our exception handling module when CustomErrors where set to RemoteOnly or On:
public class FilterConfig
{
public static void RegisterGlobalFilters(GlobalFilterCollection filters)
{
filters.Add(new HandleErrorAttribute());
}
}
This thread put me on the right track. I ended up removing the filters.Add(..) line, and now it seems to be working fine!

Resources