when trying to deploy a package from Octopus Server to Octopus Tentacle, tentacle suddenly crashes with this info in logs:
10 INFO [] [RunningScript] [Read Lock] [no locks] Trying to acquire lock.
10 FATAL Unhandled AppDomain exception occurred: Value cannot be null.
Parameter name: key
System.ArgumentNullException
at System.Collections.Concurrent.ConcurrentDictionary`2.TryAdd(TKey key, TValue value)
at Octopus.Shared.Scripts.ScriptIsolationMutex.ScriptIsolationMutexReleaser.EnterReadLock()
at Octopus.Shared.Scripts.ScriptIsolationMutex.ScriptIsolationMutexReleaser.EnterLock()
at Octopus.Shared.Scripts.RunningScript.Execute()
at System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx)
at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx)
at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state)
at System.Threading.ThreadHelper.ThreadStart()
System.ArgumentNullException: Value cannot be null.
Parameter name: key
at System.Collections.Concurrent.ConcurrentDictionary`2.TryAdd(TKey key, TValue value)
at Octopus.Shared.Scripts.ScriptIsolationMutex.ScriptIsolationMutexReleaser.EnterReadLock()
at Octopus.Shared.Scripts.ScriptIsolationMutex.ScriptIsolationMutexReleaser.EnterLock()
at Octopus.Shared.Scripts.RunningScript.Execute()
at System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx)
at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx)
at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state)
at System.Threading.ThreadHelper.ThreadStart()
Many thanks,
Pawel
I've solved it by reinstalling everything with default settings.
Related
I'm having this error repeat almost every minute, I think to the point it's hanging my web-server as sometimes it becomes unresponsive but still ping-able.
I'm not exactly sure where to start looking for a fix in reading the 3 log entries below.
Running asp.net 4 for a DNN CMS website (Dotnetnuke) on a new install of windows server 2016 standard. Prior this site was on a 2008R2 machine and this has been happening since the migration.
Application: w3wp.exe
Framework Version: v4.0.30319
Description: The process was terminated due to an unhandled exception.
Exception Info: System.IndexOutOfRangeException
at Lucene.Net.Index.IndexWriter.HandleMergeException(System.Exception, OneMerge)
at Lucene.Net.Index.IndexWriter.Merge(OneMerge)
at Lucene.Net.Index.ConcurrentMergeScheduler+MergeThread.Run()
Exception Info: Lucene.Net.Index.MergePolicy+MergeException
at Lucene.Net.Index.ConcurrentMergeScheduler.HandleMergeException(System.Exception)
at Lucene.Net.Index.ConcurrentMergeScheduler+MergeThread.Run()
at System.Threading.ExecutionContext.RunInternal(System.Threading.ExecutionContext, System.Threading.ContextCallback, System.Object, Boolean)
at System.Threading.ExecutionContext.Run(System.Threading.ExecutionContext, System.Threading.ContextCallback, System.Object, Boolean)
at System.Threading.ExecutionContext.Run(System.Threading.ExecutionContext, System.Threading.ContextCallback, System.Object)
at System.Threading.ThreadHelper.ThreadStart()
.
Faulting application name: w3wp.exe, version: 10.0.14393.0, time stamp: 0x57899b8a
Faulting module name: KERNELBASE.dll, version: 10.0.14393.0, time stamp: 0x57899809
Exception code: 0xe0434352
Fault offset: 0x0000000000017788
Faulting process id: 0x1840
Faulting application start time: 0x01d27de3c8a59141
Faulting application path: c:\windows\system32\inetsrv\w3wp.exe
Faulting module path: C:\Windows\System32\KERNELBASE.dll
Report Id: b8c443b9-7658-47b3-ae17-40d304a55c82
Faulting package full name:
Faulting package-relative application ID:
.
An unhandled exception occurred and the process was terminated.
Application ID: /LM/W3SVC/25/ROOT
Process ID: 6208
Exception: Lucene.Net.Index.MergePolicy+MergeException
Message: Exception of type 'Lucene.Net.Index.MergePolicy+MergeException' was thrown.
StackTrace: at Lucene.Net.Index.ConcurrentMergeScheduler.HandleMergeException(Exception exc)
at Lucene.Net.Index.ConcurrentMergeScheduler.MergeThread.Run()
at System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx)
at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx)
at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state)
at System.Threading.ThreadHelper.ThreadStart()
InnerException: System.IndexOutOfRangeException
Message: Index was outside the bounds of the array.
StackTrace: at Lucene.Net.Index.IndexWriter.HandleMergeException(Exception t, OneMerge merge)
at Lucene.Net.Index.IndexWriter.Merge(OneMerge merge)
at Lucene.Net.Index.ConcurrentMergeScheduler.MergeThread.Run()
I retrieve this crash log about my app.
System.NullReferenceExceptionObject reference not set to an instance of an object
Raw TouristApp.PuntiEventiView.c__AnonStorey1.<>m__0()
Foundation.NSActionDispatcher.Apply()
UIKit.UIApplication.UIApplicationMain(int, string[], intptr, intptr)(wrapper managed-to-native)
UIKit.UIApplication.Main(string[] args, IntPtr principal, IntPtr delegate)
UIKit.UIApplication.Main(string[] args, string principalClassName, string delegateClassName)
TouristApp.Application.Main(string[] args)
I have no clue about the error, in my class (PuntiEventiView) there is no call to c__AnonStorey1.m (it seems a Mono file).
Any idea?
Debug your app step by step. You're using an anonymous delegate or lambda somewhere which causes a NULL ref exception.
The c__AnonStorey1 is a compiler generated class which captures your local variables.
(Using my crystal ball here to guess, because you are not showing any code)
We have this error occurring very frequently in an MVC 5.1.3 application, when it does happen, the user has to refresh the page and it does away, so its an intermittent issue.
We have found it tricky to diagnose as it appears to happen within the framework itself. Any ideas on where to look?
This is the full stack:
System.InvalidOperationException: Collection was modified; enumeration operation may not execute.
at System.Collections.Generic.List`1.Enumerator.MoveNextRare()
at System.Web.Mvc.FilterProviderCollection.GetFilters(ControllerContext controllerContext, ActionDescriptor actionDescriptor)
at System.Web.Mvc.ControllerActionInvoker.GetFilters(ControllerContext controllerContext, ActionDescriptor actionDescriptor)
at System.Web.Mvc.Async.AsyncControllerActionInvoker.BeginInvokeAction(ControllerContext controllerContext, String actionName, AsyncCallback callback, Object state)
at System.Web.Mvc.Async.AsyncControllerActionInvoker.BeginInvokeAction(ControllerContext controllerContext, String actionName, AsyncCallback callback, Object state)
at System.Web.Mvc.Controller.<BeginExecuteCore>b__1c(AsyncCallback asyncCallback, Object asyncState, ExecuteCoreState innerState)
at System.Web.Mvc.Async.AsyncResultWrapper.WrappedAsyncVoid`1.CallBeginDelegate(AsyncCallback callback, Object callbackState)
at System.Web.Mvc.Async.AsyncResultWrapper.WrappedAsyncResultBase`1.Begin(AsyncCallback callback, Object state, Int32 timeout)
at System.Web.Mvc.Async.AsyncResultWrapper.Begin[TState](AsyncCallback callback, Object callbackState, BeginInvokeDelegate`1 beginDelegate, EndInvokeVoidDelegate`1 endDelegate, TState invokeState, Object tag, Int32 timeout, SynchronizationContext callbackSyncContext)
at System.Web.Mvc.Controller.BeginExecuteCore(AsyncCallback callback, Object state)
at System.Web.Mvc.Async.AsyncResultWrapper.WrappedAsyncResultBase`1.Begin(AsyncCallback callback, Object state, Int32 timeout)
at System.Web.Mvc.Async.AsyncResultWrapper.Begin[TState](AsyncCallback callback, Object callbackState, BeginInvokeDelegate`1 beginDelegate, EndInvokeVoidDelegate`1 endDelegate, TState invokeState, Object tag, Int32 timeout, SynchronizationContext callbackSyncContext)
at System.Web.Mvc.Controller.BeginExecute(RequestContext requestContext, AsyncCallback callback, Object state)
at System.Web.Mvc.MvcHandler.<BeginProcessRequest>b__4(AsyncCallback asyncCallback, Object asyncState, ProcessRequestState innerState)
at System.Web.Mvc.Async.AsyncResultWrapper.WrappedAsyncVoid`1.CallBeginDelegate(AsyncCallback callback, Object callbackState)
at System.Web.Mvc.Async.AsyncResultWrapper.WrappedAsyncResultBase`1.Begin(AsyncCallback callback, Object state, Int32 timeout)
at System.Web.Mvc.Async.AsyncResultWrapper.Begin[TState](AsyncCallback callback, Object callbackState, BeginInvokeDelegate`1 beginDelegate, EndInvokeVoidDelegate`1 endDelegate, TState invokeState, Object tag, Int32 timeout, SynchronizationContext callbackSyncContext)
at System.Web.Mvc.MvcHandler.BeginProcessRequest(HttpContextBase httpContext, AsyncCallback callback, Object state)
at System.Web.HttpApplication.CallHandlerExecutionStep.System.Web.HttpApplication.IExecutionStep.Execute()
at System.Web.HttpApplication.CallHandlerExecutionStep.System.Web.HttpApplication.IExecutionStep.Execute()
at System.Web.HttpApplication.ExecuteStep(IExecutionStep step, Boolean& completedSynchronously)
There has been a similar question asked before: Collection was modified; enumeration operation may not execute, but this was resolved by upgrading to MVC 5, we already on 5.
As #Erik said in a comment, obviously the collection of filters are being modified during the execution of the requests, where it should only be populated during application start up.
One possibility is that you've registered a custom IFilterProvider which is breaking the code. Here's the source code from where the error occurs (from FilterProviderCollection)
for (int i = 0; i < providers.Length; i++)
{
IFilterProvider provider = providers[i];
foreach (Filter filter in provider.GetFilters(controllerContext, actionDescriptor))
{
filters.Add(filter);
}
}
The provider.GetFilter(...) is returning a List which is later modified during enumeration in the FilterProviderCollection's foreach block.
Here's what comes to my mind for you to test:
First of all, make sure you're not registering a custom IFilterProvider (either you, or a library you might be using). If so, that is the primary suspect.
Look for all usages of GlobalFilterCollection and GlobalFilters class in your code. Any modification to this collection should only be reached from Application_Start in Global.asax or the Startup class (if you're using OWIN)
Look for clues where you're creating filters or accessing them.
Try upgrading to MVC 5.2. If it's an issue with the framework (which seems unlikely) this may fix it. (I'm using 5.2.2.0 and don't see any similar issues in either development or production)
Debug your application, and set a break point in the beginning of Application_BeginRequest and in the controller action. Try to inspect the global filters collection and see if there's any change.
Maybe registering a custom IFilterProvider and inspecting the behavior by debugging or logging can be helpful.
I am using Umbraco and for some reason out of the blue I get the Visual Studio JIT Debugger popup asking me if I want to debug and exception with w3wp.exe, I attached to it and found the following exception, I can't find any cause for this as I haven't added anything new.
Lucene.Net.Index.MergePolicy.MergeException was unhandled
Message=Exception of type 'Lucene.Net.Index.MergePolicy+MergeException' was thrown.
Source=Lucene.Net
StackTrace:
at Lucene.Net.Index.ConcurrentMergeScheduler.HandleMergeException(Exception exc)
at Lucene.Net.Index.ConcurrentMergeScheduler.MergeThread.Run()
at System.Threading.ThreadHelper.ThreadStart_Context(Object state)
at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean ignoreSyncCtx)
at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state)
at System.Threading.ThreadHelper.ThreadStart()
InnerException: Lucene.Net.Index.CorruptIndexException
Message=doc counts differ for segment _3qw: fieldsReader shows 1025 but segmentInfo shows 1
Source=Lucene.Net
StackTrace:
at Lucene.Net.Index.IndexWriter.HandleMergeException(Exception t, OneMerge merge)
at Lucene.Net.Index.IndexWriter.Merge(OneMerge merge)
at Lucene.Net.Index.ConcurrentMergeScheduler.DoMerge(OneMerge merge)
at Lucene.Net.Index.ConcurrentMergeScheduler.MergeThread.Run()
InnerException:
Does anyone have any idea what is causing this or how to fix it?
Thanks
I found the answer from this link:
http://our.umbraco.org/forum/ourumb-dev-forum/bugs/16066-Missing-LuceneNet-assembly
on Dunfee posted this reply 6 months ago
Just went through a migration from one location to another. I removed the contents of App_Data/TEMP/ExamineIndexes and after a minute or so of watching the browser spinner (while the Lucene cache was being rebuilt) the site came up.
I'm getting the following exception from structure map - "Duplicate Objects detected for Instance d54e25dc-d19c-4d70-99d1-56bd2502d203 of Type ..."
The stack trace is:
at StructureMap.InstanceCache.Set(Type pluginType, Instance Instance, Object result) in c:\code\structuremap\Source\StructureMap\InstanceCache.cs:line 60
at StructureMap.BuildSession.CreateInstance(Type pluginType, Instance instance) in c:\code\structuremap\Source\StructureMap\BuildSession.cs:line 159
at StructureMap.BuildSession.<>c_DisplayClass3.<.ctor>b_1() in c:\code\structuremap\Source\StructureMap\BuildSession.cs:line 34
at StructureMap.BuildSession.CreateInstance(Type pluginType) in c:\code\structuremap\Source\StructureMap\BuildSession.cs:line 192
at StructureMap.BuildSession.GetInstance[T]() in c:\code\structuremap\Source\StructureMap\BuildSession.cs:line 78
This exception happens only from time to time and is very difficult to reproduce.
What I've found out so far:
Calling ObjectFactory.Container.GetInstance(type) will create a new BuildSession with a new non-static cache.
The call will reach BuildSession.CreateInstance(Type pluginType, Instance instance) and no instance is in the cache yet. (https://github.com/structuremap/structuremap/blob/master/Source/StructureMap/BuildSession.cs#L149-166)
Now we'll go on to _builder.Resolve(pluginType, instance, this);
Somewhere during Resolve() the object will get into the cache and later in CreateInstance we're saving it into the cache once again, receiving the exception.
That object is configured with Singleton lifecycle. The application is WCF service hosted in IIS.
Any help with finding the cause is appreciated.