Application.Exit() throws System.InvalidOperationException: 'Collection was modified...' - shutdown

(Visual Studio 2022-Community -- .Net 5 -- Windows 10)
I have been fighting an issue for days and have finally given up and have to ask the "smart people" for help.
I'm in Visual Studio (2022-Community) debugging my application. When I click on the "close button" I close Internet connection, database connections, then exit the app. In my log file I see "=== Application closed ===" in the log so it's making it to the end of the main TRY - CATCH block. It then steps out to a "DoEvents()" (which I added just because I couldn't find anything else wrong and though that maybe some resource was waiting).
It even executes the Application.Exit() and moves to the curly-bracket after it:
applog.Info("=== Application closed ===");
}
catch (Exception ex)
{
string strError = $"Application Error{Environment.NewLine}{ex.ToString()}";
applog.Error(strError);
MessageBox.Show(strError, "Application Error", MessageBoxButtons.OK);
}
Application.DoEvents();
Application.Exit();
} //- <--- Error is happening HERE
}
}
...then throws:
System.InvalidOperationException
HResult=0x80131509
Message=Collection was modified; enumeration operation may not execute.
Source=System.Private.CoreLib
StackTrace:
at System.ThrowHelper.ThrowInvalidOperationException_InvalidOperation_EnumFailedVersion() in /_/src/libraries/System.Private.CoreLib/src/System/ThrowHelper.cs:line 361
at System.Collections.Generic.List`1.Enumerator.MoveNextRare() in /_/src/libraries/System.Private.CoreLib/src/System/Collections/Generic/List.cs:line 1109
at Microsoft.Data.Sqlite.SqliteConnectionPool.ReclaimLeakedConnections() in /_/src/Microsoft.Data.Sqlite.Core/SqliteConnectionPool.cs:line 168
at Microsoft.Data.Sqlite.SqliteConnectionPool.Clear() in /_/src/Microsoft.Data.Sqlite.Core/SqliteConnectionPool.cs:line 113
at Microsoft.Data.Sqlite.SqliteConnectionFactory.ReleasePool(SqliteConnectionPool pool, Boolean clearing) in /_/src/Microsoft.Data.Sqlite.Core/SqliteConnectionFactory.cs:line 129
at Microsoft.Data.Sqlite.SqliteConnectionPoolGroup.Clear() in /_/src/Microsoft.Data.Sqlite.Core/SqliteConnectionPoolGroup.cs:line 59
at Microsoft.Data.Sqlite.SqliteConnectionFactory.ClearPools() in /_/src/Microsoft.Data.Sqlite.Core/SqliteConnectionFactory.cs:line 146
at Microsoft.Data.Sqlite.SqliteConnectionFactory.<.ctor>b__7_1(Object _, EventArgs _) in /_/src/Microsoft.Data.Sqlite.Core/SqliteConnectionFactory.cs:line 31
at System.AppContext.OnProcessExit() in /_/src/libraries/System.Private.CoreLib/src/System/AppContext.cs:line 77
I've closed out the database connections. I even "disposed" them to make sure they were gone.
I know that the "Collection was modified..." exception means that you can't change a collection you are iterating through, but at this point my code isn't iterating through anything.
I'd appreciate any help that can be provided.

Related

Quit a specflow scenario, or avoid execution based on a precondition

I want to be able to avoid a specflow scenario execution, or quit at the first step, depending on external configuration. Is that possible?
Like
Background:
Given we have a satisfied precondition
the precondition fails and the scenario stops without returning an error
IUnitTestRuntimeProvider interface should be injected
https://docs.specflow.org/projects/specflow/en/latest/Execution/SkippingScenarios.html
however, I then run into the fact that this interface can not be resolved
https://github.com/SpecFlowOSS/SpecFlow/issues/2076
the resolution of this does not work for me in a .Net 5, XUnit project, with Visual Studio 2019
If a step does not throw an exception, then the step passes. You could refactor that particular step into a private method, then simply wrap the method call in a try-catch inside an if statement:
[Given(#"we have a satisfied precondition")]
public void GivenWeHaveASatisfiedPrecondition()
{
if (/* check your external config here */)
{
try
{
PerformPrecondition();
}
catch (Exception ex)
{
Console.WriteLine("Step failed, but continuing scenario." + Environment.NewLine + Environment.NewLine + ex);
}
}
else
{
// Failure in this step should fail the scenario.
PerformPrecondition();
}
}
private void PerformPrecondition()
{
// Your step logic goes here
}
You haven't specified where your external configuration is at, but there are a thousand ways to do this.

Get the stack trace of exception in Rhino

We have an app that allows to enter a scripts. These scripts can be in multiple files, so when there is an exception we want to be able to show a stack trace to the app admin so he can diagnose the issue.
Is there there a way to extract the stack trace from a Rhino exception?
First of all, in order to get a good stack trace, you need to be very careful when evaluating scripts in your scope. I've seen that this is overlooked by many developers where they always send the same params for sourceName and lineno, like:
ctx.evaluateString(scriptScope, script, "script", 0, null);
So in the 3rd and 4th params you have to send the file name of something that will help the developer identify the script. Then, the line number should be correct, in case you concatenate scripts.
So, once you evaluate all the scripts in your context correctly, you can be able to get a useful stack track. So just catch the exception and process the info in the stack trace:
try {
// execute script
} catch (RhinoException re) {
StackTraceElement[] stackTrace = re.getStackTrace();
if (stackTrace != null) {
for (StackTraceElement stackElement : stackTrace) {
// stackElement.getFileName();
// stackElement.getLineNumber();
}
}
}

Why Stop method is invoked many times in Topshelf

Topshelf is working as the windows service broker in our application. This morning, we find that the Stop mehtod is invoked many times. Here is the related code.
class Program
{
static void Main(string[] args)
{
ILog Log = new FileLog();
try
{
HostFactory.Run(serviceConfig =>
{
serviceConfig.Service<ServiceManager>(serviceInstance =>
{
serviceInstance.ConstructUsing(() => new ServiceManager());
serviceInstance.WhenStarted(execute => execute.Start());
serviceInstance.WhenStopped(execute => execute.Stop());
});
});
}
catch (Exception ex)
{
Console.WriteLine(ex);
Log.Error("Program.Main", ex, LogType.Error);
Console.ReadLine();
};
}
}
In the ServiceManager, we have the Stop mehtod, which will be invoked then TopShelf receives the stop signal from operation system.
class ServiceManager
{
xxx.....
public bool Stop()
{
try
{
_log.Info("The service is stopping", LogType.Info);
_service.StopExecuteProduceMessage();
Task.WaitAll(_tasks.ToArray());
_log.Info("The service is stopped", LogType.Info);
}
catch (Exception ex)
{
_log.Error("Stop", ex, LogType.Error);
}
return true;
}
}
This morning, we find the service is stopped with unclear reason. And there are many lines logging about this stop action.
I guess Topshelf invoke ServiceManager.Stop method many times. Anyone have encountered this problem before? I want to know I can trace why this happens.
Anyone can help? Many thanks.
You are experiencing this behavior because your Stop() method takes a while but is not being responsive to the request to stop.
Your method essentially looks like this:
Stop() {
log-stopping;
wait-a-while;
log-stopped;
}
While you are waiting, the status of the service remains "Running". This causes the requester (could be Windows itself or another program) to keep re-sending the stop request, resulting in multiple parallel/overlapping calls to Stop(). That accounts for the first 10 lines in the log you included.
You can see that it takes almost 20 seconds for the "wait" to complete (from 05:39:45 to 05:40:04).
After that, it looks like Topshelf may be stuck. That causes more messages to be sent. (Notice that in the next lines in your log, the stopping and starting pairs are logged simultaneously because your tasks are stopped and there is no waiting).
To fix the problem, you should:
Modify your WhenStopped() call to pass the HostControl parameter to Stop():
serviceInstance.WhenStopped((execute, hostControl) => execute.Stop(hostControl));
Update the Stop() method to take the HostControl parameter and make this call before the call to Task.WaitAll():
hostControl.RequestAdditionalTime(TimeSpan.FromSeconds(30));
This will inform Windows that your service has received the request and may be working on it for up to 30 seconds. That should avoid the repeated calls.
Reference: Topshelf documentation

CombineFn Dataflow - Step not in order, creating null pointer

I am a newbie in dataflow, please pardon if I made a newbie mistake,
I recently use dataflow/beam to process several data from pubsub, and I am using cloud-dataflow-nyc-taxi-tycoon as a starting point, but I upgrade it to sdk 2.2.0 to make it work with Big Table. I simulate it using http cloud function that send a single data to pubsub so the dataflow can ingest it, using below code
.apply("session windows on rides with early firings",
Window.<KV<String, TableRow>>into(
new GlobalWindows())
.triggering(
Repeatedly.forever(AfterPane.elementCountAtLeast(1))
)
.accumulatingFiredPanes()
.withAllowedLateness(Duration.ZERO))
.apply("group by", Combine.perKey(new LatestPointCombine()))
.apply("prepare to big table",
MapElements.via(new SimpleFunction<KV<String,TableRow>,TableRow >() {
#Override
public TableRow apply(KV<String, TableRow> input) {
TableRow tableRow = null;
try{
tableRow=input.getValue();
....
}
catch (Exception ex){
ex.printStackTrace();
}
return tableRow;
}
}))
.apply....
But it gives me an error at phase "group by"/CombineFn, after "session windows on rides with early firings", here is the logs from stackdriver
1. I create accumulator
2. I addinput
3. I mergeaccumulators
4. I extractoutput
5. I pre mutation_transform
6. W Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
7. I mergeaccumulators
8. I create accumulator
9. E Uncaught exception:
10. E Execution of work for S0 for key db2871226f7cec594ebd976e6758ac7e failed. Will retry locally.
11. I Memory is used/total/max = 105/365/4949 MB, GC last/max = 1.00/1.00 %, #pushbacks=0, gc thrashing=false
12. I create accumulator
13. I addinput
14. I mergeaccumulators
15. I extractoutput
16. I pre mutation_transform
17. I mergeaccumulators
18. I create accumulator
19. E Uncaught exception:
20. E Execution of work for S0 for key db2871226f7cec594ebd976e6758ac7e failed. Will retry locally.
21. I create accumulator
...
My questions are :
A. What I dont understand is after step 4, (extract output), why the dataflow mergeaccumulator method called first (line 7.) and later on the create accumulator were called (line 8.) here is the mergeAccumulator method I wrote
public RidePoint mergeAccumulators(Iterable<RidePoint> latestList) {
//RidePoint merged = createAccumulator();
RidePoint merged=new RidePoint();
LOG.info("mergeaccumulators");
for (RidePoint latest : latestList) {
if (latest==null){
LOG.info("latestnull");
}else
if (merged.rideId == null || latest.timestamp > merged.timestamp){
LOG.info(latest.timestamp + " latest " + latest.rideId);
merged = new RidePoint(latest);
}
}
return merged;
}
B. It seems the data is null, and I dont know what caused it, but it reach at the end of the pipeline, elements added in "session windows on rides with early firings" is showing 1 element added, but below that "group by" phase, ... gives 52 elements added,
The Detailed Uncaught exception that shown in the log, looks like this :
(90c7caea3f5d5ad4): java.lang.NullPointerException: in com.google.codelabs.dataflow.utils.RidePoint in string null of string in field status of com.google.codelabs.dataflow.utils.RidePoint
org.apache.avro.reflect.ReflectDatumWriter.write(ReflectDatumWriter.java:161)
org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:62)
org.apache.beam.sdk.coders.AvroCoder.encode(AvroCoder.java:308)
org.apache.beam.sdk.coders.Coder.encode(Coder.java:143)
com.google.cloud.dataflow.worker.WindmillStateInternals$WindmillBag.persistDirectly(WindmillStateInternals.java:575)
com.google.cloud.dataflow.worker.WindmillStateInternals$SimpleWindmillState.persist(WindmillStateInternals.java:320)
com.google.cloud.dataflow.worker.WindmillStateInternals$WindmillCombiningState.persist(WindmillStateInternals.java:952)
com.google.cloud.dataflow.worker.WindmillStateInternals.persist(WindmillStateInternals.java:216)
com.google.cloud.dataflow.worker.StreamingModeExecutionContext$StepContext.flushState(StreamingModeExecutionContext.java:513)
com.google.cloud.dataflow.worker.StreamingModeExecutionContext.flushState(StreamingModeExecutionContext.java:363)
com.google.cloud.dataflow.worker.StreamingDataflowWorker.process(StreamingDataflowWorker.java:1071)
com.google.cloud.dataflow.worker.StreamingDataflowWorker.access$1000(StreamingDataflowWorker.java:133)
com.google.cloud.dataflow.worker.StreamingDataflowWorker$8.run(StreamingDataflowWorker.java:841)
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.NullPointerException
org.apache.avro.specific.SpecificDatumWriter.writeString(SpecificDatumWriter.java:67)
org.apache.avro.generic.GenericDatumWriter.writeWithoutConversion(GenericDatumWriter.java:128)
org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:75)
org.apache.avro.reflect.ReflectDatumWriter.write(ReflectDatumWriter.java:159)
org.apache.avro.generic.GenericDatumWriter.writeField(GenericDatumWriter.java:166)
org.apache.avro.specific.SpecificDatumWriter.writeField(SpecificDatumWriter.java:90)
org.apache.avro.reflect.ReflectDatumWriter.writeField(ReflectDatumWriter.java:191)
org.apache.avro.generic.GenericDatumWriter.writeRecord(GenericDatumWriter.java:156)
org.apache.avro.generic.GenericDatumWriter.writeWithoutConversion(GenericDatumWriter.java:118)
org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:75)
org.apache.avro.reflect.ReflectDatumWriter.write(ReflectDatumWriter.java:159)
org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:62)
org.apache.beam.sdk.coders.AvroCoder.encode(AvroCoder.java:308)
org.apache.beam.sdk.coders.Coder.encode(Coder.java:143)
com.google.cloud.dataflow.worker.WindmillStateInternals$WindmillBag.persistDirectly(WindmillStateInternals.java:575)
com.google.cloud.dataflow.worker.WindmillStateInternals$SimpleWindmillState.persist(WindmillStateInternals.java:320)
com.google.cloud.dataflow.worker.WindmillStateInternals$WindmillCombiningState.persist(WindmillStateInternals.java:952)
com.google.cloud.dataflow.worker.WindmillStateInternals.persist(WindmillStateInternals.java:216)
com.google.cloud.dataflow.worker.StreamingModeExecutionContext$StepContext.flushState(StreamingModeExecutionContext.java:513)
com.google.cloud.dataflow.worker.StreamingModeExecutionContext.flushState(StreamingModeExecutionContext.java:363)
com.google.cloud.dataflow.worker.StreamingDataflowWorker.process(StreamingDataflowWorker.java:1071)
com.google.cloud.dataflow.worker.StreamingDataflowWorker.access$1000(StreamingDataflowWorker.java:133)
com.google.cloud.dataflow.worker.StreamingDataflowWorker$8.run(StreamingDataflowWorker.java:841)
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
java.lang.Thread.run(Thread.java:745)
Your question has many parts to it. But to start, here are some recommendations for debugging:
Don't swallow exceptions. Currently in your "prepare to big table" logic you have: catch (Exception ex){ ex.printStackTrace(); }. This hides the exception and is causing null elements to be returned from the function. It's better to understand and fix the exception here rather dealing with invalid data later.
Validate with the DirectRunner first. Make sure that your pipeline runs correctly on your machine using the Beam DirectRunner. This is the easiest way to understand and fix issues with the Beam model. You can run from the commandline or your favorite IDE and debugger. Then if your pipeline works on the DirectRunner but not on Dataflow, you know that there is a Dataflow-specific issue.
To touch on your specific questions:
A. What I dont understand is after step 4, (extract output), why the
dataflow mergeaccumulator method called first (line 7.) and later on
the create accumulator were called (line 8.)
Your code uses Combine.perKey, which will group elements by key value. So each unique key will cause an accumulator to be created. Dataflow also applies a set of optimizations which can parallelize and reorder independent operations, which could explain what you're seeing.
B. It seems the data is null, and I don;t know what caused it
The null values are likely those that hit an exception in your prepare to big table logic.
I'm not exactly sure what you mean with the output counts because I don't quite understand your pipeline topology. For example, your LatestPointCombine logic seems to output type RidePoint, but the "prepare to big table" function takes in a String. If you are still having trouble after following the suggestions above, you can post a Dataflow job_id and I can help investigate further.

Timer job not listed in Timer Job Definitions + Sharepoint 2007

I have created a Custom Timer job which is added when a feature gets activated. The feature does get activated without any exception. But the timer job is not being listed in Timer Job definitions on Central Admin.
The same is working on Staging Server but I am facing this in Production Server.
I am working on Sharepoint2007.
Below is what I have done in Feature Activated.
public override void FeatureActivated(SPFeatureReceiverProperties properties)
{
try
{
SPSecurity.RunWithElevatedPrivileges(delegate()
{
SPWeb parentWeb = properties.Feature.Parent as SPWeb;
UpdateEmpReferralListTimer taskJob;
SPMinuteSchedule schedule;
foreach (SPJobDefinition job in parentWeb.Site.WebApplication.JobDefinitions)
{
if (job.Name == "xyz")
job.Delete();
}
parentWeb = properties.Feature.Parent as SPWeb;
taskJob = new UpdateEmpReferralListTimer("xyz", parentWeb.Site.WebApplication);
schedule = new SPMinuteSchedule();
schedule.BeginSecond = 0;
schedule.EndSecond = 59;
taskJob.Schedule = schedule;
taskJob.Update();
});
}
catch (Exception Ex)
{
string str = Ex.Message.ToString();
}
}
Scope of Feature is "Web"
To my knowledge trying to perform tasks that require administrator privilegies (like creating timer job) should not work from site level feature activation callback if SharePoint accounts set up properly. Account that web sites run under normally does not have adminstrative privilegies.
It likely works on your staging server due to all accounts having the same privileges. Check out SharePoint logs on your production server to see what exception is (after removing code that eats exception without rethrowing/reporting it - catch (Exception Ex) {...}).
how do you know that your feature is getting activated without error. your code has try/catch block and you are doing nothing in catch block. if your code throwing any error, your catch block will suppress it.
add this line in your catch block and check sharepoint log.
Microsoft.SharePoint.Administration.SPDiagnosticsService.Local.WriteTrace(0, new SPDiagnosticsCategory("Development Debugging", TraceSeverity.Unexpected, EventSeverity.Information), TraceSeverity.Unexpected, str, null);
you can find log files in C:\Program Files\Common Files\Microsoft Shared\Web Server Extensions\14\LOGS folder

Resources