When I was processing large number of data I ended up in Cursor Not Found Error in Mongoid. And I googled and came across the below link:
DBCursor mongodb
Now I'm using skip and limit function to process 100 data at once and continue till it reaches 7350, which is the count of my documents.
As it takes too long to produce Cursor Not Found Error. I wonder if there is any quick way I can test for occurrence of this error. Thanks in advance.
maybe killOp would work, i guess.
Related
I'm currently building a fairly long flow, where I collect quantitative and qualitative input and save it in output contexts with a long lifespan (500 turns).
By the end of the flow, the bot sums up all the answers saved in the parameters my using the
#outputcontext.parametername syntax.
This seems to work 50 % of the time. Sometimes, especially if there's been a few fallbacks/repromps during the flow, it won't parse the stored value into the #outputcontext.parametername placeholder, but instead just parse the #outputcontext.parametername. I find this odd, as the parameter should still be saved in the context, and I suspect it might have something to do with memory capacity in DF itself, when too many parameters are stored at once.
Have anyone experienced similar issues, and/or found a solution?
Thanks - appreciate it!
Basically Dialogflow context saved for 25min then it will be reseted you need to use cloud functions ( fulfillment )
I am using rsolr gem to integrate solr search with my RoR app. Now for each search, I need to specify the rows parameter, which is the number of results I want to retrieve. In order to retrieve all results corresponding to a query, I set the rows parameter to a high value as mentioned in this post.
But doing that makes the processing really really slow and I am getting the following error in the rails logs:
[2014-01-11 15:51:08] ERROR WEBrick::HTTPStatus::RequestURITooLarge
[2014-01-11 15:51:08] ERROR TypeError: can't convert nil into an exact number
/home/nish/.rvm/gems/ruby-1.9.2-p320#voylla/gems/activesupport-3.1.10/lib/active_support/core_ext/time/calculations.rb:266:in `-'
/home/nish/.rvm/gems/ruby-1.9.2-p320#voylla/gems/activesupport-3.1.10/lib/active_support/core_ext/time/calculations.rb:266:in `minus_with_duration'
/home/nish/.rvm/gems/ruby-1.9.2-p320#voylla/gems/activesupport-3.1.10/lib/active_support/core_ext/time/calculations.rb:277:in `minus_with_coercion'
/home/nish/.rvm/rubies/ruby-1.9.2-p320/lib/ruby/1.9.1/webrick/accesslog.rb:42:in `setup_params'
/home/nish/.rvm/rubies/ruby-1.9.2-p320/lib/ruby/1.9.1/webrick/httpserver.rb:164:in `access_log'
/home/nish/.rvm/rubies/ruby-1.9.2-p320/lib/ruby/1.9.1/webrick/httpserver.rb:87:in `run'
/home/nish/.rvm/rubies/ruby-1.9.2-p320/lib/ruby/1.9.1/webrick/server.rb:183:in `block in start_thread'
How can I fix this issue? Thanks
Your error is related to RoR, not Solr. It's telling you the problem -- the requested URI is too large. WEBrick is not a production-caliber web server, and v1.9.3 appears to limit http request length to 2083 (per this other SO question.)
The short-term fix? Use a web server that doesn't limit your requested URI length to something so short.
However, that's just one part of the fix -- the process you're engaging in will grow in linear or worse fashion in terms of execution time relative to the number of results. Not only does the number of results affect performance, but also the size of the documents being retrieved.
Can you share your requirements that led to an implementation where all results are returned with each query?
From the Solr FAQ:
This is impractical in most cases. People typically only want to do
this when they know they are dealing with an index whose size
guarantees the result sets will be always be small enough that they
can feasibly be transmitted in a manageable amount -- but if that's
the case just specify what you consider a "manageable amount" as your
rows param and get the best of both worlds (all the results when your
assumption is right, and a sanity cap on the result size if it turns
out your assumptions are wrong)
I have been struggling with parallel and async constructs in F# for the last couple days and not sure where to go at this point. I have been programming with F# for about 4 months - certainly no expert - and I currently have a series of calculations that are implemented in F# (asp.net 4.5) and are working correctly when executed sequentially. I am running the calculations on a multi-core server and since there are millions of inputs to perform the same calculation on, I am hoping to take advantage of parallelism to speed it up.
The calculations are extremely data parallel - basically the exact calculation on different input data. I have tried a number of different avenues and I continually run into the same issue - it seems as if the parallel looping never gets to the end of the input data set. I have tried TPL, ConcurrentQueues, Parallel.Array.map/iter and all the same result: the program starts out fine and then somewhere in the middle (indeterminate) it just hangs and never completes. For simplicity I actually removed the calculation from the program and I am just calling a print method, and Here is where the code is currently at:
let runParallel =
let ids = query {for c in db.CustTable do select c.id} |> Seq.take(5)
let customerInputArray= getAllObservations ids
Array.Parallel.iter(fun c -> testParallel c) customerInputArray
let key = System.Console.ReadKey()
0
A few points...
I limited the results above to only 5 just for debugging. The actual program does not apply the Take(5).
The testParallel method is just a printfn "test".
The customerInputArray is a complex data type. It is a tuple of lists that contain records. So I am pretty sure my problem must be there...but I added exception handling and no exception is getting raised, so have no idea how to go about finding the problem.
Any help is appreciated. Thanks in advance.
EDIT: Thanks for the advice...I think it is definitely deadlock. When I remove all of the printfn, sprintfn, and string concat operations, it completes. (of course, I need those things in there.)
Is printfn, sprintfn, and string ops not thread-safe?
Another EDIT: Iteration always stops on the last item..So if my input array has 15 items, the processing stops on item 14, or seems to never get to item 15. Then everything just hangs. Does not matter what the size of the input array is..Any ideas what can be causing this? I even switched over to Parallel.ForEach (instead of Array.Parallel) and same behavior.
Update on the situation and how I resolved this issue.
I was unable to upload code from my example due to my company's firewall policy, so in the end my question did not have enough details. I failed to mention that I was using a type provider which was important information in this situation. But here is what I figured out.
I am using the F# type provider for SQL Server and was passing around its Service Types which I suspect are not thread-safe. When I replaced the ServiceTypes with plain old F# Records, the code worked fine - no more deadlocks and everything completed without error.
I'm having some issues regarding a conjoint analysis. Excuse me if some of the terms I use are wrong, but it has been some time since I last worked with SPSS - and my teacher was Danish.
Task object
I am to make a series of concept travelpackages (attributes and attribute notes/levels).
This far I've got things under control - I've reduced the number of packages from 81 to 9, with the help of 'orthogonal' design.
These 9 packages have been rated by some people (1-10), on a questionnaire.
Then I've been asked to write a syntax which evaluates my conjoint plan:
CONJOINT PLAN= 'C:\Users\MYNAME\DROBBOXFOLDER\Conjoint_cards.SAV'
/DATA='C:\Users\MYNAME\DROBBOXFOLDER\Respondents.SAV'
/SCORE=Card_1 TO Card_9
/SUBJECT=ID
/FACTORS= SMS Minutter Data Tryghed
/PRINT=ALL
/PLOT=ALL.
However I keep getting this error:
SUBJECT SUBCOMMAND -- Subject variable is not on data file.
Execution of this command stops.
At this point I've been to the dark pages of Google and back for an answer to what I am doing wrong, but nothing so far. The answer is probably staring me in the face. But I will appreciate any help or pointers as to what I'm doing wrong.
Problem solved:
So apparently one shouldn't follow a guide to the letter. My datafile didn't contain a ID, so removing this from my syntax solved the problem.
Nevrona Designs' Rave Reports is a Report Engine for use by Embarcadero's Delphi IDE.
This is what I call the Rave Endless Loop bug. In Rave Reports
version 6.5.0 (VCL10) that comes bundled with Delphi 2006, there is a
nortorious bug that plagues many Rave report developers. If you have a
non-empty dataset, and the data rows for this dataset fit exactly into a
page (that is to say there are zero widow rows), then upon PrintPreview,
Rave will get stuck in an infinite loop generating pages.
This problem has been previously reported in this newsgroup under the
following headings:
"error: generating infinite pages"; Hugo Hiram 20/9/2006 8:44PM
"Rave loop bug. Please help"; Tomas Lazar 11/07/2006 7:35PM
"Loop on full page of data?"; Tony Chistiansen 23/12/2004 3:41PM
reply to (3) by another complainant; Oliver Piche
"Endless lopp print bug"; Richso 9/11/2004 4:44PM
In each of these postings, there was no response from Nevrona, and no
solution was reported.
Possibly, the problem has also been reported on an allied newsgroup
(nevrona.public.rave.reports.general), to wit:
6. "Continuously generating report"; Jobard 20/11/2005
Although it is not clear to me if (6) is the Rave Endless loop bug or
another problem. This posting did get a reply from Nevrona, but it was
more in relation to multiple regions ("There is a problem when using
multiple regions that go over a page-break.") than the problem of zero
widows.
This is more of a work-around than a true solution. I first posted this work-around on the Nevrona newsgroup (Group=nevrona.public.rave.developer.delphi.rave; Subject="Are you suffering from the Rave Endless Loop bug?: Work-around announced."; Date=13/11/2006 7:06 PM)
So here is my solution. It is more of a work-around than a good
long-term solution, and I hope that Nevrona will give this issue some
serious attention in the near future.
Given your particular report layout, count the maximum number of rows
per page. Let us say that this is 40.
Set up a counter to count the rows within the page (as opposed to rows within the whole report). You could do this either by event script or by a CalcTotal component.
Define an OnBeforePrint scripted event handler for the main data band.
In this event handler set the FinishNewPage property of the main data band to be True when the row-per-page count is one or two below the max (in our example, this would be 38). And set it to False in all other cases. The effect of this is to give every page a non-zero number of widows (in this case 1..38), thus avoiding the condition that gives rise to the Rave Endless loop problem.
Thanks so much for this Sean - unfortunately this wouldn't work for me but I came up with another solution...
You see I have a memo at the top of the region that might expand or contract depending on how many notes the user has left in the database. This means that the number of rows that can fit on a page varies.
However. there is another solution - you use the MaxHeightLeft property of a databand.
All you do is measure the height of your databand, multiply it by 2, and put this in your MaxHeightLeft property. This will force 1 or 2 records onto the next page if it fills up that much.
thank's a lot, this thread helps me out from my problem with endless printing loop in Nevrona Rave...., I set MinHeightLeft to 0,500, this setting is work but i'm not sure that it will work for anothers result set of my query report.
Master,
The solution is MinHeightLeft to 0,500 , i use property wastefit area in true and generated the loop in the second print, but when changed the property MinHeightLeft to 0,500 the error disapear.
Thanks !
Atte
Fabiola Herrera.
Fabi_ucv#hotmail.com