I have a problem when using comprehension with a neo4j-embedded (version 3.5.3).
For exemple, this kind of query works perfectly fine with neo4j enterprise 3.5.3, but does not work with neo4j-embedded :
MATCH (myNode:MyNode {myId:'myid'})
MATCH path = ( (myNode) -[*0..]- (otherNode:MyNode) )
WHERE
ALL(n in nodes(path) where [ (n)<--(state:MyState) | state.isConnected ][0] = true)
RETURN myNode, otherNode
The error I get when using neo4j-embedded is difficult to understand, and looks like an internal error :
org.neo4j.driver.v1.exceptions.DatabaseException: This expression should not be added to a logical plan:
VarExpand(myNode, BOTH, OUTGOING, List(), otherNode, UNNAMED62, VarPatternLength(0,None), ExpandInto, UNNAMED62_NODES, UNNAMED62_RELS, Equals(ContainerIndex(PatternComprehension(None,RelationshipsPattern(RelationshipChain(NodePattern(Some(Variable( UNNAMED62_NODES)),List(),None,None),RelationshipPattern(Some(Variable( REL136)),List(),None,None,INCOMING,false,None),NodePattern(Some(Variable(state)),List(LabelName(MyState)),None,None))),None,Property(Variable(state),PropertyKeyName(isConnected))),Parameter( AUTOINT1,Integer)),True()), True(), List((Variable(n),Equals(ContainerIndex(PatternComprehension(None,RelationshipsPattern(RelationshipChain(NodePattern(Some(Variable(n)),List(),None,None),RelationshipPattern(Some(Variable( REL136)),List(),None,None,INCOMING,false,None),NodePattern(Some(Variable(state)),List(LabelName(MyState)),None,None))),None,Property(Variable(state),PropertyKeyName(isConnected))),Parameter( AUTOINT1,Integer)),True())))) {
LHS -> CartesianProduct() {
LHS -> Selection(Ands(Set(In(Property(Variable(myNode),PropertyKeyName(myId)),ListLiteral(List(Parameter( AUTOSTRING0,String))))))) {
LHS -> NodeByLabelScan(myNode, LabelName(MyNode), Set()) {}
}
RHS -> NodeByLabelScan(otherNode, LabelName(MyNode), Set()) {}
}
}
Any idea ?
It was quite a complicated issue but here is the full explanation.
First, I found that it was not specific to neo4j-embedded. The internal error exception was raised because of an assert in Neo4J, witch would trigger an exception only if the flag -ea (enable assertions) is set. And this flag is set only when running tests with maven or any IDE.
Drilling down Neo4J's code on github, I found also that this assert whas added because of some concerns on recursive comprehension pattern. (The commit is here : https://github.com/neo4j/neo4j/commit/dfbe8ce397f7b72cf7d9b9ff1500f24a5c4b70b0)
In my case, I do use comprehension pattern but not recursively, so I think everything should be fine, except when unit testing :)
I submitted the problem to Neo4J's support, and they will provide a fix in a future release.
Related
my script works on a line by line basis. The task per line implies use of a Grammar, but is not too complicated. I notice that when the input has a lot of lines, say 150_000, the memory usage is gigantic, over 6G, and my computer hangs. I used --profile on shorter input, but that gives me no clue where to look for a solution. Most names in there do not refer to my code.
Any suggestions on where I can prevent spurious memory use or find out what is causing this?
here is the core loop:
method process() {
my $promise = Promise.start({
&!start-handler();
my $intro = 1; # to skip the directory part of the file.
for $!source-name.IO.lines {
...
}
}).then({ $!DB.completed });
await $promise;
}
Thanks,
Theo
I found one thing that caused havoc. By changing the line
m:g/ << $<funname>=<[A..Z]>+ '(' <{ %fun{$<funname>}++ }> /;
by something like
if $formula ~~ m:g/ << $<funname>=<[A..Z]>+ '(' / {
for $/.list -> $funname {
%fun{$<funname>}++
};
}
the script gets a zillion times faster and runs without problems.
However, see #JonathanWorthington's reaction below. My conclusion:
I should have used:
m:g/ << $<funname>=<[A..Z]>+ '(' { %fun{$<funname>}++ } /;
Again, this used to work in previous editions of rakudo.
This is Rakudo version 2020.01 built on MoarVM version 2020.01.1
implementing Perl 6.d.
I have a sequence made up of multiple operators. There are total of 7 places where errors can be generated during this sequence processing. I'm running into an issue where the sequence does not behave as I expected and I'm looking for an elegant solution around the problem:
let inputRelay = PublishRelay<Int>()
let outputRelay = PublishRelay<Result<Int>>()
inputRelay
.map{ /*may throw multiple errors*/}
.flatmap{ /*may throw error*/ }
.map{}
.filter{}
.map{ _ -> Result<Int> in ...}
.catchError{}
.bind(to: outputRelay)
I thought that catchError would simply catch the error, allow me to convert it to failure result, but prevent the sequence from being deallocated. However, I see that the first time an error is caught, the entire sequence is deallocated and no more events go through.
Without this behavior, I'm left with a fugly Results<> all over the place, and have to branch my sequence multiple times to direct the Result.failure(Error) to the output. There are non-recoverable errors, so retry(n) is not an option:
let firstOp = inputRelay
.map{ /*may throw multiple errors*/}
.share()
//--Handle first error results--
firstOp
.filter{/*errorResults only*/}
.bind(to: outputRelay)
let secondOp = firstOp
.flatmap{ /*may throw error*/ }
.share()
//--Handle second error results--
secondOp
.filter{/*errorResults only*/}
.bind(to: outputRelay)
secondOp
.map{}
.filter{}
.map{ _ -> Result<Int> in ...}
.catchError{}
.bind(to: outputRelay)
^ Which is very bad, because there are around 7 places where errors can be thrown and I cannot just keep branching the sequence each time.
How can RxSwift operators catch all errors and emit a failure result at the end, but NOT dispose the entire sequence on first error?
The first trick to come to mind is using materialize. This would convert every Observable<T> to Observable<Event<T>>, so an Error would just be a .next(.error(Error)) and won't cause the termination of the sequence.
in this specific case, though, another trick would be needed. Putting your entire "trigger" chain within a flatMap, as well, and materializeing that specific piece. This is needed because a materialized sequence can still complete, which would cause a termination in case of a regular chain, but would not terminate a flatMapped chain (as complete == successfully done, inside a flatMap).
inputRelay
.flatMapLatest { val in
return Observable.just(val)
.map { value -> Int in
if value == 1 { throw SomeError.randomError }
return value + value
}
.flatMap { value in
return Observable<String>.just("hey\(value)")
}
.materialize()
}
.debug("k")
.subscribe()
inputRelay.accept(1)
inputRelay.accept(2)
inputRelay.accept(3)
inputRelay.accept(4)
This will output the following for k :
k -> subscribed
k -> Event next(error(randomError))
k -> Event next(next(hey4))
k -> Event next(completed)
k -> Event next(next(hey6))
k -> Event next(completed)
k -> Event next(next(hey8))
k -> Event next(completed)
Now all you have to do is filter just the "next" events from the materialized sequence.
If you have RxSwiftExt, you can simply use the errors() and elements() operators:
stream.elements()
.debug("elements")
.subscribe()
stream.errors()
.debug("errors")
.subscribe()
This will provide the following output:
errors -> Event next(randomError)
elements -> Event next(hey4)
elements -> Event next(hey6)
elements -> Event next(hey8)
When using this strategy, don't forget adding share() after your flatMap, so many subscriptions don't cause multiple pieces of processing.
You can read more about why you should use share in this situation here: http://adamborek.com/how-to-handle-errors-in-rxswift/
Hope this helps!
Yes, it's a pain. I've thought about the idea of making a new library where the grammar doesn't require the stream to end on an error, but trying to reproduce the entire Rx ecosystem for it seems pointless.
There are reactive libraries that allow you to specify Never as the error type (meaning an error can't be emitted at all,) and in RxCocoa you can use Driver (which can't error) but you are still left with the whole Result dance. "Monads in my Monads!".
To deal with it properly, you need a set of Monad transformers. With these, you can do all the mapping/flatMapping you want and not worry about looking at the errors until the very end.
I'm using the NUnit testing technique suggested in the yet to be released book "F# Deep Dives Version 12" (Sec. 2.2 "Adding Tests")
The code below executes fine compiled or interactive with MEMOIZE defined/undefined. However, executing the unit test from the GUI NUnit works fine with MEMOIZE undefined, but it fails with a "Null Reference Exception" when MEMOIZE is defined. Notice memorize() uses a Closure. I'm suspecting the exception is happening because some initialization code generated by the compiler is not getting executed when NUnit starts up. What do you think?
open System
open System.Collections.Generic
open NUnit.Framework
open FsUnit
let memoize (f: 'T -> 'U) =
let t = new Dictionary<'T, 'U>(HashIdentity.Structural)
fun n ->
if t.ContainsKey n then t.[n]
else let res = f n
t.Add(n, res)
res
//TODO: Insure J>0 & K>0 & J<K
let isMult =
#if MEMOIZE
memoize (fun (j,k) -> k % j = 0)
#else
(fun (j,k) -> k % j = 0)
#endif
type ``Given the isMult function``() =
[<TestCase(3,1,false)>]
[<TestCase(3,2,false)>]
[<TestCase(3,3,true)>]
[<TestCase(5,10,true)>]
[<TestCase(3,15,true)>]
[<TestCase(5,13,false)>]
[<TestCase(5,15,true)>]
member t.``the result is calculated correctly``(j, k, expected) =
let actual = isMult (j,k)
actual |> should equal expected
UPDATE:
The standalone NUnit application is version 2.6.3.13283.
"FinnNk" gave me an idea! I installed Nuget package "NUnitTestAdapter". Now I can test directly within VS 2013. No surprises, though. I get all tests 'green' when MEMORIZE is undefined and all tests 'red' when it is defined.
The exception is still the same: "Null Reference Exception". However, now that it executes in the IDE, I can have the debugger stop on the exception. All I can determine so far at the breakpoint is that it needs the symbols from:
C:\WINDOWS\Microsoft.Net\assembly\GAC_32\mscorlib\v4.0_4.0.0.0__b77a5c561934e089\mscorlib.pdb
I installed the new VS 2015 Preview Edition. Nothing different happens in that environment. Now that .NET Framework is open source, maybe I can zero the debugger precisely on the problem with the source code for "mscorlib".
Are you running your NUnit tests in multiple threads? Normal dictionary is not thread-safe so weird things can happen. How about if you use ConcurrentDictionary, will it give the same result?
let memoize (f: 'T -> 'U) =
let t = System.Collections.Concurrent.ConcurrentDictionary<'T, 'U>()
fun n -> t.GetOrAdd(n, f)
I have two class:
class Foo {
String doSomething(String a, String b) {
return 'Not Working'
}
}
class Bar {
String methodIWannaTest() {
return new Foo().doSomething('val1', 'val2')
}
}
And I want to replace 'doSomething' in a test but it dosent work
class BarTests {
#Test
void testMethodIWannaTest() {
Foo.metaClass.doSomething {String a, String b -> return 'Working'}
assert new Bar().methodIWannaTest() == 'Working' //THIS TEST FAIL, return 'Not Working'
}
}
*I know the test doesn't really make sens, it's just to show my point
What do I do wrong ? Is it possible to do it without using 'mockFor' ?
I would suggest you to start the test afresh. Baby Steps is what I follow. :)
Create a new grails app.
Create both Foo and Bar inside src/groovy under a package.
Create unit test case from command prompt. Put the desired test code.
Execute grails test-app
[Grails v2.2.0]
Another alternative is to use Groovy MockFor.
def mock = MockFor(Foo)
mock.demand.doSomething = {String a, String b -> return 'Working'}
mock.use {
assert new Bar().methodIWannaTest() == 'Working'
}
A downside is that there was a bug, making unit tests not clear the mock. This is fixed for 2.2.3.
I found the problem:
First, in my exemple, I forgot the '=' when defining the closure
Foo.metaClass.doSomething {String a, String b -> return 'Working'}
should be
Foo.metaClass.doSomething = {String a, String b -> return 'Working'}
While searching I also found that you cannot replace a method with optionnal parameters by a closure. Probably a bug
I tried this on GGTS 3.2.0 and got the following results.
Copied all the code and ran on Grails 2.2.2: everything worked correctly. However, then I commented out Foo.metaClass.doSomething = {String a, String b -> return 'Working'} line and, surprisingly, the test was still passing successfully. I did more changes and the metaClass just seemed to get "cached" between tests: changes had no effect.
Then I thought if it is possible that Grails were running in interactive mode. Went to Preferences > Groovy > Grails > Grails Launch and found a checkbox 'Keep external Grails running' ticked. Its tooltip even has the following words: 'Warning: experimental feature!'. Unticking that checkbox did the trick and the metaClass no longer got cached between test runs. I am still wondering why an exprimental feature would be turned on by default...
Statement coverage is said to make sure that every statement in the code is executed at least once.
Decision/branch coverage is said to test that each branch/output of a decisions is tested, i.e. all statements in both false/true branches will be executed.
But is it not the same? In Statement coverage I need to execute all statements so I guess it can be only done by running all possible ways. I know I am missing something here..
The answer by Paul isn't quite right, at least I think so (according to ISTQB's definitions). There's quite significant difference between statement, decision/branch and condition coverage.
I'll use the sample from the other answer but modified a bit, so I can show all three test coverage examples. Tests written here gives 100% test coverage for each type.
if(a || b)) {
test1 = true;
}
else {
if(c) {
test2 = true
}
}
We have two statements here - if(a||b) and if(c), to fully explain those coverage differences:
statement coverage have to test each statement at least once, so we need just two tests:
a=true b=false - that gives us path if(a||b) true -> test1 = true
a=false, b=false and c=true - that gives us path: if(a||b) false -> else -> if(c) -> test2=true.
This way we executed each and every statement.
branch/decision coverage needs one more test:
a=false, b=false, c=false - that leads us to that second if but we are executing false branch from that statement, that wasn't executed in statement coverage
That way we have all the branches tested, meaning we went through all the paths.
condition coverage needs another test:
a=false, b=true - that leads through the same path as first test but executes the other decision in OR statement (a||b) to go through it.
That way we have all conditions tested, meaning that we went through all paths (branches) and triggered it with each condition we could - first 'if' statement was true in first test because of a=true triggered it and in the last test because b=true triggered it. Of course someone can argue that case with a=true and b=true should be tested as well, but when we will check how 'or' works then we can see it isn't needed and also variable c can be of any value as in those tests it is not evaluated.
At least I interpreted it this way. If someone is still interested :)
EDIT: In most sources I found lately decision/branch coverage terms are equivalent and the term I described as decision coverage is in fact condition coverage hence that update of the answer.
If the tests have complete branch coverage then we can say it also has complete statement coverage, but not the vice versa.
100% branch coverage => 100% statement coverage
100% statement coverage does not imply 100% branch coverage
the reason is in branch coverage apart from executing all the statements, we should also verify if the tests executes all branches, which can be interpreted as covering all edges in the control flow branch
if(a){
if(b){
bool statement1 = true;
}
}
a = true, b = true will give 100% statement coverage, but not branch coverage
In the branch coverage we need to cover all the edges, which we missed in the statement coverage shown as red lines in the above image
Nice question. The explanation I often use is that an if-statement without an else-branch still has an invisible "empty" else-statement:
Plain statement coverage just insists that all statements that are actually there are really executed.
Branch coverage insists that even invisible else-branches are executed.
Similar situations occur with switch-statements without a default-case, and repeat-until loops. Branch coverage requires that a default-case is executed, and that a repeat-until is executed at least twice.
A code example:
if (passwordEnteredOK()) {
enterSystem();
}
/* Invisible else part
else {
// do nothing
}
*/
With statement coverage you just check that with a correct password you can use the system. With branch coverage you also test that with an incorrect password you will not enter the system.
You may have a statement like:
if(a || b || (c && d && !e)) {
test1 = true;
} else {
test2 = false;
}
If your code coverage says both the test1 and test2 lines are hit then you have statement coverage, but to get full branch coverage you will need to test when a is true, when a is false but b is true, when a and b are false but c and d are true and e is false, etc.
Branch coverage covers every potential combination of branch choices and so is harder to achieve 100% coverage.