Is it the correct way to create a in memory db using neo4j ? So that traverse query will hit only cache and not the disk .
approach - 1 : I tried with this :
package com.test;
import org.neo4j.graphdb.GraphDatabaseService;
import org.neo4j.graphdb.factory.GraphDatabaseFactory;
import org.neo4j.kernel.impl.util.FileUtils;
import java.io.File;
import java.io.IOException;
import java.util.HashMap;
import java.util.Map;
public class CreateDBFactory {
private static GraphDatabaseService graphDb = null;
public static final String DB = "test/db";
public static GraphDatabaseService createInMemoryDB() {
System.out.println("- Inside createInMemoryDB() - ");
if (null == graphDb) {
synchronized (GraphDatabaseService.class) {
if (null == graphDb) {
System.out.println(" - Inside if clause -");
final Map<String, String> config = new HashMap<>();
config.put("neostore.nodestore.db.mapped_memory", "50M");
config.put("string_block_size", "60");
config.put("array_block_size", "300");
graphDb = new GraphDatabaseFactory()
.newEmbeddedDatabaseBuilder(DB).setConfig(config)
.newGraphDatabase();
registerShutdownHook(graphDb);
}
}
}
return graphDb;
}
private static void registerShutdownHook(final GraphDatabaseService graphDb) {
Runtime.getRuntime().addShutdownHook(new Thread() {
#Override
public void run() {
graphDb.shutdown();
}
});
}
public static void clearDb() {
try {
if (graphDb != null) {
graphDb.shutdown();
graphDb = null;
}
FileUtils.deleteRecursively(new File(DB));
} catch (final IOException e) {
throw new RuntimeException(e);
}
}
}
Approach -2 : Using Neo4jBasicDocTest class.
Here new ImpermanentDatabaseBuilder() is not creating the target/test-data/impermanent-db folder. So not able to test " Nancy" node is created or not .
Neo4j doesn't have an 'in memory' mode in the sense that all data always stored in memory and no disk storage is used. ImpermanentGraphDatabase is the closest you'll find to anything like that, but that just creates a data directory at random and will delete it when it is shutdown.
If you are okay with using disk, you can use the above ImpermanentGraphDatabase, and just set the cache of neo4j to be really high. This will make everything be stored in memory, as well as on disk.
Related
Am new to QAF and I need to implement self-healing in our test method using healenium. I have implemented it without QAF it's working fine. Please refer to the below code.
import com.epam.healenium.SelfHealingDriver;
import io.github.bonigarcia.wdm.WebDriverManager;
import org.junit.jupiter.api.AfterAll;
import org.junit.jupiter.api.BeforeAll;
import org.openqa.selenium.Dimension;
import org.openqa.selenium.WebDriver;
import org.openqa.selenium.chrome.ChromeDriver;
import org.openqa.selenium.chrome.ChromeOptions;
import java.util.concurrent.TimeUnit;
public class BaseTest {
static protected SelfHealingDriver driver;
#BeforeAll
static public void setUp() {
WebDriverManager.chromedriver().setup();
ChromeOptions options = new ChromeOptions();
options.setHeadless(false);
//declare delegate
WebDriver delegate = new ChromeDriver(options);
driver = SelfHealingDriver.create(delegate);
driver.manage().timeouts().implicitlyWait(4, TimeUnit.SECONDS);
driver.manage().window().setSize(new Dimension(1200, 800));
}
#AfterAll
static public void afterAll() {
if (driver != null) {
driver.quit();
}
}
}
I just want to wrap this self-healing driver with a QAF web driver like above.
QAF discourage to use static class variable for driver. The code provided in question will not work for parallel execution. Driver management is taken care by qaf with thread safety and with different behavior that can be set using property selenium.singletone.
You can try following way when you want SelfHealingDriver:
public class SampleTest extends WebDriverTestCase {
#Test
public void yourTestCase(){
SelfHealingDriver driver = SelfHealingDriver.create(getDriver());
//your code goes below
}
}
SelfHealingDriver proxies actual driver. You can achieve the self heal functionality without driver proxy with listener for findelement/findChildelement. Driver listener should work without proxing driver. For example:
public class WDListener extends QAFWebDriverCommandAdapter {
private static final Map<String, Object> byToString = JSONUtil.toMap(
"{'ByCssSelector':'css selector','ByClassName':'class name','ByXPath':'xpath','ByPartialLinkText':'partial link text','ById':'id','ByLinkText':'link text','ByName':'name'}");
//this method will called when new driver object created
public void onInitialize(QAFExtendedWebDriver driver){
driver.manage().timeouts().implicitlyWait(4, TimeUnit.SECONDS);
driver.manage().window().setSize(new Dimension(1200, 800));
}
#Override
public void afterCommand(QAFExtendedWebDriver driver, CommandTracker commandTracker) {
if (DriverCommand.FIND_ELEMENT.equalsIgnoreCase(commandTracker.getCommand())
|| DriverCommand.FIND_ELEMENTS.equalsIgnoreCase(commandTracker.getCommand())
|| DriverCommand.FIND_CHILD_ELEMENT.equalsIgnoreCase(commandTracker.getCommand())
|| DriverCommand.FIND_CHILD_ELEMENTS.equalsIgnoreCase(commandTracker.getCommand())) {
Map<String, Object> parameters = commandTracker.getParameters();
if (parameters != null && parameters.containsKey("using") && parameters.containsKey("value")) {
By by = LocatorUtil
.getBy(String.format("%s=%s", parameters.get("using"), parameters.get("value")));
HealingServiceImpl healingServiceImpl = new HealingServiceImpl(new SelfHealingEngine(driver));
StackTraceElement[] stackTrace = Thread.currentThread().getStackTrace();
Object result = commandTracker.getResponce().getValue();
List<WebElement> webElements = List.class.isAssignableFrom(result.getClass())?(List<WebElement>) result:Collections.singletonList((WebElement)result)
healingServiceImpl.savePath(new PageAwareBy(driver.getTitle(),by),webElements);
}
}
}
#Override
public void onFailure(QAFExtendedWebDriver driver, CommandTracker commandTracker) {
// StackTraceElement[] stackTrace =
// commandTracker.getException().getStackTrace();
Map<String, Object> parameters = commandTracker.getParameters();
if (parameters != null && parameters.containsKey("using") && parameters.containsKey("value")) {
By by = LocatorUtil
.getBy(String.format("%s=%s", parameters.get("using"), parameters.get("value")));
StackTraceElement[] stackTrace = Thread.currentThread().getStackTrace();
HealingServiceImpl healingServiceImpl = new HealingServiceImpl(new SelfHealingEngine(driver));
Optional<By> healedBy = healingServiceImpl.healLocators(new PageAwareBy(driver.getTitle(),by), null, stackTrace) ;
if(healedBy.isPresent()) {
commandTracker.getParameters().putAll(toParams(healedBy.get()));
commandTracker.setRetry(true);
}
}
}
}
I have a small Micronaut application with a view layer (thymeleaf). Now I want to integrate Google Analytics or some other tracking tool. Of course this should be just loaded in production mode but how can I do a check for the environment in the view layer?
I start the application in the following way:
java -Dmicronaut.environments=prod -jar mywebsite.jar
You can inject the io.micronaut.context.env.Environment and pass the result of getActiveNames() as part of your view model
EDIT:
If you want to add this data to each model without touching each controller method you can add a server filter that happens before the view filter to manipulate the response, or create a piece of around advice that can manipulate the return value of the controller.
Another approach to solving this problem is to utilise #Replaces annotation to override ThymeleafViewsRenderer class and add common data in your class.
import io.micronaut.context.annotation.Replaces
import io.micronaut.context.env.Environment
import io.micronaut.core.beans.BeanMap
import io.micronaut.core.io.Writable
import io.micronaut.core.io.scan.ClassPathResourceLoader
import io.micronaut.core.util.ArgumentUtils
import io.micronaut.http.HttpRequest
import io.micronaut.views.exceptions.ViewRenderingException
import io.micronaut.views.thymeleaf.ThymeleafViewsRenderer
import io.micronaut.views.thymeleaf.WebContext
import org.thymeleaf.TemplateEngine
import org.thymeleaf.context.Context
import org.thymeleaf.context.IContext
import org.thymeleaf.exceptions.TemplateEngineException
import org.thymeleaf.templateresolver.AbstractConfigurableTemplateResolver
import javax.annotation.Nonnull
import javax.annotation.Nullable
import javax.inject.Inject
import javax.inject.Singleton
#Singleton
#Replaces(bean = ThymeleafViewsRenderer.class)
class CustomThymeleafFilter extends ThymeleafViewsRenderer {
CustomThymeleafFilter(AbstractConfigurableTemplateResolver templateResolver, TemplateEngine templateEngine, ClassPathResourceLoader resourceLoader) {
super(templateResolver, templateEngine, resourceLoader)
}
#Inject
Environment environment
#Override
#Nonnull
public Writable render(#Nonnull String viewName, #Nullable Object data) {
ArgumentUtils.requireNonNull("viewName", viewName);
return (writer) -> {
//following block adds environments variable to model
Map dataMap = variables(data)
dataMap.put("environment", environment.activeNames)
IContext context = new Context(Locale.US, dataMap);
processView(viewName, writer, context);
};
}
#Override
#Nonnull
public Writable render(#Nonnull String viewName, #Nullable Object data,
#Nonnull HttpRequest<?> request) {
ArgumentUtils.requireNonNull("viewName", viewName);
ArgumentUtils.requireNonNull("request", request);
return (writer) -> {
//following block adds environments variable to model
Map dataMap = variables(data)
dataMap.put("environment", environment.activeNames)
IContext context = new WebContext(request, Locale.US, dataMap);
processView(viewName, writer, context);
};
}
private static Map<String, Object> variables(#Nullable Object data) {
if (data == null) {
return new HashMap<>();
}
if (data instanceof Map) {
return (Map<String, Object>) data;
} else {
return BeanMap.of(data);
}
}
private void processView(String viewName, Writer writer, IContext context) {
try {
engine.process(viewName, context, writer);
} catch (TemplateEngineException e) {
throw new ViewRenderingException("Error rendering Thymeleaf view [" + viewName + "]: " + e.getMessage(), e);
}
}
}
I am using Neo4j 2.1.2 community edition. When I point neo4j to the database I created and login to webadmin, it shows the relationship type and and Labels I created in my code but when i try to fetch the nodes and relationship it does not list any thing.
See the screenshot attached.
I am creating my database using following piece of code.
import org.neo4j.graphdb.GraphDatabaseService;
import org.neo4j.graphdb.Label;
import org.neo4j.graphdb.Node;
import org.neo4j.graphdb.Relationship;
import org.neo4j.graphdb.RelationshipType;
import org.neo4j.graphdb.Transaction;
import org.neo4j.graphdb.factory.GraphDatabaseFactory;
public class HelloWorld {
private GraphDatabaseService m_databaseService;
private Node m_firstNode;
private Node m_secondNode;
private Relationship m_relationship;
public static void main(String[] args) {
new HelloWorld().startApplication();
}
private void startApplication() {
createDatabase();
createNodesAndrelationship();
shutDownDB();
}
private void createDatabase() {
m_databaseService = new GraphDatabaseFactory().newEmbeddedDatabase("E:\\ERM\\RoutingData\\neo4j");
}
private void createNodesAndrelationship() {
Transaction tx = m_databaseService.beginTx();
try {
m_firstNode = m_databaseService.createNode();
m_firstNode.addLabel(new Label() {
#Override
public String name() {
return "Demo";
}
});
m_firstNode.setProperty("message", "Hello");
System.out.println("Created First Node.");
m_secondNode = m_databaseService.createNode();
m_secondNode.setProperty("message", "world !");
m_secondNode.addLabel(new Label() {
#Override
public String name() {
return "Demo";
}
});
System.out.println("Created Second Node.");
m_relationship = m_firstNode.createRelationshipTo(m_secondNode, RelType.KNOWS);
m_relationship.setProperty("message", "bravo neo4j !");
System.out.println("Created relationship.");
tx.success();
} finally {
tx.failure();
}
System.out.println(m_firstNode.getProperty("message").toString() + " " + m_relationship.getProperty("message") + " " + m_secondNode.getProperty("message"));
}
private void shutDownDB() {
m_databaseService.shutdown();
System.out.println("Database shutdown completed.");
}
private static enum RelType implements RelationshipType {
KNOWS
}
}
Please suggest what am I doing wrong.
You are rolling back the transaction rather than committing it.
Replace tx.failure() with tx.close().
Also, this line needs to be moved before the tx.close() call, to avoid an org.neo4j.graphdb.NotInTransactionException:
System.out.println(m_firstNode.getProperty("message").toString() + " " + m_relationship.getProperty("message") + " " + m_secondNode.getProperty("message"));
I'm experimenting with Neo4J + Gremlin plugin, unfortunately I got this error (bellow) when I try to use the Gremlin console web (the console didn't runs on Gremlin language):
SEVERE: The exception contained within MappableContainerException could not be mapped to a response, re-throwing to the HTTP container
java.lang.NoClassDefFoundError: org/neo4j/server/logging/Logger
at org.neo4j.server.webadmin.console.GremlinSession.<clinit>(GremlinSession.java:42)
at org.neo4j.server.webadmin.console.GremlinSessionCreator.newSession(GremlinSessionCreator.java:35)
......
Any suggestions?
Many thanks!
Neo4J 2.0.2 server has changed the Logger class, so I modified GremlinSession.java code in order to use java.util.logging.Logger class instead of org.neo4j.server.logging.Logger. Finally the Gremlin console web works fine:
Here you will find the new GremlinSession.java code.
/**
* Copyright (c) 2002-2014 "Neo Technology,"
* Network Engine for Objects in Lund AB [http://neotechnology.com]
*
* This file is part of Neo4j.
*
* Neo4j is free software: you can redistribute it and/or modify
* it under the terms of the GNU General Public License as published by
* the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU General Public License for more details.
*
* You should have received a copy of the GNU General Public License
* along with this program. If not, see <http://www.gnu.org/licenses/>.
*/
package org.neo4j.server.webadmin.console;
import com.tinkerpop.blueprints.TransactionalGraph;
import com.tinkerpop.blueprints.impls.neo4j2.Neo4j2Graph;
import groovy.lang.Binding;
import groovy.lang.GroovyRuntimeException;
import org.codehaus.groovy.tools.shell.IO;
import org.neo4j.graphdb.Transaction;
import org.neo4j.helpers.Pair;
import org.neo4j.server.database.Database;
import java.util.logging.Logger;
import java.util.logging.Level;
import java.io.BufferedOutputStream;
import java.io.ByteArrayOutputStream;
import java.io.PrintStream;
import java.util.ArrayList;
import java.util.HashMap;
import java.util.List;
import java.util.Map;
public class GremlinSession implements ScriptSession {
private static final String INIT_FUNCTION = "init()";
private static final Logger log = Logger.getLogger(GremlinSession.class.getName());
private final Database database;
private final IO io;
private final ByteArrayOutputStream baos = new ByteArrayOutputStream();
private final List<String> initialBindings;
protected GremlinWebConsole scriptEngine;
public GremlinSession(Database database) {
this.database = database;
PrintStream out = new PrintStream(new BufferedOutputStream(baos));
io = new IO(System.in, out, out);
Map<String, Object> bindings = new HashMap<String, Object>();
bindings.put("g", getGremlinWrappedGraph());
bindings.put("out", out);
initialBindings = new ArrayList<String>(bindings.keySet());
try {
scriptEngine = new GremlinWebConsole(new Binding(bindings), io);
} catch (final Exception failure) {
scriptEngine = new GremlinWebConsole() {
#Override
public void execute(String script) {
io.out.println("Could not start Groovy during Gremlin initialization, reason:");
failure.printStackTrace(io.out);
}
};
}
}
/**
* Take some gremlin script, evaluate it in the context of this gremlin
* session, and return the result.
*
* #param script
* #return the return string of the evaluation result, or the exception
* message.
*/
#Override
public Pair<String, String> evaluate(String script) {
String result = null;
try (Transaction tx = database.getGraph().beginTx()) {
if (script.equals(INIT_FUNCTION)) {
result = init();
} else {
try {
scriptEngine.execute(script);
result = baos.toString();
} finally {
resetIO();
}
}
tx.success();
} catch (GroovyRuntimeException ex) {
log.log(Level.SEVERE, ex.toString());
result = ex.getMessage();
}
return Pair.of(result, null);
}
private String init() {
StringBuilder out = new StringBuilder();
out.append("\n");
out.append(" \\,,,/\n");
out.append(" (o o)\n");
out.append("-----oOOo-(_)-oOOo-----\n");
out.append("\n");
out.append("Available variables:\n");
for (String variable : initialBindings) {
out.append(" " + variable + "\t= ");
out.append(evaluate(variable));
}
out.append("\n");
return out.toString();
}
private void resetIO() {
baos.reset();
}
private TransactionalGraph getGremlinWrappedGraph() {
Neo4j2Graph neo4jGraph = null;
try {
neo4jGraph = new Neo4j2Graph(database.getGraph());
} catch (Exception e) {
throw new RuntimeException(e);
}
return neo4jGraph;
}
}
Im trying to import RDF-Triples into OrientDB with help of tinkerpop/blueprints.
I found the basic usage here.
Im now that far:
import info.aduna.iteration.CloseableIteration;
import org.openrdf.model.Statement;
import org.openrdf.model.ValueFactory;
import org.openrdf.sail.Sail;
import org.openrdf.sail.SailConnection;
import org.openrdf.sail.SailException;
import com.hp.hpl.jena.graph.Node;
import com.hp.hpl.jena.graph.Triple;
import com.tinkerpop.blueprints.impls.orient.OrientGraph;
import com.tinkerpop.blueprints.oupls.sail.GraphSail;
import de.hof.iisys.relationExtraction.jena.parser.impl.ParserStreamIterator;
import de.hof.iisys.relationExtraction.neo4j.importer.Importer;
public class ImporterJenaTriples extends Importer {
private OrientGraph graph = null;
private Sail sail = null;
private SailConnection sailConnection = null;
private ValueFactory valueFactory = null;
private Thread parserThread = null;
public ImporterJenaTriples(ParserStreamIterator parser, String databasePath) throws SailException {
this.parser = parser;
this.databasePath = databasePath;
this.initialize();
}
private void initialize() throws SailException {
this.graph = new OrientGraph(this.databasePath);
this.sail = new GraphSail<OrientGraph>(graph);
sail.initialize();
this.sailConnection = sail.getConnection();
this.valueFactory = sail.getValueFactory();
}
public void startImport() {
this.parserThread = new Thread(this.parser);
this.parserThread.start();
try {
Triple next = (Triple) this.parser.getIterator().next();
Node subject = next.getSubject();
Node predicate = next.getPredicate();
Node object = next.getObject();
} catch (SailException e) {
e.printStackTrace();
}
try {
CloseableIteration<? extends Statement, SailException> results = this.sailConnection.getStatements(null, null, null, false);
while(results.hasNext()) {
System.out.println(results.next());
}
} catch (SailException e) {
e.printStackTrace();
}
}
public void stopImport() throws InterruptedException {
this.parser.terminate();
this.parserThread.join();
}
}
What i need to do now is to differ the types of subject, predicate and object
but the problem is i dont know which types they are and how i have to use
the valuefactory to create the type and to add the Statement to my SailConnection.
Unfortunately i cant find an example how to use it.
Maybe someone has done it before and knows how to continue.
I guess you need to convert from Jena object types to Sesame ones and use the
The unsupported project https://github.com/afs/JenaSesame may have some code for that.
But mixing Jena and Sesame seems to make things more complicated - have you consider using the Sesame parser and getting Sesame objects that can go into the SailConnection?