I have the avro scheme (pay attention on nested array struct):
{
"namespace":"nifi",
"name":"scheme_name",
"type":"record",
"fields":[
{
"name":"values",
"type":{
"type":"array",
"items":{
"type":"array",
"items": ["float", {"type": "array", "items": ["float", "string", "null"]}]
}
}
}
]
}
I have the json file:
{"values": [[1, 1.1, 1.2, 1.3, [-1, -1.1, -1.2, -1.3], -2, 3], [2, 2.1, 2.2, 2.3, [-2, -2.1, -2.2, -2.3], -3, 4]]}
I have next nifi processor group (see picture). GetFile - just get json which you can see above. ConvertRecord - just convert recieved json by JsonTreeReader to avro by AvroRecordSetWriter. JsonTreeReader and AvroRecordSetWriter have schema registry: AvroSchemaRegistry (which contain avro scheme which you can see above).
On the step converting avro to orc nifi throws exception:
2018-10-17 13:51:56,809 ERROR [Timer-Driven Process Thread-8] o.a.n.processors.hive.ConvertAvroToORC ConvertAvroToORC[id=814f08dc-0166-1000-a46c-f69042e8ae94] ConvertAvroToORC[id=814f08dc-0166-1000-a46c-f69042e8ae94] failed to process session due to java.lang.IllegalArgumentException: Object Type for class org.apache.avro.generic.GenericData$Array not in Union declaration; Processor Administratively Yielded for 1 sec: java.lang.IllegalArgumentException: Object Type for class org.apache.avro.generic.GenericData$Array not in Union declaration
java.lang.IllegalArgumentException: Object Type for class org.apache.avro.generic.GenericData$Array not in Union declaration
at org.apache.hadoop.hive.ql.io.orc.NiFiOrcUtils.convertToORCObject(NiFiOrcUtils.java:88)
at org.apache.hadoop.hive.ql.io.orc.NiFiOrcUtils.lambda$convertToORCObject$7(NiFiOrcUtils.java:149)
at java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:193)
at java.util.Iterator.forEachRemaining(Iterator.java:116)
at java.util.Spliterators$IteratorSpliterator.forEachRemaining(Spliterators.java:1801)
at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:481)
at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:471)
at java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:708)
at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
at java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:499)
at org.apache.hadoop.hive.ql.io.orc.NiFiOrcUtils.convertToORCObject(NiFiOrcUtils.java:149)
at org.apache.hadoop.hive.ql.io.orc.NiFiOrcUtils.lambda$convertToORCObject$7(NiFiOrcUtils.java:149)
at java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:193)
at java.util.Iterator.forEachRemaining(Iterator.java:116)
at java.util.Spliterators$IteratorSpliterator.forEachRemaining(Spliterators.java:1801)
at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:481)
at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:471)
at java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:708)
at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
at java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:499)
at org.apache.hadoop.hive.ql.io.orc.NiFiOrcUtils.convertToORCObject(NiFiOrcUtils.java:149)
at org.apache.nifi.processors.hive.ConvertAvroToORC.lambda$onTrigger$0(ConvertAvroToORC.java:245)
at org.apache.nifi.controller.repository.StandardProcessSession.write(StandardProcessSession.java:2885)
at org.apache.nifi.processors.hive.ConvertAvroToORC.onTrigger(ConvertAvroToORC.java:209)
at org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:27)
at org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1165)
at org.apache.nifi.controller.tasks.ConnectableTask.invoke(ConnectableTask.java:203)
at org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:117)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
2018-10-17 13:51:56,812 WARN [Timer-Driven Process Thread-8] o.a.n.controller.tasks.ConnectableTask Administratively Yielding ConvertAvroToORC[id=814f08dc-0166-1000-a46c-f69042e8ae94] due to uncaught Exception: java.lang.IllegalArgumentException: Object Type for class org.apache.avro.generic.GenericData$Array not in Union declaration
java.lang.IllegalArgumentException: Object Type for class org.apache.avro.generic.GenericData$Array not in Union declaration
at org.apache.hadoop.hive.ql.io.orc.NiFiOrcUtils.convertToORCObject(NiFiOrcUtils.java:88)
at org.apache.hadoop.hive.ql.io.orc.NiFiOrcUtils.lambda$convertToORCObject$7(NiFiOrcUtils.java:149)
at java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:193)
at java.util.Iterator.forEachRemaining(Iterator.java:116)
at java.util.Spliterators$IteratorSpliterator.forEachRemaining(Spliterators.java:1801)
at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:481)
at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:471)
at java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:708)
at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
at java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:499)
at org.apache.hadoop.hive.ql.io.orc.NiFiOrcUtils.convertToORCObject(NiFiOrcUtils.java:149)
at org.apache.hadoop.hive.ql.io.orc.NiFiOrcUtils.lambda$convertToORCObject$7(NiFiOrcUtils.java:149)
at java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:193)
at java.util.Iterator.forEachRemaining(Iterator.java:116)
at java.util.Spliterators$IteratorSpliterator.forEachRemaining(Spliterators.java:1801)
at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:481)
at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:471)
at java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:708)
at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
at java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:499)
at org.apache.hadoop.hive.ql.io.orc.NiFiOrcUtils.convertToORCObject(NiFiOrcUtils.java:149)
at org.apache.nifi.processors.hive.ConvertAvroToORC.lambda$onTrigger$0(ConvertAvroToORC.java:245)
at org.apache.nifi.controller.repository.StandardProcessSession.write(StandardProcessSession.java:2885)
at org.apache.nifi.processors.hive.ConvertAvroToORC.onTrigger(ConvertAvroToORC.java:209)
at org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:27)
at org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1165)
at org.apache.nifi.controller.tasks.ConnectableTask.invoke(ConnectableTask.java:203)
at org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:117)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
2018-10-17 13:52:01,247 INFO [NiFi Web Server-214] o.a.n.c.s.StandardProcessScheduler Stopping ConvertRecord[id=814a4abe-0166-1000-0755-4d43aef3dc4a]
2018-10-17 13:52:01,247 INFO [NiFi Web Server-214] o.a.n.controller.StandardProcessorNode Stopping processor: class org.apache.nifi.processors.standard.ConvertRecord
2018-10-17 13:52:01,247 INFO [Timer-Driven Process Thread-8] o.a.n.c.s.TimerDrivenSchedulingAgent Stopped scheduling ConvertRecord[id=814a4abe-0166-1000-0755-4d43aef3dc4a] to run
So, please tell me where am I wrong?
Environment:
OS:SUSE Linux Enterprise Server 12 SP3 (release 12.3)
or Windows 7 Corporate SP1
Nifi version: 1.7.1
There is a bug about this in jira:
NIFI-5716. ConvertAvroToORC processor can't convert avro file which has array which consists of floats and array of float
Related
So I'm trying to deploy 2 instances of Janusgraph (v 0.3.1) to insert data in the same keyspace of ScyllaDb backend. To do that i deploy 2 janusgraph containers with docker. The first one is starting without errors and creates the keyspace in my ScyllaDb but the second one displays some errors.
So my Janusgraph containers works on a cluster while my backend works on another cluster. Actually, I tried with only one container to restart it when my keyspace Scylla is already created and I have the same problem.
I also found a script clean.groovy that allowed to force shutdown the graph instances opened. But nothing works...
5316 [main] INFO org.janusgraph.diskstorage.Backend - Initiated backend operations thread pool of size 8
5648 [main] WARN org.apache.tinkerpop.gremlin.server.GremlinServer - Graph [graph] configured at [/janusgraph-config/janusgraph.properties] could not be instantiated and will not be available in Gremlin Server. GraphFactory message: GraphFactory could not instantiate this Graph implementation [class org.janusgraph.core.JanusGraphFactory]
java.lang.RuntimeException: GraphFactory could not instantiate this Graph implementation [class org.janusgraph.core.JanusGraphFactory]
at org.apache.tinkerpop.gremlin.structure.util.GraphFactory.open(GraphFactory.java:82)
at org.apache.tinkerpop.gremlin.structure.util.GraphFactory.open(GraphFactory.java:70)
at org.apache.tinkerpop.gremlin.structure.util.GraphFactory.open(GraphFactory.java:104)
at org.apache.tinkerpop.gremlin.server.util.DefaultGraphManager.lambda$new$0(DefaultGraphManager.java:57)
at java.util.LinkedHashMap$LinkedEntrySet.forEach(LinkedHashMap.java:671)
at org.apache.tinkerpop.gremlin.server.util.DefaultGraphManager.<init>(DefaultGraphManager.java:55)
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
at org.apache.tinkerpop.gremlin.server.util.ServerGremlinExecutor.<init>(ServerGremlinExecutor.java:80)
at org.apache.tinkerpop.gremlin.server.GremlinServer.<init>(GremlinServer.java:120)
at org.apache.tinkerpop.gremlin.server.GremlinServer.<init>(GremlinServer.java:84)
at org.apache.tinkerpop.gremlin.server.GremlinServer.main(GremlinServer.java:343)
Caused by: java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.tinkerpop.gremlin.structure.util.GraphFactory.open(GraphFactory.java:78)
... 13 more
Caused by: org.janusgraph.core.JanusGraphException: A JanusGraph graph with the same instance id [0a0a01657-node11] is already open. Might required forced shutdown.
at org.janusgraph.graphdb.database.StandardJanusGraph.<init>(StandardJanusGraph.java:165)
at org.janusgraph.core.JanusGraphFactory.open(JanusGraphFactory.java:160)
at org.janusgraph.core.JanusGraphFactory.open(JanusGraphFactory.java:131)
at org.janusgraph.core.JanusGraphFactory.open(JanusGraphFactory.java:111)
... 18 more
5651 [main] INFO org.apache.tinkerpop.gremlin.server.util.ServerGremlinExecutor - Initialized Gremlin thread pool. Threads in pool named with pattern gremlin-*
5800 [main] INFO org.apache.tinkerpop.gremlin.server.util.ServerGremlinExecutor - Initialized GremlinExecutor and preparing GremlinScriptEngines instances.
7837 [gremlin-server-exec-1] ERROR org.apache.tinkerpop.gremlin.jsr223.DefaultGremlinScriptEngineManager - Could not create GremlinScriptEngine for gremlin-groovy
java.lang.IllegalStateException: javax.script.ScriptException: javax.script.ScriptException: groovy.lang.MissingPropertyException: No such property: graph for class: Script1
at org.apache.tinkerpop.gremlin.jsr223.DefaultGremlinScriptEngineManager.lambda$createGremlinScriptEngine$16(DefaultGremlinScriptEngineManager.java:464)
at java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:184)
at java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:193)
at java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1382)
at java.util.stream.ReferencePipeline$Head.forEach(ReferencePipeline.java:580)
at java.util.stream.ReferencePipeline$7$1.accept(ReferencePipeline.java:270)
at java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1382)
at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:481)
at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:471)
at java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:151)
at java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:174)
at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
at java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:418)
at org.apache.tinkerpop.gremlin.jsr223.DefaultGremlinScriptEngineManager.createGremlinScriptEngine(DefaultGremlinScriptEngineManager.java:450)
at org.apache.tinkerpop.gremlin.jsr223.DefaultGremlinScriptEngineManager.getEngineByName(DefaultGremlinScriptEngineManager.java:219)
at org.apache.tinkerpop.gremlin.jsr223.CachedGremlinScriptEngineManager.lambda$getEngineByName$0(CachedGremlinScriptEngineManager.java:57)
at java.util.concurrent.ConcurrentHashMap.computeIfAbsent(ConcurrentHashMap.java:1660)
at org.apache.tinkerpop.gremlin.jsr223.CachedGremlinScriptEngineManager.getEngineByName(CachedGremlinScriptEngineManager.java:57)
at org.apache.tinkerpop.gremlin.groovy.engine.GremlinExecutor.lambda$eval$0(GremlinExecutor.java:263)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: javax.script.ScriptException: javax.script.ScriptException: groovy.lang.MissingPropertyException: No such property: graph for class: Script1
at org.apache.tinkerpop.gremlin.groovy.jsr223.GremlinGroovyScriptEngine.eval(GremlinGroovyScriptEngine.java:397)
at javax.script.AbstractScriptEngine.eval(AbstractScriptEngine.java:264)
at org.apache.tinkerpop.gremlin.jsr223.DefaultGremlinScriptEngineManager.lambda$createGremlinScriptEngine$16(DefaultGremlinScriptEngineManager.java:460)
... 24 more
Caused by: javax.script.ScriptException: groovy.lang.MissingPropertyException: No such property: graph for class: Script1
at org.apache.tinkerpop.gremlin.groovy.jsr223.GremlinGroovyScriptEngine.eval(GremlinGroovyScriptEngine.java:713)
at org.apache.tinkerpop.gremlin.groovy.jsr223.GremlinGroovyScriptEngine.eval(GremlinGroovyScriptEngine.java:395)
... 26 more
Caused by: groovy.lang.MissingPropertyException: No such property: graph for class: Script1
at org.codehaus.groovy.runtime.ScriptBytecodeAdapter.unwrap(ScriptBytecodeAdapter.java:66)
at org.codehaus.groovy.runtime.callsite.PogoGetPropertySite.getProperty(PogoGetPropertySite.java:51)
at org.codehaus.groovy.runtime.callsite.AbstractCallSite.callGroovyObjectGetProperty(AbstractCallSite.java:310)
at Script1.run(Script1.groovy:16)
at org.apache.tinkerpop.gremlin.groovy.jsr223.GremlinGroovyScriptEngine.eval(GremlinGroovyScriptEngine.java:690)
... 27 more
7841 [main] WARN org.apache.tinkerpop.gremlin.server.util.ServerGremlinExecutor - Could not initialize gremlin-groovy GremlinScriptEngine as init script could not be evaluated
java.util.concurrent.CompletionException: java.lang.IllegalArgumentException: gremlin-groovy is not an available GremlinScriptEngine
at java.util.concurrent.CompletableFuture.reportJoin(CompletableFuture.java:375)
at java.util.concurrent.CompletableFuture.join(CompletableFuture.java:1934)
at org.apache.tinkerpop.gremlin.server.util.ServerGremlinExecutor.lambda$new$4(ServerGremlinExecutor.java:141)
at java.util.LinkedHashMap$LinkedKeySet.forEach(LinkedHashMap.java:559)
at org.apache.tinkerpop.gremlin.server.util.ServerGremlinExecutor.<init>(ServerGremlinExecutor.java:136)
at org.apache.tinkerpop.gremlin.server.GremlinServer.<init>(GremlinServer.java:120)
at org.apache.tinkerpop.gremlin.server.GremlinServer.<init>(GremlinServer.java:84)
at org.apache.tinkerpop.gremlin.server.GremlinServer.main(GremlinServer.java:343)
Caused by: java.lang.IllegalArgumentException: gremlin-groovy is not an available GremlinScriptEngine
at org.apache.tinkerpop.gremlin.jsr223.CachedGremlinScriptEngineManager.registerLookUpInfo(CachedGremlinScriptEngineManager.java:95)
at org.apache.tinkerpop.gremlin.jsr223.CachedGremlinScriptEngineManager.getEngineByName(CachedGremlinScriptEngineManager.java:58)
at org.apache.tinkerpop.gremlin.groovy.engine.GremlinExecutor.lambda$eval$0(GremlinExecutor.java:263)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
I think all those errors are created by this :
Caused by: org.janusgraph.core.JanusGraphException: A JanusGraph graph with the same instance id [0a0a01657-node11] is already open. Might required forced shutdown.
And this is the script clean.groovy
graph = JanusGraphFactory.open('janusgraph.properties')
mgmt = graph.openManagement()
instances = mgmt.getOpenInstances()
instances.iterator().findAll {
!it.contains('current')
}.each {
mgmt.forceCloseInstance(it)
}
mgmt.commit()
So did anybody solved this problem of graph ?
Thanks in advance.
I finally got answer ! So you can use ConfiguredGraphFactory but it failed for me. So for the first container i started it normally but the second one i added another configuration in the janusgraph.properties file :
graph.replace-instance-if-exists=true`
And it works perfectly !
While trying to run a docker.push on a ppc64le Jenkins slave called by an x86 Jenkins master, the following error occurs.
Remote call to JNLP4-connect connection from /
java.lang.UnsatisfiedLinkError: /tmp/libnetty-transport-native-epoll3535546627560640699.so: /tmp/libnetty-transport-native-epoll3535546627560640699.so: cannot open shared object file: No such file or directory (Possible cause: can’t load AMD 64-bit .so on a Power PC 64-bit platform
Whole log
Also: hudson.remoting.Channel$CallSiteStackTrace: Remote call to JNLP4-connect connection from IP1/IP2:PORT
at hudson.remoting.Channel.attachCallSiteStackTrace(Channel.java:1741)
at hudson.remoting.UserRequest$ExceptionResponse.retrieve(UserRequest.java:357)
at hudson.remoting.Channel.call(Channel.java:955)
at org.jfrog.hudson.pipeline.docker.utils.DockerAgentUtils.getImageIdFromAgent(DockerAgentUtils.java:303)
at org.jfrog.hudson.pipeline.steps.DockerPushStep$Execution.run(DockerPushStep.java:122)
at org.jfrog.hudson.pipeline.steps.DockerPushStep$Execution.run(DockerPushStep.java:86)
at org.jenkinsci.plugins.workflow.steps.AbstractSynchronousNonBlockingStepExecution$1$1.call(AbstractSynchronousNonBlockingStepExecution.java:47)
at hudson.security.ACL.impersonate(ACL.java:290)
at org.jenkinsci.plugins.workflow.steps.AbstractSynchronousNonBlockingStepExecution$1.run(AbstractSynchronousNonBlockingStepExecution.java:44)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
java.lang.UnsatisfiedLinkError: /tmp/libnetty-transport-native-epoll3535546627560640699.so: /tmp/libnetty-transport-native-epoll3535546627560640699.so: cannot open shared object file: No such file or directory (Possible cause: can't load AMD 64-bit .so on a Power PC 64-bit platform)
at java.lang.ClassLoader$NativeLibrary.load(Native Method)
at java.lang.ClassLoader.loadLibrary0(ClassLoader.java:1941)
at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1824)
at java.lang.Runtime.load0(Runtime.java:809)
at java.lang.System.load(System.java:1086)
at io.netty.util.internal.NativeLibraryLoader.load(NativeLibraryLoader.java:214)
at io.netty.channel.epoll.Native.loadNativeLibrary(Native.java:269)
at io.netty.channel.epoll.Native.<clinit>(Native.java:64)
at io.netty.channel.epoll.IovArray.<clinit>(IovArray.java:57)
at io.netty.channel.epoll.EpollEventLoop.<init>(EpollEventLoop.java:62)
at io.netty.channel.epoll.EpollEventLoopGroup.newChild(EpollEventLoopGroup.java:130)
at io.netty.channel.epoll.EpollEventLoopGroup.newChild(EpollEventLoopGroup.java:35)
at io.netty.util.concurrent.MultithreadEventExecutorGroup.<init>(MultithreadEventExecutorGroup.java:84)
at io.netty.util.concurrent.MultithreadEventExecutorGroup.<init>(MultithreadEventExecutorGroup.java:58)
at io.netty.util.concurrent.MultithreadEventExecutorGroup.<init>(MultithreadEventExecutorGroup.java:47)
at io.netty.channel.MultithreadEventLoopGroup.<init>(MultithreadEventLoopGroup.java:58)
at io.netty.channel.epoll.EpollEventLoopGroup.<init>(EpollEventLoopGroup.java:100)
at io.netty.channel.epoll.EpollEventLoopGroup.<init>(EpollEventLoopGroup.java:87)
at io.netty.channel.epoll.EpollEventLoopGroup.<init>(EpollEventLoopGroup.java:64)
at com.github.dockerjava.netty.NettyDockerCmdExecFactory$UnixDomainSocketInitializer.init(NettyDockerCmdExecFactory.java:220)
at com.github.dockerjava.netty.NettyDockerCmdExecFactory.init(NettyDockerCmdExecFactory.java:196)
at com.github.dockerjava.core.DockerClientImpl.withDockerCmdExecFactory(DockerClientImpl.java:161)
at com.github.dockerjava.core.DockerClientBuilder.build(DockerClientBuilder.java:45)
at org.jfrog.hudson.pipeline.docker.utils.DockerUtils.getDockerClient(DockerUtils.java:326)
at org.jfrog.hudson.pipeline.docker.utils.DockerUtils.getImageIdFromTag(DockerUtils.java:37)
at org.jfrog.hudson.pipeline.docker.utils.DockerAgentUtils$6.call(DockerAgentUtils.java:305)
at org.jfrog.hudson.pipeline.docker.utils.DockerAgentUtils$6.call(DockerAgentUtils.java:303)
at hudson.remoting.UserRequest.perform(UserRequest.java:212)
at hudson.remoting.UserRequest.perform(UserRequest.java:54)
at hudson.remoting.Request$2.run(Request.java:369)
at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:72)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at hudson.remoting.Engine$1.lambda$newThread$0(Engine.java:93)
Caused: java.io.IOException: Remote call on JNLP4-connect connection from IP1/IP2:PORT failed
at hudson.remoting.Channel.call(Channel.java:961)
at org.jfrog.hudson.pipeline.docker.utils.DockerAgentUtils.getImageIdFromAgent(DockerAgentUtils.java:303)
at org.jfrog.hudson.pipeline.steps.DockerPushStep$Execution.run(DockerPushStep.java:122)
at org.jfrog.hudson.pipeline.steps.DockerPushStep$Execution.run(DockerPushStep.java:86)
at org.jenkinsci.plugins.workflow.steps.AbstractSynchronousNonBlockingStepExecution$1$1.call(AbstractSynchronousNonBlockingStepExecution.java:47)
at hudson.security.ACL.impersonate(ACL.java:290)
at org.jenkinsci.plugins.workflow.steps.AbstractSynchronousNonBlockingStepExecution$1.run(AbstractSynchronousNonBlockingStepExecution.java:44)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Finished: FAILURE
A workaround is not to use the syntax provided in Jenkins and just wrap docker commands inside sh.
sh """
docker build . -t ${imageTag}
docker push ${imageTag}
"""
DataStax Enterprise Server 5.1.2
In a 5-node cluster one node crashed. DSE will not start on this node. Here is the error from system log:
ERROR [main] 2018-07-13 12:47:37,336 CassandraDaemon.java:705 - Exception encountered during startup
org.apache.cassandra.io.FSReadError: java.io.EOFException
at org.apache.cassandra.hints.HintsDescriptor.readFromFile(HintsDescriptor.java:235) ~[cassandra-all-3.11.0.1758.jar:3.11.0.1758]
at java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:193) ~[na:1.8.0_141]
at java.util.stream.ReferencePipeline$2$1.accept(ReferencePipeline.java:175) ~[na:1.8.0_141]
at java.util.Iterator.forEachRemaining(Iterator.java:116) ~[na:1.8.0_141]
at java.util.Spliterators$IteratorSpliterator.forEachRemaining(Spliterators.java:1801) ~[na:1.8.0_141]
at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:481) ~[na:1.8.0_141]
at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:471) ~[na:1.8.0_141]
at java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:708) ~[na:1.8.0_141]
at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234) ~[na:1.8.0_141]
at java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:499) ~[na:1.8.0_141]
at org.apache.cassandra.hints.HintsCatalog.load(HintsCatalog.java:65) ~[cassandra-all-3.11.0.1758.jar:3.11.0.1758]
at org.apache.cassandra.hints.HintsService.(HintsService.java:97) ~[cassandra-all-3.11.0.1758.jar:3.11.0.1758]
at org.apache.cassandra.hints.HintsService.(HintsService.java:88) ~[cassandra-all-3.11.0.1758.jar:3.11.0.1758]
at org.apache.cassandra.hints.HintsService.(HintsService.java:66) ~[cassandra-all-3.11.0.1758.jar:3.11.0.1758]
at org.apache.cassandra.service.StorageProxy.(StorageProxy.java:129) ~[cassandra-all-3.11.0.1758.jar:3.11.0.1758]
at java.lang.Class.forName0(Native Method) ~[na:1.8.0_141]
at java.lang.Class.forName(Class.java:264) ~[na:1.8.0_141]
at org.apache.cassandra.service.StorageService.initServer(StorageService.java:638) ~[cassandra-all-3.11.0.1758.jar:3.11.0.1758]
at org.apache.cassandra.service.StorageService.initServer(StorageService.java:623) ~[cassandra-all-3.11.0.1758.jar:3.11.0.1758]
at org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:393) ~[cassandra-all-3.11.0.1758.jar:3.11.0.1758]
at com.datastax.bdp.server.DseDaemon.setup(DseDaemon.java:465) ~[dse-core-5.1.2.jar:5.1.2]
at org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:599) ~[cassandra-all-3.11.0.1758.jar:3.11.0.1758]
at com.datastax.bdp.DseModule.main(DseModule.java:93) [dse-core-5.1.2.jar:5.1.2]
Caused by: java.io.EOFException: null
at java.io.RandomAccessFile.readInt(RandomAccessFile.java:803) ~[na:1.8.0_141]
at org.apache.cassandra.hints.HintsDescriptor.deserialize(HintsDescriptor.java:344) ~[cassandra-all-3.11.0.1758.jar:3.11.0.1758]
at org.apache.cassandra.hints.HintsDescriptor.readFromFile(HintsDescriptor.java:231) ~[cassandra-all-3.11.0.1758.jar:3.11.0.1758]
Can someone from datastax please advise? I have been unable to find any useful suggestions on the web. Thanks for your time.
RL
I located the offending hints' files by referring to datastax default directories page. In my case they were in /var/lib/cassandra/hints. There were two 0-byte files in that directory. I created a temp directory, moved them to that, and started dse with no problem.
For some unknown (to me) reason I had to reinstall all opscenter agents used in this 5-node cluster (opscenter 6.1.5) and repair took around 20 hours since this node had been offline for around 8 days.
I have installed Grails Elasticsearch plugin, but when I run a unit test just for creating a new instance of a domain class, I'm getting the following error in the console.
OS: ubuntu 14.04
Grails: 2.4.4
ElasticSearch: 1.5.0
elasticSearch.disableAutoIndex=true
elasticSearch.cluster.name = 'myInstanceName'
environments {
development {
grails.logging.jul.usebridge = true
/**
* Possible values : "local", "node", "dataNode", "transport"
* If set to null, "node" mode is used by default.
*/
elasticSearch.client.mode = 'local'
}
test{
elasticSearch.client.mode = 'local'
}
SEVERE: Unexpected problem transforming call sites
java.lang.IllegalStateException: Unexpected problem processing bytes for class
at org.springsource.loaded.ConstantPoolChecker2.readConstantPool(ConstantPoolChecker2.java:196)
at org.springsource.loaded.ConstantPoolChecker2.<init>(ConstantPoolChecker2.java:128)
at org.springsource.loaded.ConstantPoolChecker2.getReferences(ConstantPoolChecker2.java:102)
at org.springsource.loaded.MethodInvokerRewriter.rewrite(MethodInvokerRewriter.java:300)
at org.springsource.loaded.MethodInvokerRewriter.rewriteUsingCache(MethodInvokerRewriter.java:156)
at org.springsource.loaded.TypeRegistry.methodCallRewriteUseCacheIfAvailable(TypeRegistry.java:828)
at org.springsource.loaded.agent.SpringLoadedPreProcessor.preProcess(SpringLoadedPreProcessor.java:321)
at org.springsource.loaded.agent.ClassPreProcessorAgentAdapter.transform(ClassPreProcessorAgentAdapter.java:102)
at sun.instrument.TransformerManager.transform(TransformerManager.java:188)
at sun.instrument.InstrumentationImpl.transform(InstrumentationImpl.java:424)
at java.lang.ClassLoader.defineClass1(Native Method)
at java.lang.ClassLoader.defineClass(ClassLoader.java:800)
at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
at java.net.URLClassLoader.defineClass(URLClassLoader.java:449)
at java.net.URLClassLoader.access$100(URLClassLoader.java:71)
at java.net.URLClassLoader$1.run(URLClassLoader.java:361)
at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
at groovy.lang.GroovyClassLoader.loadClass(GroovyClassLoader.java:655)
at groovy.lang.GroovyClassLoader.loadClass(GroovyClassLoader.java:765)
I'm trying to register a rest api published with swagger into the wso2 Api Manager 1.8.
In the wso2 publisher I do "Import swagger definition"
It shows me all the apis under the "Resources", but if I do "save" I get the exception below in the log.
How can I fix it?
THANKS
Alberto
My env is:
Linux hqldvwsos1 2.6.32-504.el6.x86_64 #1 SMP Tue Sep 16 01:56:35 EDT 2014 x86_64 x86_64 x86_64 GNU/Linux
java version "1.7.0_75"
OpenJDK Runtime Environment (rhel-2.5.4.0.el6_6-x86_64 u75-b13)
OpenJDK 64-Bit Server VM (build 24.75-b04, mixed mode)
TID: [0] [AM] [2015-02-10 10:18:14,592] ERROR {org.wso2.carbon.apimgt.impl.UserAwareAPIProvider} - Error while retrieving Swagger Definition for SDRESTAPI-1.1.1 {org.wso2.carbon.apimgt.impl.UserAwareAPIProvider}
org.wso2.carbon.registry.core.exceptions.ResourceNotFoundException: Resource does not exist at path /_system/governance/apimgt/applicationdata/api-docs/SDRESTAPI-1.1.1-admin/1.2/default/budget-codes
at org.wso2.carbon.registry.core.jdbc.EmbeddedRegistry.get(EmbeddedRegistry.java:532)
at org.wso2.carbon.registry.core.caching.CacheBackedRegistry.getGhostResourceFromCache(CacheBackedRegistry.java:194)
at org.wso2.carbon.registry.core.caching.CacheBackedRegistry.get(CacheBackedRegistry.java:167)
at org.wso2.carbon.registry.core.session.UserRegistry.getInternal(UserRegistry.java:620)
at org.wso2.carbon.registry.core.session.UserRegistry.access$400(UserRegistry.java:60)
at org.wso2.carbon.registry.core.session.UserRegistry$5.run(UserRegistry.java:603)
at org.wso2.carbon.registry.core.session.UserRegistry$5.run(UserRegistry.java:600)
at java.security.AccessController.doPrivileged(Native Method)
at org.wso2.carbon.registry.core.session.UserRegistry.get(UserRegistry.java:600)
at org.wso2.carbon.apimgt.impl.APIProviderImpl.getSwagger12Definition(APIProviderImpl.java:2431)
at org.wso2.carbon.apimgt.impl.UserAwareAPIProvider.getSwagger12Definition(UserAwareAPIProvider.java:33)
at org.wso2.carbon.apimgt.hostobjects.APIProviderHostObject.jsFunction_getSwagger12Resource(APIProviderHostObject.java:755)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.mozilla.javascript.MemberBox.invoke(MemberBox.java:126)
at org.mozilla.javascript.FunctionObject.call(FunctionObject.java:386)
at org.mozilla.javascript.optimizer.OptRuntime.call1(OptRuntime.java:32)
at org.jaggeryjs.rhino.publisher.modules.api.c3._c_anonymous_4(/publisher/modules/api/add.jag:75)
at org.jaggeryjs.rhino.publisher.modules.api.c3.call(/publisher/modules/api/add.jag)
at org.mozilla.javascript.ScriptRuntime.applyOrCall(ScriptRuntime.java:2430)
at org.mozilla.javascript.BaseFunction.execIdCall(BaseFunction.java:269)
at org.mozilla.javascript.IdFunctionObject.call(IdFunctionObject.java:97)
at org.mozilla.javascript.optimizer.OptRuntime.call2(OptRuntime.java:42)
at org.jaggeryjs.rhino.publisher.modules.api.c0._c_anonymous_32(/publisher/modules/api/module.jag:104)
at org.jaggeryjs.rhino.publisher.modules.api.c0.call(/publisher/modules/api/module.jag)
at org.mozilla.javascript.optimizer.OptRuntime.call1(OptRuntime.java:32)
at org.jaggeryjs.rhino.publisher.site.blocks.item_design.ajax.c0._c_anonymous_2(/publisher/site/blocks/item-design/ajax/add.jag:209)
at org.jaggeryjs.rhino.publisher.site.blocks.item_design.ajax.c0.call(/publisher/site/blocks/item-design/ajax/add.jag)
at org.mozilla.javascript.optimizer.OptRuntime.call0(OptRuntime.java:23)
at org.jaggeryjs.rhino.publisher.site.blocks.item_design.ajax.c0._c_script_0(/publisher/site/blocks/item-design/ajax/add.jag:18)
at org.jaggeryjs.rhino.publisher.site.blocks.item_design.ajax.c0.call(/publisher/site/blocks/item-design/ajax/add.jag)
at org.mozilla.javascript.ContextFactory.doTopCall(ContextFactory.java:394)
at org.mozilla.javascript.ScriptRuntime.doTopCall(ScriptRuntime.java:3091)
at org.jaggeryjs.rhino.publisher.site.blocks.item_design.ajax.c0.call(/publisher/site/blocks/item-design/ajax/add.jag)
at org.jaggeryjs.rhino.publisher.site.blocks.item_design.ajax.c0.exec(/publisher/site/blocks/item-design/ajax/add.jag)
at org.jaggeryjs.scriptengine.engine.RhinoEngine.execScript(RhinoEngine.java:570)
at org.jaggeryjs.scriptengine.engine.RhinoEngine.exec(RhinoEngine.java:273)
at org.jaggeryjs.jaggery.core.manager.WebAppManager.execute(WebAppManager.java:432)
at org.jaggeryjs.jaggery.core.JaggeryServlet.doGet(JaggeryServlet.java:24)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:735)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:848)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:305)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)
at org.apache.catalina.core.ApplicationDispatcher.invoke(ApplicationDispatcher.java:749)
at org.apache.catalina.core.ApplicationDispatcher.processRequest(ApplicationDispatcher.java:487)
at org.apache.catalina.core.ApplicationDispatcher.doForward(ApplicationDispatcher.java:379)
at org.apache.catalina.core.ApplicationDispatcher.forward(ApplicationDispatcher.java:339)
at org.jaggeryjs.jaggery.core.JaggeryFilter.doFilter(JaggeryFilter.java:21)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:243)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)
at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:222)
at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:123)
at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:472)
at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:171)
at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:99)
at org.wso2.carbon.tomcat.ext.valves.CompositeValve.continueInvocation(CompositeValve.java:178)
at org.wso2.carbon.tomcat.ext.valves.CarbonTomcatValve$1.invoke(CarbonTomcatValve.java:47)
at org.wso2.carbon.webapp.mgt.TenantLazyLoaderValve.invoke(TenantLazyLoaderValve.java:56)
at org.wso2.carbon.tomcat.ext.valves.TomcatValveContainer.invokeValves(TomcatValveContainer.java:47)
at org.wso2.carbon.tomcat.ext.valves.CompositeValve.invoke(CompositeValve.java:141)
at org.wso2.carbon.tomcat.ext.valves.CarbonStuckThreadDetectionValve.invoke(CarbonStuckThreadDetectionValve.java:156)
at org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:936)
at org.wso2.carbon.tomcat.ext.valves.CarbonContextCreatorValve.invoke(CarbonContextCreatorValve.java:52)
at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:118)
at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:407)
at org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1004)
at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:589)
at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.run(NioEndpoint.java:1653)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
TID: [0] [AM] [2015-02-10 10:18:14,597] ERROR {JAGGERY.site.blocks.item-design.ajax.add:jag} - org.wso2.carbon.apimgt.api.APIManagementException: Error while retrieving Swagger Definition for SDRESTAPI-1.1.1 {JAGGERY.site.blocks.item-design.ajax.add:jag}
----- UPdate n1 ----
I found some inconsistency in the swagger definition.
For example I had
apis:
- path: /api/budget-codes
description: Budget Codes
file:
apiVersion: "1.0"
swaggerVersion: "1.2"
basePath: /fsd
resourcePath: /api/budget
Now I changed the first "budget-codes" into "budget" and I'm not getting the exception I documented. I can save it, but if I go to the implementation step or if I save and re-read the Design step, I get
TID: [0] [AM] [2015-02-10 11:15:19,655] ERROR {JAGGERY.site.blocks.item-design.ajax.add:jag} - java.lang.ClassCastException: [Ljava.lang.String; cannot be cast to [B {JAGGERY.site.blocks.item-design.ajax.add:jag}
Can you please help me?
Thanks
Alberto,
Did you edit your swagger definition after importing? If you made changes make sure authorization header is in the following format.
authorizations:
oauth2:
type: oauth2
scopes: []