error code :
Failed to close task manager due to the following error:
java.lang.NoClassDefFoundError: Could not initialize class org.rocksdb.DBOptions
at org.apache.kafka.streams.state.internals.RocksDBStore.openDB(RocksDBStore.java:133)
at org.apache.kafka.streams.state.internals.TimestampedSegment.openDB(TimestampedSegment.java:49)
at org.apache.kafka.streams.state.internals.TimestampedSegments.getOrCreateSegment(TimestampedSegments.java:50)
at org.apache.kafka.streams.state.internals.TimestampedSegments.getOrCreateSegment(TimestampedSegments.java:25)
at org.apache.kafka.streams.state.internals.AbstractSegments.getOrCreateSegmentIfLive(AbstractSegments.java:84)
at org.apache.kafka.streams.state.internals.AbstractRocksDBSegmentedBytesStore.put(AbstractRocksDBSegmentedBytesStore.java:146)
at org.apache.kafka.streams.state.internals.RocksDBWindowStore.put(RocksDBWindowStore.java:61)
at org.apache.kafka.streams.state.internals.RocksDBWindowStore.put(RocksDBWindowStore.java:27)
at org.apache.kafka.streams.state.internals.ChangeLoggingWindowBytesStore.put(ChangeLoggingWindowBytesStore.java:111)
at org.apache.kafka.streams.state.internals.ChangeLoggingWindowBytesStore.put(ChangeLoggingWindowBytesStore.java:34)
at org.apache.kafka.streams.state.internals.CachingWindowStore.putAndMaybeForward(CachingWindowStore.java:106)
at org.apache.kafka.streams.state.internals.CachingWindowStore.lambda$initInternal$0(CachingWindowStore.java:86)
at org.apache.kafka.streams.state.internals.NamedCache.flush(NamedCache.java:151)
at org.apache.kafka.streams.state.internals.NamedCache.flush(NamedCache.java:109)
at org.apache.kafka.streams.state.internals.ThreadCache.flush(ThreadCache.java:124)
at org.apache.kafka.streams.state.internals.CachingWindowStore.flush(CachingWindowStore.java:291)
at org.apache.kafka.streams.state.internals.WrappedStateStore.flush(WrappedStateStore.java:84)
at org.apache.kafka.streams.state.internals.MeteredWindowStore.lambda$flush$4(MeteredWindowStore.java:200)
at org.apache.kafka.streams.processor.internals.metrics.StreamsMetricsImpl.maybeMeasureLatency(StreamsMetricsImpl.java:801)
at org.apache.kafka.streams.state.internals.MeteredWindowStore.flush(MeteredWindowStore.java:200)
at org.apache.kafka.streams.processor.internals.ProcessorStateManager.flush(ProcessorStateManager.java:282)
at org.apache.kafka.streams.processor.internals.StreamTask.suspend(StreamTask.java:647)
at org.apache.kafka.streams.processor.internals.StreamTask.close(StreamTask.java:745)
at org.apache.kafka.streams.processor.internals.AssignedStreamsTasks.closeTask(AssignedStreamsTasks.java:81)
at org.apache.kafka.streams.processor.internals.AssignedStreamsTasks.closeTask(AssignedStreamsTasks.java:37)
at org.apache.kafka.streams.processor.internals.AssignedTasks.shutdown(AssignedTasks.java:256)
at org.apache.kafka.streams.processor.internals.AssignedStreamsTasks.shutdown(AssignedStreamsTasks.java:535)
at org.apache.kafka.streams.processor.internals.TaskManager.shutdown(TaskManager.java:292)
at org.apache.kafka.streams.processor.internals.StreamThread.completeShutdown(StreamThread.java:1133)
at org.apache.kafka.streams.processor.internals.StreamThread.run(StreamThread.java:682)
Local works fine.
In the environment where the error occurs, the error occurs when uploading it to docker.
kafka streams version : 2.5.0
Related
When running the following commands, an error is thrown by the add command. The rdf4j framework is used for communicating with a graphdb Knowledge Base:
import org.eclipse.rdf4j.model.Model;
Model model;
//statements have been added to the model
public RepositoryConnection connection;
...
connection.add(model); <-- error is thrown
An error is thrown in add command
2022-01-24 09:46:02 [http-nio-8080-exec-5] ERROR restapi.SubmitRMService - unable to rollback transaction. HTTP error code 404
org.eclipse.rdf4j.repository.RepositoryException: unable to rollback transaction. HTTP error code 404
at org.eclipse.rdf4j.http.client.RDF4JProtocolSession.rollbackTransaction(RDF4JProtocolSession.java:786)
at org.eclipse.rdf4j.repository.http.HTTPRepositoryConnection.rollback(HTTPRepositoryConnection.java:354)
at org.eclipse.rdf4j.repository.base.AbstractRepositoryConnection.conditionalRollback(AbstractRepositoryConnection.java:335)
at org.eclipse.rdf4j.repository.base.AbstractRepositoryConnection.add(AbstractRepositoryConnection.java:379)
The code is running successfully in two other environments except one we are setting with kubernetes. We ensured that /opt/graphdb/home has write permissions.
Question: I just do not understand the 404 RepositoryException error. The code can successfully run queries before the .add(model), so the connection with the graphdb is ok.
Getting the following error when I try to launch a Dataflow SQL job:
Failed to start the VM, launcher-____, used for launching because of status code: INVALID_ARGUMENT, reason: Error: Message: Invalid value for field 'resource.networkInterfaces[0].network': 'global/networks/default'. The referenced network resource cannot be found. HTTP Code: 400.
This issue just started today.
Adding the default network solved the issue.
I have a simple streaming Flink Scala job which connects to a Kafka topic and maps its
org.apache.avro.generic.GenericRecord messages and map into Json string.
When it is running in IntelliJ it ingests the topic well and printing out the jsons.
When I run it in docker-compose I got the following exception:
com.esotericsoftware.kryo.KryoException: Error constructing instance of class: org.apache.avro.Schema$LockableArrayList
Serialization trace:
types (org.apache.avro.Schema$UnionSchema)
schema (org.apache.avro.Schema$Field)
fieldMap (org.apache.avro.Schema$RecordSchema)
schema (org.apache.avro.generic.GenericData$Record)
at com.twitter.chill.Instantiators$$anon$1.newInstance(KryoBase.scala:136)
at com.esotericsoftware.kryo.Kryo.newInstance(Kryo.java:1061)
at com.esotericsoftware.kryo.serializers.CollectionSerializer.create(CollectionSerializer.java:89)
at com.esotericsoftware.kryo.serializers.CollectionSerializer.read(CollectionSerializer.java:93)
at com.esotericsoftware.kryo.serializers.CollectionSerializer.read(CollectionSerializer.java:22)
at com.esotericsoftware.kryo.Kryo.readObject(Kryo.java:679)
at com.esotericsoftware.kryo.serializers.ObjectField.read(ObjectField.java:106)
at com.esotericsoftware.kryo.serializers.FieldSerializer.read(FieldSerializer.java:528)
at com.esotericsoftware.kryo.Kryo.readObject(Kryo.java:679)
at com.esotericsoftware.kryo.serializers.ObjectField.read(ObjectField.java:106)
at com.esotericsoftware.kryo.serializers.FieldSerializer.read(FieldSerializer.java:528)
at com.esotericsoftware.kryo.Kryo.readClassAndObject(Kryo.java:761)
at com.esotericsoftware.kryo.serializers.MapSerializer.read(MapSerializer.java:143)
at com.esotericsoftware.kryo.serializers.MapSerializer.read(MapSerializer.java:21)
at com.esotericsoftware.kryo.Kryo.readObject(Kryo.java:679)
at com.esotericsoftware.kryo.serializers.ObjectField.read(ObjectField.java:106)
at com.esotericsoftware.kryo.serializers.FieldSerializer.read(FieldSerializer.java:528)
at com.esotericsoftware.kryo.Kryo.readObject(Kryo.java:679)
at com.esotericsoftware.kryo.serializers.ObjectField.read(ObjectField.java:106)
at com.esotericsoftware.kryo.serializers.FieldSerializer.read(FieldSerializer.java:528)
at com.esotericsoftware.kryo.Kryo.readObject(Kryo.java:657)
at org.apache.flink.api.java.typeutils.runtime.kryo.KryoSerializer.copy(KryoSerializer.java:262)
at org.apache.flink.api.java.typeutils.runtime.TupleSerializer.copy(TupleSerializer.java:111)
at org.apache.flink.api.java.typeutils.runtime.TupleSerializer.copy(TupleSerializer.java:37)
at org.apache.flink.streaming.runtime.tasks.OperatorChain$CopyingChainingOutput.pushToOperator(OperatorChain.java:635)
at org.apache.flink.streaming.runtime.tasks.OperatorChain$CopyingChainingOutput.collect(OperatorChain.java:612)
at org.apache.flink.streaming.runtime.tasks.OperatorChain$CopyingChainingOutput.collect(OperatorChain.java:592)
at org.apache.flink.streaming.api.operators.AbstractStreamOperator$CountingOutput.collect(AbstractStreamOperator.java:727)
at org.apache.flink.streaming.api.operators.AbstractStreamOperator$CountingOutput.collect(AbstractStreamOperator.java:705)
at org.apache.flink.streaming.api.operators.StreamSourceContexts$NonTimestampContext.collect(StreamSourceContexts.java:104)
at org.apache.flink.streaming.api.operators.StreamSourceContexts$NonTimestampContext.collectWithTimestamp(StreamSourceContexts.java:111)
at org.apache.flink.streaming.connectors.kafka.internals.AbstractFetcher.emitRecordWithTimestamp(AbstractFetcher.java:398)
at org.apache.flink.streaming.connectors.kafka.internal.KafkaFetcher.emitRecord(KafkaFetcher.java:185)
at org.apache.flink.streaming.connectors.kafka.internal.KafkaFetcher.runFetchLoop(KafkaFetcher.java:150)
at org.apache.flink.streaming.connectors.kafka.FlinkKafkaConsumerBase.run(FlinkKafkaConsumerBase.java:715)
at org.apache.flink.streaming.api.operators.StreamSource.run(StreamSource.java:100)
at org.apache.flink.streaming.api.operators.StreamSource.run(StreamSource.java:63)
at org.apache.flink.streaming.runtime.tasks.SourceStreamTask$LegacySourceFunctionThread.run(SourceStreamTask.java:208)
Caused by: java.lang.IllegalAccessException: Class com.twitter.chill.Instantiators$ can not access a member of class org.apache.avro.Schema$LockableArrayList with modifiers "public"
at sun.reflect.Reflection.ensureMemberAccess(Reflection.java:102)
at java.lang.reflect.AccessibleObject.slowCheckMemberAccess(AccessibleObject.java:296)
at java.lang.reflect.AccessibleObject.checkAccess(AccessibleObject.java:288)
at java.lang.reflect.Constructor.newInstance(Constructor.java:413)
at com.twitter.chill.Instantiators$.$anonfun$normalJava$1(KryoBase.scala:170)
at com.twitter.chill.Instantiators$$anon$1.newInstance(KryoBase.scala:133)
... 37 more
I tried forcing Avro serialization with:
val env: StreamExecutionEnvironment = StreamExecutionEnvironment.getExecutionEnvironment
env.getConfig.disableForceKryo()
env.getConfig.enableForceAvro()
but got the same error.
Based on [this][1] I'm using all Flink related dependencies as "provided" with no good result.
What can be the difference between running the job in the IDE and in
Docker?
How can I fix the job to be able to read the Kafka topic from
Docker?
How shall I setup Docker for this?
What can I handle Kryo/Avro serialization issue?
SS
[1]: http://www.alternatestack.com/development/com-esotericsoftware-kryo-kryoexception-unusual-solution-upgrading-flink/
While running jython py script with adminApp.install to install on Docker WAS, I'm getting the Error:
WASX7017E: Exception received while running file
"deployWebSphere_with_install_standAlone.py"; exception information:
com.ibm.websphere.management.filetransfer.client.TransferFailedException:
401 Unauthorized (for:
C:\Users\devopsjava\AppData\Local\Temp\app5674632847255103189.ear).
I tried give full control on this path,
I tried set custom property:com.ibm.websphere.management.filetransfer.serverBasicAuth=true
(https://www-01.ibm.com/support/docview.wss?uid=swg1PK71800)
My code:
options = ['-node ',node,'-cell ',cell,"-server ",server]
print AdminApp.install(packageFile, options)
I always get :
WASX7017E: Exception received while running file
"deployWebSphere_with_install_standAlone.py"; exception information:
com.ibm.websphere.management.filetransfer.client.TransferFailedException:
401 Unauthorized (for:
C:\Users\devopsjava\AppData\Local\Temp\app5674632847255103189.ear).
Maybe, I need to set some more property, or there is problem with security between docker domain and the outside world ?
I am trying to work on grails through command prompt.
I am using the below versions:
| Grails Version: 3.2.3
| Groovy Version: 2.4.7
| JVM Version: 1.8.0_112
The following error is thrown:
| Error Error occurred running Grails CLI: This is usually a temporary
error dur ing hostname resolution and means that the local server did
not receive a respon se from an authoritative server (repo.grails.org)
(Use --stacktrace to see the f ull trace)
Upon using stacktrace the following is displayed:
| Error Error occurred running Grails CLI: This is usually a temporary
error dur ing hostname resolution and means that the local server did
not receive a respon se from an authoritative server (repo.grails.org)
(NOTE: Stack trace has been fi ltered. Use --verbose to see entire
trace.)
java.net.UnknownHostException: This is usually a temporary error
during hostname resolution and means that the local server did not
receive a response from an a uthoritative server (repo.grails.org)
at org.apache.http.impl.conn.SystemDefaultDnsResolver.resolve(SystemDefa
ultDnsResolver.java:45)
at org.apache.http.impl.conn.DefaultClientConnectionOperator.resolveHost
name(DefaultClientConnectionOperator.java:278)
at org.apache.http.impl.conn.DefaultClientConnectionOperator.openConnect
ion(DefaultClientConnectionOperator.java:162)
at org.apache.http.impl.conn.ManagedClientConnectionImpl.open(ManagedCli
entConnectionImpl.java:294)
at org.apache.http.impl.client.DefaultRequestDirector.tryConnect(Default
RequestDirector.java:643)
at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultReq
uestDirector.java:479)
at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpCl
ient.java:906)
at org.apache.http.impl.client.DecompressingHttpClient.execute(Decompres
singHttpClient.java:137)
at org.eclipse.aether.transport.http.HttpTransporter.execute(HttpTranspo
rter.java:287)
at org.eclipse.aether.transport.http.HttpTransporter.implGet(HttpTranspo
rter.java:243)
at org.eclipse.aether.spi.connector.transport.AbstractTransporter.get(Ab
stractTransporter.java:59)
at org.eclipse.aether.connector.basic.BasicRepositoryConnector$GetTaskRu
nner.runTask(BasicRepositoryConnector.java:447)
at org.eclipse.aether.connector.basic.BasicRepositoryConnector$TaskRunne
r.run(BasicRepositoryConnector.java:350)
at org.eclipse.aether.util.concurrency.RunnableErrorForwarder$1.run(Runn
ableErrorForwarder.java:67)
at org.eclipse.aether.connector.basic.BasicRepositoryConnector$DirectExe
cutor.execute(BasicRepositoryConnector.java:581)
at org.eclipse.aether.connector.basic.BasicRepositoryConnector.get(Basic
RepositoryConnector.java:249)
at org.eclipse.aether.internal.impl.DefaultArtifactResolver.performDownl
oads(DefaultArtifactResolver.java:520)
at org.eclipse.aether.internal.impl.DefaultArtifactResolver.resolve(Defa
ultArtifactResolver.java:421)
at org.eclipse.aether.internal.impl.DefaultArtifactResolver.resolveArtif
acts(DefaultArtifactResolver.java:246)
at org.eclipse.aether.internal.impl.DefaultArtifactResolver.resolveArtif
act(DefaultArtifactResolver.java:223)
at org.apache.maven.repository.internal.DefaultArtifactDescriptorReader.
loadPom(DefaultArtifactDescriptorReader.java:320)
at org.apache.maven.repository.internal.DefaultArtifactDescriptorReader.
readArtifactDescriptor(DefaultArtifactDescriptorReader.java:217)
at org.eclipse.aether.internal.impl.DefaultDependencyCollector.resolveCa
chedArtifactDescriptor(DefaultDependencyCollector.java:535)
at org.eclipse.aether.internal.impl.DefaultDependencyCollector.getArtifa
ctDescriptorResult(DefaultDependencyCollector.java:519)
at org.eclipse.aether.internal.impl.DefaultDependencyCollector.processDe
pendency(DefaultDependencyCollector.java:409)
at org.eclipse.aether.internal.impl.DefaultDependencyCollector.processDe
pendency(DefaultDependencyCollector.java:363)
at org.eclipse.aether.internal.impl.DefaultDependencyCollector.process(D
efaultDependencyCollector.java:351)
at org.eclipse.aether.internal.impl.DefaultDependencyCollector.collectDe
pendencies(DefaultDependencyCollector.java:254)
at org.eclipse.aether.internal.impl.DefaultRepositorySystem.resolveDepen
dencies(DefaultRepositorySystem.java:341)
at org.springframework.boot.cli.compiler.grape.AetherGrapeEngine.resolve
(AetherGrapeEngine.java:302)
at org.springframework.boot.cli.compiler.grape.AetherGrapeEngine.resolve
(AetherGrapeEngine.java:284)
at org.springframework.boot.cli.compiler.grape.AetherGrapeEngine.resolve
(AetherGrapeEngine.java:276)
at org.grails.cli.boot.GrailsDependencyVersions.<init>(GrailsDependencyV
ersions.groovy:53)
at org.grails.cli.boot.GrailsDependencyVersions.<init>(GrailsDependencyV
ersions.groovy:49)
at org.grails.cli.profile.repository.MavenProfileRepository.<init>(Maven
ProfileRepository.groovy:53)
at org.grails.cli.GrailsCli.createMavenProfileRepository(GrailsCli.groov
y:333)
at org.grails.cli.GrailsCli.execute(GrailsCli.groovy:234)
at org.grails.cli.GrailsCli.main(GrailsCli.groovy:159)
| Error Error occurred running Grails CLI: This is usually a temporary
error dur ing hostname resolution and means that the local server did
not receive a respon se from an authoritative server (repo.grails.org)
Kindly help me resolve the above issue.
Most likely reason is your internet is not working or your are behind a proxy. See http://docs.grails.org/latest/guide/conf.html#proxyConfig for how to configure your proxy settings