I have the following SQL Migration Skript, but it fails when I try to insert the Varchar 'a n'. Upper or lower case is not important, the only important part appears to be that the ' n' are the last two characters. It works fine when executed manually with Toad and sqlplus.
The default flyway placeholder settings are unchanged.
Flyway Version: 2.3 and executed with Ant
Oracle Database 12.1.02.0
CREATE TABLE TEST(
TEST VARCHAR2(40) NOT NULL
);
-- This works
INSERT INTO TEST (TEST) VALUES ( 'a b');
INSERT INTO TEST (TEST) VALUES ( 'a an');
INSERT INTO TEST (TEST) VALUES ( 'a nn');
INSERT INTO TEST (TEST) VALUES ( 'a n ');
-- This does not
INSERT INTO TEST (TEST) VALUES ( 'a n');
The complete stacktrace:
build.xml:38: Flyway Error: com.googlecode.flyway.core.command.FlywaySqlScriptException: Error executing statement at line 12: INSERT INTO TEST (TEST) VALUES ( ' n');
at com.googlecode.flyway.ant.AbstractFlywayTask.execute(AbstractFlywayTask.java:331)
at org.apache.tools.ant.UnknownElement.execute(UnknownElement.java:291)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
at java.lang.reflect.Method.invoke(Unknown Source)
at org.apache.tools.ant.dispatch.DispatchUtils.execute(DispatchUtils.java:106)
at org.apache.tools.ant.Task.perform(Task.java:348)
at org.apache.tools.ant.Target.execute(Target.java:392)
at org.apache.tools.ant.Target.performTasks(Target.java:413)
at org.apache.tools.ant.Project.executeSortedTargets(Project.java:1399)
at org.apache.tools.ant.Project.executeTarget(Project.java:1368)
at org.apache.tools.ant.helper.DefaultExecutor.executeTargets(DefaultExecutor.java:41)
at org.eclipse.ant.internal.launching.remote.EclipseDefaultExecutor.executeTargets(EclipseDefaultExecutor.java:32)
at org.apache.tools.ant.Project.executeTargets(Project.java:1251)
at org.eclipse.ant.internal.launching.remote.InternalAntRunner.run(InternalAntRunner.java:424)
at org.eclipse.ant.internal.launching.remote.InternalAntRunner.main(InternalAntRunner.java:138)
Caused by: java.sql.SQLSyntaxErrorException: ORA-00933: SQL command not properly ended
at oracle.jdbc.driver.T4CTTIoer.processError(T4CTTIoer.java:445)
at oracle.jdbc.driver.T4CTTIoer.processError(T4CTTIoer.java:396)
at oracle.jdbc.driver.T4C8Oall.processError(T4C8Oall.java:879)
at oracle.jdbc.driver.T4CTTIfun.receive(T4CTTIfun.java:450)
at oracle.jdbc.driver.T4CTTIfun.doRPC(T4CTTIfun.java:192)
at oracle.jdbc.driver.T4C8Oall.doOALL(T4C8Oall.java:531)
at oracle.jdbc.driver.T4CStatement.doOall8(T4CStatement.java:193)
at oracle.jdbc.driver.T4CStatement.executeForRows(T4CStatement.java:1033)
at oracle.jdbc.driver.OracleStatement.doExecuteWithTimeout(OracleStatement.java:1329)
at oracle.jdbc.driver.OracleStatement.executeInternal(OracleStatement.java:1909)
at oracle.jdbc.driver.OracleStatement.execute(OracleStatement.java:1871)
at oracle.jdbc.driver.OracleStatementWrapper.execute(OracleStatementWrapper.java:318)
at com.googlecode.flyway.core.dbsupport.JdbcTemplate.executeStatement(JdbcTemplate.java:230)
at com.googlecode.flyway.core.dbsupport.SqlScript.execute(SqlScript.java:90)
at com.googlecode.flyway.core.resolver.sql.SqlMigrationExecutor.execute(SqlMigrationExecutor.java:72)
at com.googlecode.flyway.core.command.DbMigrate$2.doInTransaction(DbMigrate.java:243)
at com.googlecode.flyway.core.command.DbMigrate$2.doInTransaction(DbMigrate.java:241)
at com.googlecode.flyway.core.util.jdbc.TransactionTemplate.execute(TransactionTemplate.java:72)
at com.googlecode.flyway.core.command.DbMigrate.applyMigration(DbMigrate.java:241)
at com.googlecode.flyway.core.command.DbMigrate.access$700(DbMigrate.java:44)
at com.googlecode.flyway.core.command.DbMigrate$1.doInTransaction(DbMigrate.java:186)
at com.googlecode.flyway.core.command.DbMigrate$1.doInTransaction(DbMigrate.java:135)
at com.googlecode.flyway.core.util.jdbc.TransactionTemplate.execute(TransactionTemplate.java:72)
at com.googlecode.flyway.core.command.DbMigrate.migrate(DbMigrate.java:135)
at com.googlecode.flyway.core.Flyway$1.execute(Flyway.java:872)
at com.googlecode.flyway.core.Flyway$1.execute(Flyway.java:819)
at com.googlecode.flyway.core.Flyway.execute(Flyway.java:1200)
at com.googlecode.flyway.core.Flyway.migrate(Flyway.java:819)
at com.googlecode.flyway.ant.MigrateTask.doExecuteWithMigrationConfig(MigrateTask.java:162)
at com.googlecode.flyway.ant.AbstractMigrationLoadingTask.doExecute(AbstractMigrationLoadingTask.java:271)
at com.googlecode.flyway.ant.AbstractFlywayTask.execute(AbstractFlywayTask.java:329)
... 16 more
This has been fixed as part of 3.0
Related
Deployed NEO4j on a hardened RHEL7 using ansible.
No constraint can be created even though the DB is empty. Any idea on how to debug this or what could be the cause? Other objects can be created. It seems to be a problem with some core library...
We have the same install on another instance which works fine, so I suppose it must be something related to the deployment...even though there were not install errors
Thanks.
Error logs below:
020-07-09 12:11:13.898+0000 ERROR [o.n.b.v.r.ErrorReporter] Client triggered an unexpected error [Neo.DatabaseError.Schema.ConstraintCreationFailed]: Unable to create CONSTRAINT ON ( equipment:Equipment ) ASSERT equipment.UUID IS UNIQUE:
, reference 0a5adcde-dbeb-4ef0-b9aa-9bf32ef58e63.
2020-07-09 12:11:13.898+0000 ERROR [o.n.b.v.r.ErrorReporter] Client triggered an unexpected error [Neo.DatabaseError.Schema.ConstraintCreationFailed]: Unable to create CONSTRAINT ON ( equipment:Equipment ) ASSERT equipment.UUID IS UNIQUE:
, reference 0a5adcde-dbeb-4ef0-b9aa-9bf32ef58e63. Unable to create CONSTRAINT ON ( equipment:Equipment ) ASSERT equipment.UUID IS UNIQUE:
org.neo4j.kernel.impl.query.QueryExecutionKernelException: Unable to create CONSTRAINT ON ( equipment:Equipment ) ASSERT equipment.UUID IS UNIQUE:
at org.neo4j.bolt.v1.runtime.TransactionStateMachineSPI$1.start(TransactionStateMachineSPI.java:158)
at org.neo4j.bolt.v1.runtime.TransactionStateMachine$State.startExecution(TransactionStateMachine.java:444)
at org.neo4j.bolt.v1.runtime.TransactionStateMachine$State$1.execute(TransactionStateMachine.java:259)
at org.neo4j.bolt.v1.runtime.TransactionStateMachine$State$1.run(TransactionStateMachine.java:240)
at org.neo4j.bolt.v1.runtime.TransactionStateMachine.run(TransactionStateMachine.java:81)
at org.neo4j.bolt.v1.runtime.BoltStateMachine$State$2.run(BoltStateMachine.java:457)
at org.neo4j.bolt.v1.runtime.BoltStateMachine.run(BoltStateMachine.java:225)
at org.neo4j.bolt.v1.messaging.BoltMessageRouter.lambda$onRun$3(BoltMessageRouter.java:93)
at org.neo4j.bolt.runtime.MetricsReportingBoltConnection.lambda$enqueue$0(MetricsReportingBoltConnection.java:69)
at org.neo4j.bolt.runtime.DefaultBoltConnection.processNextBatch(DefaultBoltConnection.java:195)
at org.neo4j.bolt.runtime.MetricsReportingBoltConnection.processNextBatch(MetricsReportingBoltConnection.java:87)
at org.neo4j.bolt.runtime.DefaultBoltConnection.processNextBatch(DefaultBoltConnection.java:143)
at org.neo4j.bolt.runtime.ExecutorBoltScheduler.executeBatch(ExecutorBoltScheduler.java:170)
at org.neo4j.bolt.runtime.ExecutorBoltScheduler.lambda$scheduleBatchOrHandleError$2(ExecutorBoltScheduler.java:153)
at java.util.concurrent.CompletableFuture$AsyncSupply.run(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
at java.lang.Thread.run(Unknown Source)
Caused by: org.neo4j.kernel.impl.query.QueryExecutionKernelException: Unable to create CONSTRAINT ON ( equipment:Equipment ) ASSERT equipment.UUID IS UNIQUE:
at org.neo4j.cypher.internal.javacompat.ExecutionEngine.executeQuery(ExecutionEngine.java:65)
at org.neo4j.bolt.v1.runtime.TransactionStateMachineSPI$1.start(TransactionStateMachineSPI.java:144)
... 17 more
Caused by: org.neo4j.cypher.CypherExecutionException: Unable to create CONSTRAINT ON ( equipment:Equipment ) ASSERT equipment.UUID IS UNIQUE:
at org.neo4j.cypher.internal.compatibility.v3_4.ExceptionTranslationSupport$class.translateException(ExceptionTranslationSupport.scala:35)
at org.neo4j.cypher.internal.compatibility.v3_4.ExceptionTranslatingQueryContext.translateException(ExceptionTranslatingQueryContext.scala:41)
at org.neo4j.cypher.internal.compatibility.v3_4.ExceptionTranslatingQueryContext.createUniqueConstraint(ExceptionTranslatingQueryContext.scala:128)
at org.neo4j.cypher.internal.runtime.interpreted.UpdateCountingQueryContext.createUniqueConstraint(UpdateCountingQueryContext.scala:125)
at org.neo4j.cypher.internal.compatibility.v3_4.runtime.executionplan.procs.ProcedureCallOrSchemaCommandExecutionPlanBuilder$$anonfun$6.apply(ProcedureCallOrSchemaCommandExecutionPlanBuilder.scala:80)
at org.neo4j.cypher.internal.compatibility.v3_4.runtime.executionplan.procs.ProcedureCallOrSchemaCommandExecutionPlanBuilder$$anonfun$6.apply(ProcedureCallOrSchemaCommandExecutionPlanBuilder.scala:78)
at org.neo4j.cypher.internal.compatibility.v3_4.runtime.executionplan.procs.PureSideEffectExecutionPlan.run(PureSideEffectExecutionPlan.scala:55)
at org.neo4j.cypher.internal.compatibility.LatestRuntimeVariablePlannerCompatibility$ExecutionPlanWrapper$$anonfun$run$1.apply(LatestRuntimeVariablePlannerCompatibility.scala:128)
at org.neo4j.cypher.internal.compatibility.LatestRuntimeVariablePlannerCompatibility$ExecutionPlanWrapper$$anonfun$run$1.apply(LatestRuntimeVariablePlannerCompatibility.scala:124)
at org.neo4j.cypher.exceptionHandler$runSafely$.apply(exceptionHandler.scala:89)
at org.neo4j.cypher.internal.compatibility.LatestRuntimeVariablePlannerCompatibility$ExecutionPlanWrapper.run(LatestRuntimeVariablePlannerCompatibility.scala:124)
at org.neo4j.cypher.internal.PreparedPlanExecution.execute(PreparedPlanExecution.scala:29)
at org.neo4j.cypher.internal.ExecutionEngine.execute(ExecutionEngine.scala:119)
at org.neo4j.cypher.internal.javacompat.ExecutionEngine.executeQuery(ExecutionEngine.java:61)
... 18 more
Caused by: org.neo4j.kernel.api.exceptions.schema.CreateConstraintFailureException: Unable to create constraint org.neo4j.kernel.api.schema.constaints.UniquenessConstraintDescriptor#6: Existing data does not satisfy CONSTRAINT ON ( label[1]:label[1] ) ASSERT label[1].property[0] IS UNIQUE: Failed to populate index for :Equipment(UUID) [provider: {key=lucene+native, version=2.0}] [labelId: 1, properties [0]]
at org.neo4j.kernel.impl.newapi.Operations.indexBackedConstraintCreate(Operations.java:1239)
at org.neo4j.kernel.impl.newapi.Operations.uniquePropertyConstraintCreate(Operations.java:943)
at org.neo4j.internal.kernel.api.SchemaWrite.uniquePropertyConstraintCreate(SchemaWrite.java:66)
at org.neo4j.cypher.internal.runtime.interpreted.TransactionBoundQueryContext.createUniqueConstraint(TransactionBoundQueryContext.scala:670)
at org.neo4j.cypher.internal.compatibility.v3_4.ExceptionTranslatingQueryContext$$anonfun$createUniqueConstraint$1.apply$mcZ$sp(ExceptionTranslatingQueryContext.scala:128)
at org.neo4j.cypher.internal.compatibility.v3_4.ExceptionTranslatingQueryContext$$anonfun$createUniqueConstraint$1.apply(ExceptionTranslatingQueryContext.scala:128)
at org.neo4j.cypher.internal.compatibility.v3_4.ExceptionTranslatingQueryContext$$anonfun$createUniqueConstraint$1.apply(ExceptionTranslatingQueryContext.scala:128)
at org.neo4j.cypher.internal.compatibility.v3_4.ExceptionTranslationSupport$class.translateException(ExceptionTranslationSupport.scala:33)
... 31 more
Caused by: org.neo4j.kernel.api.exceptions.schema.UniquePropertyValueValidationException: Existing data does not satisfy CONSTRAINT ON ( label[1]:label[1] ) ASSERT label[1].property[0] IS UNIQUE: Failed to populate index for :Equipment(UUID) [provider: {key=lucene+native, version=2.0}] [labelId: 1, properties [0]]
at org.neo4j.kernel.impl.api.state.ConstraintIndexCreator.awaitConstrainIndexPopulation(ConstraintIndexCreator.java:244)
at org.neo4j.kernel.impl.api.state.ConstraintIndexCreator.createUniquenessConstraintIndex(ConstraintIndexCreator.java:135)
at org.neo4j.kernel.impl.newapi.Operations.indexBackedConstraintCreate(Operations.java:1224)
... 38 more
Caused by: org.neo4j.kernel.api.exceptions.index.IndexPopulationFailedKernelException: Failed to populate index for :Equipment(UUID) [provider: {key=lucene+native, version=2.0}] [labelId: 1, properties [0]]
at org.neo4j.kernel.impl.api.index.IndexPopulationFailure$1.asIndexPopulationFailure(IndexPopulationFailure.java:47)
at org.neo4j.kernel.impl.api.index.FailedIndexProxy.failureCause(FailedIndexProxy.java:80)
at org.neo4j.kernel.impl.api.index.FailedIndexProxy.awaitStoreScanCompleted(FailedIndexProxy.java:75)
at org.neo4j.kernel.impl.api.index.FlippableIndexProxy.awaitStoreScanCompleted(FlippableIndexProxy.java:325)
at org.neo4j.kernel.impl.api.index.AbstractDelegatingIndexProxy.awaitStoreScanCompleted(AbstractDelegatingIndexProxy.java:122)
at org.neo4j.kernel.impl.api.state.ConstraintIndexCreator.awaitConstrainIndexPopulation(ConstraintIndexCreator.java:232)
... 40 more
Caused by: java.lang.NoSuchMethodError: org.apache.lucene.codecs.blocktree.BlockTreeTermsWriter.validateAutoPrefixSettings(II)V
at org.apache.lucene.codecs.autoprefix.AutoPrefixPostingsFormat.(AutoPrefixPostingsFormat.java:77)
at org.apache.lucene.codecs.autoprefix.AutoPrefixPostingsFormat.(AutoPrefixPostingsFormat.java:52)
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(Unknown Source)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(Unknown Source)
at java.lang.reflect.Constructor.newInstance(Unknown Source)
at java.lang.Class.newInstance(Unknown Source)
at org.apache.lucene.util.NamedSPILoader.reload(NamedSPILoader.java:72)
at org.apache.lucene.util.NamedSPILoader.(NamedSPILoader.java:51)
at org.apache.lucene.util.NamedSPILoader.(NamedSPILoader.java:38)
at org.apache.lucene.codecs.PostingsFormat$Holder.(PostingsFormat.java:49)
at org.apache.lucene.codecs.PostingsFormat.forName(PostingsFormat.java:112)
at org.apache.lucene.codecs.lucene62.Lucene62Codec.(Lucene62Codec.java:167)
at org.apache.lucene.codecs.lucene62.Lucene62Codec.(Lucene62Codec.java:82)
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(Unknown Source)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(Unknown Source)
at java.lang.reflect.Constructor.newInstance(Unknown Source)
at java.lang.Class.newInstance(Unknown Source)
at org.apache.lucene.util.NamedSPILoader.reload(NamedSPILoader.java:72)
at org.apache.lucene.util.NamedSPILoader.(NamedSPILoader.java:51)
at org.apache.lucene.util.NamedSPILoader.(NamedSPILoader.java:38)
at org.apache.lucene.codecs.Codec$Holder.(Codec.java:47)
at org.apache.lucene.codecs.Codec.getDefault(Codec.java:143)
at org.apache.lucene.index.LiveIndexWriterConfig.(LiveIndexWriterConfig.java:121)
at org.apache.lucene.index.IndexWriterConfig.(IndexWriterConfig.java:151)
at org.neo4j.kernel.api.impl.index.IndexWriterConfigs.standard(IndexWriterConfigs.java:81)
at org.neo4j.kernel.api.impl.index.IndexWriterConfigs.population(IndexWriterConfigs.java:119)
at org.neo4j.kernel.api.impl.index.IndexWriterConfigs.population(IndexWriterConfigs.java:114)
at org.neo4j.kernel.api.impl.index.partition.WritableIndexPartitionFactory.createPartition(WritableIndexPartitionFactory.java:45)
at org.neo4j.kernel.api.impl.index.AbstractLuceneIndex.open(AbstractLuceneIndex.java:103)
at org.neo4j.kernel.api.impl.index.WritableAbstractDatabaseIndex.open(WritableAbstractDatabaseIndex.java:62)
at org.neo4j.kernel.api.impl.schema.populator.LuceneIndexPopulator.create(LuceneIndexPopulator.java:51)
at org.neo4j.kernel.impl.index.schema.fusion.InstanceSelector.consume(InstanceSelector.java:147)
at org.neo4j.kernel.impl.index.schema.fusion.InstanceSelector.forAll(InstanceSelector.java:90)
at org.neo4j.kernel.impl.index.schema.fusion.FusionIndexPopulator.create(FusionIndexPopulator.java:56)
at org.neo4j.kernel.impl.api.index.MultipleIndexPopulator$IndexPopulation.create(MultipleIndexPopulator.java:525)
at org.neo4j.kernel.impl.api.index.MultipleIndexPopulator.lambda$create$0(MultipleIndexPopulator.java:144)
at org.neo4j.kernel.impl.api.index.MultipleIndexPopulator.forEachPopulation(MultipleIndexPopulator.java:408)
at org.neo4j.kernel.impl.api.index.MultipleIndexPopulator.create(MultipleIndexPopulator.java:141)
at org.neo4j.kernel.impl.api.index.IndexPopulationJob.run(IndexPopulationJob.java:103)
at org.neo4j.kernel.impl.api.index.IndexPopulationJobController$IndexPopulationJobWrapper.run(IndexPopulationJobController.java:82)
at org.neo4j.kernel.impl.scheduler.ThreadPool.lambda$submit$0(ThreadPool.java:58)
at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
at java.util.concurrent.FutureTask.run(Unknown Source)
... 3 more
It would seem the database might not be empty due to this error:
Existing data does not satisfy CONSTRAINT ON ( label[1]:label[1] )
ASSERT label[1].property[0] IS UNIQUE
It complains that the constraint cannot be created as the existing data does not satisfy the constraint.
You can find the culprit with this query:
MATCH ( e:Equipment )
WITH e.UUID as id, count(*) as count
WHERE count > 1
RETURN id;
In the end, it was a problem with the deployment. A newer lucene library was deployed, one that did not contain that method.
Try to use BeamSQL for unnest the nested type of PCollection. Lets assume the PCollection which have the Employees and its details. Here details are in nested collection. So if we use the BeamSQL like "SELECT PCOLLECTION.details FROM PCOLLECTION" then getting nested type of details as array collection in the separate PCollection. However when I want to get specific column from the nested type collection as details, then getting error like unable to find the column name. Tried the BeamSQL like (similar like BigQuery SQL) "SELECT X.address FROM PCOLLECTION, Unnest(details) as X" then getting nullpointer exception. Used 2.12.0 apache beam version.
Appreciate some one please help on this.
Below is the sample data of details nested Value (details has email, phone columns. so per row, 'n' no of list of details. Here it has two list of details):
WARNING: printValue:Row:[[Row:[lourdurajan#gmail.com, 9840618047], Row:[lourdurajan#sanmina.com, 9840618047]]]
Here is the Java stacktrace for second select statement:
SELECT `X`.`email`
FROM `beam`.`PCOLLECTION` AS `PCOLLECTION`,
UNNEST(`PCOLLECTION`.`details`) AS `X`
May 08, 2019 11:23:30 AM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(email=[$3])
LogicalCorrelate(correlation=[$cor0], joinType=[inner], requiredColumns=[{2}])
BeamIOSourceRel(table=[[beam, PCOLLECTION]])
Uncollect
LogicalProject(details=[$cor0.details_2])
LogicalValues(tuples=[[{ 0 }]])
May 08, 2019 11:23:30 AM org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..4=[{inputs}], email=[$t3])
BeamUnnestRel(unnestIndex=[2])
BeamIOSourceRel(table=[[beam, PCOLLECTION]])
[WARNING]
java.lang.NullPointerException
at org.apache.beam.sdk.extensions.sql.impl.utils.CalciteUtils.toSchema(CalciteUtils.java:171)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamUnnestRel$Transform.expand(BeamUnnestRel.java:93)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamUnnestRel$Transform.expand(BeamUnnestRel.java:87)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:537)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:488)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:66)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.lambda$buildPCollectionList$0(BeamSqlRelUtils.java:47)
at java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:193)
at java.util.Iterator.forEachRemaining(Iterator.java:116)
at java.util.Spliterators$IteratorSpliterator.forEachRemaining(Spliterators.java:1801)
at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:481)
at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:471)
at java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:708)
at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
at java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:499)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.buildPCollectionList(BeamSqlRelUtils.java:48)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:64)
at org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:36)
at org.apache.beam.sdk.extensions.sql.SqlTransform.expand(SqlTransform.java:111)
at org.apache.beam.sdk.extensions.sql.SqlTransform.expand(SqlTransform.java:79)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:537)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:488)
at org.apache.beam.sdk.values.PCollection.apply(PCollection.java:370)
at com.sanmina.BeamSQLUnnest.main(BeamSQLUnnest.java:217)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.codehaus.mojo.exec.ExecJavaMojo$1.run(ExecJavaMojo.java:282)
at java.lang.Thread.run(Thread.java:748)
You can achieve this using BigQueryIO.
String Query ="SELECT `X`.`email`
FROM `beam`.`PCOLLECTION` AS `PCOLLECTION`,
UNNEST(`PCOLLECTION`.`details`) AS `X`"
BigQueryIO.readTableRows().fromQuery(query).usingStandardSql()
My Presto plugin has two components:
An EventListener that logs stats of queries made to Presto
Some UDFs for hashing (MD5, SHA1) common datatypes (varchar, long) and returning result as varchar
I've already done:
Integration-test (mvn verify) of query-logging component
Unit-test of hashing UDFs
In integration test of UDF component, I'm tyring to assert value returned by my UDF by firing a query
Connection connection = DriverManager.getConnection(PRESTO_URL, PRESTO_USER, PRESTO_PASSWORD);
Statement statement = connection.createStatement();
ResultSet resultSet = statement.executeQuery("SELECT SHA1(9223372036854775807) AS hash");
assertEquals("458B642B137E2C76E0B746C6FA43E64C3D4C47F1", resultSet.getString("hash"));
This gives following stack-trace (PrestoResultSet.checkValidRow)
NOTE: ExtendedHashFunctionsTestIT.java:39 line is the one containing resultSet.getString(..)
java.sql.SQLException: Not on a valid row
at com.facebook.presto.jdbc.PrestoResultSet.checkValidRow(PrestoResultSet.java:1658)
at com.facebook.presto.jdbc.PrestoResultSet.column(PrestoResultSet.java:1690)
at com.facebook.presto.jdbc.PrestoResultSet.getString(PrestoResultSet.java:370)
at com.company.plugin.it.udfs.scalar.hash.ExtendedHashFunctionsTestIT.testBody(ExtendedHashFunctionsTestIT.java:39)
at com.company.plugin.it.AbstractIntegrationTest.test(AbstractIntegrationTest.java:77)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.testng.internal.MethodInvocationHelper.invokeMethod(MethodInvocationHelper.java:85)
at org.testng.internal.Invoker.invokeMethod(Invoker.java:639)
at org.testng.internal.Invoker.invokeTestMethod(Invoker.java:816)
at org.testng.internal.Invoker.invokeTestMethods(Invoker.java:1124)
at org.testng.internal.TestMethodWorker.invokeTestMethods(TestMethodWorker.java:125)
at org.testng.internal.TestMethodWorker.run(TestMethodWorker.java:108)
at org.testng.TestRunner.privateRun(TestRunner.java:774)
at org.testng.TestRunner.run(TestRunner.java:624)
at org.testng.SuiteRunner.runTest(SuiteRunner.java:359)
at org.testng.SuiteRunner.runSequentially(SuiteRunner.java:354)
at org.testng.SuiteRunner.privateRun(SuiteRunner.java:312)
at org.testng.SuiteRunner.run(SuiteRunner.java:261)
at org.testng.SuiteRunnerWorker.runSuite(SuiteRunnerWorker.java:52)
at org.testng.SuiteRunnerWorker.run(SuiteRunnerWorker.java:86)
at org.testng.TestNG.runSuitesSequentially(TestNG.java:1191)
at org.testng.TestNG.runSuitesLocally(TestNG.java:1116)
at org.testng.TestNG.run(TestNG.java:1024)
at org.apache.maven.surefire.testng.TestNGExecutor.run(TestNGExecutor.java:62)
at org.apache.maven.surefire.testng.TestNGDirectoryTestSuite.execute(TestNGDirectoryTestSuite.java:141)
at org.apache.maven.surefire.Surefire.run(Surefire.java:180)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.maven.surefire.booter.SurefireBooter.runSuitesInProcess(SurefireBooter.java:350)
at org.apache.maven.surefire.booter.SurefireBooter.main(SurefireBooter.java:1021)
I'm perplexed because error occurs only when I try to access ResultSet; commenting out the assertEquals(..) statement lets the test run successfully.
What's even more disturbing is that stack-trace is same for virtually any query:
SELECT 'some_string' AS some_alias
SHOW SCHEMAS
I'm using Spotify's dockerfile-maven-plugin for the tests. I've put up relevant files in this GitHub Gist
ExtendedHashFunctionsTestIT.java (simplified integration-test file)
pom.xml
Dockerfile
It laments me to think that the mistake that ate up my 2 days was this
Connection connection = DriverManager.getConnection(PRESTO_URL, PRESTO_USER, PRESTO_PASSWORD);
Statement statement = connection.createStatement();
ResultSet resultSet = statement.executeQuery("SELECT SHA1(9223372036854775807) AS hash");
resultSet.next();
assertEquals("458B642B137E2C76E0B746C6FA43E64C3D4C47F1", resultSet.getString("hash"));
That 4th line: I had missed resultSet.next() prior to resultSet.getString(..) :-(
With this code i got an error in Jenkins pipeline. I don`t get it why?
Am I missing something?
node {
stage 'test'
def whatThe = someFunc('textToFunc')
{def whatThe2 = someFunc2('textToFunc2')}
}
def someFunc(String text){
echo text
text
}
def someFunc2(String text2){
echo text2
text2
}
Error:
java.lang.NoSuchMethodError: **No such DSL method 'someFunc'** found among [archive, bat, build, catchError, checkout, deleteDir, dir, echo, emailext, emailextrecipients, error, fileExists, git, input, isUnix, load, mail, node, parallel, properties, pwd, readFile, readTrusted, retry, sh, sleep, stage, stash, step, svn, timeout, timestamps, tool, unarchive, unstash, waitUntil, withCredentials, withEnv, wrap, writeFile, ws]
at org.jenkinsci.plugins.workflow.cps.DSL.invokeMethod(DSL.java:124)
at org.jenkinsci.plugins.workflow.cps.CpsScript.invokeMethod(CpsScript.java:117)
at groovy.lang.MetaClassImpl.invokeMethodOnGroovyObject(MetaClassImpl.java:1280)
at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1174)
at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1024)
at org.codehaus.groovy.runtime.callsite.PogoMetaClassSite.call(PogoMetaClassSite.java:42)
at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:48)
at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:113)
at com.cloudbees.groovy.cps.sandbox.DefaultInvoker.methodCall(DefaultInvoker.java:15)
at WorkflowScript.run(WorkflowScript:4)
at ___cps.transform___(Native Method)
at com.cloudbees.groovy.cps.impl.ContinuationGroup.methodCall(ContinuationGroup.java:55)
at com.cloudbees.groovy.cps.impl.FunctionCallBlock$ContinuationImpl.dispatchOrArg(FunctionCallBlock.java:106)
at com.cloudbees.groovy.cps.impl.FunctionCallBlock$ContinuationImpl.fixArg(FunctionCallBlock.java:79)
at sun.reflect.GeneratedMethodAccessor878.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
at java.lang.reflect.Method.invoke(Unknown Source)
at com.cloudbees.groovy.cps.impl.ContinuationPtr$ContinuationImpl.receive(ContinuationPtr.java:72)
at com.cloudbees.groovy.cps.impl.ClosureBlock.eval(ClosureBlock.java:40)
at com.cloudbees.groovy.cps.Next.step(Next.java:58)
at com.cloudbees.groovy.cps.Continuable.run0(Continuable.java:154)
at org.jenkinsci.plugins.workflow.cps.CpsThread.runNextChunk(CpsThread.java:164)
at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup.run(CpsThreadGroup.java:360)
at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup.access$100(CpsThreadGroup.java:80)
at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup$2.call(CpsThreadGroup.java:236)
at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup$2.call(CpsThreadGroup.java:226)
at org.jenkinsci.plugins.workflow.cps.CpsVmExecutorService$2.call(CpsVmExecutorService.java:47)
at java.util.concurrent.FutureTask.run(Unknown Source)
at hudson.remoting.SingleLaneExecutorService$1.run(SingleLaneExecutorService.java:112)
at jenkins.util.ContextResettingExecutorService$1.run(ContextResettingExecutorService.java:28)
at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
at java.util.concurrent.FutureTask.run(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
at java.lang.Thread.run(Unknown Source)
Finished: FAILURE
remove the extra brackets from around the sumfunc2 invocation:
node {
stage 'test'
def whatThe = someFunc('textToFunc')
def whatThe2 = someFunc2('textToFunc2')
}
def someFunc(String text){
echo text
text
}
def someFunc2(String text2){
echo text2
text2
}
Update:
In Groovy if a method's last argument is of type Closure, then when calling the method the closure can be outside of the brackets like:
def foo(whatever, Closure c) {}
// Can be invoked as
foo(whatever, {
// This is the second argument of foo of type Closure
})
// It is also the same as writing
foo(whatever) {
// This is the second argument of foo of type Closure
}
The reason that the original throws is because the following code
def whatThe = someFunc('textToFunc')
{def whatThe2 = someFunc2('textToFunc2')}
is the same code as
def whatThe = someFunc('textToFunc') {
def whatThe2 = someFunc2('textToFunc2')
}
This means that what the interpreter will be looking for is
someFunc(String text, Closure c)
and there is no such method
Since this answer is the first one I found when I lookup the "No such DSL method" message, I would like to add that it might also be the interface that is not matching. In my case the first parameter was a list, but I tried to call the method with an array. So please check your interface matches what you expect, and your parameters are passed in correctly.
In my case what was happening is that I was referencing a variable using: ${} that was expanding to an empty string:
env.VAR1=${VAR2}
in my case, var2 didn't really exists, so what I had to do was actually:
env.VAR1=env.VAR2.
Silly mistake from my end.
You may have also forgotten to in include your library...
i.e.
#Library('shared-library') _
I've got a data stream that send a packet every 10 seconds. I need a query for Esper that generate event (with last packet received) when the device is down.
I assume device lost if no data arrives for 30sec but I need to get only 1 notify every 5h until the device came back, to avoid spam
Here is my base query:
select * from pattern [every tick=MyInputStream-> (timer:interval(30 sec) and not MyInputStream)] output every 5 hours
But it doesn't work, after first event i got:
Event:
-> {tick=BeanEventBean eventType=BeanEventType name=MyInputStream clazz=mypackage.MyInputStream bean=MyInputStream [field1=1, field2=2]}
Event:
ERROR - Unexpected exception invoking listener update method on listener class 'GenericListener' : NullPointerException : null
java.lang.NullPointerException at cep.listener.GenericListener.update(GenericListener.java:11)
at com.espertech.esper.core.service.StatementResultServiceImpl.dispatchInternal(StatementResultServiceImpl.java:381)
at com.espertech.esper.core.service.StatementResultServiceImpl.processDispatch(StatementResultServiceImpl.java:256)
at com.espertech.esper.core.service.StatementResultServiceImpl.execute(StatementResultServiceImpl.java:242)
at com.espertech.esper.core.service.UpdateDispatchViewBase.execute(UpdateDispatchViewBase.java:75)
at com.espertech.esper.core.service.UpdateDispatchFutureSpin.execute(UpdateDispatchFutureSpin.java:85)
at com.espertech.esper.dispatch.DispatchServiceImpl.dispatchFromQueue(DispatchServiceImpl.java:52)
at com.espertech.esper.dispatch.DispatchServiceImpl.dispatch(DispatchServiceImpl.java:31)
at com.espertech.esper.core.service.EPRuntimeImpl.dispatch(EPRuntimeImpl.java:1340)
at com.espertech.esper.core.service.EPRuntimeImpl.processTimeEvent(EPRuntimeImpl.java:540)
at com.espertech.esper.core.service.EPRuntimeImpl.processEvent(EPRuntimeImpl.java:425)
at com.espertech.esper.core.service.EPRuntimeImpl.sendEvent(EPRuntimeImpl.java:197)
at com.espertech.esper.core.service.EPRuntimeImpl.timerCallback(EPRuntimeImpl.java:171)
at com.espertech.esper.timer.EPLTimerTask.run(EPLTimerTask.java:61)
at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
at java.util.concurrent.FutureTask.runAndReset(Unknown Source)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(Unknown Source)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
at java.lang.Thread.run(Unknown Source)
Also I think that my query is not 100% correct for my problem.
Can anyone help me?
The exception indicates that the listener code throws the NullPointer. You'd want to check your code.
Check context partitions for the start and end type logic.