I am trying to run this synapseml opencv to pre processing and I get this error. It's simple use case of reading images and transforming .
from synapse.ml.opencv import *
image_transformer = (ImageTransformer().setOutputCol("transformed")
.resize(1024, True)
)
images_transformed = image_transformer.transform(images_df)
images_transformed.show()
Error
Py4JJavaError: An error occurred while calling o666.showString.
: org.apache.spark.SparkException: Job aborted due to stage failure: Task 1 in stage 3.0 failed 4 times, most recent failure: Lost task 1.3 in stage 3.0 (TID 8) (vm-94657109 executor 2): org.apache.spark.SparkException: Failed to execute user defined function(functions$$$Lambda$4740/1915249168: (struct<origin:string,height:int,width:int,nChannels:int,mode:int,data:binary>) => struct<origin:string,height:int,width:int,nChannels:int,mode:int,data:binary>)
at org.apache.spark.sql.catalyst.expressions.GeneratedClass$GeneratedIteratorForCodegenStage1.project_doConsume_0$(Unknown Source)
at org.apache.spark.sql.catalyst.expressions.GeneratedClass$GeneratedIteratorForCodegenStage1.serializefromobject_doConsume_0$(Unknown Source)
at org.apache.spark.sql.catalyst.expressions.GeneratedClass$GeneratedIteratorForCodegenStage1.processNext(Unknown Source)
at org.apache.spark.sql.execution.BufferedRowIterator.hasNext(BufferedRowIterator.java:43)
at org.apache.spark.sql.execution.WholeStageCodegenExec$$anon$1.hasNext(WholeStageCodegenExec.scala:755)
at org.apache.spark.sql.execution.SparkPlan.$anonfun$getByteArrayRdd$1(SparkPlan.scala:383)
at org.apache.spark.rdd.RDD.$anonfun$mapPartitionsInternal$2(RDD.scala:905)
at org.apache.spark.rdd.RDD.$anonfun$mapPartitionsInternal$2$adapted(RDD.scala:905)
at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:52)
at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:374)
at org.apache.spark.rdd.RDD.iterator(RDD.scala:338)
at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:90)
at org.apache.spark.scheduler.Task.run(Task.scala:131)
at org.apache.spark.executor.Executor$TaskRunner.$anonfun$run$3(Executor.scala:497)
at org.apache.spark.util.Utils$.tryWithSafeFinally(Utils.scala:1439)
at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:500)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: CvException [org.opencv.core.CvException: cv::Exception: /home/jason/opencv/opencv/opencv-3.2.0/modules/core/src/matrix.cpp:307: error: (-215) s >= 0 in function setSize
]
at org.opencv.core.Mat.n_Mat(Native Method)
at org.opencv.core.Mat.<init>(Mat.java:37)
at com.microsoft.azure.synapse.ml.opencv.ImageTransformer$.row2mat(ImageTransformer.scala:294)
at com.microsoft.azure.synapse.ml.opencv.ImageTransformer$.$anonfun$decodeImage$4(ImageTransformer.scala:326)
at scala.Option.map(Option.scala:230)
at com.microsoft.azure.synapse.ml.opencv.ImageTransformer$.decodeImage(ImageTransformer.scala:326)
at com.microsoft.azure.synapse.ml.opencv.ImageTransformer.$anonfun$transform$9(ImageTransformer.scala:632)
at org.apache.spark.injections.UDFUtils$$anon$1.call(UDFUtils.scala:23)
at org.apache.spark.sql.functions$.$anonfun$udf$91(functions.scala:4854)
... 19 more
Related
I created maven project in Jenkins and it has a step to deploy war to JBoss using Wildfy Deployer Plugin with following details -
WAR/EAR files = ./target/test.war
Hostname = localhost
Port - 8181
But I run job it results into following error -
2019-10-14 13:42:48.720+0000 [id=185] INFO hudson.model.Run#execute: deploy-war-to-jboss #4 main build action completed: FAILURE
java.lang.IllegalStateException: Unable to initialize command context.
at org.jboss.as.cli.scriptsupport.CLI.connect(CLI.java:152)
at org.jboss.as.cli.scriptsupport.CLI.connect(CLI.java:135)
at org.jenkinsci.plugins.wildfly.WildflyBuilder.perform(WildflyBuilder.java:136)
at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20)
at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:741)
at hudson.model.Build$BuildExecution.build(Build.java:206)
at hudson.model.Build$BuildExecution.doRun(Build.java:163)
at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:504)
at hudson.model.Run.execute(Run.java:1815)
at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
at hudson.model.ResourceController.execute(ResourceController.java:97)
at hudson.model.Executor.run(Executor.java:429)
Caused by: org.jboss.as.cli.CliInitializationException: Failed to load org.jboss.as.cli.impl.CommandContextFactoryImpl
at org.jboss.as.cli.CommandContextFactory.getInstance(CommandContextFactory.java:41)
at org.jboss.as.cli.scriptsupport.CLI.connect(CLI.java:149)
... 11 more
Caused by: java.lang.ClassNotFoundException: org.jboss.as.cli.impl.CommandContextFactoryImpl
at java.net.URLClassLoader.findClass(Unknown Source)
at java.lang.ClassLoader.loadClass(Unknown Source)
at java.lang.ClassLoader.loadClass(Unknown Source)
at org.eclipse.jetty.webapp.WebAppClassLoader.loadClass(WebAppClassLoader.java:555)
at java.lang.ClassLoader.loadClass(Unknown Source)
at org.jboss.as.cli.CommandContextFactory.getInstance(CommandContextFactory.java:39)
... 12 more
2019-10-14 13:44:06.945+0000 [id=230] INFO hudson.model.Run#execute: deploy-war-to-jboss #5 main build action completed: FAILURE
Please help to understand what's wrong here.
I am using Google test 1.8 and have the xml report in JUnit format ( I read from other sources)
generated using ./a.out --gtest_output=xml:result.xml in my Linux machine.
however when I add this report as part of post build action , choosing Publish JUnit test result report, in the text box in jenkins, I specified "result.xml " file .
Built it and getting the following error
ERROR: Step ‘Publish JUnit test result report’ aborted due to exception:
java.io.IOException: remote file operation failed
2 FAILED TESTS
Build step 'Execute shell' marked build as failure
Recording test results
ERROR: Step ‘Publish JUnit test result report’ aborted due to exception:
java.io.IOException: remote file operation failed: /home/nelrtu/workspace /NELRTU_CI_BUILD at hudson.remoting.Channel#d88e287:SAME_SLAVE: java.io.IOException: Remote call on SOME_SLAVE failed
at hudson.FilePath.act(FilePath.java:986)
at hudson.FilePath.act(FilePath.java:968)
at hudson.tasks.junit.JUnitParser.parseResult(JUnitParser.java:103)
at hudson.tasks.junit.JUnitResultArchiver.parse(JUnitResultArchiver.java:126)
at hudson.tasks.junit.JUnitResultArchiver.perform(JUnitResultArchiver.java:148)
at hudson.tasks.BuildStepCompatibilityLayer.perform(BuildStepCompatibilityLayer.java:78)
at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20)
at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:782)
at hudson.model.AbstractBuild$AbstractBuildExecution.performAllBuildSteps(AbstractBuild.java:723)
at hudson.model.Build$BuildExecution.post2(Build.java:185)
at hudson.model.AbstractBuild$AbstractBuildExecution.post(AbstractBuild.java:668)
at hudson.model.Run.execute(Run.java:1763)
at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
at hudson.model.ResourceController.execute(ResourceController.java:98)
at hudson.model.Executor.run(Executor.java:410)
Caused by: java.io.IOException: Remote call on SOME_SLAVE failed
at hudson.remoting.Channel.call(Channel.java:789)
at hudson.FilePath.act(FilePath.java:979)
... 14 more`enter code here`
Caused by: java.lang.Error: Unable to load resource hudson/tasks/junit/Messages.properties
at hudson.remoting.RemoteClassLoader.findResource(RemoteClassLoader.java:397)
at java.lang.ClassLoader.getResource(ClassLoader.java:1096)
at java.lang.Class.getResource(Class.java:2267)
at org.jvnet.localizer.ResourceBundleHolder.get(ResourceBundleHolder.java:107)
at org.jvnet.localizer.ResourceBundleHolder.get(ResourceBundleHolder.java:128)
at org.jvnet.localizer.ResourceBundleHolder.get(ResourceBundleHolder.java:128)
at org.jvnet.localizer.ResourceBundleHolder.format(ResourceBundleHolder.java:170)
at hudson.tasks.junit.Messages.JUnitResultArchiver_NoTestReportFound(Messages.java:368)
at hudson.tasks.junit.JUnitParser$ParseResultCallable.invoke(JUnitParser.java:140)
at hudson.tasks.junit.JUnitParser$ParseResultCallable.invoke(JUnitParser.java:107)
at hudson.FilePath$FileCallableWrapper.call(FilePath.java:2719)
at hudson.remoting.UserRequest.perform(UserRequest.java:152)
at hudson.remoting.UserRequest.perform(UserRequest.java:50)
at hudson.remoting.Request$2.run(Request.java:332)
at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:68)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
at ......remote call to RTU_CI_SLAVE(Native Method)
at hudson.remoting.Channel.attachCallSiteStackTrace(Channel.java:1416)
at hudson.remoting.UserResponse.retrieve(UserRequest.java:252)
at hudson.remoting.Channel.call(Channel.java:781)
... 15 more
Caused by: java.util.concurrent.ExecutionException: java.io.IOException: Failed to write to /home/nelrtu/.jenkins/cache/jars/FB/BD3521667259673E8728695E2AC347.jar
at hudson.remoting.AsyncFutureImpl.get(AsyncFutureImpl.java:75)
at hudson.remoting.FutureAdapter.get(FutureAdapter.java:55)
at hudson.remoting.RemoteClassLoader.findResource(RemoteClassLoader.java:389)
at java.lang.ClassLoader.getResource(ClassLoader.java:1096)
at java.lang.Class.getResource(Class.java:2267)
at org.jvnet.localizer.ResourceBundleHolder.get(ResourceBundleHolder.java:107)
at org.jvnet.localizer.ResourceBundleHolder.get(ResourceBundleHolder.java:128)
at org.jvnet.localizer.ResourceBundleHolder.get(ResourceBundleHolder.java:128)
at org.jvnet.localizer.ResourceBundleHolder.format(ResourceBundleHolder.java:170)
at hudson.tasks.junit.Messages.JUnitResultArchiver_NoTestReportFound(Messages.java:368)
at hudson.tasks.junit.JUnitParser$ParseResultCallable.invoke(JUnitParser.java:140)
at hudson.tasks.junit.JUnitParser$ParseResultCallable.invoke(JUnitParser.java:107)
at hudson.FilePath$FileCallableWrapper.call(FilePath.java:2719)
at hudson.remoting.UserRequest.perform(UserRequest.java:152)
at hudson.remoting.UserRequest.perform(UserRequest.java:50)
at hudson.remoting.Request$2.run(Request.java:332)
at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:68)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.io.IOException: Failed to write to /home/<User-director>/.jenkins/cache/jars/FB/BD3521667259673E8728695E2AC347.jar
at hudson.remoting.FileSystemJarCache.retrieve(FileSystemJarCache.java:112)
at hudson.remoting.JarCacheSupport$1.run(JarCacheSupport.java:64)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at hudson.remoting.AtmostOneThreadExecutor$Worker.run(AtmostOneThreadExecutor.java:110)
... 1 more
Caused by: java.io.IOException: Remote call on channel failed
at hudson.remoting.Channel.call(Channel.java:789)
at hudson.remoting.RemoteInvocationHandler.invoke(RemoteInvocationHandler.java:249)
at com.sun.proxy.$Proxy3.writeJarTo(Unknown Source)
at hudson.remoting.FileSystemJarCache.retrieve(FileSystemJarCache.java:74)
... 5 more
Caused by: java.lang.OutOfMemoryError: PermGen space
at sun.misc.Unsafe.defineClass(Native Method)
at sun.reflect.ClassDefiner.defineClass(ClassDefiner.java:63)
at sun.reflect.MethodAccessorGenerator$1.run(MethodAccessorGenerator.java:399)
at sun.reflect.MethodAccessorGenerator$1.run(MethodAccessorGenerator.java:396)
at java.security.AccessController.doPrivileged(Native Method)
at sun.reflect.MethodAccessorGenerator.generate(MethodAccessorGenerator.java:395)
at sun.reflect.MethodAccessorGenerator.generateSerializationConstructor(MethodAccessorGenerator.java:113)
at sun.reflect.ReflectionFactory.newConstructorForSerialization(ReflectionFactory.java:331)
at java.io.ObjectStreamClass.getSerializableConstructor(ObjectStreamClass.java:1376)
at java.io.ObjectStreamClass.access$1500(ObjectStreamClass.java:72)
at java.io.ObjectStreamClass$2.run(ObjectStreamClass.java:493)
at java.io.ObjectStreamClass$2.run(ObjectStreamClass.java:468)
at java.security.AccessController.doPrivileged(Native Method)
at java.io.ObjectStreamClass.<init>(ObjectStreamClass.java:468)
at java.io.ObjectStreamClass.lookup(ObjectStreamClass.java:365)
at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1131)
at java.io.ObjectOutputStream.writeObject(ObjectOutputStream.java:347)
at java.util.HashMap.writeObject(HashMap.java:1133)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at java.io.ObjectStreamClass.invokeWriteObject(ObjectStreamClass.java:988)
at java.io.ObjectOutputStream.writeSerialData(ObjectOutputStream.java:1493)
at java.io.ObjectOutputStream.writeOrdinaryObject(ObjectOutputStream.java:1429)
at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1175)
at java.io.ObjectOutputStream.defaultWriteFields(ObjectOutputStream.java:1541)
at java.io.ObjectOutputStream.writeSerialData(ObjectOutputStream.java:1506)
at java.io.ObjectOutputStream.writeOrdinaryObject(ObjectOutputStream.java:1429)
at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1175)
at java.io.ObjectOutputStream.defaultWriteFields(ObjectOutputStream.java:1541)
at java.io.ObjectOutputStream.writeSerialData(ObjectOutputStream.java:1506)
at ......remote call to channel(Native Method)
at hudson.remoting.Channel.attachCallSiteStackTrace(Channel.java:1416)
at hudson.remoting.UserResponse.retrieve(UserRequest.java:252)
at hudson.remoting.Channel.call(Channel.java:781)
... 8 more
Finished: FAILURE
I pressume , jenkins is accepting the Xml format generated by Google test , since I see the log message as "Recording test results"
I am building a docker image using maven plugin com.spotify. its giving error when I execute below command:
mvn clean package docker:build
[ERROR] Failed to execute goal io.fabric8:docker-maven-plugin:0.16.4:build (default-cli) on project xbs-docker: Execution default-cli of goal io.fabric8:docker-maven-plugin:0.16.4:build failed: No given, no DOCKER_HOST environment variable, no read/writable '/var/run/docker.sock' or '//./pipe/docker_engine' and no external provider like Docker machine configured -> [Help 1]
When I set DOCKER_HOST=tcp://localhost:2375, its giving below error :
Failed to execute goal io.fabric8:docker-maven-plugin:0.16.4:build (default-cli) on project <project_name>: Execution default-cli of goal io.fabric8:docker-maven-plugin:0.16.4:build failed. NullPointerException -> [Help 1]
org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute goal io.fabric8:docker-maven-plugin:0.16.4:build (default-cli) on project <project_name>: Execution default-cli of goal io.fabric8:docker-maven-plugin:0.16.4:build failed.
at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:212)
at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:153)
at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:145)
at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:116)
at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:80)
at org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build(SingleThreadedBuilder.java:51)
at org.apache.maven.lifecycle.internal.LifecycleStarter.execute(LifecycleStarter.java:128)
at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:307)
at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:193)
at org.apache.maven.DefaultMaven.execute(DefaultMaven.java:106)
at org.apache.maven.cli.MavenCli.execute(MavenCli.java:863)
at org.apache.maven.cli.MavenCli.doMain(MavenCli.java:288)
at org.apache.maven.cli.MavenCli.main(MavenCli.java:199)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced(Launcher.java:289)
at org.codehaus.plexus.classworlds.launcher.Launcher.launch(Launcher.java:229)
at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode(Launcher.java:415)
at org.codehaus.plexus.classworlds.launcher.Launcher.main(Launcher.java:356)
Caused by: org.apache.maven.plugin.PluginExecutionException: Execution default-cli of goal io.fabric8:docker-maven-plugin:0.16.4:build failed.
at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo(DefaultBuildPluginManager.java:145)
at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:207)
... 20 more
Caused by: java.lang.NullPointerException
at io.fabric8.maven.docker.access.DockerConnectionDetector.getCertPath(DockerConnectionDetector.java:108)
at io.fabric8.maven.docker.AbstractDockerMojo.createDockerAccess(AbstractDockerMojo.java:277)
at io.fabric8.maven.docker.AbstractDockerMojo.execute(AbstractDockerMojo.java:191)
at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo(DefaultBuildPluginManager.java:134)
... 21 more
Please help me to resolve this issue.
I'm using Spark1.6
I'm running a simple df.show(2) method and got errors like
An error occurred while calling o143.showString.
: org.apache.spark.SparkException: Job aborted due to stage
failure: Task 6 in stage 6.0 failed 4 times, most recent failure:
Lost task 6.3 in stage 6.0
ExecutorLostFailure (executor 2 exited caused by one of the
running tasks) Reason: Slave lost
When I did persist, through spark UI I saw the shuffleWrite memory is very high and took a long time and still returned errors.
Through some search, I found these might be the out of memory problem.
Following this link out of memory error Java
I did a repartition up to 1000, still not so helpful.
I set up the SparkConf as
conf = (SparkConf().set("spark.driver.maxResultSize", "150g").set("spark.serializer", "org.apache.spark.serializer.KryoSerializer"))
My server side memory could be up to 200GB
Do yo have any good idea to do this or point me to related links. Pyspark will be most helpful
Here is the error log from YARN:
Application application_1477088172315_0118 failed 2 times due to
AM Container for appattempt_1477088172315_0118_000006 exited
with exitCode: 10
For more detailed output, check application tracking page: Then,
click on links to logs of each attempt.
Diagnostics: Exception from container-launch.
Container id: container_1477088172315_0118_06_000001
Exit code: 10
Stack trace: ExitCodeException exitCode=10:
at org.apache.hadoop.util.Shell.runCommand(Shell.java:582)
at org.apache.hadoop.util.Shell.run(Shell.java:479)
at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:773)
at org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.launchContainer(DefaultContainerExecutor.java:212)
at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:302)
at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:82)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Container exited with a non-zero exit code 10
Failing this attempt. Failing the application.
Here is the error info from notebook:
Py4JJavaError: An error occurred while calling o71.showString.
: org.apache.spark.SparkException: Job aborted due to stage failure: Task 1 in stage 15.0 failed 4 times, most recent failure: Lost task 1.3 in stage 15.0 (): ExecutorLostFailure (executor 26 exited caused by one of the running tasks) Reason: Slave lost
Driver stacktrace:
at org.apache.spark.scheduler.DAGScheduler.org$apache$spark$scheduler$DAGScheduler$$failJobAndIndependentStages(DAGScheduler.scala:1431)
at org.apache.spark.scheduler.DAGScheduler$$anonfun$abortStage$1.apply(DAGScheduler.scala:1419)
at org.apache.spark.scheduler.DAGScheduler$$anonfun$abortStage$1.apply(DAGScheduler.scala:1418)
at scala.collection.mutable.ResizableArray$class.foreach(ResizableArray.scala:59)
at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:47)
at org.apache.spark.scheduler.DAGScheduler.abortStage(DAGScheduler.scala:1418)
at org.apache.spark.scheduler.DAGScheduler$$anonfun$handleTaskSetFailed$1.apply(DAGScheduler.scala:799)
at org.apache.spark.scheduler.DAGScheduler$$anonfun$handleTaskSetFailed$1.apply(DAGScheduler.scala:799)
at scala.Option.foreach(Option.scala:236)
at org.apache.spark.scheduler.DAGScheduler.handleTaskSetFailed(DAGScheduler.scala:799)
at org.apache.spark.scheduler.DAGSchedulerEventProcessLoop.doOnReceive(DAGScheduler.scala:1640)
at org.apache.spark.scheduler.DAGSchedulerEventProcessLoop.onReceive(DAGScheduler.scala:1599)
at org.apache.spark.scheduler.DAGSchedulerEventProcessLoop.onReceive(DAGScheduler.scala:1588)
at org.apache.spark.util.EventLoop$$anon$1.run(EventLoop.scala:48)
at org.apache.spark.scheduler.DAGScheduler.runJob(DAGScheduler.scala:620)
at org.apache.spark.SparkContext.runJob(SparkContext.scala:1832)
at org.apache.spark.SparkContext.runJob(SparkContext.scala:1845)
at org.apache.spark.SparkContext.runJob(SparkContext.scala:1858)
at org.apache.spark.sql.execution.SparkPlan.executeTake(SparkPlan.scala:212)
at org.apache.spark.sql.execution.Limit.executeCollect(basicOperators.scala:165)
at org.apache.spark.sql.execution.SparkPlan.executeCollectPublic(SparkPlan.scala:174)
at org.apache.spark.sql.DataFrame$$anonfun$org$apache$spark$sql$DataFrame$$execute$1$1.apply(DataFrame.scala:1499)
at org.apache.spark.sql.DataFrame$$anonfun$org$apache$spark$sql$DataFrame$$execute$1$1.apply(DataFrame.scala:1499)
at org.apache.spark.sql.execution.SQLExecution$.withNewExecutionId(SQLExecution.scala:56)
at org.apache.spark.sql.DataFrame.withNewExecutionId(DataFrame.scala:2086)
at org.apache.spark.sql.DataFrame.org$apache$spark$sql$DataFrame$$execute$1(DataFrame.scala:1498)
at org.apache.spark.sql.DataFrame.org$apache$spark$sql$DataFrame$$collect(DataFrame.scala:1505)
at org.apache.spark.sql.DataFrame$$anonfun$head$1.apply(DataFrame.scala:1375)
at org.apache.spark.sql.DataFrame$$anonfun$head$1.apply(DataFrame.scala:1374)
at org.apache.spark.sql.DataFrame.withCallback(DataFrame.scala:2099)
at org.apache.spark.sql.DataFrame.head(DataFrame.scala:1374)
at org.apache.spark.sql.DataFrame.take(DataFrame.scala:1456)
at org.apache.spark.sql.DataFrame.showString(DataFrame.scala:170)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at py4j.reflection.MethodInvoker.invoke(MethodInvoker.java:231)
at py4j.reflection.ReflectionEngine.invoke(ReflectionEngine.java:381)
at py4j.Gateway.invoke(Gateway.java:259)
at py4j.commands.AbstractCommand.invokeMethod(AbstractCommand.java:133)
at py4j.commands.CallCommand.execute(CallCommand.java:79)
at py4j.GatewayConnection.run(GatewayConnection.java:209)
at java.lang.Thread.run(Thread.java:745)
Thank you
I have a problem with the OpsHub Visual Studio Migration Utility. It sometimes halts at a apparently randome changeset during migration, and I am not able to get it to continue. The error message of the failed changeset is:
OH-SCM-009: Error occurred while sync. Source parameters not found to
download the source
file:24::3117::$/Solution/Project/ProjectFolder/SomeFile.cs
I have succeded with migration of a couple of project by repeating the whole thing. But for larger projects I never get through the complete migration. And the problematic changeset is different each time.
Looking at the log, i could find some exceptions thrown like this:
08/11/2016 12:10:30,922 ERROR [DefaultQuartzScheduler_Worker-5] (org.jbpm.graph.def.GraphElement) - action threw exception: script evaluation exception
org.jbpm.graph.def.DelegationException: script evaluation exception
at org.jbpm.graph.action.Script.eval(Script.java:139)
at org.jbpm.graph.action.Script.eval(Script.java:73)
at org.jbpm.graph.action.Script.execute(Script.java:62)
at org.jbpm.graph.def.GraphElement.executeAction(GraphElement.java:259)
at org.jbpm.graph.def.GraphElement.executeActions(GraphElement.java:215)
at org.jbpm.graph.def.GraphElement.fireAndPropagateEvent(GraphElement.java:185)
at org.jbpm.graph.def.GraphElement.fireEvent(GraphElement.java:169)
at org.jbpm.graph.def.Transition.take(Transition.java:138)
at org.jbpm.graph.def.Node.leave(Node.java:393)
at org.jbpm.graph.exe.Token.signal(Token.java:192)
at org.jbpm.graph.exe.Token.signal(Token.java:140)
at org.jbpm.graph.exe.ProcessInstance.signal(ProcessInstance.java:270)
at com.opshub.eai.ProcessEngine.executeTask(ProcessEngine.java:190)
at com.opshub.eai.ProcessEngine.executeProcess(ProcessEngine.java:60)
at com.opshub.eai.SyncEaiProcess.syncEvent(SyncEaiProcess.java:230)
at com.opshub.eai.SyncEaiProcess.sync(SyncEaiProcess.java:137)
at com.opshub.eai.OIMEventGenerator.process(OIMEventGenerator.java:428)
at com.opshub.jobs.core.OHJob.execute(OHJob.java:125)
at org.quartz.core.JobRunShell.run(JobRunShell.java:202)
at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:525)
Caused by: com.opshub.eai.core.exceptions.OIMSCMAdapterException: OH-SCM-009: Error occurred while sync. Source parameters not found to download the source file:24::3117::$/Solution/Project/ApiControllers/PersonServiceController.cs
at com.opshub.eai.core.adapters.OIMSCMAdapter.sync(OIMSCMAdapter.java:409)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:483)
at bsh.Reflect.invokeMethod(Unknown Source)
at bsh.Reflect.invokeObjectMethod(Unknown Source)
at bsh.Name.invokeMethod(Unknown Source)
at bsh.BSHMethodInvocation.eval(Unknown Source)
at bsh.BSHPrimaryExpression.eval(Unknown Source)
at bsh.BSHVariableDeclarator.eval(Unknown Source)
at bsh.BSHTypedVariableDeclaration.eval(Unknown Source)
at bsh.Interpreter.eval(Unknown Source)
at bsh.Interpreter.eval(Unknown Source)
at bsh.Interpreter.eval(Unknown Source)
at org.jbpm.graph.action.Script.eval(Script.java:130)
... 19 more
Caused by: java.lang.RuntimeException: Source parameters not found to download the source file:24::3117::$/Solution/Project/ApiControllers/PersonServiceController.cs
at com.opshub.eai.tfssource.adapter.TFSVersionControlConnector.checkInAll(TFSVersionControlConnector.java:166)
at com.opshub.eai.tfssource.adapter.TFSSourceAdapter.checkIn(TFSSourceAdapter.java:246)
at com.opshub.eai.tfssource.adapter.TFSSourceAdapter$$EnhancerByCGLIB$$2a6a7d04.CGLIB$checkIn$17(<generated>)
at com.opshub.eai.tfssource.adapter.TFSSourceAdapter$$EnhancerByCGLIB$$2a6a7d04$$FastClassByCGLIB$$cee96b8e.invoke(<generated>)
at net.sf.cglib.proxy.MethodProxy.invokeSuper(MethodProxy.java:167)
at com.opshub.eai.core.adapters.caching.AdapterCacheHandler.intercept(AdapterCacheHandler.java:185)
at com.opshub.eai.tfssource.adapter.TFSSourceAdapter$$EnhancerByCGLIB$$2a6a7d04.checkIn(<generated>)
at com.opshub.eai.core.adapters.OIMSCMAdapter.sync(OIMSCMAdapter.java:325)
... 34 more
Caused by: Source parameters not found to download the source file:24::3117::$/Solution/Project/ApiControllers/PersonServiceController.cs
at org.apache.axis.message.SOAPFaultBuilder.createFault(SOAPFaultBuilder.java:222)
at org.apache.axis.message.SOAPFaultBuilder.endElement(SOAPFaultBuilder.java:129)
at org.apache.axis.encoding.DeserializationContext.endElement(DeserializationContext.java:1087)
at com.sun.org.apache.xerces.internal.parsers.AbstractSAXParser.endElement(AbstractSAXParser.java:609)
at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl.scanEndElement(XMLDocumentFragmentScannerImpl.java:1782)
at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl$FragmentContentDriver.next(XMLDocumentFragmentScannerImpl.java:2973)
at com.sun.org.apache.xerces.internal.impl.XMLDocumentScannerImpl.next(XMLDocumentScannerImpl.java:606)
at com.sun.org.apache.xerces.internal.impl.XMLNSDocumentScannerImpl.next(XMLNSDocumentScannerImpl.java:117)
at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl.scanDocument(XMLDocumentFragmentScannerImpl.java:510)
at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:848)
at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:777)
at com.sun.org.apache.xerces.internal.parsers.XMLParser.parse(XMLParser.java:141)
at com.sun.org.apache.xerces.internal.parsers.AbstractSAXParser.parse(AbstractSAXParser.java:1213)
at com.sun.org.apache.xerces.internal.jaxp.SAXParserImpl$JAXPSAXParser.parse(SAXParserImpl.java:649)
at com.sun.org.apache.xerces.internal.jaxp.SAXParserImpl.parse(SAXParserImpl.java:333)
at org.apache.axis.encoding.DeserializationContext.parse(DeserializationContext.java:227)
at org.apache.axis.SOAPPart.getAsSOAPEnvelope(SOAPPart.java:696)
at org.apache.axis.Message.getSOAPEnvelope(Message.java:435)
at org.apache.axis.handlers.soap.MustUnderstandChecker.invoke(MustUnderstandChecker.java:62)
at org.apache.axis.client.AxisClient.invoke(AxisClient.java:206)
at org.apache.axis.client.Call.invokeEngine(Call.java:2784)
at org.apache.axis.client.Call.invoke(Call.java:2767)
at org.apache.axis.client.Call.invoke(Call.java:2443)
at org.apache.axis.client.Call.invoke(Call.java:2366)
at org.apache.axis.client.Call.invoke(Call.java:1812)
at com.opshub.tfs.test.BasicHttpBinding_ITFSWebServiceStub.checkinAllStatus(BasicHttpBinding_ITFSWebServiceStub.java:6282)
at com.opshub.eai.tfssource.adapter.TFSVersionControlConnector.checkInAll(TFSVersionControlConnector.java:155)
... 41 more
08/11/2016 12:10:30,924 ERROR [DefaultQuartzScheduler_Worker-5] (com.opshub.eai.SyncEaiProcess) - Error in executing process definition : SCM Default Integration Workflow at node callMethodOpsHub-012013: Process execution failed on node: callMethod, in process definition: 'SCM Default Integration Workflow'; nested exception is : script evaluation exception
com.opshub.exceptions.eai.EAIProcessEngineException: OpsHub-012013: Process execution failed on node: callMethod, in process definition: 'SCM Default Integration Workflow'; nested exception is : script evaluation exception
at com.opshub.eai.ProcessEngine.executeTask(ProcessEngine.java:212)
at com.opshub.eai.ProcessEngine.executeProcess(ProcessEngine.java:60)
at com.opshub.eai.SyncEaiProcess.syncEvent(SyncEaiProcess.java:230)
at com.opshub.eai.SyncEaiProcess.sync(SyncEaiProcess.java:137)
at com.opshub.eai.OIMEventGenerator.process(OIMEventGenerator.java:428)
at com.opshub.jobs.core.OHJob.execute(OHJob.java:125)
at org.quartz.core.JobRunShell.run(JobRunShell.java:202)
at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:525)
Caused by: org.jbpm.graph.def.DelegationException: script evaluation exception
at org.jbpm.graph.action.Script.eval(Script.java:139)
at org.jbpm.graph.action.Script.eval(Script.java:73)
at org.jbpm.graph.action.Script.execute(Script.java:62)
at org.jbpm.graph.def.GraphElement.executeAction(GraphElement.java:259)
at org.jbpm.graph.def.GraphElement.executeActions(GraphElement.java:215)
at org.jbpm.graph.def.GraphElement.fireAndPropagateEvent(GraphElement.java:185)
at org.jbpm.graph.def.GraphElement.fireEvent(GraphElement.java:169)
at org.jbpm.graph.def.Transition.take(Transition.java:138)
at org.jbpm.graph.def.Node.leave(Node.java:393)
at org.jbpm.graph.exe.Token.signal(Token.java:192)
at org.jbpm.graph.exe.Token.signal(Token.java:140)
at org.jbpm.graph.exe.ProcessInstance.signal(ProcessInstance.java:270)
at com.opshub.eai.ProcessEngine.executeTask(ProcessEngine.java:190)
... 7 more
Caused by: com.opshub.eai.core.exceptions.OIMSCMAdapterException: OH-SCM-009: Error occurred while sync. Source parameters not found to download the source file:24::3117::$/Solution/Project/ApiControllers/PersonServiceController.cs
at com.opshub.eai.core.adapters.OIMSCMAdapter.sync(OIMSCMAdapter.java:409)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:483)
at bsh.Reflect.invokeMethod(Unknown Source)
at bsh.Reflect.invokeObjectMethod(Unknown Source)
at bsh.Name.invokeMethod(Unknown Source)
at bsh.BSHMethodInvocation.eval(Unknown Source)
at bsh.BSHPrimaryExpression.eval(Unknown Source)
at bsh.BSHVariableDeclarator.eval(Unknown Source)
at bsh.BSHTypedVariableDeclaration.eval(Unknown Source)
at bsh.Interpreter.eval(Unknown Source)
at bsh.Interpreter.eval(Unknown Source)
at bsh.Interpreter.eval(Unknown Source)
at org.jbpm.graph.action.Script.eval(Script.java:130)
... 19 more
Caused by: java.lang.RuntimeException: Source parameters not found to download the source file:24::3117::$/Solution/Project/ApiControllers/PersonServiceController.cs
at com.opshub.eai.tfssource.adapter.TFSVersionControlConnector.checkInAll(TFSVersionControlConnector.java:166)
at com.opshub.eai.tfssource.adapter.TFSSourceAdapter.checkIn(TFSSourceAdapter.java:246)
at com.opshub.eai.tfssource.adapter.TFSSourceAdapter$$EnhancerByCGLIB$$2a6a7d04.CGLIB$checkIn$17(<generated>)
at com.opshub.eai.tfssource.adapter.TFSSourceAdapter$$EnhancerByCGLIB$$2a6a7d04$$FastClassByCGLIB$$cee96b8e.invoke(<generated>)
at net.sf.cglib.proxy.MethodProxy.invokeSuper(MethodProxy.java:167)
at com.opshub.eai.core.adapters.caching.AdapterCacheHandler.intercept(AdapterCacheHandler.java:185)
at com.opshub.eai.tfssource.adapter.TFSSourceAdapter$$EnhancerByCGLIB$$2a6a7d04.checkIn(<generated>)
at com.opshub.eai.core.adapters.OIMSCMAdapter.sync(OIMSCMAdapter.java:325)
... 34 more
Caused by: Source parameters not found to download the source file:24::3117::$/Solution/Project/ApiControllers/PersonServiceController.cs
at org.apache.axis.message.SOAPFaultBuilder.createFault(SOAPFaultBuilder.java:222)
at org.apache.axis.message.SOAPFaultBuilder.endElement(SOAPFaultBuilder.java:129)
at org.apache.axis.encoding.DeserializationContext.endElement(DeserializationContext.java:1087)
at com.sun.org.apache.xerces.internal.parsers.AbstractSAXParser.endElement(AbstractSAXParser.java:609)
at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl.scanEndElement(XMLDocumentFragmentScannerImpl.java:1782)
at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl$FragmentContentDriver.next(XMLDocumentFragmentScannerImpl.java:2973)
at com.sun.org.apache.xerces.internal.impl.XMLDocumentScannerImpl.next(XMLDocumentScannerImpl.java:606)
at com.sun.org.apache.xerces.internal.impl.XMLNSDocumentScannerImpl.next(XMLNSDocumentScannerImpl.java:117)
at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl.scanDocument(XMLDocumentFragmentScannerImpl.java:510)
at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:848)
at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:777)
at com.sun.org.apache.xerces.internal.parsers.XMLParser.parse(XMLParser.java:141)
at com.sun.org.apache.xerces.internal.parsers.AbstractSAXParser.parse(AbstractSAXParser.java:1213)
at com.sun.org.apache.xerces.internal.jaxp.SAXParserImpl$JAXPSAXParser.parse(SAXParserImpl.java:649)
at com.sun.org.apache.xerces.internal.jaxp.SAXParserImpl.parse(SAXParserImpl.java:333)
at org.apache.axis.encoding.DeserializationContext.parse(DeserializationContext.java:227)
at org.apache.axis.SOAPPart.getAsSOAPEnvelope(SOAPPart.java:696)
at org.apache.axis.Message.getSOAPEnvelope(Message.java:435)
at org.apache.axis.handlers.soap.MustUnderstandChecker.invoke(MustUnderstandChecker.java:62)
at org.apache.axis.client.AxisClient.invoke(AxisClient.java:206)
at org.apache.axis.client.Call.invokeEngine(Call.java:2784)
at org.apache.axis.client.Call.invoke(Call.java:2767)
at org.apache.axis.client.Call.invoke(Call.java:2443)
at org.apache.axis.client.Call.invoke(Call.java:2366)
at org.apache.axis.client.Call.invoke(Call.java:1812)
at com.opshub.tfs.test.BasicHttpBinding_ITFSWebServiceStub.checkinAllStatus(BasicHttpBinding_ITFSWebServiceStub.java:6282)
at com.opshub.eai.tfssource.adapter.TFSVersionControlConnector.checkInAll(TFSVersionControlConnector.java:155)
... 41 more
Upgrading to the newest version solved my problem. The failing version was 2.2.4 and the new version is 2.2.5.