I want to try setup the development environment on Windows with Eclipse. But failed at the final step after successfully setting project id and gs bucket. The errors seems related to network (I have set proxy in Eclipse), I guess Maven need to set proxy, but how? Can someone confirm it? Thanks.
"Error encountered when trying to create project"
java.lang.reflect.InvocationTargetException
at org.eclipse.jface.operation.ModalContext.run(ModalContext.java:423)
at org.eclipse.jface.wizard.WizardDialog.run(WizardDialog.java:1059)
at com.google.cloud.dataflow.eclipse.ui.wizard.NewDataflowProjectWizard.performFinish(NewDataflowProjectWizard.java:50)
at org.eclipse.jface.wizard.WizardDialog.finishPressed(WizardDialog.java:853)
at org.eclipse.jface.wizard.WizardDialog.buttonPressed(WizardDialog.java:438)
at org.eclipse.jface.dialogs.Dialog$2.widgetSelected(Dialog.java:619)
at org.eclipse.swt.widgets.TypedListener.handleEvent(TypedListener.java:248)
at org.eclipse.swt.widgets.EventTable.sendEvent(EventTable.java:84)
at org.eclipse.swt.widgets.Display.sendEvent(Display.java:4353)
at org.eclipse.swt.widgets.Widget.sendEvent(Widget.java:1061)
at org.eclipse.swt.widgets.Display.runDeferredEvents(Display.java:4172)
at org.eclipse.swt.widgets.Display.readAndDispatch(Display.java:3761)
at org.eclipse.jface.window.Window.runEventLoop(Window.java:832)
at org.eclipse.jface.window.Window.open(Window.java:808)
at org.eclipse.ui.actions.NewProjectAction.run(NewProjectAction.java:117)
at org.eclipse.jface.action.Action.runWithEvent(Action.java:519)
at org.eclipse.jface.action.ActionContributionItem.handleWidgetSelection(ActionContributionItem.java:595)
at org.eclipse.jface.action.ActionContributionItem.access$2(ActionContributionItem.java:511)
at org.eclipse.jface.action.ActionContributionItem$5.handleEvent(ActionContributionItem.java:420)
at org.eclipse.swt.widgets.EventTable.sendEvent(EventTable.java:84)
at org.eclipse.swt.widgets.Display.sendEvent(Display.java:4353)
at org.eclipse.swt.widgets.Widget.sendEvent(Widget.java:1061)
at org.eclipse.swt.widgets.Display.runDeferredEvents(Display.java:4172)
at org.eclipse.swt.widgets.Display.readAndDispatch(Display.java:3761)
at org.eclipse.e4.ui.internal.workbench.swt.PartRenderingEngine$9.run(PartRenderingEngine.java:1151)
at org.eclipse.core.databinding.observable.Realm.runWithDefault(Realm.java:332)
at org.eclipse.e4.ui.internal.workbench.swt.PartRenderingEngine.run(PartRenderingEngine.java:1032)
at org.eclipse.e4.ui.internal.workbench.E4Workbench.createAndRunUI(E4Workbench.java:148)
at org.eclipse.ui.internal.Workbench$5.run(Workbench.java:636)
at org.eclipse.core.databinding.observable.Realm.runWithDefault(Realm.java:332)
at org.eclipse.ui.internal.Workbench.createAndRunWorkbench(Workbench.java:579)
at org.eclipse.ui.PlatformUI.createAndRunWorkbench(PlatformUI.java:150)
at org.eclipse.ui.internal.ide.application.IDEApplication.start(IDEApplication.java:135)
at org.eclipse.equinox.internal.app.EclipseAppHandle.run(EclipseAppHandle.java:196)
at org.eclipse.core.runtime.internal.adaptor.EclipseAppLauncher.runApplication(EclipseAppLauncher.java:134)
at org.eclipse.core.runtime.internal.adaptor.EclipseAppLauncher.start(EclipseAppLauncher.java:104)
at org.eclipse.core.runtime.adaptor.EclipseStarter.run(EclipseStarter.java:380)
at org.eclipse.core.runtime.adaptor.EclipseStarter.run(EclipseStarter.java:235)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
at java.lang.reflect.Method.invoke(Unknown Source)
at org.eclipse.equinox.launcher.Main.invokeFramework(Main.java:648)
at org.eclipse.equinox.launcher.Main.basicRun(Main.java:603)
at org.eclipse.equinox.launcher.Main.run(Main.java:1465)
Caused by: java.lang.IllegalStateException: org.eclipse.core.runtime.CoreException: Could not resolve artifact com.google.cloud.dataflow:google-cloud-dataflow-java-archetypes-examples:pom:LATEST
at com.google.cloud.dataflow.eclipse.core.project.DataflowProjectCreator.run(DataflowProjectCreator.java:207)
at org.eclipse.jface.operation.ModalContext$ModalContextThread.run(ModalContext.java:122)
Caused by: org.eclipse.core.runtime.CoreException: Could not resolve artifact com.google.cloud.dataflow:google-cloud-dataflow-java-archetypes-examples:pom:LATEST
at org.eclipse.m2e.core.internal.embedder.MavenImpl$5.call(MavenImpl.java:776)
at org.eclipse.m2e.core.internal.embedder.MavenImpl$5.call(MavenImpl.java:1)
at org.eclipse.m2e.core.internal.embedder.MavenExecutionContext.executeBare(MavenExecutionContext.java:176)
at org.eclipse.m2e.core.internal.embedder.MavenExecutionContext.execute(MavenExecutionContext.java:151)
at org.eclipse.m2e.core.internal.embedder.MavenExecutionContext.execute(MavenExecutionContext.java:99)
at org.eclipse.m2e.core.internal.embedder.MavenImpl.resolve(MavenImpl.java:743)
at org.eclipse.m2e.core.internal.embedder.MavenImpl.resolve(MavenImpl.java:720)
at com.google.cloud.dataflow.eclipse.core.project.DataflowArtifactRetriever.archetypePom(DataflowArtifactRetriever.java:54)
at com.google.cloud.dataflow.eclipse.core.project.DataflowProjectCreator.defaultArchetypeVersions(DataflowProjectCreator.java:250)
at com.google.cloud.dataflow.eclipse.core.project.DataflowProjectCreator.run(DataflowProjectCreator.java:204)
... 1 more
"Couldn't get archetype versions to use as default"
org.eclipse.core.runtime.CoreException: Could not resolve artifact com.google.cloud.dataflow:google-cloud-dataflow-java-archetypes-examples:pom:LATEST
at org.eclipse.m2e.core.internal.embedder.MavenImpl$5.call(MavenImpl.java:776)
at org.eclipse.m2e.core.internal.embedder.MavenImpl$5.call(MavenImpl.java:1)
at org.eclipse.m2e.core.internal.embedder.MavenExecutionContext.executeBare(MavenExecutionContext.java:176)
at org.eclipse.m2e.core.internal.embedder.MavenExecutionContext.execute(MavenExecutionContext.java:151)
at org.eclipse.m2e.core.internal.embedder.MavenExecutionContext.execute(MavenExecutionContext.java:99)
at org.eclipse.m2e.core.internal.embedder.MavenImpl.resolve(MavenImpl.java:743)
at org.eclipse.m2e.core.internal.embedder.MavenImpl.resolve(MavenImpl.java:720)
at com.google.cloud.dataflow.eclipse.core.project.DataflowArtifactRetriever.archetypePom(DataflowArtifactRetriever.java:54)
at com.google.cloud.dataflow.eclipse.core.project.DataflowProjectCreator.defaultArchetypeVersions(DataflowProjectCreator.java:250)
at com.google.cloud.dataflow.eclipse.core.project.DataflowProjectCreator.run(DataflowProjectCreator.java:204)
at org.eclipse.jface.operation.ModalContext$ModalContextThread.run(ModalContext.java:122)
Contains: Failed to resolve version for com.google.cloud.dataflow:google-cloud-dataflow-java-archetypes-examples:pom:LATEST: Could not find metadata com.google.cloud.dataflow:google-cloud-dataflow-java-archetypes-examples/maven-metadata.xml in local (C:\Users\ztang16\.m2\repository)
org.eclipse.aether.resolution.VersionResolutionException: Failed to resolve version for com.google.cloud.dataflow:google-cloud-dataflow-java-archetypes-examples:pom:LATEST: Could not find metadata com.google.cloud.dataflow:google-cloud-dataflow-java-archetypes-examples/maven-metadata.xml in local (C:\Users\ztang16\.m2\repository)
at org.apache.maven.repository.internal.DefaultVersionResolver.resolveVersion(DefaultVersionResolver.java:313)
at org.eclipse.aether.internal.impl.DefaultArtifactResolver.resolve(DefaultArtifactResolver.java:302)
at org.eclipse.aether.internal.impl.DefaultArtifactResolver.resolveArtifacts(DefaultArtifactResolver.java:246)
at org.eclipse.aether.internal.impl.DefaultArtifactResolver.resolveArtifact(DefaultArtifactResolver.java:223)
at org.eclipse.aether.internal.impl.DefaultRepositorySystem.resolveArtifact(DefaultRepositorySystem.java:294)
at org.eclipse.m2e.core.internal.embedder.MavenImpl$5.call(MavenImpl.java:753)
at org.eclipse.m2e.core.internal.embedder.MavenImpl$5.call(MavenImpl.java:1)
at org.eclipse.m2e.core.internal.embedder.MavenExecutionContext.executeBare(MavenExecutionContext.java:176)
at org.eclipse.m2e.core.internal.embedder.MavenExecutionContext.execute(MavenExecutionContext.java:151)
at org.eclipse.m2e.core.internal.embedder.MavenExecutionContext.execute(MavenExecutionContext.java:99)
at org.eclipse.m2e.core.internal.embedder.MavenImpl.resolve(MavenImpl.java:743)
at org.eclipse.m2e.core.internal.embedder.MavenImpl.resolve(MavenImpl.java:720)
at com.google.cloud.dataflow.eclipse.core.project.DataflowArtifactRetriever.archetypePom(DataflowArtifactRetriever.java:54)
at com.google.cloud.dataflow.eclipse.core.project.DataflowProjectCreator.defaultArchetypeVersions(DataflowProjectCreator.java:250)
at com.google.cloud.dataflow.eclipse.core.project.DataflowProjectCreator.run(DataflowProjectCreator.java:204)
at org.eclipse.jface.operation.ModalContext$ModalContextThread.run(ModalContext.java:122)
Caused by: org.eclipse.aether.transfer.MetadataNotFoundException: Could not find metadata com.google.cloud.dataflow:google-cloud-dataflow-java-archetypes-examples/maven-metadata.xml in local (C:\Users\ztang16\.m2\repository)
at org.eclipse.aether.internal.impl.DefaultMetadataResolver.resolve(DefaultMetadataResolver.java:247)
at org.eclipse.aether.internal.impl.DefaultMetadataResolver.resolveMetadata(DefaultMetadataResolver.java:205)
at org.apache.maven.repository.internal.DefaultVersionResolver.resolveVersion(DefaultVersionResolver.java:250)
... 15 more
You seem to be hitting an issue when resolving the latest version of our Maven archetype for the examples project. The most inner exception seems to be:
org.eclipse.aether.transfer.MetadataNotFoundException:
Could not find metadata com.google.cloud.dataflow:google-cloud-dataflow-java-archetypes-examples/maven-metadata.xml
in local (C:\Users\YOUR-USERNAME\.m2\repository)
This indicates a problem in your local repository.
We have not seen this specific error before, but tangentially related problems have been resolved by manually deleting the following directory on your local machine and retrying.
C:\Users\YOUR-USERNAME\.m2\repository\com\google\cloud\dataflow\dataflow:google-cloud-dataflow-java-archetypes-examples
This would force Maven to reinitialize that part of the local repository, and perhaps recreate the missing metadata file.
That said, we have slightly changed how our Eclipse plugin interacts with the m2e plugin. The new code doesn't take this path through m2e at all. It is quite likely that this would be a non-issue with an updated code. We tentatively plan to release a new version next week, which would contain this change.
Now, there also could be an issue with a HTTP proxy configuration in your environment. I found this StackOverflow question, which explains how to configure advanced options of the M2Eclipse plugin, including the proxy settings. Dataflow Eclipse plugin is built on top of M2Eclipse and I would expect those setting to apply automatically.
Related
I have problem uploading the findings of minIO securecodebox outputs to OWASP DefectDojo.
Screenshot of Error
https://drive.google.com/file/d/1PqVOazjr7r_1oMPf6SQsh8_iPFgnqkjC/view?usp=sharing
I try following these steps
https://github.com/DefectDojo/django-DefectDojo/blob/dev/readme-docs/KUBERNETES.md
then
https://docs.securecodebox.io/docs/hooks/defectdojo/
This is the link for the scanners
https://github.com/secureCodeBox/secureCodeBox/tree/main/scanners
The Error:
2022-03-07 07:23:54 INFO DefectDojoPersistenceProvider:35 - Downloading Scan Result ence provider
2022-03-07 07:23:56 INFO DefectDojoPersistenceProvider:39 - Uploading Findings to DefectDojo at: http://defectdojo.default.minikube.local:8080/ tDojo at: http://defectdojo.default.minikube.local:8080/
Exception in thread "main" org.springframework.web.client.ResourceAccessException: I/O error on GET request for "http://defectdojo.default.minikube.locarror on GET request for "http://defectdojo.default.minikube.local:8080/api/v2/users/": defectdojo.default.minikube.local; nested exception is java.net.UnknownHostException: defectdojo.default.minikube.local
at org.springframework.web.client.RestTemplate.doExecute(RestTemplate.java:785)
at org.springframework.web.client.RestTemplate.execute(RestTemplate.java:751) rnalSearch(GenericDefectDojoService.java:151)
at org.springframework.web.client.RestTemplate.exchange(RestTemplate.java:621) ch(GenericDefectDojoService.java:167)
at io.securecodebox.persistence.defectdojo.service.GenericDefectDojoService.intechUnique(GenericDefectDojoService.java:187)rnalSearch(GenericDefectDojoService.java:151) ionedEngagementsStrategy.java:82)
at io.securecodebox.persistence.defectdojo.service.GenericDefectDojoService.search(GenericDefectDojoService.java:167)
at io.securecodebox.persistence.defectdojo.service.GenericDefectDojoService.searchUnique(GenericDefectDojoService.java:187)
at io.securecodebox.persistence.strategies.VersionedEngagementsStrategy.run(VersionedEngagementsStrategy.java:82)
at io.securecodebox.persistence.DefectDojoPersistenceProvider.main(DefectDojoPersistenceProvider.java:42)
Caused by: java.net.UnknownHostException: defectdojo.default.minikube.local
at java.base/java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:229)
at java.base/java.net.Socket.connect(Socket.java:609)
at java.base/java.net.Socket.connect(Socket.java:558)
at java.base/sun.net.NetworkClient.doConnect(NetworkClient.java:182)
at java.base/sun.net.www.http.HttpClient.openServer(HttpClient.java:474)
at java.base/sun.net.www.http.HttpClient.openServer(HttpClient.java:569)
at java.base/sun.net.www.http.HttpClient.(HttpClient.java:242)
at java.base/sun.net.www.http.HttpClient.New(HttpClient.java:341)
at java.base/sun.net.www.http.HttpClient.New(HttpClient.java:362)
at java.base/sun.net.www.protocol.http.HttpURLConnection.getNewHttpClient(HttpURLConnection.java:1253)
at java.base/sun.net.www.protocol.http.HttpURLConnection.plainConnect0(HttpURLConnection.java:1187)
at java.base/sun.net.www.protocol.http.HttpURLConnection.plainConnect(HttpURLConnection.java:1081)
at java.base/sun.net.www.protocol.http.HttpURLConnection.connect(HttpURLConnection.java:1015)
at org.springframework.http.client.SimpleBufferingClientHttpRequest.executeInternal(SimpleBufferingClientHttpRequest.java:76)
at org.springframework.http.client.AbstractBufferingClientHttpRequest.executeInternal(AbstractBufferingClientHttpRequest.java:48)
at org.springframework.http.client.AbstractClientHttpRequest.execute(AbstractClientHttpRequest.java:66)
at org.springframework.web.client.RestTemplate.doExecute(RestTemplate.java:776)
... 7 more
Thank you for the reponse!
there is a dedicated DefectDojo Hook which will do it for you.
You just need to install in on a cluster with some basic configuration.
Installing the DefectDojo persistenceProvider hook will add a ReadAndWrite Hook to your namespace.
kubectl create secret generic defectdojo-credentials --from-literal="username=admin" --from-literal="apikey=08b7..."
helm upgrade --install dd secureCodeBox/persistence-defectdojo
--set="defectdojo.url=https://defectdojo-django.default.svc"
The hook will automatically import the scan results into an engagement in DefectDojo. If the engagement doesn't exist the hook will create the engagement (CI/CD engagement) and all objects required for it (product & product type). The hook will then pull the imported information from DefectDojo and use them to replace the findings inside secureCodeBox.
More https://docs.securecodebox.io/docs/hooks/defectdojo
I have WebSphere 8.5.5.9 and want to run Jenkins on it, because of that, I have installed jenkins.war (the last version 2.19.2), but when I try to start it, I get the error bellow and the application(Jenkins) doesn't start.
OS: Red Hat Enterprise Linux Server release 6.7 (Santiago) - 64 bits
[11/14/16 14:34:55:824 EST] 0000007c BootFailure E Failed to initialize Jenkins
hudson.util.HudsonFailedToLoad: java.lang.NullPointerException
at hudson.WebAppMain$3.run(WebAppMain.java:248)
Caused by: java.lang.NullPointerException
at com.ibm.ws.webcontainer.webapp.WebApp.getInitParameter(WebApp.java:2791)
at com.ibm.wsspi.webcontainer.facade.ServletContextFacade.getInitParameter(ServletContextFacade.java:264)
at jenkins.util.SystemProperties.tryGetValueFromContext(SystemProperties.java:322)
at jenkins.util.SystemProperties.getString(SystemProperties.java:121)
at jenkins.util.SystemProperties.getInteger(SystemProperties.java:258)
at jenkins.model.Jenkins.<init>(Jenkins.java:599)
at hudson.model.Hudson.<init>(Hudson.java:85)
at hudson.model.Hudson.<init>(Hudson.java:81)
at hudson.WebAppMain$3.run(WebAppMain.java:231)
Does somebody have any idea what could be the rouse cause?
Thanks
Looking into other log from WebSphere I found an erro like this:
restricted programmatic session cookie
looking new solutions to this new error, I found a solution:
Just remove on item from programmatic session cookie configuration
Cookie Domain:*
Cookie Name: "JSESSIONID"
Cookie Path: /
from:
Security > Global security > Programmatic session cookie configuration.
After that, I could start Jenkins with success.
I'm trying to deploy wlapp file in MFP Liberty Server using ant task.
I created the build.xml then run "ant" in cmd.
C:\IBM\MobileFirst_Platform_Server\shortcuts>ant
Unable to locate tools.jar. Expected to find it in C:\IBM\Installation Manager\e
clipse\jre_7.0.9030.20160210_1426\lib\tools.jar
Buildfile: C:\IBM\MobileFirst_Platform_Server\shortcuts\build.xml
install:
[wladm] The REST service URL http://IP:9080/worklightconsol
e cannot be accessed in a secure way. You may want to use a https URL instead, o
r retry with the 'secure' option set to 'false'.
BUILD FAILED
C:\IBM\MobileFirst_Platform_Server\shortcuts\build.xml:9: com.ibm.worklight.admi
n.restclient.RESTException: The REST service URL http://IP:9080
/worklightconsole cannot be accessed in a secure way. You may want to use a http
s URL instead, or retry with the 'secure' option set to 'false'.
at com.ibm.worklight.admin.restclient.RESTClient.checkURI(RESTClient.jav
a:1001)
at com.ibm.worklight.admin.restclient.RESTClient.getPOSTResponse(RESTCli
ent.java:1433)
at com.ibm.worklight.admin.restclient.RESTClient.getPOSTFileResponse(RES
TClient.java:1464)
at com.ibm.worklight.admin.commands.DeployApp.getResponse(DeployApp.java
:41)
at com.ibm.worklight.admin.restclient.ActionClient.execute(ActionClient.
java:84)
at com.ibm.worklight.admin.ant.types.AbstractActionElement.executeComman
d(AbstractActionElement.java:76)
at com.ibm.worklight.admin.ant.types.ActionElement.executeCommands(Actio
nElement.java:43)
at com.ibm.worklight.admin.ant.WladmTask.executeCommands(WladmTask.java:
705)
at com.ibm.worklight.admin.ant.WladmTask.execute(WladmTask.java:459)
at org.apache.tools.ant.UnknownElement.execute(UnknownElement.java:291)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.
java:95)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAcces
sorImpl.java:56)
at java.lang.reflect.Method.invoke(Method.java:620)
at org.apache.tools.ant.dispatch.DispatchUtils.execute(DispatchUtils.jav
a:106)
at org.apache.tools.ant.Task.perform(Task.java:348)
at org.apache.tools.ant.Target.execute(Target.java:392)
at org.apache.tools.ant.Target.performTasks(Target.java:413)
at org.apache.tools.ant.Project.executeSortedTargets(Project.java:1399)
at org.apache.tools.ant.Project.executeTarget(Project.java:1368)
at org.apache.tools.ant.helper.DefaultExecutor.executeTargets(DefaultExe
cutor.java:41)
at org.apache.tools.ant.Project.executeTargets(Project.java:1251)
at org.apache.tools.ant.Main.runBuild(Main.java:811)
at org.apache.tools.ant.Main.startAnt(Main.java:217)
at org.apache.tools.ant.launch.Launcher.run(Launcher.java:280)
at org.apache.tools.ant.launch.Launcher.main(Launcher.java:109)
Total time: 1 second
I don't know where to set the "secure to false" but I tried this one:
wladm --url= --user= ... [--passwordfile=...] [--secure=false]
It says:
Secure mode is enabled, but an option '--passwordfile' is missing.
Anyone know where to change the secure to false since the url is just http and not https?
I don't think you're supposed to keep the square brackets in the command.
Try without. Simply --secure=false.
For more details, please refer to the wladm usage documentation at http://www.ibm.com/support/knowledgecenter/SSHS8R_7.1.0/com.ibm.worklight.appadmin.doc/admin/r_invoking_the_wladm_program.html.
Downloaded Galago 3.5 bin version and tried to index wiki-small.corpus following this guide. Strangely I get a File Not Found Exception for the .index file when trying to run the build index command. This error goes away when I explicitly use the inputPath and indexPath but instead now I get this exception -
Created executor: org.lemurproject.galago.tupleflow.execution.LocalCheckpointedStageExecutor#69107c05
Running without server!
Use --server=true to enable web-based status page.
Stage inputSplit completed with 0 errors.
Mar 14, 2014 3:26:01 PM org.lemurproject.galago.core.parse.UniversalParser process
INFO: Processing split: /Users/nanz/Downloads/wiki-small.corpus
java.lang.RuntimeException: java.lang.reflect.InvocationTargetException
at org.lemurproject.galago.core.parse.UniversalParser.process(UniversalParser.java:137)
at org.lemurproject.galago.core.parse.UniversalParser.process(UniversalParser.java:52)
at org.lemurproject.galago.core.types.DocumentSplit$FileIdOrder$TupleUnshredder.processTuple(DocumentSplit.java:2033)
at org.lemurproject.galago.core.types.DocumentSplit$FileIdOrder$DuplicateEliminator.processTuple(DocumentSplit.java:1989)
at org.lemurproject.galago.core.types.DocumentSplit$FileIdOrder$ShreddedBuffer.copyTuples(DocumentSplit.java:1705)
at org.lemurproject.galago.core.types.DocumentSplit$FileIdOrder$ShreddedBuffer.copyUntilFileId(DocumentSplit.java:1732)
at org.lemurproject.galago.core.types.DocumentSplit$FileIdOrder$ShreddedBuffer.copyUntil(DocumentSplit.java:1740)
at org.lemurproject.galago.core.types.DocumentSplit$FileIdOrder$ShreddedReader.run(DocumentSplit.java:1940)
at org.lemurproject.galago.tupleflow.FileOrderedReader.run(FileOrderedReader.java:76)
at org.lemurproject.galago.tupleflow.execution.LocalCheckpointedStageExecutor$LocalExecutionStatus.run(LocalCheckpointedStageExecutor.java:96)
at java.lang.Thread.run(Thread.java:695)
Caused by: java.lang.reflect.InvocationTargetException
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:39)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27)
at java.lang.reflect.Constructor.newInstance(Constructor.java:513)
at org.lemurproject.galago.core.parse.UniversalParser.constructParserWithSplit(UniversalParser.java:213)
at org.lemurproject.galago.core.parse.UniversalParser.process(UniversalParser.java:132)
... 10 more
Caused by: java.lang.NullPointerException
at org.lemurproject.galago.core.index.KeyValueReader.getManifest(KeyValueReader.java:35)
at org.lemurproject.galago.core.index.corpus.CorpusReader.init(CorpusReader.java:41)
at org.lemurproject.galago.core.index.corpus.CorpusReader.(CorpusReader.java:32)
at org.lemurproject.galago.core.parse.CorpusSplitParser.(CorpusSplitParser.java:33)
... 16 more
Stage parsePostings completed with 1 errors.
java.lang.Exception: java.lang.RuntimeException: java.lang.reflect.InvocationTargetException
Exception in thread "main" java.util.concurrent.ExecutionException: Stage threw an exception:
at org.lemurproject.galago.tupleflow.execution.JobExecutor$JobExecutionStatus.waitForStages(JobExecutor.java:1062)
at org.lemurproject.galago.tupleflow.execution.JobExecutor$JobExecutionStatus.run(JobExecutor.java:971)
at org.lemurproject.galago.tupleflow.execution.JobExecutor.runWithoutServer(JobExecutor.java:1122)
at org.lemurproject.galago.tupleflow.execution.JobExecutor.runLocally(JobExecutor.java:1177)
at org.lemurproject.galago.core.tools.AppFunction.runTupleFlowJob(AppFunction.java:101)
at org.lemurproject.galago.core.tools.apps.BuildIndex.run(BuildIndex.java:789)
at org.lemurproject.galago.core.tools.AppFunction.run(AppFunction.java:55)
at org.lemurproject.galago.core.tools.App.run(App.java:82)
at org.lemurproject.galago.core.tools.App.run(App.java:73)
at org.lemurproject.galago.core.tools.App.main(App.java:69)
Caused by: java.lang.Exception: java.lang.RuntimeException: java.lang.reflect.InvocationTargetException
at org.lemurproject.galago.tupleflow.execution.LocalCheckpointedStageExecutor$LocalExecutionStatus.run(LocalCheckpointedStageExecutor.java:99)
at java.lang.Thread.run(Thread.java:695)
I tried building the source code and I got the same results in that case as well. Can somebody point out where I am going wrong ? Hardly anybody seems to have faced this issue so there's not much I get via a simple Google search.
Solved. Just in case someone else faces this issue, one of my friends figured it out that Galago would not work directly on the wiki-small.corpus file as it tries to look for corpus.keys which do not exist for this. Just replace this .corpus file instead with the directory of documents and everything will work just fine. Do specify the indexPath and inputPath parameters explicitly. Use "galago build help" to view the exact syntax. Cheers.
I know this is late, but the wiki-small.corpus file from the textbook's website was built with an old version of galago, namely the 1.0 series, which is preserved in this google code repository: https://code.google.com/p/galagosearch/
The newer releases of Galago (2.0 ... 3.5 ...3.7) are part of newer development under the Lemur Project on sourceforge, and the corpus format has since changed. If you had a corpus file built with Galago 3.5, your commands should have worked.
I've setup a file to file source/sink , just as a test of basic flume functionality.
Im currently using the "exec" source, with the command being "tail -F mytmpfile".
In my script, I continuously echo "....." >> mytmpfile , so that the tail command constitutes a stream.
However, I've started seeing the following exception in the flume logs:
java.lang. IllegalStateException: Channel closed [channel=c1]. Due to
java.lang.NullPointerException: null
at org.apache.flume.channel.file.FileChannel.createTransaction(FileChannel.java:353)
at org.apache.flume.channel.BasicChannelSemantics.getTransaction(BasicChannelSemantics.java:122)
at org.apache.flume.sink.RollingFileSink.process(RollingFileSink.java:183)
at org.apache.flume.sink.DefaultSinkProcessor.process(DefaultSinkProcessor.java:68)
at org.apache.flume.SinkRunner$PollingRunner.run(SinkRunner.java:147)
at java.lang.Thread.run(Thread.java:662) Caused by: java.lang.NullPointerException
at org.apache.flume.channel.file.Log.writeCheckpoint(Log.java:895)
at org.apache.flume.channel.file.Log.replay(Log.java:406)
at org.apache.flume.channel.file.FileChannel.start(FileChannel.java:303)
at org.apache.flume.lifecycle.LifecycleSupervisor$MonitorRunnable.run(LifecycleSupervisor.java:236)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:439)
at java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:317)
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:150)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:98)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.runPeriodic(ScheduledThreadPoolExecutor.java:180)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:204)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:895)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:918)
... 1 more
Any thoughts on where this NullPointerException is coming from? It appears from scanning the code that maybe it related to a missing folder or directory. But I cant find the exact line on the git hub branches.
This is using apache-flume-1.3.1.23-...
In the past I've had problems with file channels, and they've normally boiled down to two problems:
1) If you're running multiple agents on the same box, make sure you configure them to have separate dataDirs and checkpointDir.
2) On Linux boxes, check that your tmpfs isn't near its capacity. If it's getting full, flume will complain. Try stopping the flume agent, unmount tmpfs, enlarge it, remount and restart the agent.