When I run the jar in the GCE, it had the following error:
java -jar mySimple.jar --project=myProjcet
Aug 13, 2015 1:22:26 AM com.google.cloud.dataflow.sdk.runners.DataflowPipelineRunner detectClassPathResourcesToStage
SEVERE: Unable to convert url (rsrc:./) to file.
Aug 13, 2015 1:22:26 AM simple.SimpleV1 main
SEVERE: Failed to construct instance from factory method com.google.cloud.dataflow.sdk.runners.BlockingDataflowPipelineRunner#fromOptions
I am working on Eclipse(window). And it succeeded to run dataflow through the eclipse. Packaging the project to Runable jar file and uploaded to the GCE (ubuntu). And i had errors when i run the jar file on the GCE(ubuntu).
the runner is BlockingDataflowPipelineRunner(batch mode).
there are other options in source code.
the follow is manifest.
Manifest-Version: 1.0
Rsrc-Class-Path: ./ httpclient-4.3.6.jar httpcore-4.3.3.jar commons-lo
gging-1.1.3.jar commons-codec-1.6.jar mybatis-3.2.8.jar mysql-connect
or-java-5.1.34.jar ibatis2-common-2.1.7.597.jar ibatis2-dao-2.1.7.597
.jar ibatis2-sqlmap-2.1.7.597.jar geoip-api-1.2.14.jar google-api-cli
ent-java6-1.20.0.jar google-api-client-1.20.0.jar google-oauth-client
-1.20.0.jar guava-jdk5-13.0.jar google-oauth-client-java6-1.20.0.jar
google-oauth-client-jetty-1.20.0.jar jetty-6.1.26.jar jetty-util-6.1.
26.jar servlet-api-2.5-20081211.jar google-http-client-jackson2-1.20.
0.jar google-http-client-1.20.0.jar jsr305-1.3.9.jar joda-time-2.8.1.
jar slf4j-api-1.7.7.jar slf4j-jdk14-1.7.7.jar commons-csv-1.1.jar aws
-java-sdk-sqs-1.10.5.1.jar aws-java-sdk-core-1.10.5.1.jar google-clou
d-dataflow-java-sdk-all-0.4.150710.jar google-api-services-dataflow-v
1b3-rev4-1.19.1.jar google-cloud-dataflow-java-proto-library-all-0.4.
150612.jar protobuf-java-2.5.0.jar google-api-services-bigquery-v2-re
v187-1.19.1.jar google-api-services-compute-v1-rev46-1.19.1.jar googl
e-api-services-pubsub-v1beta2-rev1-1.19.1.jar google-api-services-sto
rage-v1-rev25-1.19.1.jar google-api-services-datastore-protobuf-v1bet
a2-rev1-2.1.2.jar google-http-client-protobuf-1.15.0-rc.jar google-ht
tp-client-jackson-1.15.0-rc.jar jackson-annotations-2.4.2.jar jackson
-databind-2.4.2.jar avro-1.7.7.jar jackson-core-asl-1.9.13.jar jackso
n-mapper-asl-1.9.13.jar paranamer-2.3.jar snappy-java-1.0.5.jar commo
ns-compress-1.9.jar jetty-server-9.2.10.v20150310.jar javax.servlet-a
pi-3.1.0.jar jetty-http-9.2.10.v20150310.jar jetty-io-9.2.10.v2015031
0.jar jetty-jmx-9.2.10.v20150310.jar jetty-util-9.2.10.v20150310.jar
jackson-core-2.6.0.jar
Class-Path: .
Rsrc-Main-Class: simple.SimpleV1
Main-Class: org.eclipse.jdt.internal.jarinjarloader.JarRsrcLoader
When exporting a Runnable JAR file using Eclipse, there are three ways to package your project:
Extract required libraries into generated JAR
Package required libraries into generated JAR
Copy required libraries into a sub-folder next to the generated JAR
All 3 options, have the same usage pattern when executing, e.g.
java -jar myrunnable.jar --myCommandLineOption1=...
Currently, only option 1 is compatible with how the Dataflow SDK for Java is able to detect resources to stage because it is dependent on them being file URIs from a URLClassLoader.
For an explanation of how the Runnable Jars are created and more specific details of why this was problematic, read further below.
An alternative solution to using the Runnable Jars, is to execute your project using mvn exec.
Option 1
This creates a jar which copies all the class files & resources in each individual jar into a single jar. This allows for a manifest where the entire classpath is composed of file based URIs:
Manifest-Version: 1.0
Main-Class: com.google.cloud.dataflow.starter.StarterPipeline
Class-Path: .
Option 2
This creates a jar file with additional jars embedded within it. It uses a custom main entry point (org.eclipse.jdt.internal.jarinjarloader.JarRsrcLoader) which knows how to read the custom manifest entries (Rsrc-Class-Path & Rsrc-Main-Class) and creates a classloader with non file based URIs. Since the Dataflow SDK for Java currently only knows how to handle file based resources and doesn't know how to interpret the rsrc:... URIs, you get the exception that your seeing.
Manifest-Version: 1.0
Rsrc-Class-Path: ./ httpclient-4.3.6.jar ...
Class-Path: .
Rsrc-Main-Class: simple.SimpleV1
Main-Class: org.eclipse.jdt.internal.jarinjarloader.JarRsrcLoader
Option 3
This creates a jar file which contains your project resources and then creates a folder along side the runnable jar containing all your projects dependent jars. This allows for a more complex standard manifest listing all your project dependencies.
Manifest-Version: 1.0
Main-Class: com.google.cloud.dataflow.starter.StarterPipeline
Class-Path: . runnable_lib/google-cloud-dataflow-java-sdk-all-manual_build.jar ...
The Class-Path manifest is not returned part of the URLClassLoader and hence these classes are not discoverable. Furthermore, those jars are only meant to be loaded by classes from that jar which can lead to a jar loading hierarchy. More details are available here: http://docs.oracle.com/javase/7/docs/technotes/tools/findingclasses.html
Related
I would like to:
use the Frege programming language to write a simple "Hello World" piece of code,
then using the Frege compiler generating the equivalent Java source code,
then building an executable Jar file to run from the command line,
all the previous steps should be "controlled" by Gradle.
I am able to generate the source code (items 1. and 2. from the previous list), but I am not able to specify a "package" structure of the Java source code in output, i.e. I can not see the package Java statement as the first line of code in the generate Java source code. I can specify to the Frege compiler where to put the generated code though (via the -d argument).
I think this is the reason why when building an executable Jar, then launching it, I am seeing similar errors (according to different attempts on Gradle tasks) e.g.: no main manifest attribute.
The Frege source code is saved in a file named HelloFrege.fr, the generated Java source code is in a file named HelloFrege.java (I verified the file contains the expected main method).
Here there's a version of the Gradle "Jar task":
//create a single Jar with all dependencies
task fatJar(type: Jar) {
manifest {
attributes 'Implementation-Title': 'Hello Frege Jar Example',
'Implementation-Version': version,
'Main-Class': 'HelloFrege'
}
baseName = project.name + '-all'
from { configurations.compile.collect { it.isDirectory() ? it : zipTree(it) } }
with jar
}
Here there's another version of the Gradle "Jar" task:
jar {
manifest {
attributes 'Main-Class': 'HelloFrege'
}
}
How can I solve this problem? I would like to avoid to manually add the package reference to the automatically generated Java source code file.
If your module name in Frege is unqualified such as HelloWorld, you will not see the package statement generated in Java. The module name will become the class name and the package will be empty or default package.
If your module name is qualified such as foo.bar.HelloWorld, then foo.bar will be the package name and HelloWorld will be the class name in the generated Java source.
The rule is that the last part of the module name becomes the class name and the qualifiers form the package name in the generated Java source.
I am not sure what gradle can do for you in this regard, but without gradle, the following should at least be possible:
... build your jar, as before ...
jar -uvfe project-all.jar HelloFrege
java -jar project-all.jar # run it
This, of course, is just another way to create a manifest. If this works, then it would be time to investigate why gradle refuses to do it.
Postscriptum: After thinking another minute about what the problem might be, it occurs to me that you may think that the source file name/path has anything to do with the java package name. This is not so in Frege, though it is good practice to have the file path match the package name, and the file base name match the class name (just like in Java). In addition, to remove some confusion, use the module keyword in frege. As explained by Marimuthu, the Java package and class name is derived from the frege module name.
Example:
$ cat Foo.fr
module my.org.Baz where
...
$ java -jar fregec.jar -d bin Foo.fr
This generates the Baz class in package my.org, and creates the class file in bin/my/org/Baz.class
I am posting here my findings so far. The combination of Gradle commands that works for me is the following one (calling it from the command line typing gradle clean generateJavaSrcFromFregeSrc fatJar):
task generateJavaSrcFromFregeSrc {
ant.java(jar:"lib/frege3.21.586-g026e8d7.jar",fork:true) {
arg(value: "-j") // do not run the java compiler
arg(value: "-d")
arg(value: "src/main/java") // the place where to put the generated source code (paired to the -d argument)
arg(value: "src/main/frege/HelloFrege.fr")
}
}
jar {
manifest {
attributes 'Main-Class': 'org.wathever.HelloFrege'
}
}
task fatJar(type: Jar) {
from files(sourceSets.main.output.classesDir)
from files(sourceSets.main.output.resourcesDir)
//from {configurations.compile.collect {zipTree(it)}} // this does not include the autogenerated source code
baseName = project.name + '-fatJar'
from { configurations.compile.collect { it.isDirectory() ? it : zipTree(it) } }
with jar
}
The manifest details need to be specified in the jar block of code, if I specify them in the task fatJar then when running the jar I get no main manifest attribute, in [...].
If I use just the block of code jar with the property from("$projectDir") { include 'lib/**'} to include the Frege jar, then I get errors like java.lang.ClassNotFoundException (I think because the Jar is included as it is and not as a set of .class files).
The folder src/main/java/org/wathever needs to be there before running Gradle (additional info: the Maven convention prefix src/main/java with as a suffix the "Java package" as specified inside the HelloFrege.fr source code: module org.wathever.HelloFrege where)
Some useful details I found:
How to build a fat Jar
Another how to build a fat Jar
An "Hello Frege" example without the Gradle management
The Gradle documentation on how to use the Jar task
I am running into the following error when trying to run ant:
Problem: failed to create task or type runtarget
I am building on a mac 10.8.3.
Prior research has suggested that I add ant-contrib-0.3.jar to my ANT_HOME installation directory, which I have done (that had actually gotten rid of another 'failed to create task or type' error)
I used ant-contrib-0.3.jar because research suggested that this jar is mapped to the line:
< taskdef resource="net/sf/antcontrib/antcontrib.properties" />
which is in the build.xml file I am using.
The project builds on windows machines ( I even got it to build using https://code.google.com/p/winant/ ) but am trying to get it built on a mac. I am thus not looking to change the build.xml file.
An example of the run target line is:
<target name="setPASProps" depends="" description="setup the properties">
<property name="systemname" value="PAS"/>
<runtarget target="setSystemProps"/>
</target>
Here is some info from running ant -diagnostics
-------------------------------------------
ANT PROPERTIES
-------------------------------------------
ant.version: Apache Ant(TM) version 1.8.2 compiled on June 20 2012
ant.java.version: 1.7
Is this the Apache Harmony VM? no
Is this the Kaffe VM? no
Is this gij/gcj? no
ant.core.lib: /usr/share/ant/lib/ant.jar
ant.home: /usr/share/ant
-------------------------------------------
ANT_HOME/lib jar listing
-------------------------------------------
ant.home: /usr/share/ant
ant-antlr.jar (5756 bytes)
ant-contrib-0.3.jar (17708 bytes)
ant-jmf.jar (6745 bytes)
ant-junit.jar (102350 bytes)
ant-junit4.jar (7126 bytes)
ant-launcher.jar (12321 bytes)
ant-swing.jar (7563 bytes)
ant-testutil.jar (15198 bytes)
ant.jar (1937098 bytes)
Thanks !
It would be helpful if you posted your build.xml too.
You usually get this error if Ant sees a task, but there's a problem with the definition.
Here's my recommendation:
In your project create a directory antlib/ant-contrib.
Download this zip file. Ant-contrib is a wee bit strange is that there is a separate jar for C compiling and for all of the other Ant tasks. The latest version is 1.0b3. When you unzip this zip file, you will see ant-contrib-1.0b3.jar inside this folder.
Put that ant-contrib-1.0b3.jar inside the antlib/ant-contrib folder.
Now, in your build.xml, use the following <taskdef/>:
<taskdef resource="net/sf/antcontrib/antlib.xml">
<classpath>
<fileset dir="${basedir}/antlib/ant-contrib"/>
</classpath>
</taskdef>
Make sure this is not inside a target. You want this to be executed before any targets are executed.
I like doing the definition this way because the Ant Contrib jar file becomes part of my project, and anyone who needs to run my project will not have to manually install the Ant Contrib jar before they can use my project.
Take a look at your build.xml and see where that <Runtarget> task is being used. I've never used it, and the documentation for this task is so clear and helpful. Actually, I'm not even sure if it works. If you are still having problems, you try to see if you can remove the defined <target/> that contains this task, and see if that gets rid of the issue.
Removed ant-contrib-0.3.jar and added ant-contrib-1.0b3.jar and ant-contrib.jar to my ANT_HOME directory since these are the jars installed with https://code.google.com/p/winant/ (and it was working on windows machines).
This did the trick.
There is a webproject with a batchfile that generates all files needed on the targetserver and puts them in a folder "/Deployable" .
The batch file is quite involved because the project contains a pluginsystem and all plugins need to be copied to a certain location.
When I use webdeploy to deploy to the targetserver it happens what you expect: there are some of the needed assemblies copied over, but not the files as specified in the batchfile.
My plan is now to first execute the batchfile and then use webdeploy to copy the folder "/Deployable" to the targetserver. Can this be done with webdeploy?
This is what I see in Visual Studio deploy menue:
This is the resulting publish profile
<?xml version="1.0" encoding="utf-8"?> <publishData>
<publishProfile publishUrl="http://myserver/msdeployagentservice"
deleteExistingFiles="False"
ftpAnonymousLogin="False"
ftpPassiveMode="True"
msdeploySite="mysite/"
msdeploySiteID=""
msdeployRemoteSitePhysicalPath=""
msdeployAllowUntrustedCertificate="False"
msdeploySkipExtraFilesOnServer="False"
msdeployMarkAsApp="False"
profileName="publish_to_myserver"
publishMethod="MSDeploy"
replaceMatchingFiles="True"
userName="myuser"
savePWD="True" userPWD="xxx" SelectedForPublish="True" />
</publishData>>
I think there is an ability to add third-party files into webdeploy package by modifying .csproj file, however, I have never had to use it.
Alternatively, you can easily achieve the same result by using MSDeploy's command-line client and its sync verb, by specifying your /Deployable folder as the -source argument and your target server's msdeploy service as the -dest, e.g.:
$(WebDeployToolPath)\msdeploy -verb:sync -source:dirPath='Deployable\' -dest:dirPath='$(DeployDirectoryLocalPath)',computerName=$(DeployTargetURL),userName='$(DeployUserName)',password='$(Password)',authType='Basic' -verbose -allowUntrusted
Substitutions:
$(WebDeployToolPath) - full path to folder with msdeploy executable (e.g. c:\Program Files (x86)\IIS\Microsoft Web Deploy V3\msdeploy.exe )
Deployable\ - full path to the folder you wnat to copy to the target server.
$(DeployDirectoryLocalPath) - full local path to the target folder on the target server.
$(DeployTargetURL) - web deploy service URL (e.g. https://192.168.142.55:8172/MsDeploy.axd or http://myserver/msdeployagentservice)
$(DeployUserName) - username to be used for deployment (should be admin for Win 2003)
$(Password) - user's password.
That's it - this command will synchronize Deployable\ folder with $(DeployDirectoryLocalPath) folder (i.e. make the content exactly match).
You can wrap it into an msbuild target in your .csproj file:
<PropertyGroup>
<DeployTargetURL Condition="'$(DeployTargetURL)'==''">https://192.168.142.55:8172/MsDeploy.axd</DeployTargetURL>
<DeployUserName Condition="'$(DeployUserName)'==''">tergetServer\Administrator</DeployUserName>
<Password Condition="'$(Password)'==''">AdminPassword</Password>
<WebDeployToolPath Condition="'$(WebDeployToolPath)'==''">c:\Program Files (x86)\IIS\Microsoft Web Deploy V3\msdeploy.exe</WebDeployToolPath>
<Target Name="Deploy">
<Exec Command=""$(WebDeployToolPath)"\msdeploy -verb:sync -source:dirPath='Deployable\' -dest:dirPath='$(DeployDirectoryLocalPath)',computerName=$(DeployTargetURL),userName='$(DeployUserName)',password='$(Password)',authType='Basic' -verbose -allowUntrusted " />
</Target>
And than run it from the command-line in the following way:
%windir%\Microsoft.NET\Framework\v4.0.30319\MSBuild.exe MyProject.proj /t:Deploy*
I'm trying to create an OSGi wrapper for the newest version of jTDS. I'm trying to add the wrapping process to the existing jTDS build process (Ant-based). I've downloaded the latest bnd.jar and added the following to the jTDS build.xml:
<taskdef resource="aQute/bnd/ant/taskdef.properties" classpath="bnd.jar"/>
<bndwrap trace="true" definitions="${basedir}/bnd" output="${build}/${ant.project.name}-${version}.osgi.jar">
<fileset dir="${build}" includes="*.jar"/>
</bndwrap>
I've also got a very simple bnd definition defined:
version=1.2.6
Export-Package: net.sourceforge.jtds*;version=${version}
Bundle-Version: ${version}
Bundle-Name: net.sourceforge.jtds
When I execute the dist task in Ant, it should be creating a JAR with the proper OSGi manifest. It IS creating another JAR, but the manifest is identical to the original.
If I execute the same wrap directly against the bnd JAR:
java -jar bnd.jar wrap -p bnd\jtds-1.2.6.bnd -o build\jtds-1.2.6.osgi.jar build\jtds-1.2.6.jar
I get the correct OSGi manifest.
What is going wrong during the Ant build?
It seems to be a problem with the latest version of bnd, found here. The Ant WrapTask was retooled some and just doesn't seem to work (maybe it's just misconfigured; documentation hasn't kept up with code).
I dropped in version 1.50.0 instead and everything worked as expected both through the bnd.jar and through Ant.
I am working on a project that uses j2me polish and now i am receiving an error i.e Using Default Jar Packager
creating JAD file [D:\MobileDevelopment\risingkarachi5_21_2012\dist\BlackBerry-5.0-de_DE-risingkarachi.jad].
jdp: Reading template...
jdp: Collecting java classes from D:\MobileDevelopment\risingkarachi5_21_2012\build\real\BlackBerry\5.0\de_DE\source...
jdp: Collecting resources from D:\MobileDevelopment\risingkarachi5_21_2012\build\real\BlackBerry\5.0\de_DE\resources...
jdp: Writing files to project D:\MobileDevelopment\risingkarachi5_21_2012\build\real\BlackBerry\5.0\de_DE\J2ME_Polish_Beispiel.jdp...
jdp: Creating D:\MobileDevelopment\risingkarachi5_21_2012\build\real\BlackBerry\5.0\de_DE\J2ME_Polish_Beispiel.jdp
JDP: Note: project is obfuscated, the generated JDP will not contain the necessary subproject for the imported files.
jdp: Creating D:\MobileDevelopment\risingkarachi5_21_2012\build\real\BlackBerry\5.0\de_DE\J2ME_Polish_BeispielLIB.jdp
rapc: Converting jar to cod for device [BlackBerry/5.0]
rapc: Resolving
rapc: Optimizing
rapc: c: Error!: Data contribution too large: 435372
rapc-call: [java, -Xmx512m, -cp, C:\Program Files (x86)\Research In Motion\BlackBerry JDE 5.0.0\bin\rapc.jar, net.rim.tools.compiler.Compiler, import=C:\Program Files (x86)\Research In Motion\BlackBerry JDE 5.0.0\lib\net_rim_api.jar, codename=BlackBerry-5.0-de_DE-risingkarachi, BlackBerry-5.0-de_DE-risingkarachi.rapc, BlackBerry-5.0-de_DE-risingkarachi.jar]
D:\MobileDevelopment\risingkarachi5_21_2012\build.xml:99: rapc failed and returned 97
BUILD FAILED (total time: 13 seconds)
I have used all blackberry properties in build.xml and i have followed
http://www.enough.de/products/j2me-polish/documentation/platforms/blackberry/ link to to create the blackberry build but still not resolved.
my issue has been resolved, this issue occurs when you are using some resources like classes or images are more then 64kb and my single class was 296kb so i was receiving this error and then i divided my that class into multiple small classes.
Thanks