Building a Jar of a Frege project using Gradle - ant

I would like to:
use the Frege programming language to write a simple "Hello World" piece of code,
then using the Frege compiler generating the equivalent Java source code,
then building an executable Jar file to run from the command line,
all the previous steps should be "controlled" by Gradle.
I am able to generate the source code (items 1. and 2. from the previous list), but I am not able to specify a "package" structure of the Java source code in output, i.e. I can not see the package Java statement as the first line of code in the generate Java source code. I can specify to the Frege compiler where to put the generated code though (via the -d argument).
I think this is the reason why when building an executable Jar, then launching it, I am seeing similar errors (according to different attempts on Gradle tasks) e.g.: no main manifest attribute.
The Frege source code is saved in a file named HelloFrege.fr, the generated Java source code is in a file named HelloFrege.java (I verified the file contains the expected main method).
Here there's a version of the Gradle "Jar task":
//create a single Jar with all dependencies
task fatJar(type: Jar) {
manifest {
attributes 'Implementation-Title': 'Hello Frege Jar Example',
'Implementation-Version': version,
'Main-Class': 'HelloFrege'
}
baseName = project.name + '-all'
from { configurations.compile.collect { it.isDirectory() ? it : zipTree(it) } }
with jar
}
Here there's another version of the Gradle "Jar" task:
jar {
manifest {
attributes 'Main-Class': 'HelloFrege'
}
}
How can I solve this problem? I would like to avoid to manually add the package reference to the automatically generated Java source code file.

If your module name in Frege is unqualified such as HelloWorld, you will not see the package statement generated in Java. The module name will become the class name and the package will be empty or default package.
If your module name is qualified such as foo.bar.HelloWorld, then foo.bar will be the package name and HelloWorld will be the class name in the generated Java source.
The rule is that the last part of the module name becomes the class name and the qualifiers form the package name in the generated Java source.

I am not sure what gradle can do for you in this regard, but without gradle, the following should at least be possible:
... build your jar, as before ...
jar -uvfe project-all.jar HelloFrege
java -jar project-all.jar # run it
This, of course, is just another way to create a manifest. If this works, then it would be time to investigate why gradle refuses to do it.
Postscriptum: After thinking another minute about what the problem might be, it occurs to me that you may think that the source file name/path has anything to do with the java package name. This is not so in Frege, though it is good practice to have the file path match the package name, and the file base name match the class name (just like in Java). In addition, to remove some confusion, use the module keyword in frege. As explained by Marimuthu, the Java package and class name is derived from the frege module name.
Example:
$ cat Foo.fr
module my.org.Baz where
...
$ java -jar fregec.jar -d bin Foo.fr
This generates the Baz class in package my.org, and creates the class file in bin/my/org/Baz.class

I am posting here my findings so far. The combination of Gradle commands that works for me is the following one (calling it from the command line typing gradle clean generateJavaSrcFromFregeSrc fatJar):
task generateJavaSrcFromFregeSrc {
ant.java(jar:"lib/frege3.21.586-g026e8d7.jar",fork:true) {
arg(value: "-j") // do not run the java compiler
arg(value: "-d")
arg(value: "src/main/java") // the place where to put the generated source code (paired to the -d argument)
arg(value: "src/main/frege/HelloFrege.fr")
}
}
jar {
manifest {
attributes 'Main-Class': 'org.wathever.HelloFrege'
}
}
task fatJar(type: Jar) {
from files(sourceSets.main.output.classesDir)
from files(sourceSets.main.output.resourcesDir)
//from {configurations.compile.collect {zipTree(it)}} // this does not include the autogenerated source code
baseName = project.name + '-fatJar'
from { configurations.compile.collect { it.isDirectory() ? it : zipTree(it) } }
with jar
}
The manifest details need to be specified in the jar block of code, if I specify them in the task fatJar then when running the jar I get no main manifest attribute, in [...].
If I use just the block of code jar with the property from("$projectDir") { include 'lib/**'} to include the Frege jar, then I get errors like java.lang.ClassNotFoundException (I think because the Jar is included as it is and not as a set of .class files).
The folder src/main/java/org/wathever needs to be there before running Gradle (additional info: the Maven convention prefix src/main/java with as a suffix the "Java package" as specified inside the HelloFrege.fr source code: module org.wathever.HelloFrege where)
Some useful details I found:
How to build a fat Jar
Another how to build a fat Jar
An "Hello Frege" example without the Gradle management
The Gradle documentation on how to use the Jar task

Related

error while reading extension file 'intellij_info_bundled.bzl'

we are trying to create a Scala project which uses Spark also but we are facing issue Encountered error while reading extension file 'intellij_info_bundled.bzl': no such package '#intellij_aspect//': No WORKSPACE file found in C:/users//_bazel_user/i45wuf6d/external/intellij_aspect. Is it has something missing in Intellij?
Scala file
package src.main.scala
object HelloWorld extends App {
def main(args: Array[String]) {
println("Hello, world!")
}
}
Build file
package(default_visibility = ["//visibility:public"])
load("#io_bazel_rules_scala//scala:scala.bzl", "scala_library", "scala_test")
scala_library(
name = "hello-world",
srcs = glob(["src/main/scala/*.scala"]),
)
scala_test(
name = "Hello_test",
srcs = glob(["src/main/scala/*.scala"]),
size = "small", # Expect this test to run quickly
)
Work Space
workspace(name = "scala_example")
rules_scala_version="7522c866450cf7810eda443e91ff44d2a2286ba1" # update this as needed
http_archive(
name = "io_bazel_rules_scala",
url = "https://github.com/bazelbuild/rules_scala/archive/%s.zip"%rules_scala_version,
type = "zip",
strip_prefix= "rules_scala-%s" % rules_scala_version
)
load("#io_bazel_rules_scala//scala:scala.bzl", "scala_repositories")
scala_repositories()`enter code here`
# register default scala toolchain
load("#io_bazel_rules_scala//scala:toolchains.bzl", "scala_register_toolchains")
scala_register_toolchains()
Command and Error from console
Command: C:\ProgramData\chocolatey\bin\bazel.exe build --tool_tag=ijwb:IDEA:community --keep_going --curses=no --color=yes --experimental_ui=no --progress_in_terminal_title=no --aspects=#intellij_aspect//:intellij_info_bundled.bzl%intellij_info_aspect --override_repository=intellij_aspect=C:\Users\ADMIN.IdeaIC2017.3\config\plugins\ijwb\aspect --output_groups=intellij-compile-java,intellij-compile-py -- //...:all
INFO: Loading complete. Analyzing...
ERROR: Encountered error while reading extension file 'intellij_info_bundled.bzl': no such package '#intellij_aspect//': No WORKSPACE file found in C:/users/admin/appdata/local/temp/_bazel_sandhya/criyrv6d/external/intellij_aspect.
INFO: Found 3 targets...
WARNING: failed to create one or more convenience symlinks for prefix 'bazel-':
cannot create symbolic link bazel-out -> C:/users/admin/appdata/local/temp/_bazel_sandhya/criyrv6d/execroot/scala_example/bazel-out: Cannot create junction (name=C:\users\admin\scalaprojects\example1\bazel-out, target=C:\users\admin\appdata\local\temp_bazel_sandhya\criyrv6d\execroot\scala_example\bazel-out): ERROR: src/main/native/windows/file-jni.cc(86): nativeCreateJunction(C:\users\admin\scalaprojects\example1\bazel-out, C:\users\admin\appdata\local\temp_bazel_sandhya\criyrv6d\execroot\scala_example\bazel-out): ERROR: src/main/native/windows/file.cc(128): CreateJunction(\?\C:\users\admin\scalaprojects\example1\bazel-out): Cannot create a file when that file already exists.
cannot create symbolic link bazel-out -> C:/users/admin/appdata/local/temp/_bazel_sandhya/criyrv6d/execroot/scala_example/bazel-out: Cannot create junction (name=C:\users\admin\scalaprojects\example1\bazel-out, target=C:\users\admin\appdata\local\temp_bazel_sandhya\criyrv6d\execroot\scala_example\bazel-out): ERROR: src/main/native/windows/file-jni.cc(86): nativeCreateJunction(C:\users\admin\scalaprojects\example1\bazel-out, C:\users\admin\appdata\local\temp_bazel_sandhya\criyrv6d\execroot\scala_example\bazel-out): ERROR: src/main/native/windows/file.cc(128): CreateJunction(\?\C:\users\admin\scalaprojects\example1\bazel-out): Cannot create a file when that file already exists.
.
INFO: Building...
ERROR: command succeeded, but not all targets were analyzed.
INFO: Elapsed time: 18.108s, Critical Path: 0.05s
Make failed
This is a sample Helloworld program only
In general, like #Ittai, I would suggest you open an issue in the intellij plugin github repo.
Unfortunately, your version of the plugin is no longer supported. I, too, previously ran into an issue with an older version of the plugin and was recommended to upgrade to the latest version. Which resolved the specific issue I was facing.
When reporting the issue make sure to include the following bits of information:
intellij build number
plugin version number
rules_scala version
operating system (it seems your using Windows, while most users use unix based systems)
bazel release number
how you have opened the intellij project (BUILD file, WORKSPACE, .blazeproject)
Additionally, to verify this is in fact an issue with the plugin, I would also suggest you try to reproduce this issue on a Unix based system. It seems you are using Intellij
compile on Windows. This may be Windows specific issue with aspects not being recognized.
When attempting to reproduce, make sure to clone your repository in a separate directory, close the intellij project, and reopen the project

detectClassPathResourcesToStage - Unable to convert url

When I run the jar in the GCE, it had the following error:
java -jar mySimple.jar --project=myProjcet
Aug 13, 2015 1:22:26 AM com.google.cloud.dataflow.sdk.runners.DataflowPipelineRunner detectClassPathResourcesToStage
SEVERE: Unable to convert url (rsrc:./) to file.
Aug 13, 2015 1:22:26 AM simple.SimpleV1 main
SEVERE: Failed to construct instance from factory method com.google.cloud.dataflow.sdk.runners.BlockingDataflowPipelineRunner#fromOptions
I am working on Eclipse(window). And it succeeded to run dataflow through the eclipse. Packaging the project to Runable jar file and uploaded to the GCE (ubuntu). And i had errors when i run the jar file on the GCE(ubuntu).
the runner is BlockingDataflowPipelineRunner(batch mode).
there are other options in source code.
the follow is manifest.
Manifest-Version: 1.0
Rsrc-Class-Path: ./ httpclient-4.3.6.jar httpcore-4.3.3.jar commons-lo
gging-1.1.3.jar commons-codec-1.6.jar mybatis-3.2.8.jar mysql-connect
or-java-5.1.34.jar ibatis2-common-2.1.7.597.jar ibatis2-dao-2.1.7.597
.jar ibatis2-sqlmap-2.1.7.597.jar geoip-api-1.2.14.jar google-api-cli
ent-java6-1.20.0.jar google-api-client-1.20.0.jar google-oauth-client
-1.20.0.jar guava-jdk5-13.0.jar google-oauth-client-java6-1.20.0.jar
google-oauth-client-jetty-1.20.0.jar jetty-6.1.26.jar jetty-util-6.1.
26.jar servlet-api-2.5-20081211.jar google-http-client-jackson2-1.20.
0.jar google-http-client-1.20.0.jar jsr305-1.3.9.jar joda-time-2.8.1.
jar slf4j-api-1.7.7.jar slf4j-jdk14-1.7.7.jar commons-csv-1.1.jar aws
-java-sdk-sqs-1.10.5.1.jar aws-java-sdk-core-1.10.5.1.jar google-clou
d-dataflow-java-sdk-all-0.4.150710.jar google-api-services-dataflow-v
1b3-rev4-1.19.1.jar google-cloud-dataflow-java-proto-library-all-0.4.
150612.jar protobuf-java-2.5.0.jar google-api-services-bigquery-v2-re
v187-1.19.1.jar google-api-services-compute-v1-rev46-1.19.1.jar googl
e-api-services-pubsub-v1beta2-rev1-1.19.1.jar google-api-services-sto
rage-v1-rev25-1.19.1.jar google-api-services-datastore-protobuf-v1bet
a2-rev1-2.1.2.jar google-http-client-protobuf-1.15.0-rc.jar google-ht
tp-client-jackson-1.15.0-rc.jar jackson-annotations-2.4.2.jar jackson
-databind-2.4.2.jar avro-1.7.7.jar jackson-core-asl-1.9.13.jar jackso
n-mapper-asl-1.9.13.jar paranamer-2.3.jar snappy-java-1.0.5.jar commo
ns-compress-1.9.jar jetty-server-9.2.10.v20150310.jar javax.servlet-a
pi-3.1.0.jar jetty-http-9.2.10.v20150310.jar jetty-io-9.2.10.v2015031
0.jar jetty-jmx-9.2.10.v20150310.jar jetty-util-9.2.10.v20150310.jar
jackson-core-2.6.0.jar
Class-Path: .
Rsrc-Main-Class: simple.SimpleV1
Main-Class: org.eclipse.jdt.internal.jarinjarloader.JarRsrcLoader
When exporting a Runnable JAR file using Eclipse, there are three ways to package your project:
Extract required libraries into generated JAR
Package required libraries into generated JAR
Copy required libraries into a sub-folder next to the generated JAR
All 3 options, have the same usage pattern when executing, e.g.
java -jar myrunnable.jar --myCommandLineOption1=...
Currently, only option 1 is compatible with how the Dataflow SDK for Java is able to detect resources to stage because it is dependent on them being file URIs from a URLClassLoader.
For an explanation of how the Runnable Jars are created and more specific details of why this was problematic, read further below.
An alternative solution to using the Runnable Jars, is to execute your project using mvn exec.
Option 1
This creates a jar which copies all the class files & resources in each individual jar into a single jar. This allows for a manifest where the entire classpath is composed of file based URIs:
Manifest-Version: 1.0
Main-Class: com.google.cloud.dataflow.starter.StarterPipeline
Class-Path: .
Option 2
This creates a jar file with additional jars embedded within it. It uses a custom main entry point (org.eclipse.jdt.internal.jarinjarloader.JarRsrcLoader) which knows how to read the custom manifest entries (Rsrc-Class-Path & Rsrc-Main-Class) and creates a classloader with non file based URIs. Since the Dataflow SDK for Java currently only knows how to handle file based resources and doesn't know how to interpret the rsrc:... URIs, you get the exception that your seeing.
Manifest-Version: 1.0
Rsrc-Class-Path: ./ httpclient-4.3.6.jar ...
Class-Path: .
Rsrc-Main-Class: simple.SimpleV1
Main-Class: org.eclipse.jdt.internal.jarinjarloader.JarRsrcLoader
Option 3
This creates a jar file which contains your project resources and then creates a folder along side the runnable jar containing all your projects dependent jars. This allows for a more complex standard manifest listing all your project dependencies.
Manifest-Version: 1.0
Main-Class: com.google.cloud.dataflow.starter.StarterPipeline
Class-Path: . runnable_lib/google-cloud-dataflow-java-sdk-all-manual_build.jar ...
The Class-Path manifest is not returned part of the URLClassLoader and hence these classes are not discoverable. Furthermore, those jars are only meant to be loaded by classes from that jar which can lead to a jar loading hierarchy. More details are available here: http://docs.oracle.com/javase/7/docs/technotes/tools/findingclasses.html

Missing ant property while configuring worklight database via ant task

I am integrating worklight 6.1 official deployment ant task into gradle build script.
We will utilize below script to run flexible continuous integration process.
def antTaskLibClasspath = "ant-task-lib/worklight-ant-deployer.jar"
def verifyDatabaseArguments(String database){
switch(project.ext[database + 'Type']){
case 'derby':
def dbNameProp = database + 'Name'
def dbDirPathProp = database + 'Dir'
def dbProperties = ['databaseName':project.ext[dbNameProp], 'databaseDir':project.ext[dbDirPathProp] ]
.each { key, value ->
if(!value?.trim()){
def errMsg = "Missing mandatory parameter : ${key}"
println errMsg
throw new StopExecutionException(errMsg)
}
}
project.ext[dbDirPathProp] = Eval.me(""" "${project.ext[dbDirPathProp]}" """).replace("\\", "/")
def dbFile = file(project.ext[dbDirPathProp] + "/${project.ext[dbNameProp]}")
if(!dbFile.exists()){
if(dbFile.mkdirs()){
println "Create folders ${project.ext[dbDirPathProp]} for ${database} derby database"
}else {
def errMsg = "Can not create folders for ${database} derby database"
println errMsg
throw new StopExecutionException(errMsg)
}
}
break
case 'oracle':
break
default:
errMsg = "Unsupported database type"
println errMsg
throw new StopExecutionException(errMsg)
break
}
}
verifyDatabaseArguments("worklightDatabase")
verifyDatabaseArguments("worklightReportsDatabase")
task configure {
description "Configure database for worklight server"
doLast {
ant.typedef(name:'configureDatabase',
classname:'com.ibm.worklight.config.ant.database.ConfigureDatabaseTask',
classpath:antTaskLibClasspath
)
ant.configureDatabase(kind: "Worklight"){
switch(worklightDatabaseType){
case 'derby':
derby(database:worklightDatabaseName, datadir:worklightDatabaseDir)
break
}
}
ant.configureDatabase(kind: "WorklightReports"){
switch(worklightReportsDatabaseType){
case 'derby':
derby(database:worklightReportsDatabaseName, datadir:worklightReportsDatabaseDir)
break
}
}
}
}
And these are properties which were referenced in the script overhead. I've set them in gradle.properties where the build.gradle is located.
worklightDatabaseType=derby
# derby database name
worklightDatabaseName=WRKLGHT
# derby database file directory
worklightDatabaseDir=${System.properties['user.home']}/.derby/ibm
worklightReportsDatabaseType=derby
# derby database name
worklightReportsDatabaseName=WLREPORT
# derby database file directory
worklightReportsDatabaseDir=${System.properties['user.home']}/.derby/ibm
The command line shows ant error message which I can't figure out what's going wrong after I try to configure database with command $ > gradle configure.
Line 75 is the place where I start the first invocation of ant.configureDatabase( ...
Did I miss any mandatory configuration which may not documented in the worklight 6.1?
My jdk version is jdk7_60, gradle version 2.4 without wrapper.
Any suggestion/comment is appreciated, thank you~
By the way, if any consultant from IBM mobilefirst team is watching,
please consider to ask your product team to rename ant task which has dash symbol in it's name.
It seems that Groovy AntBuilder can not load ant element name includes dash symbol,
so we have to unzip worklight ant task jar archives and search for the task class we need in defaults.properties,
then redefine ant task using class name manually in gradle script. That's not convenient.
The build progress of worklight project is complex. We don't like to use ant to setup build automation.
Appended after Peter post his suggestion:
Thank you, Peter.
I have step over the ant library issue after adopting your way.
But it stucks again, and I couldn't interpret the error message.
At first I assume this issue is because worklight 6.1 ant deploy task may not be compatible with gradle,
so I wrote ant build script below to test if my assumption is correct,
which implements the same build logic and variable as previous gradle script does.
build.xml
<?xml version="1.0" encoding="UTF-8"?>
<project basedir="." default="install">
<loadproperties srcfile="build.properties" />
<taskdef resource="com/worklight/ant/defaults.properties">
<classpath>
<pathelement location="worklight-ant-task-libs/worklight-ant-deployer.jar"/>
</classpath>
</taskdef>
<target name="configure">
<configuredatabase kind="Worklight">
<derby database="${worklightDatabaseName}" datadir="${worklightDatabaseDir}"/>
</configuredatabase>
<configuredatabase kind="WorklightReports">
<derby database="${worklightReportsDatabaseName}" datadir="${worklightReportsDatabaseDir}"/>
</configuredatabase>
</target>
</project>
build.properties
# derby database name
worklightDatabaseName=WRKLGHT
# derby database file directory
worklightDatabaseDir=${user.home}/.derby/ibm
# derby database name
worklightReportsDatabaseName=WLREPORT
# derby database file directory
worklightReportsDatabaseDir=${user.home}/.derby/ibm
And my build progress fails again.
I have upload the ant error message log file to google drive.
Can anyone give me some suggestion to help me figure it out?
Thank you~
I'm going to use a lot of guesswork here, but according to this section of the Ant Manual, ant.library.dir is only set by Ant's Launcher class, which Gradle's Ant integration probably doesn't use.
I suspect that the Worklight Ant task you're using assumes that property is set and throws an exception. You can check this by running
gradle --stacktrace configure
assuming that configure is the task you want to run. This will show you where the exception is being thrown from (and any causes too).
You might be best off adding an unpacked Ant distribution to your source tree (or perhaps just its lib directory) and adding the following entry to a gradle.properties file in the root of your project:
systemProp.ant.library.dir=<path to Ant>/lib
Of course you should replace <path to Ant> with the actual path to an Ant installation.

Running Ant build from Gradle, how to pass -lib argument?

I want to slowly migrate to a better build and dependency resolution process, we are currently using Ant with local file dependencies.
We chose to migrate to gradle ,so as a first step I would like to simply run my current ant build from a gradle sript. But i dont know how to pass the -lib classpath to ant. Im getting errors of missing dependencies.
This is my gradle.build:
apply plugin: 'java'
task someProperties {
ext.LIBS_CATW = "backend/java-src/lib"
ext.LIB_SERVLET = "/usr/local/apache-tomcat-7.0.32/lib/servlet-api.jar"
}
dependencies {
compile files('/usr/local/apache-tomcat-7.0.32/lib/servlet-api.jar')
compile fileTree(dir: 'backend/java-src/lib', include: '*.jar')
}
I use this shell script to run ant from command line.
#!/bin/bash
export JAVA_HOME="/System/Library/Frameworks/JavaVM.framework/Versions/1.6.0/Home"
ROOT_DIR="/Users/poolebu/catwizardMultitenant/catwizard/catwBackend/branches/branchSpringSecurity/"
DIR_MT="$ROOT_DIR"
LIBS_CATW="$ROOT_DIR/backend/java-src/lib"
LIB_SERVLET="/usr/local/apache-tomcat-7.0.32/lib/servlet-api.jar"
ANT_TARGET="allButFlex"
cd $DIR_MT
ant -lib $LIBS_CATW -lib $LIB_SERVLET $ANT_TARGET
This is one of the many dependency errors I´m getting
[ant:javac] /Users/poolebu/catwizardMultitenant/catwizard/catwBackend/branches/branchSpringSecurity/backend/java-src/catw-common/src/com/bamboo/common/factory/SpringFactory.java:5: error: package flex.messaging does not exist
[ant:javac] import flex.messaging.FlexFactory;
[ant:javac] ^

grails (GGTS): can I automate copying of a class from target-eclipse to a deployment folder?

I have a class in src/Java whose class file needs to be copied (or compiled) to a classes folder in my externalConfig folder from the target-eclipse folder - which is not visible in GGTS.
My specific requirement is that I need the class file to be available for Liquibase to invoke as a CustomTaskChange on a deployment platform.
Is there something I can code that would copy any class files in a specific package name into my target classes folder? Any solution that avoids having to go outside GGTS would be welcome. I have tried the Resouce perspective, but target-eclipse isn't shown.
Regards, John
You can use the grails script to do execute by an event. You can use the command
grails create-script scriptClassName....
or by naming convention, create a file called _Event.groovy under the /scripts directory. In this groovy file, create event methods. For example (this would be called when a war file is created:
In scripts/_Event.groovy
eventCreateWarStart = { warName, myDir ->
println 'EVENT CALLED!'
File libDir = new File("${myDir}/WEB-INF/lib/")
if (grailsEnv != "development") {
libDir.eachFileMatch( ~/^(my_class_name).*\.class$/) { File classToCopy ->
println "Put code to copy file ${classToCopy}"
}
}
}

Resources