When running a Dataflow unittest that uses DataflowAssert I get the exception
java.lang.RuntimeException: DataflowAssert requires that JUnit and Hamcrest be linked in.
What dependencies do I need to add to my pom file to fix this exception?
Add the following dependencies to your pom file to link in the required libraries
<dependency>
<groupId>junit</groupId>
<artifactId>junit</artifactId>
<version>4.11</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.hamcrest</groupId>
<artifactId>hamcrest-all</artifactId>
<version>1.3</version>
<scope>test</scope>
</dependency>
Related
I could access my mysql table using JDBCIO and Avro Coder. Now I am trying to load my hive database using JdbcIO.
Below exception has thrown while connecting to hive from dataflow. Any help from the beam geeks would be really helpful.
Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/avro/reflect/AvroSchema
at org.apache.beam.sdk.coders.AvroCoder$AvroDeterminismChecker.recurse(AvroCoder.java:426)
at org.apache.beam.sdk.coders.AvroCoder$AvroDeterminismChecker.check(AvroCoder.java:419)
at org.apache.beam.sdk.coders.AvroCoder.<init>(AvroCoder.java:259)
at org.apache.beam.sdk.coders.AvroCoder.of(AvroCoder.java:120)
at com.google.cloud.bigquery.csv.loader.GoogleSQLPipeline.main(GoogleSQLPipeline.java:101)
Caused by: java.lang.ClassNotFoundException: org.apache.avro.reflect.AvroSchema
at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:335)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
... 5 more
Below code snippet is trying to access hive :
dataflowPipeline
.apply(JdbcIO.<Customer>read()
.withDataSourceConfiguration(JdbcIO.DataSourceConfiguration
.create("org.apache.hive.jdbc.HiveDriver", "jdbc:hive2://<ip>/mydb")
.withUsername("username").withPassword("password"))
.withQuery(
"select c_customer_id,c_first_name,c_last_name,c_preferred_cust_flag,c_birth_day,c_birth_month,c_birth_year,c_birth_country,c_customer_sk,c_current_cdemo_sk,c_current_hdemo_sk from customer")
.withRowMapper(new JdbcIO.RowMapper<Customer>() {
#Override
public Customer mapRow(ResultSet resultSet) throws Exception
POM Dependecies :
<dependencies>
<dependency>
<groupId>com.google.cloud.dataflow</groupId>
<artifactId>google-cloud-dataflow-java-sdk-all</artifactId>
<version>2.0.0</version>
</dependency>
<!-- https://mvnrepository.com/artifact/org.apache.beam/beam-sdks-java-io-jdbc -->
<dependency>
<groupId>org.apache.beam</groupId>
<artifactId>beam-sdks-java-io-jdbc</artifactId>
<version>2.0.0</version>
</dependency>
<!-- https://mvnrepository.com/artifact/org.apache.hive/hive-jdbc -->
<dependency>
<groupId>org.apache.hive</groupId>
<artifactId>hive-jdbc</artifactId>
<version>1.2.1</version>
</dependency>
<dependency>
<groupId>jdk.tools</groupId>
<artifactId>jdk.tools</artifactId>
<version>1.8.0_131</version>
<scope>system</scope>
<systemPath>${JAVA_HOME}/lib/tools.jar</systemPath>
</dependency>
<!-- https://mvnrepository.com/artifact/org.apache.hadoop/hadoop-common -->
<dependency>
<groupId>org.apache.hadoop</groupId>
<artifactId>hadoop-common</artifactId>
<version>2.8.1</version>
</dependency>
<dependency>
<groupId>com.google.guava</groupId>
<artifactId>guava</artifactId>
<version>18.0</version>
</dependency>
<!-- slf4j API frontend binding with JUL backend -->
<dependency>
<groupId>org.slf4j</groupId>
<artifactId>slf4j-jdk14</artifactId>
<version>1.7.14</version>
</dependency>
</dependencies>
Added dependency for Avro 1.8.1 and manually imported import org.apache.avro.reflect.AvroSchema; Now that issue is solved. Again, issues related to com.google.protobuf.GeneratedMessageV3 is solved by manually importing this class.
Added the dataflow dependency to the project. The project builds , but on start up ( using jetty ) I get a runtime exception
<dependency>
<groupId>com.google.cloud.dataflow</groupId>
<artifactId>google-cloud-dataflow-java-sdk-all</artifactId>
<version>1.9.0</version>
</dependency>
caused by: java.lang.ClassNotFoundException: com.google.auth.http.HttpTransportFactory
at org.codehaus.plexus.classworlds.strategy.SelfFirstStrategy.loadClass(SelfFirstStrategy.java:50)
at org.codehaus.plexus.classworlds.realm.ClassRealm.unsynchronizedLoadClass(ClassRealm.java:259)
at org.codehaus.plexus.classworlds.realm.ClassRealm.loadClass(ClassRealm.java:235)
at org.codehaus.plexus.classworlds.realm.ClassRealm.loadClass(ClassRealm.java:227)
at org.eclipse.jetty.webapp.WebAppClassLoader.loadClass(WebAppClassLoader.java:487)
at org.eclipse.jetty.webapp.WebAppClassLoader.loadClass(WebAppClassLoader.java:428)
if i remove the dependency. The start up has not issues.
Any idea why the dataflow dependency is causing startup error.
Added the exclusion for the conflicting dependency, and it works.
<dependency>
<groupId>com.google.cloud.dataflow</groupId>
<artifactId>google-cloud-dataflow-java-sdk-all</artifactId>
<version>1.9.0</version>
<exclusions>
<exclusion>
<groupId>com.google.auth</groupId>
<artifactId>google-auth-library-oauth2-http</artifactId>
</exclusion>
</exclusions>
</dependency>
We are trying to validate a JSON response to see if it matches a schema placed in the Eclipse ClassPath. This is designed to be a Maven project and I believe we have all the dependencies in place. But we are always getting the following error:
java.lang.NoClassDefFoundError: com/github/fge/jsonschema/main/JsonSchemaFactory
at com.jayway.restassured.module.jsv.JsonSchemaValidatorSettings.<init>(JsonSchemaValidatorSettings.java:58)
at com.jayway.restassured.module.jsv.JsonSchemaValidator$JsonSchemaValidatorFactory.createSettings(JsonSchemaValidator.java:277)
at com.jayway.restassured.module.jsv.JsonSchemaValidator$JsonSchemaValidatorFactory.create(JsonSchemaValidator.java:289)
at com.jayway.restassured.module.jsv.JsonSchemaValidator.matchesJsonSchema(JsonSchemaValidator.java:166)
at com.jayway.restassured.module.jsv.JsonSchemaValidator.matchesJsonSchemaInClasspath(JsonSchemaValidator.java:117)
at com.macys.xapi.ProductServices.TestProductServices.ohGodImDoomed(TestProductServices.java:76)
Caused by: java.lang.ClassNotFoundException: com.github.fge.jsonschema.main.JsonSchemaFactory
at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
We have the following dependencies in place:
<dependency>
<groupId>com.jayway.restassured</groupId>
<artifactId>rest-assured</artifactId>
<version>2.9.0</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>com.jayway.restassured</groupId>
<artifactId>json-schema-validator</artifactId>
<version>2.9.0</version>
</dependency>
<dependency>
<groupId>com.github.fge</groupId>
<artifactId>jackson-coreutils</artifactId>
<version>1.8</version>
</dependency>
<dependency>
<groupId>com.github.fge</groupId>
<artifactId>json-schema-core</artifactId>
<version>1.2.5</version>
</dependency>
Not sure what we are missing here - can someone please help?
Vishnu, Just add following dependencies and remove all above dependencies.
<dependency>
<groupId>io.rest-assured</groupId>
<artifactId>rest-assured</artifactId>
<version>3.0.1</version>
</dependency>
<dependency>
<groupId>io.rest-assured</groupId>
<artifactId>json-schema-validator</artifactId>
<version>3.0.1</version>
</dependency>
Since about a week ago, running tests with InProcessServer on 4.0.0.BUILD-SNAPSHOT results in the following exception:
Caused by: java.lang.NoClassDefFoundError: org/neo4j/ogm/testutil/TestServer
at org.springframework.data.neo4j.server.InProcessServer.<init>(InProcessServer.java:25) ~[spring-data-neo4j-4.0.0.BUILD-SNAPSHOT-tests.jar:na]
at com.ninjasquare.server.test.integration.IntegrationTestConfig.neo4jServer(IntegrationTestConfig.java:43) ~[test-classes/:na]
Swithing the test dependency back to 4.0.0.M1 resolves the issue:
<dependency>
<groupId>org.springframework.data</groupId>
<artifactId>spring-data-neo4j</artifactId>
<version>4.0.0.M1</version>
<type>test-jar</type>
</dependency>
I assume it's something to do with some refactoring work on SDN4/OGM?
Thanks.
Yes, in recent snapshots, the OGM has been separated from SDN. You'll need to include these two dependencies now to use the test utilities.
<dependency>
<groupId>org.neo4j</groupId>
<artifactId>neo4j-ogm</artifactId>
<version>1.1.0</version>
<type>test-jar</type>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.neo4j.test</groupId>
<artifactId>neo4j-harness</artifactId>
<version>${neo4j.version}</version>
<scope>test</scope>
</dependency>
I am trying to setup Maven, JSF and Primefaces project. But when i run the project i get the following error
com.sun.faces.config.ConfigurationException:
Source Document: jar:file:/D:/Personal%20Work/eclipse%2032%20Bit/workspace/Java%20EE
/Spring/.metadata/.plugins/org.eclipse.wst.server.core/tmp0/wtpwebapps
/ch18_SpringWebFlowAndJSF/WEB-INF/lib/primefaces-3.5.jar!/META-INF/faces-config.xml
Cause: Class 'org.primefaces.component.fileupload.FileUploadRenderer' is missing
a runtime dependency: java.lang.NoClassDefFoundError: org/apache/commons/fileupload
/FileItem
Here is my POM snippet
<properties>
<java-version>1.6</java-version>
<org.springframework-version>3.2.3.RELEASE</org.springframework-version>
<org.aspectj-version>1.6.10</org.aspectj-version>
<org.slf4j-version>1.7.5</org.slf4j-version>
<jsf-version>2.2.0</jsf-version>
</properties>
<dependency>
<groupId>org.springframework.webflow</groupId>
<artifactId>spring-faces</artifactId>
<version>2.3.2.RELEASE</version>
</dependency>
<dependency>
<groupId>com.sun.faces</groupId>
<artifactId>jsf-api</artifactId>
<version>${jsf-version}</version>
</dependency>
<dependency>
<groupId>com.sun.faces</groupId>
<artifactId>jsf-impl</artifactId>
<version>${jsf-version}</version>
</dependency>
<dependency>
<groupId>org.primefaces</groupId>
<artifactId>primefaces</artifactId>
<version>3.5</version>
</dependency>
</dependencies>
<repositories>
<repository>
<id>prime-repo</id>
<name>PrimeFaces Maven Repository</name>
<url>http://repository.primefaces.org</url>
<layout>default</layout>
</repository>
</repositories>
Why i am getting this error?
Thanks
java doc says
NoClassDefFoundError Thrown if the Java Virtual Machine or a
ClassLoader instance tries to load in the definition of a class (as
part of a normal method call or as part of creating a new instance
using the new expression) and no definition of the class could be
found.
The searched-for class definition existed when the currently executing
class was compiled, but the definition can no longer be found.
org.apache.commons.fileupload.FileItem is a class in FileUpload component of Apache Commons which is missing in your class path. Add following maven dependency for FileUpload component in your pom.xml.
<dependency>
<groupId>commons-fileupload</groupId>
<artifactId>commons-fileupload</artifactId>
<version>x.x</version>
</dependency>
you can also check BalusC's this answer,