NoSuchMethodError while running spark streaming - twitter

Exception in thread "main" java.lang.NoSuchMethodError:
com.fasterxml.jackson.module.scala.deser.BigDecimalDeserializer$.handledType()Ljava/lang/Class;
I am creating a simple spark streaming application using twitter as source. But get stuck at this error. I am using spark 1.6.0 and scala 2.10.4.

This exception usually indicates that you have multiple versions of the library available at run-time, and the chosen one isn't the one you're relying on. Perhaps two libraries compiled into an uber jar.
To avoid this, you can take precedence for your versions of the JAR by specifying them explicitly via spark.executor.extraClassPath and spark.driver.extraClassPath. More can be read in Spark Configuration

Related

custom codec not found in plugin

I have a grails application with multiple internally developed plugins. Since upgrading from 4.x to 5.2.3, codecs are not found in one plugin, but are found in others. Specifically, I can place the same file (UsernameListCodec.groovy, package name changed from one plugin to the next but otherwise no changes) in grails-app/utils in one plugin and it works; when placed in grails-app/utils in another plugin it fails with MissingMethodException.
What could cause this? The plugins are fairly different in terms of what they provide, but very similar in terms of how they're built, published, etc. Clearly this is something I'm doing wrong (since the codec works in another plugin) but I don't even know where to begin looking. Does a plugin need to do something in particular to be able to provide custom codecs as of grails 5?

I see strange errors in my Dataflow job that may be related to library versioning

Errors range from 404s, IOExceptions, or encoding exceptions. They can be buried in the error stack, and occasionally suggest a versioning problem.
How can I prevent or address this class of errors?
The Dataflow service's SDKs and worker take dependencies on common third-party components, which themselves import various dependencies. Version collisions can result in unexpected behavior in the service. If you are using any of these packages in your code, be aware that some libraries are not forward-compatible and you may need to pin to the listed versions that will be in scope during execution. In order to determine whether your JAR has a conflicting version in use, consider inspecting the dependency tree of your project. Consult the list of specifically pinned versions if you suspect a problem here, and also avoid using "latest" for any of these libraries.

Exclude dependencies from a local Grails plugin

I'm not seeing anything resembling an answer to this on the googlytubes so here goes...
We make use of several local plugins in our grails project. One of our plugins recently has a dependency on SLF4J. Our main webapp (that uses the plugin) also has a dependency on SLF4J. This results in the entirely harmless but nevertheless irritating warning at runtime:
Error SLF4J: Class path contains multiple SLF4J bindings.
Typically I'd just define an "excludes" on the plugins SLF4J dependency, but since this is a local plugin I don't see any way to do so. I tried...
grails.plugin.location.'localpluginname' = '../localplugindir'
grails.project.dependency.resolution = {
plugins {
runtime("com.ourcompany:localpluginname:1.0") {
excludes('slf4j-api')
}
}
}
...but then it tries to actually resolve said plugin on the remote repositories, and fails. We also don't want to exclude the dependency directly in the plugin because the plugin may be used in other projects which do not provide the dependency already.
Before anyone suggests we deploy our local plugin to a local maven repo in order to do this, let me get it out of the way that we don't want to do that. We have them local for a reason...so we can rapidly make changes and see said changes. We'd rather live with the annoying warning messages than add in the increased pain of deploying on every change.
The warning you are getting is not related to having multiple versions of the slf4j-api present in the classpath. SLF4J API is designed to bind with one and only one underlying logging framework at a time. If more than one binding is present on the class path, SLF4J will emit a warning, listing the location of those bindings. You can read more about it in the SLF4J warning or error messages and their meanings document.
The solution proposed by Slf4J is: When multiple bindings are available on the class path, select one and only one binding you wish to use, and remove the other bindings.
In your case the best way would be to exclude the Slf4J binding directly from the plugin. The plugin should not rely on having its own binding but rather assume that the Grails application will provide the binding (which is what it will do)

Codenameone with WSDL NB - ClassDefNotFound

I have used NB to add a "client web service" to a Codename one app through the NB interface. This works fine in the simulator.
The WSDL classes are generated during build automatically and I have them landing in com.myco.myapp.generated package.
Having checked the generated JAR the WSDL classes are there all ok.
But when I push this to the "build for Android" to codename1, run on the device I get
An Internal application error occurred : java.lang.NoClassDefFoundError: com.myco.myapp.generated.SimpleStockList_Service
But the class is definitely there in the JAR.
I am sure its something to do with the JAR and its manifest, but never really had to get behind the scenes with Ant and JARs and builds to know what to do.
As the classes are generated during ant build, I can not pack them up into a library. (tried that and get fail due to 2 instances of same class.)
Codename One doesn't support binary libraries at this time, you will need to integrate the source code into the build process. There are many complexities involved in supporting binary libraries in such a setup.
Thanking Shai for his help.
Ultimate answer is not to use WSDL as moving objects relies Serialization which is not included in the small Java package.
Due to this I created a custom servlet which codename1 ConnectionRequest can deal with via a standard HTTP request.
This is how I achieved it
http://www.jamesarbrown.com/?p=164

ant taskdef - java.lang.UnsatisfiedLinkError: JSTAF (Library is already loaded in another ClassLoader)

I have written two different Tasks which need to load the same library, which contains some Classes that are imported in these tasks:
<taskdef name="someTask1" classname="somepackage.someTask1" classpath="Tasks1.jar;/pathtolib/MyJar.jar" />
<taskdef name="someTask2" classname="somepackage.someTask2" classpath="Task2s.jar;/pathtojar/MyJar.jar"/>
Because i can not load the same library two times, i get the following error:
21: java.lang.UnsatisfiedLinkError: MyJar (Library is already loaded in another ClassLoader)
How can i solve this problem?
This is a native library?
If no, then please post the exact stack trace and information about the tasks. Because you shouldn't be getting this error from a normal JAR.
If yes, then you're running into one of the consequences of Java's "sandbox" design: The underlying OS normally allows a shared library to be loaded once only. And different classloaders, because they potentially represent an isolation point between different applications, are not permitted to share native libraries (because that could open access to cross-application attacks).
Ant makes this a bit worse because it uses a separate classloader for each taskdef (see this blog post). And I think the best solution, as described in that blog, is to create an antlib specification that combines all of your tasks.

Resources