How to Serialise a byte array into AVRO in Java? - avro

I am trying to serialise array of bytes in avro.
But, I am getting ClassCastException during the process. Any help for the same would be highly appreciated.
java.lang.ClassCastException: class [B cannot be cast to class java.util.Collection ([B and java.util.Collection are in module java.base of loader 'bootstrap')
Here is my schema.
{
"name": "my-data",
"type": {
"type": "array",
"items": "bytes"
}
}

Wrap your byte array into ByteBuffer e.g : ByteBuffer.wrap(your_byte_array). This is what causing ClassCastException.

Related

Parse a json {"result": DIFFERENTS TYPES}} using differents types nested in the result

I'm trying to know how I can use a {"result": {}} json with diferents types, I don't have any idea how to construct the class or reused a class Result.
Think of it as having a Map<String, dynamic> before parsing it as encoding/decoding it as JSON. This would allow you to have the following structure:
{
"result": {
"int": 2,
"String": "Sting vaue",
"bool": true
}
}

Dart Built Value Deserialize List of Objects

I have an API that's returning a list of objects...
[{}, {}, {}, ...]
I already have a defined and working built_value model for each object. However, now I need to deserialize the list.
I currently am trying something like this:
List<Map<String, dynamic>> json = JSON.decode(DEMO_TASK);
json.expand<Task>((Map<String, dynamic> map) => _serializers.deserializeWith<Task>(Task.serializer, map));
However, that causes issues since it says _serializers.deserializeWith return type Task isn't an Iterable<Task> as defined by the closure.
How do I go about deserializing the list. I'm sure I'm missing something super basic.
In case you want to have more general approach you can use this code snippet:
In case anybody needs this functionality I leave here code snippet of how to handle this situation (code should be placed in serializers.dart file):
Serializers standardSerializers = (serializers.toBuilder()..addPlugin(StandardJsonPlugin())).build();
T deserialize<T>(dynamic value) =>
standardSerializers.deserializeWith<T>(standardSerializers.serializerForType(T), value);
BuiltList<T> deserializeListOf<T>(dynamic value) =>
BuiltList.from(value.map((value) => deserialize<T>(value)).toList(growable: false));
So if you have json file
[
{
"name": "test1",
"result": "success"
},
{
"name": "test2",
"result": "fail"
}
]
And built value class:
import 'package:built_value/built_value.dart';
import 'package:built_value/serializer.dart';
part 'test_class.g.dart';
abstract class TestClass implements Built<TestClass, TestClassBuilder> {
String get name;
String get result;
TestClass._();
factory TestClass([updates(TestClassBuilder b)]) = _$TestClass;
static Serializer<TestClass> get serializer => _$testClassSerializer;
}
You can use method deserializeListOf from above as:
import 'package:path_to_your_serializers_file/serializers.dart';
final BuiltList<TestClass> listOfTestClasses = deserializeListOf<TestClass>(json);
Yup. I missed something basic. I was thinking I was using a stream, but since it's a list you just have to use the .map function on a list.
List<Map<String, dynamic>> json = JSON.decode(DEMO_TASK);
List<Task> tasks = json.map<Task>((Map<String, dynamic> map) => _serializers.deserializeWith<Task>(Task.serializer, map)).toList();

How to correctly wrap a Flux inside a Mono object

I have a web-service which returns student and enrolled class details.
{
"name": "student-name",
"classes": [
{
"className": "reactor-101",
"day": "Tuesday"
},
{
"className": "reactor-102",
"day": "Friday"
}
]
}
The DTO for this class is as below:
public class Student {
private String name;
private Flux<StudentClass> classes;
#Data
#AllArgsConstructor
#JsonInclude(JsonInclude.Include.NON_DEFAULT)
public static class StudentClass {
private String className;
private String day;
}
}
The main REST controller logic to fetch the student is as follows:
Flux<StudentClass> studentClassFlux = studentClassRepository.getStudentClass(studentName);
return Mono.just(new Student(studentName, studentClassFlux));
The problem with this is, I get the following output after making the REST call:
{
"name": "student-name",
"classes": {
"prefetch": 32,
"scanAvailable": true
}
}
I can achieve the desired output by blocking on the flux request to get completed and then convert the output to list.
List<StudentClass> studentClassList = studentClassRepository.getStudentClass(studentName)..toStream().collect(Collectors.toList());
return Mono.just(new Student(studentName, studentClassList)); // Change the Student#classes from flux to list
I am new to reactive-programming.
What is the correct way of using the Flux & Mono here to get the desired output?
Reactive types aren't meant to be serialized when wrapped in each other.
In that particular case, you probably want your Student object to contain a List<StudentClass>. You can achieve that like this:
public Mono<Student> findStudent(String studentName) {
return studentClassRepository
.getStudentClass(studentName)
.collectList()
.map(studentClasses -> new Student(studentName, studentClasses));
}
I think, in the case that you really need a Flux in your result, you would want to break down the API so that you have separate methods to retrieve the entities.
One for student properties, and another for their classes. The student GET method could be a Mono, while the classes would return a Flux.

How do I use logical types in Avro IDL?

I tried the following code in Avro IDL which references the logical type timestamp-millis and it doesn't work.
Is there an import required to use logical types in Avro IDL? Or are logical types not useable, and I need to use the primitive type (long in this case) instead?
protocol test {
record test {
timestamp-millis time;
}
}
Results in:
Exception in thread "main" org.apache.avro.compiler.idl.ParseException: Undefined name 'timestamp', at line 3, column 9
This works of course:
protocol test {
record test {
long time;
}
}
You can use a generic annotation:
protocol test {
record test {
#logicalType("timestamp-millis")
long time;
}
}
There's actually also a shorthand you can use for timestamp-millis and a handful of other logical types (the documentation hasn't been released yet, see here for the full list of aliases):
protocol test {
record test {
timestamp_ms time;
}
}
LogicalType timestamp-millis is used with type long, so you may use schema like this:
{
"type" : "record",
"name" : "Test",
"fields" : [ {
"name" : "time",
"type" : {
"type" : "long",
"logicalType" : "timestamp-millis"
}
}]
}

Xamarin.Forms deserialising json to an object

var myObjectList = (List<MyObject>)JsonConvert.DeserializeObject(strResponseMessage, typeof(List<MyObject>));
the above works to deserialise a JSON string to a list of custom objects when the JSON has the following format
[
{
"Name": "Value"
},
{
"Name": "Value"
},
{
"Name": "Value"
},
"Name": "Value"
}
]
I don't know how to do the same when the format is like this
{
"ReturnedData" : [
{
"Name": "Value"
},
{
"Name": "Value"
},
{
"Name": "Value"
},
"Name": "Value"
}
]
}
I can get the data like this
JObject information = JObject.Parse(strResponseMessage);
foreach (dynamic data in information)
{
//convert to object here
}
and that works for Android but it seems that you cannot use a type of 'dynamic' for iOS as I get the error:
Object type Microsoft.CSharp.RuntimeBinder.CSharpInvokeMemberBinder cannot be converted to target type: System.Object[]
What step am I missing to convert the second JSON string to the first?
If JsonConvert is JSON.Net just instead of List use
public class MyClass {
public List<MyObject> ReturnedData { get; set; }
}
You can't use the dynamic keyword on iOS as its forbidden to generate code as it states in this link.
Quote:-
No Dynamic Code Generation
Since the iPhone's kernel prevents an application from generating code dynamically Mono on the iPhone does not support any form of dynamic code generation.
These include:
The System.Reflection.Emit is not available.
Quote:-
System.Reflection.Emit
The lack of System.Reflection. Emit means that no code that depends on runtime code generation will work. This includes things like:
The Dynamic Language Runtime.
Any languages built on top of the Dynamic Language Runtime.
Apparently there is some support creeping in from v7.2 as can be seen in this link - See #Rodja answer. - however - its very experimental and has flaws preventing this from fully working.
Your best approach would be to process the JObject - without - reying on the dynamic keyword and you will be alright on both Android and iOS.
Thanks for your replies - I was able to solve it as follows
JObject information = JObject.Parse(strResponseMessage);
string json = JsonConvert.SerializeObject(strResponseMessage["ReturnedData "]);
var myObjectList = (List<MyObject>)JsonConvert.DeserializeObject(json , typeof(List<MyObject>));
Works perfectly!

Resources