Avro serialization hanging on DataFileWriter close method - avro

Question

I am trying to use avro serialization but when I have multiple records to serialize, the application hangs on DataFileWriter close method, however it works with a small amount of records.
final PipedOutputStream pipedOutputStream = new PipedOutputStream();
PipedInputStream pipedInputStream = new PipedInputStream(
pipedOutputStream);
DatumWriter<DW> userDatumWriter = new SpecificDatumWriter<DW>(DW.class);
DataFileWriter<DW> dataFileWriter = new DataFileWriter<DW>(
userDatumWriter);
dataFileWriter.create(payload.get(0).getSchema(), pipedOutputStream);
for (DW currentRecord : payload) {
dataFileWriter.append(currentRecord);
}
dataFileWriter.close();
return pipedInputStream;
I tried to flush after adding 10 records at a time, but then it hangs on the flush method.
Can anyone help me with this?

Answer 1

Solved by returning a ByteArrayOutputStream as follows:
Schema schema = ReflectData.get().getSchema(DW.class);
ByteArrayOutputStream outputStream = new ByteArrayOutputStream();
ReflectDatumWriter<Object> reflectDatumWriter = new ReflectDatumWriter<Object>(
schema);
DataFileWriter<Object> writer = new DataFileWriter<Object>(
reflectDatumWriter).create(schema, outputStream);
for (DW currentRecord : payload) {
writer.append(currentRecord);
}
writer.close();
return outputStream.toByteArray();