i want to convert a ZipOutputStream into ZipInputStream using PipedOutputStream and PipedInputStream. I am receiving ZipOutputStream from DataHandler, and i don't want to write it on file and then open and read it, rather i want to convert this ZipOutputStream into ZipInputStream for my further processing, like extracting ZipEntry and save it in some file.
public String documentReceived(#WebParam(name = "document")final DataHandler dataHandler) {
...
ZipOutputStream zout = new ZipOutputStream(dataHandler.getOutputStream());
final PipedInputStream pin = new PipedInputStream();
final PipedOutputStream pout = new PipedOutputStream(pin);
??? need help here to convert a ZipOutputStream into
??? ZipInputStream
...
}
Any help will be appreciated... thanks
Related
I want to create a HttpResponse that streams a local file.
I want to use a MemoryStream, so that I can delete the file afterwards (well actually before returning the repsonse).
I always end up with an empty response although the stream seems to be valid.
Working with a FileStream in API Controller works, though.
public HttpResponseMessage GetExcelFile(Guid id)
{
// this model is needed to internally create an .xls file that represents this model
var exportModel = this.myService.GetExport(id);
// this approach does not work -> respone always empty although memory stream has content
// var stream = new MemoryStream();
// internally creates a .xls file (using lib) and returns its content as memory stream
// this.myService.ConvertToStream(exportModel, stream));
// this works fine
var stream = File.OpenRead(#"D:\test0815.xls");
var result = this.Request.CreateResponse(HttpStatusCode.OK);
result.Content = new StreamContent(stream);
result.Content.Headers.ContentType = new MediaTypeHeaderValue("application/octet-stream");
result.Content.Headers.ContentLength = stream.Length;
result.Content.Headers.ContentDisposition = new ContentDispositionHeaderValue("attachment")
{
FileName = $"{exportModel.Name}-Sheet.xls"
};
return result;
}
this is my method that actually converts to memorystream:
private MemoryStream SaveToStream(MemoryStream stream)
{
using (FileStream source = File.Open(
#"D:\test0815.xls",
FileMode.Open))
{
Console.WriteLine("Source length: {0}", source.Length.ToString());
// Copy source to destination.
source.CopyTo(stream);
}
return stream;
}
I also tried writing to memory stream but this did not work either.
It seems that result.Content = new StreamContent(stream); is just not working with an memory stream.
Any ideas?
finally I found a working solution:
var memoryStream = new MemoryStream((int)fileStream.Length);
fileStream.CopyTo(memoryStream);
fileStream.Close();
memoryStream.Seek(0, SeekOrigin.Begin);
HttpContent content = new StreamContent(memoryStream);
var result = this.Request.CreateResponse(HttpStatusCode.OK);
result.Content =content;
result.Content.Headers.ContentType = new MediaTypeHeaderValue("application/octet-stream");
I have a controller action that creates a zip file and sends back to user for download. The problem is that the zip file gets created but it is empty. Somehow it's not writing the image files to the MemoryStream. I wonder what I am missing. If I write the zip file to the disk everything will work as expected, but I'd rather not save files to the disk if I can avoid it. This is what I have tried using dotnetzip:
public ActionResult DownloadGraphs()
{
var state = Session["State"];
using (ZipFile zip = new ZipFile())
{
if (state == "IA")
{
zip.AddFile(Server.MapPath("~/Content/DataVizByState/FallGraphs/Watermarked/Fall_IA.jpg"), "");
zip.AddFile(Server.MapPath("~/Content/DataVizByState/SpringGraphs/Watermarked/Spring_IA.jpg"), "");
}
MemoryStream output = new MemoryStream();
zip.Save(output);
output.Seek(0, SeekOrigin.Begin);
var fileName = state + "Graphs.zip";
return File(output, "application/zip", fileName);
}
}
This forces download in the view based on click of a button:
$('#graphDwnldBtn').click(function (evt) {
window.location = '#Url.Action("DownloadGraphs", "DataSharing")';
})
Do I need to use StreamWriter or Reader or something? This is the first time I have ever attempted something like this and it's been cobbled together by reading various stackoverflow posts...
Dumb mistakes: Session["State"] is an object, so the state variable was coming out as object instead of a string like I need it to be for my conditional statement to evaluate correctly. I cast state to a string to fix it. Fixed code:
public ActionResult DownloadGraphs()
{
var state = Session["State"].ToString();
using (ZipFile zip = new ZipFile())
{
if (state == "IA")
{
zip.AddFile(Server.MapPath("~/Content/DataVizByState/FallGraphs/Watermarked/Fall_IA.jpg"), "");
zip.AddFile(Server.MapPath("~/Content/DataVizByState/SpringGraphs/Watermarked/Spring_IA.jpg"), "");
}
MemoryStream output = new MemoryStream();
zip.Save(output);
output.Seek(0, SeekOrigin.Begin);
var fileName = state + "Graphs.zip";
return File(output, "application/zip", fileName);
}
}
Our client is uploading files into GCS, but they are zipped. Is there any way, using the Java Dataflow SDK, in which we can run through all the zipped files, unzip the file, combine all the resulting .csv files into one file, and then only do the TextIO transforms?
EDIT
To answer jkffs's questions,
Well I don't really need to combine them all into a single file, it would just be much easier from a reading perspective.
They are ZIP files, not GZ or BZ or anything else. Each ZIP contains multiple files. The files names are not really significant, and yes, I would actually prefer it TextIO transparently decompresses and concatenates all the files, on a per-archive basis.
Hope that helps!
because I had the same problem and only came to this 1 year old and quite incomplete solution. Here is a full example on how to unzip files on google dataflow:
public class SimpleUnzip {
private static final Logger LOG = LoggerFactory.getLogger(SimpleUnzip.class);
public static void main(String[] args){
Pipeline p = Pipeline.create(
PipelineOptionsFactory.fromArgs(args).withValidation().create());
GcsUtilFactory factory = new GcsUtilFactory();
GcsUtil util = factory.create(p.getOptions());
try{
List<GcsPath> gcsPaths = util.expand(GcsPath.fromUri("gs://tlogdataflow/test/*.zip"));
List<String> paths = new ArrayList<String>();
for(GcsPath gcsp: gcsPaths){
paths.add(gcsp.toString());
}
p.apply(Create.of(paths))
.apply(ParDo.of(new UnzipFN()));
p.run();
}
catch(Exception e){
LOG.error(e.getMessage());
}
}
public static class UnzipFN extends DoFn<String,Long>{
private static final long serialVersionUID = 2015166770614756341L;
private long filesUnzipped=0;
#Override
public void processElement(ProcessContext c){
String p = c.element();
GcsUtilFactory factory = new GcsUtilFactory();
GcsUtil u = factory.create(c.getPipelineOptions());
byte[] buffer = new byte[100000000];
try{
SeekableByteChannel sek = u.open(GcsPath.fromUri(p));
InputStream is = Channels.newInputStream(sek);
BufferedInputStream bis = new BufferedInputStream(is);
ZipInputStream zis = new ZipInputStream(bis);
ZipEntry ze = zis.getNextEntry();
while(ze!=null){
LOG.info("Unzipping File {}",ze.getName());
WritableByteChannel wri = u.create(GcsPath.fromUri("gs://tlogdataflow/test/" + ze.getName()), getType(ze.getName()));
OutputStream os = Channels.newOutputStream(wri);
int len;
while((len=zis.read(buffer))>0){
os.write(buffer,0,len);
}
os.close();
filesUnzipped++;
ze=zis.getNextEntry();
}
zis.closeEntry();
zis.close();
}
catch(Exception e){
e.printStackTrace();
}
c.output(filesUnzipped);
}
private String getType(String fName){
if(fName.endsWith(".zip")){
return "application/x-zip-compressed";
}
else {
return "text/plain";
}
}
}
}
Dataflow / Apache Beam support ZIP-compressed files in TextIO automatically: TextIO.read().from(filepattern) will automatically decompress files matching the filepattern according to their extension, and .zip is one of the supported formats - in that case it will implicitly concatenate all files inside the .zip into a single file, and parse lines of text from that.
You can also specify compression type explicitly using TextIO.read().from(filepattern).withCompressionType(...) if the files don't have an extension.
I have started using Xsd2Code and to date have been deserializing XML straight from an actual file.
What I need to do now is deserialize the xml from a local variable.
Here's a code snippet.
using (FileStream getResponseDataFromFile = new FileStream (#:\Temp\Output\DeclarationResponse.xml", FileMode.Open))
{
XmlSerializer serializeGbResponseXML = new XmlSerializer(typeof(declarationGbResponse));
declarationGbResponse myResponse = (declarationGbResponse)serializeGbResponseXML.Deserialize(getResponseDataFromFile);
foreach (var acceptanceResponseItem in myResponse.acceptanceResponse)
{
........
}
What I need to do is replace loading the XML from a FileSteam c:\temp... and instead parse it from a local variable then deserialize it from that variable. I can then use the class created by Xsd2Code and display and use the various properties.
This will work for you as long as "payloadXML" is a well formed xml string.
public myType DeserializeEstimatePayload(string payloadXML)
{
myType est = null;
XmlSerializer xmlSerializer = new XmlSerializer(typeof(myType ));
MemoryStream memStream = new MemoryStream(Encoding.Unicode.GetBytes(payloadXML));
est = (myType )xmlSerializer.Deserialize(memStream);
xmlSerializer = null; memStream = null;
return est;
}
I would like to save the PhotoResult from the cameraCaptureTask into my class which I'm using as a collection and then saving into Isolated Storage:
void cameraCaptureTask_Completed(object sender, PhotoResult e)
This is part of an ObservableCollection. I want to save the photo into this collection.
[DataMember]
public Image VehicleImage
{
get
{
return _vehicleImage;
}
set
{
if (value != _vehicleImage)
{
_vehicleImage = value;
NotifyPropertyChanged("VehicleImage");
}
}
}
I'm using the example from: http://www.blog.ingenuitynow.net and in the example it works fine, but it is setting up an individual Isolated Storage and I would just like to join to my existing collection.
I'm thinking that I can't use the Image type. What would be the best way to accomplish what I'm hoping to do?
Just to answer the comment below. This is what the .Save is doing:
public static void Save<T>(string name, T objectToSave)
{
using (IsolatedStorageFile storageFile = IsolatedStorageFile.GetUserStoreForApplication())
using (IsolatedStorageFileStream storageFileStream = new IsolatedStorageFileStream(name, System.IO.FileMode.Create, storageFile))
{
DataContractSerializer serializer = new DataContractSerializer(typeof(T));
serializer.WriteObject(storageFileStream, objectToSave);
}
}
I think I finally figured out your issue. In your ObservableCollection I personally would not keep an image in there. Instead I would keep a BitmapSource to use less resources, however you may have reasoning why your doing that.
My Process
Convert the Image.Source(BitmapSource) to a byte[]
Save the byte[] to storage
Load the byte[] from storage
Convert the byte[] to and a Image.Source(BitmapSource)
Save Generic To Isolated Storage (In my utility class: IsolatedStorage_Utility.cs)
public static void Save<T>(string fileName, T item)
{
using (IsolatedStorageFile storage = IsolatedStorageFile.GetUserStoreForApplication())
{
using (IsolatedStorageFileStream fileStream = new IsolatedStorageFileStream(fileName, FileMode.Create, storage))
{
DataContractSerializer serializer = new DataContractSerializer(typeof(T));
serializer.WriteObject(fileStream, item);
}
}
}
Load Generic To Isolated Storage (In my utility class: IsolatedStorage_Utility.cs)
public static T Load<T>(string fileName)
{
using (IsolatedStorageFile storage = IsolatedStorageFile.GetUserStoreForApplication())
{
using (IsolatedStorageFileStream fileStream = new IsolatedStorageFileStream(fileName, FileMode.Open, storage))
{
DataContractSerializer serializer = new DataContractSerializer(typeof(T));
return (T)serializer.ReadObject(fileStream);
}
}
}
Convert BitmapSource to byte[] (In my utility class: Image_Utility.cs)
public static byte[] ImageToByteArray(BitmapSource bitmapSource)
{
using (MemoryStream stream = new MemoryStream())
{
WriteableBitmap writableBitmap = new WriteableBitmap(bitmapSource);
Extensions.SaveJpeg(writableBitmap, stream, bitmapSource.PixelWidth, bitmapSource.PixelHeight, 0, 100);
return stream.ToArray();
}
}
Convert byte[] to BitmapSource (In my utility class: Image_Utility.cs)
public static BitmapSource ByteArrayToImage(byte[] bytes)
{
BitmapImage bitmapImage = null;
using (MemoryStream stream = new MemoryStream(bytes, 0, bytes.Length))
{
bitmapImage = new BitmapImage();
bitmapImage.SetSource(stream);
}
return bitmapImage;
}
Example
private void TestImageConversion(object sender, RoutedEventArgs e)
{
byte[] image1AsByteArray = Image_Utility.ImageToByteArray((BitmapSource)Image1.Source);
IsolatedStorage_Utility.Save<byte[]>("Image1.jpg", image1AsByteArray);
BitmapSource image1AsBitmapImage = Image_Utility.ByteArrayToImage(IsolatedStorage_Utility.Load<byte[]>("Image1.jpg"));
Image2.Source = image1AsBitmapImage;
}
Keep in mind this is a jpg saving. If you want to save a png thn you need to use a library of CodePlex or create your own PNGEncoder.
I hope this helps!
Actually in that blog, he knows the ImageFileName stored recently so he is able to retreive the same image from the Isolated storage. i dont think so that example helps you according to your comment.
But If you want store the Picture along with the object means you have to serialize whole object along with the picture taken.
serializing the picture is achieved by converting stream you got to byte[] array and you can convert from byte[] array to BitmapImage again.)
Image conversion and serialization is expalianed here in this link
use this sample and you can serialize with whole object.
In this example I'm excepting that you got ObservableCollection where you want to store all of the images lets say it's name is VehicleImages.
So at the cameraCaptureTask_Completed you load all of the data from IsolatedStorage to VehicleImages and now you add the new VehicleImage to VehicleImages and save it to IsolatedStorage.
Code for save and load:
public static void Save<T>(string name, T objectToSave)
{
using (IsolatedStorageFile storageFile = IsolatedStorageFile.GetUserStoreForApplication())
{
using (IsolatedStorageFileStream storageFileStream = new IsolatedStorageFileStream(name, System.IO.FileMode.Create, storageFile))
{
DataContractSerializer serializer = new DataContractSerializer(typeof(T));
serializer.WriteObject(storageFileStream, objectToSave);
}
}
}
public ObservableCollection<T> Read<T>(string name)
{
using (IsolatedStorageFile storageFile = IsolatedStorageFile.GetUserStoreForApplication())
{
using (IsolatedStorageFileStream storageFileStream = new IsolatedStorageFileStream(name, System.IO.FileMode.Open, storageFile))
{
DataContractSerializer serializer = new DataContractSerializer(typeof(T));
return (ObservableCollection<T>)serializer.ReadObject(storageFileStream);
}
}
}