I'm building a rails app that takes information about products from an XML datafeed hosted on a 3rd party server. This XML is sent gzipped, and I'm having serious difficulty in getting anywhere with it.
I've spent a fair bit of time with Google on this, but the results of my searching seem to be more about Sending Gzipped output rather than receiving a Gzipped input.
The closed I've come to a solution came from StackOverflow, but I'm still getting errors.
What I'm trying to do in the first instance is print the XML data to the browser, then I can start with the processing of it. Here's my current code:
def load_data
url = "http://xml.domain.com/datafeed/"
xml_input = Net::HTTP.get(URI.parse(url))
zstream = Zlib::Inflate.new
#xml_output = zstream.inflate(xml_input)
zstream.finish
zstream.close
end
The error I'm getting from it is:
Zlib::DataError in Cron/get datafeedController#load_data
incorrect header check
I guess this means that the data isn't in the format that is expected, but I can't find information about how to do this properly anywhere. Two things I've ruled out is that the URL is valid and the response is Gzipped, but I'm stuck with how to get past this.
Any help would be greatly appreciated :-)
Sorted!
file = Net::HTTP.get(URI.parse(url))
gz = Zlib::GzipReader.new(StringIO.new(file))
whole_xml = gz.read
Then to load into Hpricot to do the XML parsing:
hp = Hpricot(whole_xml)
Related
I'm sorry, it may be a couple issues and not sure how to phrase the question properly. OK, tried a number of solutions over 2 days to no avail...thank you in advance for your help!
A photo is sent from an iPad app using POST API in the format of base64 (no meta data, just the base64 blob). I'm trying to simply decode and save locally.
I'm testing using Postman:
...com/api/register?first_name=John&photo=/9j/4AAQSkZJRgABAQAAAQAB...[base 64 image of about 400kb]
In Laravel, I am using Intervention
$jpg_url = "image-".time().".jpg";
$path = "/public/".$jpg_url;
$base=base64_decode($customer['photo']);
Image::make($base)->save($path);
and getting an "Unable to init from given binary data" error.
Here's what I don't quite understand and would appreciate a TIL5 explanation:
- When I save the POST from iPad directly into the DB with the following:
$photo = $customer->photo = $customer['photo']
The blob in mysql looks good, I can manually copy and paste it to a web decoder fine.
However, when I use postman, $photo has "+" in the base64 changed into spaces and the image doesn't render
Is this a datatype issue? I'm receiving a long blob that is trying to be converted into string? What is the best practice of receiving images from a mobile app?
So I've search everywhere. Xamarin Docs, goggle, here, W3.
All I need to do is store some small data in an XML file.
I created the XML, got the code lined up and when i go to build it.
IOS.....Can't find file.
I've googled the answer countless times, and they all say the same thing, Make sure it is set as Content or make sure it is "Embedded Resource" I've tried it both ways, It can't find the file to access it. Is IOS really that stupid? No issues in Android, took it 30 secs. Add it to the Assets and boom there it is.
But How to get IOS to Recognize xml file(find it)?
the code is this
XDocuent doc = new XDocument.Load("StoredLogs.xml") <that line is where it throws the error, through all the break points that it is.
After this it steps through a loop to bind the data in the xml to an object
Logs a.Id = x.Element("Id).Value......
a.name......... and so
All i want is basic offline storage.
iOS really that stupid?
Yes :P
When you add the XML file as an EmbeddedResource, you need to read it from the assembly instead of the path
For example:
var readme = typeof(NameSpace.App).GetTypeInfo().Assembly
.GetManifestResourceStrean("resourcename.xml");
using (var sr = new StreamReader(readme)) {
//Read the stream
}
I am using Spreadsheet to parse xls files. It's working great. but today one of my user has uploaded a file and it's getting an error "unknown encoding name - MACINTOSH"
My code is:
book = Spreadsheet.open file_path
sheet1 = book.worksheet 0
This is the excel file which is getting the error:
https://www.dropbox.com/s/jv37pk5rpiy9259/testlisttextnonames2.xls
Can you guys please help me to solve this issue.
Running into the same problem. Here is the best i can deduce:
"Macintosh" is not a known encoding in Ruby 1.9+. Try opening a console and running "Encoding.find('Macintosh')". You'll get the same error.
So what is available? In the console: "Encoding.list". One of the options is MacRoman. I'm guessing this is a close second.
So if we change the mapping in lib/spreadsheet/excel/internals.rb for 10000 and 32768 to map to "MACROMAN", it should work.
Tested locally and it does.
I opened a pull request: https://github.com/zdavatz/spreadsheet/pull/51
Some reference links:
* https://en.wikipedia.org/wiki/Mac_OS_Roman
* http://bugs.python.org/issue843590
Firstly, I am aware that there are quite a few questions that are similar to this one in SO. I have read most, if not all of them, over the past week. But I still can't make this work for me.
I am developing a Ruby on Rails app that allows users to upload mp3 files to Amazon S3. The upload itself works perfectly, but a progress bar would greatly improve user experience on the website.
I am using the aws-sdk gem which is the official one from Amazon. I have looked everywhere in its documentation for callbacks during the upload process, but I couldn't find anything.
The files are uploaded one at a time directly to S3 so it doesn't need to load it into memory. No multiple file upload necessary either.
I figured that I may need to use JQuery to make this work and I am fine with that.
I found this that looked very promising: https://github.com/blueimp/jQuery-File-Upload
And I even tried following the example here: https://github.com/ncri/s3_uploader_example
But I just could not make it work for me.
The documentation for aws-sdk also BRIEFLY describes streaming uploads with a block:
obj.write do |buffer, bytes|
# writing fewer than the requested number of bytes to the buffer
# will cause write to stop yielding to the block
end
But this is barely helpful. How does one "write to the buffer"? I tried a few intuitive options that would always result in timeouts. And how would I even update the browser based on the buffering?
Is there a better or simpler solution to this?
Thank you in advance.
I would appreciate any help on this subject.
The "buffer" object yielded when passing a block to #write is an instance of StringIO. You can write to the buffer using #write or #<<. Here is an example that uses the block form to upload a file.
file = File.open('/path/to/file', 'r')
obj = s3.buckets['my-bucket'].objects['object-key']
obj.write(:content_length => file.size) do |buffer, bytes|
buffer.write(file.read(bytes))
# you could do some interesting things here to track progress
end
file.close
After read the source code of the AWS gem, I've adapted (or mostly copy) the multipart upload method to yield the current progress based on how many chunks have been uploaded
s3 = AWS::S3.new.buckets['your_bucket']
file = File.open(filepath, 'r', encoding: 'BINARY')
file_to_upload = "#{s3_dir}/#{filename}"
upload_progress = 0
opts = {
content_type: mime_type,
cache_control: 'max-age=31536000',
estimated_content_length: file.size,
}
part_size = self.compute_part_size(opts)
parts_number = (file.size.to_f / part_size).ceil.to_i
obj = s3.objects[file_to_upload]
begin
obj.multipart_upload(opts) do |upload|
until file.eof? do
break if (abort_upload = upload.aborted?)
upload.add_part(file.read(part_size))
upload_progress += 1.0/parts_number
# Yields the Float progress and the String filepath from the
# current file that's being uploaded
yield(upload_progress, upload) if block_given?
end
end
end
The compute_part_size method is defined here and I've modified it to this:
def compute_part_size options
max_parts = 10000
min_size = 5242880 #5 MB
estimated_size = options[:estimated_content_length]
[(estimated_size.to_f / max_parts).ceil, min_size].max.to_i
end
This code was tested on Ruby 2.0.0p0
OK, I have a situation where I use loadvars to load an xml file, which has the address of 2 swf's on the server. I can trace the xml items so know they are loading. So I'm trying to load the swf's but am running into problems. First problem, the swf always gives an error when I publish from flash, "TypeError: Error #2007: Parameter url must be non-null." as it can't find the mention of the xml file, as it's just being published. I'm not sure this affects the next issue which is the swf not loading when I try to load it like this:
var loader:Loader = new Loader();
var defaultSWF:URLRequest = new URLRequest(loadswf1); // where I know loadswf1 = "myswf.swf"
loader.load(defaultSWF);
any advice appreciated,
Thanks
Sorry if I'm stating the obvious but there's very little code to go by... Are you starting to load your SWFs after the XML loading has completed?