We are having issues with the Postscript code generated by the "Canon iR-ADV C5235/5240 PS3" printer driver. We print a test document (3 A4 pages of Lorem Ipsum text) and get the following Postscript output in the Windows spool directory:
http://files.etvdzs.info/original.ps
We have tried various Postscript viewers/converters, and they are unable to handle this file. The reason is that the file contains sections of binary data at the beginning and end, and also the following byte sequence at position 0x1060b:
cd ca 10 02 00 1a 00 01 82 6f ff ff 00 00 00 00 00 00 00 00 01
If we remove these three binary sequences, we get the resulting file, which works fine in most Postscript viewers/converters we have tried:
http://files.etvdzs.info/cleaned.ps
Has anybody else encountered similar issues with Canon printer drivers? Does anybody know what these binary sequences signify, or what format they are in?
The binary sequences are CPCA codes. One can download documentation about the data structures used in CPCA after registering here:
https://www.developersupport.canon.com/user/register
It is then reasonably straightforward to write a program that strips the CPCA codes out of the file. The file can then be loaded successfully by 3rd-party Postscript viewers/converters.
Related
When I drag a file attachment from Outlook Online (Web, rendered in Firefox) to my app, I receive the following drop elements:
But how do I get the attachment contents (the raw bytes of the attachment) from these? None of the elements are big enough to contain the 250K file that is attached, and the IStream content consists just of these 16 bytes:
00 00 00 00 01 00 00 00 00 00 00 00 00 00 00 00
My suspicion is that it is this IStream that I must use to stream the bytes from the browser to my application, but I can't fathom how to convert these bytes to a stream in my Delphi application.
Edit: JSON from application/x-moz-custom-clipdata:
{
"itemType":"attachment",
"attachmentFile":{
"attachmentItemId":"AAM...BASE64...H6M=",
"name":"<FileName>.pdf",
"size":254330,
"fileType":5,
"type":"ItemIdAttachment:#Exchange",
"ContentType":"application/pdf"
}
}
Use TOleStream from Vcl.AxCtrls to convert from IStream to TStream and read the bytes.
you must user Graph API to get attachemen file
I have some Lua code which appears to be an attempt to secure the code by obscurity. My understanding of the loadstring() function is a text string is composed of Lua source code text and then converted to executable Lua code by the loadstring() method.
With the following Lua source, I tried to read the contents of the variable code by invoking print on the variable code; while I did see some valid source text in the converted string, a majority of the characters were not displayed (I assume ones with character codes below 40 and above 176). Note that there are some particularly high values in there for ASCII, e.g. 231 is obviously in the extended set, being the trademark sign. Additionally, there are several null characters in there. All this makes me doubt if it is indeed ASCII.
Could someone please tell me if the string is valid Lua source, and how to be able to get Lua to return the string as printable characters so that I can see what this code does?
When I run my version with print in the Lua console on Windows I get many empty boxes, presumably the console can only print pure ASCII?
Note that the code is executed using Lua version 5.0.2
code='\27\76\117\97\80\1\4\4\4\6\8\9\9\8\182\9\147\104\231\245\125\65\12\0\0\0\64\108\117\97\101\109\103\46\108\117\97\0\1\0\0\0\0\0\0\5\23\0\0\0\8\0\0\0\16\0\0\0\17\0\0\0\17\0\0\0\17\0\0\0\17\0\0\0\17\0\0\0\18\0\0\0\18\0\0\0\19\0\0\0\20\0\0\0\21\0\0\0\35\0\0\0\35\0\0\0\26\0\0\0\49\0\0\0\49\0\0\0\37\0\0\0\59\0\0\0\59\0\0\0\54\0\0\0\61\0\0\0\66\0\0\0\2\0\0\0\4\0\0\0\104\52\120\0\1\0\0\0\22\0\0\0\7\0\0\0\77\111\100\117\108\101\0\12\0\0\0\22\0\0\0\0\0\0\0\12\0\0\0\4\13\0\0\0\122\122\97\78\111\100\101\78\97\109\101\115\0\4\6\0\0\0\90\90\65\48\49\0\4\6\0\0\0\90\90\65\48\50\0\4\14\0\0\0\122\122\97\84\101\120\116\90\101\105\108\101\110\0\4\12\0\0\0\122\122\97\80\111\115\105\116\105\111\110\0\3\0\0\0\0\0\0\240\63\4\8\0\0\0\122\122\97\84\101\120\116\0\4\1\0\0\0\0\4\20\0\0\0\122\122\97\67\117\114\114\101\110\116\84\101\120\116\86\97\108\117\101\0\4\9\0\0\0\122\122\97\83\101\116\117\112\0\4\10\0\0\0\122\122\97\83\101\108\101\99\116\0\4\9\0\0\0\122\122\97\82\101\115\101\116\0\4\0\0\0\0\0\0\0\2\0\0\0\0\1\0\7\14\0\0\0\3\0\0\0\3\0\0\0\4\0\0\0\4\0\0\0\4\0\0\0\5\0\0\0\5\0\0\0\5\0\0\0\5\0\0\0\4\0\0\0\5\0\0\0\7\0\0\0\7\0\0\0\8\0\0\0\4\0\0\0\7\0\0\0\115\116\114\116\98\108\0\0\0\0\0\13\0\0\0\16\0\0\0\40\102\111\114\32\103\101\110\101\114\97\116\111\114\41\0\5\0\0\0\11\0\0\0\12\0\0\0\40\102\111\114\32\115\116\97\116\101\41\0\5\0\0\0\11\0\0\0\2\0\0\0\118\0\5\0\0\0\11\0\0\0\0\0\0\0\2\0\0\0\4\7\0\0\0\98\117\102\102\101\114\0\4\1\0\0\0\0\0\0\0\0\14\0\0\0\65\0\0\1\7\0\0\1\0\0\0\1\3\128\1\2\222\0\128\1\5\0\0\4\198\0\0\5\83\1\2\4\7\0\0\4\29\0\0\1\84\254\127\0\5\0\0\1\27\0\1\1\27\128\0\0\0\0\0\0\26\0\0\0\1\1\0\4\18\0\0\0\27\0\0\0\28\0\0\0\28\0\0\0\29\0\0\0\29\0\0\0\30\0\0\0\32\0\0\0\32\0\0\0\32\0\0\0\32\0\0\0\32\0\0\0\33\0\0\0\33\0\0\0\33\0\0\0\33\0\0\0\33\0\0\0\27\0\0\0\35\0\0\0\2\0\0\0\8\0\0\0\122\122\97\70\105\108\101\0\0\0\0\0\17\0\0\0\6\0\0\0\122\101\105\108\101\0\3\0\0\0\16\0\0\0\1\0\0\0\7\0\0\0\77\111\100\117\108\101\0\5\0\0\0\4\5\0\0\0\114\101\97\100\0\0\4\14\0\0\0\122\122\97\84\101\120\116\90\101\105\108\101\110\0\4\12\0\0\0\122\122\97\80\111\115\105\116\105\111\110\0\3\0\0\0\0\0\0\240\63\0\0\0\0\18\0\0\0\148\3\128\0\139\62\0\1\153\0\1\1\85\128\125\0\20\0\128\0\148\2\128\0\4\0\0\2\6\63\1\2\4\0\0\3\70\191\1\3\73\128\1\2\4\0\0\2\4\0\0\3\70\191\1\3\140\191\1\3\201\128\126\2\212\251\127\0\27\128\0\0\0\0\0\0\37\0\0\0\1\2\0\7\21\0\0\0\39\0\0\0\39\0\0\0\39\0\0\0\39\0\0\0\39\0\0\0\40\0\0\0\40\0\0\0\40\0\0\0\40\0\0\0\43\0\0\0\46\0\0\0\46\0\0\0\46\0\0\0\46\0\0\0\46\0\0\0\46\0\0\0\46\0\0\0\46\0\0\0\46\0\0\0\46\0\0\0\49\0\0\0\3\0\0\0\6\0\0\0\118\97\108\117\101\0\0\0\0\0\20\0\0\0\9\0\0\0\110\111\100\101\78\97\109\101\0\0\0\0\0\20\0\0\0\20\0\0\0\122\122\97\83\101\108\101\99\116\101\100\80\111\115\105\116\105\111\110\0\10\0\0\0\20\0\0\0\1\0\0\0\7\0\0\0\77\111\100\117\108\101\0\7\0\0\0\4\8\0\0\0\122\122\97\84\101\120\116\0\4\14\0\0\0\122\122\97\84\101\120\116\90\101\105\108\101\110\0\4\20\0\0\0\122\122\97\67\117\114\114\101\110\116\84\101\120\116\86\97\108\117\101\0\4\5\0\0\0\67\97\108\108\0\4\5\0\0\0\90\90\65\48\0\4\14\0\0\0\58\65\99\116\105\118\97\116\101\78\111\100\101\0\3\0\0\0\0\0\0\240\63\0\0\0\0\21\0\0\0\4\0\0\2\4\0\0\3\198\190\1\3\6\128\1\3\201\0\125\2\4\0\0\2\4\0\0\3\134\190\1\3\201\0\126\2\0\0\0\2\197\0\0\3\1\1\0\4\0\128\0\5\65\1\0\6\147\1\2\4\1\1\0\5\0\0\1\6\147\129\2\5\129\1\0\6\89\0\2\3\27\128\0\0\0\0\0\0\54\0\0\0\1\0\0\4\19\0\0\0\56\0\0\0\56\0\0\0\56\0\0\0\56\0\0\0\56\0\0\0\56\0\0\0\56\0\0\0\56\0\0\0\56\0\0\0\57\0\0\0\57\0\0\0\57\0\0\0\57\0\0\0\57\0\0\0\57\0\0\0\57\0\0\0\57\0\0\0\57\0\0\0\59\0\0\0\0\0\0\0\1\0\0\0\7\0\0\0\77\111\100\117\108\101\0\7\0\0\0\4\5\0\0\0\67\97\108\108\0\4\13\0\0\0\122\122\97\78\111\100\101\78\97\109\101\115\0\3\0\0\0\0\0\0\240\63\4\14\0\0\0\58\65\99\116\105\118\97\116\101\78\111\100\101\0\4\4\0\0\0\97\108\108\0\3\0\0\0\0\0\0\0\0\3\0\0\0\0\0\0\0\64\0\0\0\0\19\0\0\0\5\0\0\0\4\0\0\1\198\190\0\1\6\191\0\1\193\0\0\2\147\128\0\1\1\1\0\2\65\1\0\3\89\0\2\0\5\0\0\0\4\0\0\1\198\190\0\1\6\192\0\1\193\0\0\2\147\128\0\1\1\1\0\2\65\1\0\3\89\0\2\0\27\128\0\0\23\0\0\0\34\0\0\0\202\0\0\1\10\0\1\2\65\0\0\3\129\0\0\4\95\0\0\2\137\0\125\1\10\0\0\2\137\128\126\1\201\63\127\1\73\64\128\1\73\64\129\1\98\0\0\2\0\128\0\0\137\128\129\1\162\0\0\2\0\128\0\0\137\0\130\1\226\0\0\2\0\128\0\0\137\128\130\1\27\0\1\1\27\128\0\0';
return loadstring(code)();
This string is valid chunk of Lua code precompiled into bytecode. Header say it's for Lua 5.0. It's not a text, it doesn't need decoding, so can be run directly with loadstring()
To provide a few more details than Vlad's answer for anyone who may come across this posting.
The Lua loadstring() function accepts a string of characters that are either Lua source text or Lua bytecode. It appears that the function determines type of the text by looking at the first character of the string to see if it is an escape character (0x1b or decimal 27) or not.
The loadstring() function returns an anonymous function so in the code sample:
code='\27\76\117\97\80\1\4\4\4\6\8\9\9\8\182\9\147\104\231\245\125\65\12\0\0\0\64\108\117\97\101\109\103\46\108\117\97\0\1\0\0\0\0\0\0\5\23\0\0\0\8\0\0\0\16\0\0\0\17\0\0\0\17\0\0\0\17\0\0\0\17\0\0\0\17\0\0\0\18\0\0\0\18\0\0\0\19\0\0\0\20\0\0\0\21\0\0\0\35\0\0\0\35\0\0\0\26\0\0\0\49\0\0\0\49\0\0\0\37\0\0\0\59\0\0\0\59\0\0\0\54\0\0\0\61\0\0\0\66\0\0\0\2\0\0\0\4\0\0\0\104\52\120\0\1\0\0\0\22\0\0\0\7\0\0\0\77\111\100\117\108\101\0\12\0\0\0\22\0\0\0\0\0\0\0\12\0\0\0\4\13\0\0\0\122\122\97\78\111\100\101\78\97\109\101\115\0\4\6\0\0\0\90\90\65\48\49\0\4\6\0\0\0\90\90\65\48\50\0\4\14\0\0\0\122\122\97\84\101\120\116\90\101\105\108\101\110\0\4\12\0\0\0\122\122\97\80\111\115\105\116\105\111\110\0\3\0\0\0\0\0\0\240\63\4\8\0\0\0\122\122\97\84\101\120\116\0\4\1\0\0\0\0\4\20\0\0\0\122\122\97\67\117\114\114\101\110\116\84\101\120\116\86\97\108\117\101\0\4\9\0\0\0\122\122\97\83\101\116\117\112\0\4\10\0\0\0\122\122\97\83\101\108\101\99\116\0\4\9\0\0\0\122\122\97\82\101\115\101\116\0\4\0\0\0\0\0\0\0\2\0\0\0\0\1\0\7\14\0\0\0\3\0\0\0\3\0\0\0\4\0\0\0\4\0\0\0\4\0\0\0\5\0\0\0\5\0\0\0\5\0\0\0\5\0\0\0\4\0\0\0\5\0\0\0\7\0\0\0\7\0\0\0\8\0\0\0\4\0\0\0\7\0\0\0\115\116\114\116\98\108\0\0\0\0\0\13\0\0\0\16\0\0\0\40\102\111\114\32\103\101\110\101\114\97\116\111\114\41\0\5\0\0\0\11\0\0\0\12\0\0\0\40\102\111\114\32\115\116\97\116\101\41\0\5\0\0\0\11\0\0\0\2\0\0\0\118\0\5\0\0\0\11\0\0\0\0\0\0\0\2\0\0\0\4\7\0\0\0\98\117\102\102\101\114\0\4\1\0\0\0\0\0\0\0\0\14\0\0\0\65\0\0\1\7\0\0\1\0\0\0\1\3\128\1\2\222\0\128\1\5\0\0\4\198\0\0\5\83\1\2\4\7\0\0\4\29\0\0\1\84\254\127\0\5\0\0\1\27\0\1\1\27\128\0\0\0\0\0\0\26\0\0\0\1\1\0\4\18\0\0\0\27\0\0\0\28\0\0\0\28\0\0\0\29\0\0\0\29\0\0\0\30\0\0\0\32\0\0\0\32\0\0\0\32\0\0\0\32\0\0\0\32\0\0\0\33\0\0\0\33\0\0\0\33\0\0\0\33\0\0\0\33\0\0\0\27\0\0\0\35\0\0\0\2\0\0\0\8\0\0\0\122\122\97\70\105\108\101\0\0\0\0\0\17\0\0\0\6\0\0\0\122\101\105\108\101\0\3\0\0\0\16\0\0\0\1\0\0\0\7\0\0\0\77\111\100\117\108\101\0\5\0\0\0\4\5\0\0\0\114\101\97\100\0\0\4\14\0\0\0\122\122\97\84\101\120\116\90\101\105\108\101\110\0\4\12\0\0\0\122\122\97\80\111\115\105\116\105\111\110\0\3\0\0\0\0\0\0\240\63\0\0\0\0\18\0\0\0\148\3\128\0\139\62\0\1\153\0\1\1\85\128\125\0\20\0\128\0\148\2\128\0\4\0\0\2\6\63\1\2\4\0\0\3\70\191\1\3\73\128\1\2\4\0\0\2\4\0\0\3\70\191\1\3\140\191\1\3\201\128\126\2\212\251\127\0\27\128\0\0\0\0\0\0\37\0\0\0\1\2\0\7\21\0\0\0\39\0\0\0\39\0\0\0\39\0\0\0\39\0\0\0\39\0\0\0\40\0\0\0\40\0\0\0\40\0\0\0\40\0\0\0\43\0\0\0\46\0\0\0\46\0\0\0\46\0\0\0\46\0\0\0\46\0\0\0\46\0\0\0\46\0\0\0\46\0\0\0\46\0\0\0\46\0\0\0\49\0\0\0\3\0\0\0\6\0\0\0\118\97\108\117\101\0\0\0\0\0\20\0\0\0\9\0\0\0\110\111\100\101\78\97\109\101\0\0\0\0\0\20\0\0\0\20\0\0\0\122\122\97\83\101\108\101\99\116\101\100\80\111\115\105\116\105\111\110\0\10\0\0\0\20\0\0\0\1\0\0\0\7\0\0\0\77\111\100\117\108\101\0\7\0\0\0\4\8\0\0\0\122\122\97\84\101\120\116\0\4\14\0\0\0\122\122\97\84\101\120\116\90\101\105\108\101\110\0\4\20\0\0\0\122\122\97\67\117\114\114\101\110\116\84\101\120\116\86\97\108\117\101\0\4\5\0\0\0\67\97\108\108\0\4\5\0\0\0\90\90\65\48\0\4\14\0\0\0\58\65\99\116\105\118\97\116\101\78\111\100\101\0\3\0\0\0\0\0\0\240\63\0\0\0\0\21\0\0\0\4\0\0\2\4\0\0\3\198\190\1\3\6\128\1\3\201\0\125\2\4\0\0\2\4\0\0\3\134\190\1\3\201\0\126\2\0\0\0\2\197\0\0\3\1\1\0\4\0\128\0\5\65\1\0\6\147\1\2\4\1\1\0\5\0\0\1\6\147\129\2\5\129\1\0\6\89\0\2\3\27\128\0\0\0\0\0\0\54\0\0\0\1\0\0\4\19\0\0\0\56\0\0\0\56\0\0\0\56\0\0\0\56\0\0\0\56\0\0\0\56\0\0\0\56\0\0\0\56\0\0\0\56\0\0\0\57\0\0\0\57\0\0\0\57\0\0\0\57\0\0\0\57\0\0\0\57\0\0\0\57\0\0\0\57\0\0\0\57\0\0\0\59\0\0\0\0\0\0\0\1\0\0\0\7\0\0\0\77\111\100\117\108\101\0\7\0\0\0\4\5\0\0\0\67\97\108\108\0\4\13\0\0\0\122\122\97\78\111\100\101\78\97\109\101\115\0\3\0\0\0\0\0\0\240\63\4\14\0\0\0\58\65\99\116\105\118\97\116\101\78\111\100\101\0\4\4\0\0\0\97\108\108\0\3\0\0\0\0\0\0\0\0\3\0\0\0\0\0\0\0\64\0\0\0\0\19\0\0\0\5\0\0\0\4\0\0\1\198\190\0\1\6\191\0\1\193\0\0\2\147\128\0\1\1\1\0\2\65\1\0\3\89\0\2\0\5\0\0\0\4\0\0\1\198\190\0\1\6\192\0\1\193\0\0\2\147\128\0\1\1\1\0\2\65\1\0\3\89\0\2\0\27\128\0\0\23\0\0\0\34\0\0\0\202\0\0\1\10\0\1\2\65\0\0\3\129\0\0\4\95\0\0\2\137\0\125\1\10\0\0\2\137\128\126\1\201\63\127\1\73\64\128\1\73\64\129\1\98\0\0\2\0\128\0\0\137\128\129\1\162\0\0\2\0\128\0\0\137\0\130\1\226\0\0\2\0\128\0\0\137\128\130\1\27\0\1\1\27\128\0\0';
return loadstring(code)();
you have a text string that contains Lua bytecode, as indicated by the leading escape character of \27, and then a call to loadstring() to create a function which is then executed.
The first few characters of the text string contain the precompiled Lua header (see Lua 5.2 Bytecode and Virtual Machine). The length of this header varies depending on the version of Lua. However the first few characters seem to be fairly standard. code='\27\76\117\97\80 ... contains the escape character (0x1b or decimal 27), the capital letter L (decimal 76), the lower case letter u (decimal 117), the lower case letter a (decimal 97), and the Lua version (decimal 80 is 0x50 indicating version 5.0).
The following example is from Lua 5.2 Bytecode and Virtual Machine.
What exactly is in the bytecode? Here is the hexdump of hello.luac
(made by hd on my system).
00000000 1b 4c 75 61 52 00 01 04 04 04 08 00 19 93 0d 0a |.LuaR...........|
00000010 1a 0a 00 00 00 00 00 00 00 00 00 01 04 07 00 00 |................|
00000020 00 01 00 00 00 46 40 40 00 80 00 00 00 c1 80 00 |.....F##........|
00000030 00 96 c0 00 01 5d 40 00 01 1f 00 80 00 03 00 00 |.....]#.........|
00000040 00 04 06 00 00 00 48 65 6c 6c 6f 00 04 06 00 00 |......Hello.....|
The format is not officially documented, and needs to be
reverse-engineered. The necessary material is in the Lua source code,
of course, in several places, mainly ldump.c and lundump.c. I have
also cross-checked with NFI and LAT, but any remaining errors are
mine.
The code starts with an 18-byte file header, which is the same for all
official Lua 5.2 bytecode compiled on a machine like yours, whether by
luac or load or loadfile. Lua 5.1 only had a 12-byte header, similar
to the first 12 bytes of this one.
Byte numbers are in origin-1 decimal (mostly showing the arithmetic)
and origin-0 hex.
1 x00: 1b 4c 75 61 LUA_SIGNATURE from lua.h.
5 x04: 52 00
Binary-coded decimal 52 for the Lua version, 00 to say the bytecode is
compatible with the "official" PUC-Rio implementation.
5+2 x06: 01 04
04 04 08 00 Six system parameters. On x386 machines they mean:
little-endian, 4-byte integers, 4-byte VM instructions, 4-byte size_t
numbers, 8-byte Lua numbers, floating-point. These parameters must all
match up between the bytecode file and the Lua interpreter, otherwise
the bytecode is invalid.
7+6 x0c: 19 93 0d 0a 1a 0a
Present in all
bytecode produced by Lua 5.2 from PUC-Rio. Described in lundump.h as
"data to catch conversion errors". Might be constructed from
binary-coded decimal 1993 (the year it all started), Windows line
terminator, MS-DOS text file terminator, Unix line terminator.
After these 18 bytes come the functions defined in the file. Each function
starts with an 11-byte function header.
13+6 x12: 00 00 00 00 Line number in source code where chunk starts.
0 for the main chunk.
19+4 x16: 00 00 00 00 Line number in source
code where chunk stops. 0 for the main chunk.
23+4 x1a: 00 01 04
Number of parameters, vararg flag, number of registers used by this
function (not more than 255, obviously). Local variables are stored in
registers; there may not be more than 200 of them (see lparser.c).
We're building an application on Ruby on Rails 4 and it's going really fluently, except for this pain-giving issue
In our application we get a Multipart Response from a SOAP request. This response comes from an external party and sends us a 7zip compressed file. The response has 2 parts, where the second contains a application/octet-stream.
The parts are of the Mail:part type.
When I save the contents of 'part.body.decoded' to a file, I can't open the file with 7zip, because it says the file is damaged.
When I open the generated file in the hex-editor the first lines seem ok.
p part.body.decoded.encoding.name # gives => "ASCII-8BIT"
File.open("file.7z", "wb") do |f|
f.write(part.body.decoded)
end
Here are the first HEX lines.
37 7A BC AF 27 1C 00 03 EF F2 1B 17 EA FA 03 00
00 00 00 00 68 00 00 00 00 00 00 00 BD D9 66 08
00 1E 10 88 27 43 3B A7 F5 C7 75 19 5D F6 A6 E0
As a test I created a 7z file myself and read it in a string and wrote it back to a file. This works and gives me a valid 7zip.
Any ideas on how to tackle this?
Update: when I put the contents of part.body.decoded in a string
str = part.body.decoded #and put the string
p str # I get the string as below (I've deleted the middle section of the string because it has to many characters to post it here)
"7z\xBC\xAF'\x1C\x00\x03\xFCpd\x01\xEA\xFA\x03\x00\x00\x00\x00\x00h\x00\x00\x00\x00\x00\x00\x006E\x91\x1F\x00\x1E\x10\x88'C;\xA7\xF5\xC7u\x19]\xF6\xA6\xE0\xACC\xD9\xA4\xA0:,4\xE9\xA1\xAD%\x05\\a\xFBq\xA4Z\xEC7\xCC\x8C\x91\xBC?\x80H)\xFBa\xB8\xB8u\xBE\xB2\x1C|`\xA7P|\xC3\xA9\x82\xB3\r\n\xA2\x15\xA2\xDAn\xBB\xB9
Solved :-) The Savon 2 Soap Gem depends on the Mail Gem (max. version 2.5). In the Mail gem there is a bug, which touches the raw binary reponse (it ads line-ends). I've forked the Mail gem and made adaptions (to be found in the higher version of Mail (which Savon doesn't use)) and added 'my' Mail gem to my gemfile.
Thnx to all commenters for their tips!
I'd like to parse a proprietary message with the following format. The messages are of variable length from 8bytes to 16 bytes. If the message is more than 8 bytes, it is put into multi packet message.
Byte0 Byte1 Byte2 Byte3 Byte4 Byte5 Byte6 Byte7-14 Byte15
Header Header Flags SlaveID Sequence Command Command Data Chksum
I send these messages to a device and expect a response from said device with similar format. I know what the messages are supposed to say, but I'm really struggling to find a away to parse the data in real time and display the information in somesort of GUI framework. Currently, I'm planning on dumping the data into a .txt/.csv file and post-processing with matlab.
I'm a hardware guy. This is not my strong suit.
Example:
Command, Reponse
0xAC 04 01 01 00 8C 01 01 chksum
0x04 AC 83 01 00 8C 01 01 0B D9 chksum
0x8c01 command to test power supply 0x01
Device echos the command and power supply it was checking and also returns the bit-shifted value of the power supply voltage 0x0BD9
I'm trying to get the url of the *.flv from any youtube video. I used wireshark to analyize the traffic. I have also an addon for firefox which downloads the videos from youtube. It has a feature where I can see the direct url to the *.flv video.
So far I got this :
http://r1---sn-i5onxoxu-i5hl.c.youtube.com (it's the host) This matches to what the downloaders url is.
followed by
"videoplayback?" and then some video specific stuff I guess. And that's exactly what I cannot figure out. The downloader starts with the host, followed by this and then some other stuff.
What I did was capturing the packages. Then I searched in http requests for "/videoplayback?...". I just added it to the host but it doesn't work.
Can someone help me? What do I have to put after the host? And where do I find it?
This is what I figured out so far.
I watched the same video Wireshark - Using Dumpcap to capture to disk by Mike Pennacchi.
After capturing the packets and saving the file, apply the following display filter:
http.request.full_uri contains "range"
There are 3 packets displayed in my file:
735
2708
4615
Select the first packet, go to Packet details and expand Hypertext Transfer Protocol.
Right-click on Full request URI [truncated]: and select Copy -> Value and paste the url in a text file.
Copy also the url's (see Note) from the other packets and you get a list similar to this one:
http://r2---sn-5hn7zn7k.c.youtube.com/videoplayback?algorithm=throttle-factor&burst=40&cp=U0hWR1FUTl9GT0NONl9JTlJBOlBwSjNTRTA0azVw&cpn=B84QkSbXB7EAaDfs&expire=1372190015&factor=1.25&fexp=935500%2C910100%2C929227%2C916613%2C921047%2C928201%2C901208%2C929123%2C929915%2C929906%2C929907%2C929125%2C925714%2C929917%2C929919%2C931202%2C912512%2C912515%2C912521%2C906838%2C904488%2C906840%2C931910%2C931913%2C932227%2C904830%2C919373%2C933701%2C904122%2C900816%2C926403%2C909421%2C912711%2C935102&id=6ca6b1fa4c861ff9&ip=my ip address&ipbits=8&itag=34&keepalive=yes&key=yt1&ms=au&mt=1372168727&mv=m&newshard=yes&range=13-1781759&ratebypass=yes&signature=D8D28CBCD4AAC5F46901397219F62CB04D3C6290.114129F2F93249AC4D88DF184C7D47C9929B3232&source=youtube&sparams=algorithm%2Cburst%2Ccp%2Cfactor%2Cid%2Cip%2Cipbits%2Citag%2Csource%2Cupn%2Cexpire&sver=3&upn=9RmkbstttTM
http://r2---sn-5hn7zn7k.c.youtube.com/videoplayback?algorithm=throttle-factor&burst=40&cp=U0hWR1FUTl9GT0NONl9JTlJBOlBwSjNTRTA0azVw&cpn=B84QkSbXB7EAaDfs&expire=1372190015&factor=1.25&fexp=935500%2C910100%2C929227%2C916613%2C921047%2C928201%2C901208%2C929123%2C929915%2C929906%2C929907%2C929125%2C925714%2C929917%2C929919%2C931202%2C912512%2C912515%2C912521%2C906838%2C904488%2C906840%2C931910%2C931913%2C932227%2C904830%2C919373%2C933701%2C904122%2C900816%2C926403%2C909421%2C912711%2C935102&id=6ca6b1fa4c861ff9&ip=my ip address&ipbits=8&itag=34&keepalive=yes&key=yt1&ms=au&mt=1372168727&mv=m&newshard=yes&range=1781760-3563519&ratebypass=yes&signature=D8D28CBCD4AAC5F46901397219F62CB04D3C6290.114129F2F93249AC4D88DF184C7D47C9929B3232&source=youtube&sparams=algorithm%2Cburst%2Ccp%2Cfactor%2Cid%2Cip%2Cipbits%2Citag%2Csource%2Cupn%2Cexpire&sver=3&upn=9RmkbstttTM
http://r2---sn-5hn7zn7k.c.youtube.com/videoplayback?algorithm=throttle-factor&burst=40&cp=U0hWR1FUTl9GT0NONl9JTlJBOlBwSjNTRTA0azVw&cpn=B84QkSbXB7EAaDfs&expire=1372190015&factor=1.25&fexp=935500%2C910100%2C929227%2C916613%2C921047%2C928201%2C901208%2C929123%2C929915%2C929906%2C929907%2C929125%2C925714%2C929917%2C929919%2C931202%2C912512%2C912515%2C912521%2C906838%2C904488%2C906840%2C931910%2C931913%2C932227%2C904830%2C919373%2C933701%2C904122%2C900816%2C926403%2C909421%2C912711%2C935102&id=6ca6b1fa4c861ff9&ip=my ip address&ipbits=8&itag=34&keepalive=yes&key=yt1&ms=au&mt=1372168727&mv=m&newshard=yes&range=3563520-5347327&ratebypass=yes&signature=D8D28CBCD4AAC5F46901397219F62CB04D3C6290.114129F2F93249AC4D88DF184C7D47C9929B3232&source=youtube&sparams=algorithm%2Cburst%2Ccp%2Cfactor%2Cid%2Cip%2Cipbits%2Citag%2Csource%2Cupn%2Cexpire&sver=3&upn=9RmkbstttTM
Check the range; it is in these long url's:
range=13-1781759
range=1781760-3563519
range=3563520-5347327
Copy the first url and paste it in your browser.
Now you can download and save the first fileas videoplayback_Pennacchi01.
Repeat the steps for parts 2 and 3.
There are a couple of steps to go, before you can play the file.
Open the file videoplayback_Pennacchi01 in a hex editor.
The file starts with:
12 00 03 4B 00 00 00 00 00 00 00 02 00 0A 6F 6E ...K..........on
Prepend the the FLV header:
46 4C 56 01 05 00 00 00 09 00 00 00 00 FLV..........
Now the file looks like:
46 4C 56 01 05 00 00 00 09 00 00 00 00 12 00 03 FLV.............
4B 00 00 00 00 00 00 00 02 00 0A 6F 6E 4D 65 74 K..........onMet
Move you cursor to the end of file 01.
Open files 02 and 03 and copy and paste the content of the files into file 01.
Now you are ready to play the video.
Note 1
These links expire after some time: expire=1372190015
Convert Unix timestamp to Readable Date/time: Tue, 25 Jun 2013 19:53:35 GMT
You can also reconstruct the FLV file from the capture file.
Go to File -> Export Objects -> HTTP
look for Content Type video/x-flv.
Select the first one, hit Save As ans save the file.
Save also the other 2 parts.
Merge the files in order and prepend the FLV header.
Note 2
It used to be very easy. Apply display filter (http.request.method == "GET") and grab the URL from GET request containing /get_video?video.... But things have changed...