I want to develop some simulation software. It produces long arrays of data. Is it good idea to store this data in mkv file with custom codec ? The goal of it is to get fast random access to data and avoid headache with handling big arrays(bigger then 32bit address space)
And if so, is there are any simple mkv c++ library ?
Also, mkv is a specific application of EBML, a sort of binary xml language, optimized for media. If you decided the features are right for you, EBML would be what you would use, which would allow you to customize for your specific application.
mkv is the file extension for the Matroska format, which would help you with your search.
Here is the Matroska source code page, which includes links to EBML and Matroska c libraries.
http://www.matroska.org/team/source-code.html
I'm pretty sure the things you get from mkv are nowhere near as sophisticated for scientific (simulation) data as HDF5. It was designed for exactly the use case you describe.
Related
I'm writing a program (C#) to read, convert, display, adjust and output point cloud data.
I can make every part of the program except for one - I am required to read in a proprietary file format. The data is coming straight from a laser scanner and we cannot get any closer to the stream than what is output to the proprietary file in binary.
I have an SDK from the manufacturer/proprietor that is well outside my scope of ability to deal with.
Firstly it is written in C++, which I can read and write to some degree but this all appears incredibly complex (there are hundred of header/source files).
Secondly, the SDK documentation says that I must create my SLN using CMake which is a nightmare for me also.
Thirdly, the documentation is scarce and horrid.
Basically my question is this:
I know that after a certain amount of header information I should find thousands of lines of "lineref,x,y,z,r,g,b,time,intensity".
Can I bypass the SDK and find another way to read in this file type?
Or, must an SDK from the proprietor be used to interact with their file type due to some sort of encryption?
I want record screen (by capturing 15 screenshots per second). This part I know how to do. But I don't know how to write this to some popular video format. Best option which I found is write frames to separated PNG files and use commandline Mencoder which can convert them to many output formats. But maybe someone have another idea?
Requirements:
Must be multi-platform solutions (I'm using Free Pascal / Lazarus). Windows, Linux, MacOS
Exists some librarys for that?
Could be complex commandline application which record screen for me too, but I must have possibility to edit frames before converting whole raw data to popular video format
All materials which could give me some idea are appreciated. API, librarys, anything even in other languages than FPC (I would try rewrite it or find some equivalent)
I considered also writting frames to video RAW format and then use Mencoder (he can handle it) or other solution, but can't find any API/doc for video RAW data
Regards
Argalatyr mentioned ffmpeg already.
There are two ways that you can get that to work:
By spawning an new process. All you have to do is prepare the right input (could be a series of jpeg images for example), and the right commandline parameters. After that you just call ffmpeg.exe and wait for it to finish.
ffmpeg makes use of some dll's that do the actual work. You can use those dll's directly from within your Delphi application. It's a bit more work, because it's more low-level, but in the end it'll give you a finer control over what happens, and what you show the user while you're processing.
Here are some solutions to check out:
FFVCL Commercial. Actually looks quite good, but I was too greedy to spend money on this.
Open Source Delphi headers for FFMpeg. I've tried it, but I never managed to get it to work.
I ended up pulling the DLL wrappers from an open source karaoke program (UltraStar Deluxe). I had to remove some dependencies, but in the end it worked like a charm. The relevant (pascal) code can be found here:
http://ultrastardx.svn.sourceforge.net/viewvc/ultrastardx/trunk/src/lib/ffmpeg-0.10/
There was some earlier discussion with a Delphi component here. It's a very simple component that sometimes generates some weird movies. Maybe a start.
How can I write a script or program to manipulate Adobe Photoshop files? I'd like to be able to do something like read a Adobe PSD file, rename the layers, and save it back to a PSD format.
The files look to be saved with a combination of XML and serialized data. I looked at the file's code and see that it has <x:xmpmeta near the start, did some google searching to find the wikipedia article about xmp - Extensible Metadata Platform, but I'm unclear if that is the format for the entire file or just for the metadata portion.
I saw that there is a PSD parser class for PHP available, and not a bad article about how to use it, although it seems like it is just for reading / converting and not for writing / saving.
But I'd like to know:
What format are these files stored in?
Where are the guidelines for interfacing with that format?
Are there some classes / tools available for manipulating that file format? Any language would be fine for a start.
I'm happy to do more research on my own but I'm hoping for some guidance to know what I should be looking for.
I'm not familiar with it myself, but there is an official SDK for Photoshop available that should let you do all that and more with .psd files.
There are not so many options. The general advice would be to look into buying Adobe InDesign Server. In some cases it can be cost prohibitive and you might be interested in 3-party SDKs. Unfortunately there are a few options in the market. One of them is Graphics Mill image processing SDK (http://www.graphicsmill.com/photoshop-psd).
Disclaimer: I work for Aurigma which runs Graphics Mill project.
I am trying to deserialize an old file format that was serialized in Delphi, it uses binary seralization. I know nothing about the structure of the file except some very high level records that are in it.
What steps would you take to solve this problem? Any tools etc?
A good hexeditor, and use the gray matter to identify structures.
If you get a hint what kind of file it is, you can search for more specialized tools.
Running the unix/Linux "file" command can be good too (*) See Barry's comment below for how it works. It can be a quick check for common filetypes like DBF,ZIP etc hidden by using a different extension.
(*) there are 3rd party builds for windows, but they might lag in versions. If you can do it on a recent *nix distro, it is advised to do so.
The serialization process simply loops over all published properties and streams their value to a text file. If you do not know the exact classes that were streamed to the file you will have a very hard time deserializing the file. (if not impossible)
A good hex editor is first. If the file is read without buffering (eg read directly from a TFileStream) you could gain some information when using ProcMon from SysInternals; You can see exactly what data is read in what chunks and thus determine more quickly where the boundaries are between the structures you already identified.
How can I get the summary information for file images in Delphi?
You will have to use parsers for each file type. The simple solution would be to use something like GraphicEx and to load each supported image into a temporary object, extract the information you want then dispose of it.
For EXIF information (the information attached by modern day cameras) you might want to use a different component. I know there are a few components floating around that will give you access to this special format, however this data is not supported by all image types and is normally seen in JPEG files.
You don't need anything.
It's included in Windows (Win32 COM)
See on Win32 api group (classic question...)