As someone commented that it's impossible to dump the contents of a ".blend" file, I started to write a parser for ".blend" files ;-)
So far I can parse all the file blocks and the SDNA. However when it's about to read the file blocks' data into the corresponding structure fields, I'm running into a problem that I don't really know how to solve (due to lack of proper documentation):
If the n-th field of a structure has type index i, how do I decide whether type i is a structure recursively, or a simple type?
For example FileGlobal has a field short subversion, where the type index for short is 2. Obviously short is not a composite structure, but how does the parser decide?
In my parser's SDNA STRC index 2 is ListBase, while TYPE index 2 is short. Obviously the type index is valid in both tables, but have a different meaning...
So is my parser wrong, or what is the trick? For reference, here are some details of the SDNA I'm seeing:
file-header: BLENDER version 279 little-endian 64-bit ptr
...
file-block: code DNA1 size 90700 addr 55b81c6e7c80 SDNA 0 count 1
SDNA: names 4291
SDNA: NAME #0=*next
SDNA: NAME #1=*prev
SDNA: NAME #2=*data
...
SDNA: types 706
SDNA: TYPE #0=char
SDNA: TYPE #1=uchar
SDNA: TYPE #2=short
...
SDNA: TLEN #0=1
SDNA: TLEN #1=1
SDNA: TLEN #2=2
...
SDNA: structures 621
SDNA: STRC #0=stype 12 nfields 2
...
SDNA: STRC #2=stype 14 nfields 2
SDNA: STRC field #0=ftype 11 name 3
SDNA: STRC field #1=ftype 11 name 4
...
Insights?
As there is no answer yet, I'm trying to answer my question piece-by-piece (as I learn it):
First, the SDNA index given in the file block seems to be the index of the type in the STRC table. However the type number of the type at index 0 is 12. That fixed offset (12) continues until index 12, where the type is 26. So actually you need another table (inverted index) that matches the type number to the index of the type in the STRC table.
As the first type number in that table is 12, it seems the first twelve type numbers are used for these primitive types: char, uchar, short, ushort, int, long, ulong, float, double, int64_t, uint64_t, void
However these types do not exist in STRC. So type #2 is not present in the inverted index, so it must be a primitive (non-composite) type.
Unfortunately there seem to be exceptions from the rules above: The field types in STRC are not (like the SDNA index) an index into SDNA, but a type number, so you need to use the inverted index there. When using a recursive routine to decode (like me) the routine must know whether processing a "type index" or a "type number".
Related
Consider this type definition:
type ChoiceOne =
| A
| B
and this binding
let c1 = A
I assume the correct way to translate this into natural language would be "the value A of type ChoiceOne is constructed and bound to the name c1".
Now take a look at this next type definition and binding:
type ChoiceTwo =
| A of int
| B of float
let c1 = A 1
I now see several ways of describing what is going on here:
"the value A of type ChoiceTwo is constructed ..." ( and what about the 1 ? )
"the value 1 of type ChoiceTwo is constructed ..." ( and what about the A ? )
"the value A of type ChoiceTwo with the ( inner value ? associated value ? ... ) 1 is constructed ..."
I guess option 3 describes the situation best, but how can I correctly designate the value 1 with respect to the (other?) value A of type ChoiceTwo ?
Here's the terminology that I use, which I believe is widely accepted, using my own example:
type MyDU =
| A of int
| B
A is a case
A is also a function that takes an int and returns a value of type MyDU
Therefore, A 1 is a value of type MyDU
B is a case
B is also a value of type MyDU
The only words needed here are "case", "value", "function" and "type". Don't use the word "construct". In general, when talking about F# code, if you can accurately replace any words that you use with the ones that I recommend, then do so!
This is all splitting hairs. Nobody really cares about how exactly to say these things outside of a quiz in college, and even then, I would recommend maybe transferring to a different college :-)
But if you really want to split hairs, here we go.
In your very first example, the value A is not "constructed". Since it doesn't have parameters, there can be only one instance of it, and that instance always exists, so there is no need to "construct" it.
To be sure, A may still be referred to as a "constructor", but it's also a "value" at the same time.
In the second example, it is sometimes said that "a value A 1 was constructed", but more often "a value A constructed with parameter 1"
Starting with these frequencies:
A:7 F:6 H:1 M:2 N:4 U:5
at a later step I have 5 6 7 7, where one of the 7's is the "A". Which 7 branch I pick to be a 0 or a 1 is arbitrary.
So how do I get uniquely decodable code word?
You need to send the code to the receiver, not the frequencies. You can arbitrarily assign 0's and 1's to all of the branches, and then send the codes for each symbol before the coded symbols themselves. There are many possible Huffman codes from the same set of frequencies.
More commonly only the code lengths in bits for each symbol are sent. In this case those are A:2 F:2 H:4 M:4 N:3 U:2. Then a canonical code is used on both ends that depends only on the lengths. In this case, starting with 0's, the canonical code would be:
A: 00
F: 01
U: 10
N: 110
H: 1110
M: 1111
where codes of equal length are assigned to the symbols in lexicographical order. Note that the Huffman tree that was built is not needed. All that is needed is the number of bits for each symbol.
I am trying some things in Dafny. I want to code a simple datastructure that holds an uncompressed image in memory:
datatype image' = image(width: int, height: int, data: array<byte>)
newtype byte = b: int | 0 <= b <= 255
Actually using it:
method Main() {
var dat := [1,2,3];
var im := image(1, 3, dat);
}
datatype image' = image(width: int, height: int, data: array<byte>)
newtype byte = b: int | 0 <= b <= 255
leads Dafny to complain:
stdin.dfy(3,24): Error: incorrect type of datatype constructor argument (found seq, expected array)
1 resolution/type errors detected in stdin.dfy
I might also want to demand that the byte array is not null, and the size of the byte array is equal to width * height * 3 (to store three bytes representing the RGB value of that pixel).
What way should I enforce this? I looked into newtype, which lets you put some constraints on variables with a certain type, but this works only for numeric types.
Dafny supports both immutable sequences (which are like mathematical sequences of elements) and mutable arrays (which are, like in C and Java, pointers to elements). The error you're getting is telling you that you're calling the image constructor with a seq<byte> value where an array<byte> value is expected.
You can fix the problem by replacing your definition of dat with:
var dat := new byte[3];
dat[0], dat[1], dat[2] := 1, 2, 3;
However, the more typical thing, if you're using a datatype (which is immutable), would be to use a sequence. So, you probably want to instead change your definition of image to:
datatype image = image(width: int, height: int, data: seq<byte>)
Btw, note that Dafny allows you to name a type and one of its constructors the same, so there's no reason to name one of them with a prime (unless you want to, of course).
Another matter of style is to use a half-open interval in your definition of byte:
newtype byte = b: int | 0 <= b < 256
Since half-open intervals are prevalent in computer science, Dafny's syntax favors them. For example, for a sequence s, the expression s[52..57] denotes a subsequence of s of length 5 (that is, 57 minus 52) starting in s at index 52. One more thing, you can also leave out the type int of b if you want, since Dafny will infer it:
newtype byte = b | 0 <= b < 256
You also asked about the possibility of adding a type constraint, so that the sequence in your datatype will always be of length 3. As you discovered, you cannot do this with a newtype, because newtype (at least for now) only works with numeric types. You can (almost) use a subset type, however. This would be done as follows:
type triple = s: seq<byte> | |s| == 3
(In this example, the first vertical bar is like the one in the newtype declaration and says "such that", whereas the next two denote the length operator on sequences.) The trouble with this declaration is that types must be nonempty and Dafny isn't convinced that there are any values that satisfy the constraint of triple. Well, Dafny is not trying very hard. The plan is to add a witness clause to the type (and newtype) declaration, so that a programmer can show Dafny a value that belongs to the triple type. However, this support is waiting for some implementation changes that will allow customized initial values, so you cannot use this constraint at this time.
Not that you want it here, but Dafny would let you give a weaker constraint that admits the empty sequence:
type triple = s: seq<byte> | |s| <= 3
So, instead, if you want to talk about that an image value has a data component of length 3, then introduce a predicate:
predicate GoodImage(img: image)
{
|img.data| == 3
}
and use this predicate in specifications like pre- and postconditions.
Program safely,
Rustan
I have an ets table which you can imagine there are two columns, "Key" and Value. Value is an integer.
When I tried:
Ans = ets:select(Table_name, MS),
Ans + 1.
where Ans equals to the expected Value.
I got a error:
** exception error: an error occurred when evaluating an arithmetic expression
Can I not do any arithmetic operation on the ets:select return value?
in the ets module spec: select(Tab, MatchSpec) -> [Match], the reurn value is a list of match. If your match specification define one single integer value as return, you will receive a list of integer. In your case, it seems that this list will always have a length of 1 element, if it must be the case, you can write:
[Ans] = ets:select(Table_name, MS),
Ans + 1.
But beware that this code will crash if the returned list is empty or have more than 1 element.
ets:select(Table_name, MS)
will return result in a List.
So instead, the following would work:
[Ans] = ets:select(Table_name, MS),
Ans + 1.
I have many text files of this format
....
<snip>
'FOP' 0.19 1 24 1 25 7 8 /
'FOP' 0.18 1 24 1 25 9 11 /
/
TURX
560231
300244
70029
200250
645257
800191
900333
600334
770291
300335
220287
110262 /
SUBTRACT
'TURX' 'TURY'/
</snip>
......
where the portions I snipped off contain other various data in various formats. The file format is inconsistent (machine generated), the only thing one is assured of is the keyword TURX which may appear more than once. If it appears alone on one line, then the next few lines will contain numbers that I need to fetch into an array. The last number will have a space then a forward slash (/). I can then use this array in other operations afterwards.
How do I "search" or parse a file of unknown format in fortran, and how do I get a loop to fetch the rest of the data, please? I am really new to this and I HAVE to use fortran. Thanks.
Fortran 95 / 2003 have a lot of string and file handling features that make this easier.
For example, this code fragment to process a file of unknown length:
use iso_fortran_env
character (len=100) :: line
integer :: ReadCode
ReadLoop: do
read (75, '(A)', iostat=ReadCode ) line
if ( ReadCode /= 0 ) then
if ( ReadCode == iostat_end ) then
exit ReadLoop
else
write ( *, '( / "Error reading file: ", I0 )' ) ReadCode
stop
end if
end if
! code to process the line ....
end do ReadLoop
Then the "process the line" code can contain several sections depending on a logical variable "Have_TURX". If Have_TRUX is false you are "seeking" ... test whether the line contains "TURX". You could use a plain "==" if TURX is always at the start of the string, or for more generality you could use the intrinsic function "index" to test whether the string "line" contains TURX.
Once the program is in the mode Have_TRUX is true, then you use "internal I/O" to read the numeric value from the string. Since the integers have varying lengths and are left-justified, the easiest way is to use "list-directed I/O": combining these:
read (line, *) integer_variable
Then you could use the intrinsic function "index" again to test whether the string also contains a slash, in which case you change Have_TRUX to false and end reading mode.
If you need to put the numbers into an array, it might be necessary to read the file twice, or to backspace the file, because you will have to allocate the array, and you can't do that until you know the size of the array. Or you could pop the numbers into a linked list, then when you hit the slash allocate the array and fill it from the linked list. Or if there is a known maximum number of values you could use a temporary array, then transfer the numbers to an allocatable output array. This is assuming that you want the output argument of the subroutine be an allocatable array of the correct length, and the it returns one group of numbers per call:
integer, dimension (:), allocatable, intent (out) :: numbers
allocate (numbers (1: HowMany) )
P.S. There is a brief summary of the language features at http://en.wikipedia.org/wiki/Fortran_95_language_features and the gfortran manual has a summary of the intrinsic procedures, from which you can see what built in functions are available for string handling.
I'll give you a nudge in the right direction so that you can finish your project.
Some basics:
Do/While as you'll need some sort of loop
structure to loop through the file
and then over the numbers. There's
no for loop in Fortran, so use this
type.
Read
to read the strings.
To start you need something like this:
program readlines
implicit none
character (len=30) :: rdline
integer,dimension(1000) :: array
! This sets up a character array with 30 positions and an integer array with 1000
!
open(18,file='fileread.txt')
do
read(18,*) rdline
if (trim(rdline).eq.'TURX') exit !loop until the trimmed off portion matches TURX
end do
See this thread for way to turn your strings into integers.
Final edit: Looks like MSB has got most of what I just found out. The iostat argument of the read is the key to it. See this site for a sample program.
Here was my final way around it.
PROGRAM fetchnumbers
implicit none
character (len=50) ::line, numdata
logical ::is_numeric
integer ::I,iost,iost2,counter=0,number
integer, parameter :: long = selected_int_kind(10)
integer, dimension(1000)::numbers !Can the number of numbers be up to 1000?
open(20,file='inputfile.txt') !assuming file is in the same location as program
ReadLoop: do
read(20,*,iostat=iost) line !read data line by line
if (iost .LT. 0) exit !end of file reached before TURX was found
if (len_trim(line)==0) cycle ReadLoop !ignore empty lines
if (index(line, 'TURX').EQ.1) then !prepare to begin capturing
GetNumbers: do
read(20, *,iostat=iost2)numdata !read in the numbers one by one
if (.NOT.is_numeric(numdata)) exit !no more numbers to read
if (iost2 .LT. 0) exit !end of file reached while fetching numbers
read (numdata,*) number !read string value into a number
counter = counter + 1
Storeloop: do I =1,counter
if (I<counter) cycle StoreLoop
numbers(counter)=number !storing data into array
end do StoreLoop
end do GetNumbers
end if
end do ReadLoop
write(*,*) "Numbers are:"
do I=1,counter
write(*,'(I14)') numbers(I)
end do
END PROGRAM fetchnumbers
FUNCTION is_numeric(string)
IMPLICIT NONE
CHARACTER(len=*), INTENT(IN) :: string
LOGICAL :: is_numeric
REAL :: x
INTEGER :: e
is_numeric = .FALSE.
READ(string,*,IOSTAT=e) x
IF (e == 0) is_numeric = .TRUE.
END FUNCTION is_numeric