Original Message:
I need to multiply two 64 bit numbers, but Lua is losing precision
with big numbers. (for example 99999999999999999 is shown as
100000000000000000) After multiplying I need a 64 bit solution,
so I need a way to limit the solution to 64 bits. (I know, if the
solution would be precise, I could just use % 0x10000000000000000,
so that would work too)
EDIT: With Lua 5.3 and the new 64 bit integer support, this problem doesn't exist anymore. Neat.
Lua uses double-precision floating points for all math, including integer arithmetic (see http://lua-users.org/wiki/FloatingPoint). This gives you about 53 bits of precision, which (as you've noticed) is less than you need.
There are a couple of different ways to get better precision in Lua. Your best bet is to find the most active such effort and piggy-back off it. In that case, your question has already been answered; check out What is the standard (or best supported) big number (arbitrary precision) library for Lua?
If your Lua distribution has packages for it, the easy answer is lmapm.
If you use LuaJIT in place of Lua, you get access to all C99 built-in types, including long long which is usually 64 bits.
local ffi = require 'ffi'
-- Needed to parse constants that do not fit in a double:
ffi.cdef 'long long strtoll(const char *restrict str, char **restrict endptr, int base);'
local a = ffi.C.strtoll("99999999999999999", nil, 10)
print(a)
print(a * a)
=> 3803012203950112769LL (assuming the result is truncated to 64 bits)
Related
I am currently trying to work around with Lua 5.1 bytecode. I've gotten pretty far, and understand a lot. However, I am stuck with a question on instructions and numbers. I understand that the size of the instruction and number are located and defined in the header, but I am not sure how to get the actual number from the 4 bytes (or whatever size is specified in the header).
I've looked at output from ChunkSpy and I don't really understand how it went from those bytes to the number. I'd look in the source but I don't want to just copy it, I want to understand it. If anyone could tell me a bit about it or even point me in the right direction I'd be very grateful.
Thank you!
From A No-Frills Introduction to Lua 5.1 VM Instructions, numbers are stored in the constants pool.
The first byte is 3=LUA_TNUMBER.
The next bytes are the number, with the length as given in the header. Interpretation is based on the length, byte order and the integral flag as given in the header.
Typically, non-integral with 8 bytes means IEEE 754 64-bit double.
Deserializing bytes to double involves extracting the bits for the mantissa and exponent, and combining them with arithmetic operations. Perhaps you want that as a challenge and to start from a description of the standard: What Every Computer Scientist Should Know About Floating-Point Arithmetic, "Formats and Operations" section.
What is the number format length in bytes?
This is a "multi type" data format. Is it 4 bytes? 8 bytes? How much? How can I get it programmatically? Does the length depend on the OS/processor type?
Here https://www.lua.org/pil/2.3.html the documentation says this is a double precision type. That is, it has 64 bits. Am I right?
Like #Roddy said, it's slightly complicated with the integer type. Moreover, it depends on how your Lua is compiled.
Basically, in Lua 5.3, there are two types, the integer type lua_Integer and the number type lua_Number. You can get their lengths programatically from within Lua by parsing a chunk header:
local chunk = string.dump(function() end)
print("lua_Integer", chunk:byte(16))
print("lua_Number", chunk:byte(17))
Typically both lengths will be 8 bytes. However on some embedded platforms you can find Luas where the lua_Number type is a float (4 bytes), a 32 bit integer or even weirder things.
It depends on the version of Lua, and of course, how it's compiled.
5.3 has true integers, typically 64 bits. https://www.lua.org/manual/5.3/manual.html
The type number uses two internal representations, or two subtypes,
one called integer and the other called float.
...
Standard Lua uses 64-bit integers and double-precision (64-bit)
floats, but you can also compile Lua so that it uses 32-bit integers
and/or single-precision (32-bit) floats.
Earlier versions always use 64-bit double-precision floating point, which effectively accurately represents up to 52-bit integers. Your link... https://www.lua.org/pil/2.3.html
according to the Lua reference (for integers)
In case of overflows in integer arithmetic, all operations wrap around, according to the usual rules of two-complement arithmetic. (In other words, they return the unique representable integer that is equal modulo 2^64 to the mathematical result.)
and for floating point
With the exception of exponentiation and float division, the arithmetic operators work as follows: If both operands are integers, the operation is performed over integers and the result is an integer. Otherwise, if both operands are numbers or strings that can be converted to numbers (see ยง3.4.3), then they are converted to floats, the operation is performed following the usual rules for floating-point arithmetic (usually the IEEE 754 standard), and the result is a float.
Lua as a language does not define what you ask for. The data type used for representing numbers may differ from version to version (note that the link to the free online version of "Programming in Lua" is about Lua 5.0), but primarily this is defined by the way Lua is compiled, as others already said.
Look at luaconf.h for all the details.
Regarding your actual problem (converting hex-string to numbers), you could look at the result of tonumber() on various input strings, compared to known results:
function hexConvertibeBytes()
local i, s = 0, ''
repeat
i, s = i + 1, s .. 'FF'
local n = tonumber( s, 16 )
until n ~= 256^i - 1
return i - 1
end
We can use string.pack as follows:
s = string.pack("J",0)
number_of_bytes = #s
How could I handle operations with a number like:
48534588306961133067968196965257961415756656521818848750723547477673457670019632882524164647651492025728980571833579341743988603191694784406703
Nothing that I've tried worked so far... unsigned long, long long, etc...
What you need is a library that provides support for operations on integers of arbitrary length. However, from what I've been able to find out, there are no such libraries written in Objective-C.
You are nevertheless in luck as Objective-C is a superset of C. That makes it possible for you to use C libraries such as those described in the answers to this somewhat dated SO question.
Also, since the Clang compiler supports C++ and combining Objective-C and C++ code, you can probably use something like big int.
Note that none of the built-in types is even close to being big enough to represent numbers with as many digits as your examples. The biggest available integer type is unsigned long long, if you don't need negative numbers, and its size is 8 bytes/64 bits, which gives you a range of 0-18446744073709551615, or 20 digits max.
You could use JKBigInteger instead, it is a Objective-C wrapper around LibTomMath C library. And really easy to use and understand.
In your case:
JKBigInteger *int = [[JKBigInteger alloc] initWithString:#"48534588306961133067968196965257961415756656521818848750723547477673457670019632882524164647651492025728980571833579341743988603191694784406703"];
You can try here : http://gmplib.org/
GMP is a free library for arbitrary precision arithmetic, operating on signed integers, rational numbers, and floating point numbers. There is no practical limit to the precision except the ones implied by the available memory in the machine GMP runs on. GMP has a rich set of functions, and the functions have a regular interface.
I'm not a Pascal newbie, but I still don't know until now why Delphi and Free Pascal usually declares parameters and returned values as signed integers whereas I see them should always be positive. For example:
Pos() returns type of Integer. Is it possible to be a negative?
SetLength() declares the NewLength parameter as a type of Integer. Is there a negative length for string?
System.THandle declared as Longint. Is there a negative number for handles?
There are many decisions like those in Delphi and Free Pascal. What considerations were behind this?
In Pascal, Integer (signed) is the base type. All other integer number types are a subranges of integer. (this is not entirely true in Borland dialects, given longint in TP and int64 in Delphi, but close enough).
An important reason for that if the intermediate result of calculations gets negative, and you calculate with unsigned integers, range check errors will trigger, and since most older programming languages DON'T assume 2-complement integers, the result (with range checks off) might even be corrupt.
The THandle case is much simpler. Delphi didn't have a proper 32-bit unsigned till D4, but only a 31-bit cardinal. (since 32-bit unsigned integer is not a subrange of integer, the later unsigned ints are a subset of int64, which moved the problem to uint64 which was only added in D2010 or so)
So in many places in the headers signed types are used where the winapi uses unsigned types, probably to avoid the 32th bit getting accidentally corrupt in those versions, and the custom stuck.
But the winapi case is different from the general case.
Added later Some Pascal (and Modula2/3) implementations circumvent this trap by setting the integer at a size larger than the wordsize, and require all numeric types to declare a proper subrange, like in the below program.
The first holds the primary assumption that everything is a subset of integer, and the second allows the compiler to scale nearly everything down again to fit in registers, specially if the CPU has some operations for larger than word operations. (like x86 where 32-bit * 32-bit mul gives a 64-bit result, or can detect wordsize overflows using status bits (e.g. to generate range exceptions for adds without doing a full 2*wordsize add)
var x : 0..20;
y : -10..10;
begin
// any expression of x and y has a range -10..20
Turbo Pascal and Delphi emulate an integer type twice the wordsize for their 16-bit and 32-bit offerings. The handling of the highest unsigned type is hacky at best.
Well, for a start THandle is declared incorrectly. It's unsigned in the Windows headers and should be so in Delphi. In fact I think this was corrected in a recent release of Delphi.
I'd imagine that the preference for signed over unsigned is largely historical and not particularly significant. However, I can think of one example where it is important. Consider the for loop:
for i := 0 to Count-1 do
If i is unsigned and Count is 0 then this loop runs from 0 to $FFFFFFFF which is not what you want. Using a signed integer loop variable avoids that problem.
Pascal is a victim of its syntax here. The equivalent C or C++ loop has no such trouble
for (unsigned int i=0; i<Count; i++)
due to the syntactic difference and use of a comparison operator as stopping condition.
This could also be the reason why Length() on a string or dynamic array returns a signed value. And so for consistency, SetLength() should accept signed values. And given that the return value of Pos() is used to index strings, it should be signed also.
Here's another Stack Overflow discussion of the topic: Should I use unsigned integers for counting members?
Of course, I'm speculating wildly here. Perhaps there was no design and just out of habit the precedent of using signed values was set and became enshrined.
Some string related search functions return -1 when nothing is found.
I believe the reasoning behind this is that MaxInt is 2GB which is the maximum size for strings in 32 bit Delphi. This because a single process can have up to 2GB memory
There are many reasons for using signed integers, even some that might apply when you do not intend to return a negative value.
Imagine I write code that calls Pos, and I want to do math with the results. Would you rather have a negative result (Pos('x',s)-5) raise a range-check exception, underflow and become a very large unsigned number around 4 billion, or go negative, if Pos('x',s) returns 1? Either one is a source of problems for new users who seldom think about these cases, but the long-established tradition is that by using Integer results, it's your job to check for negative and zero results and not use them as string offsets. There is an advantage for beginning and for advanced programmers, in using Integer, and not having "negative" values roll under and become large unsigned values or raise range exceptions.
Secondly, remember that in beginning programming, one usually introduces Integer (signed) types long before one introduces unsigned types like Cardinal. Beginners often work with functions like Pos, and it makes sense to use the type that will create the least-unfriendly set of side effects. There are no negative side effects to having a range larger than the one you absolutely need (the range you probably need for Pos is 1 to maximum-string-length-in-delphi). There is zero benefit in 32-bit Delphi to using the Cardinal type for Pos, and there definitely ARE downsides to choosing it.
Once you get to 64-bit delphi, however, you could theoretically have strings LARGER than an Integer can hold, and moving to Cardinal wouldn't fix all your potential problems. However, the chance of anyone having a 2+ GB string is probably nil, and Delphi 64-bit compiler doesn't allow a >2 GB string, anyway. In my testing, I can achieve an almost 1 GB String in 64 bit Delphi. So the practical length limit for a Win64 string is about a billion (1073741814) characters, which is using nearly 2 GB of actual RAM. At that limit, I either get EIntOverflow or EAccessViolation, and it seems I am hitting Delphi run time library (RTL) bugs, not properly defined limits, so your mileage may vary.
R15b on Windows gives:
>trunc(1.9999999999999999999).
2
For that matter, just typing the float returns:
> 1.9999999999999999999.
2.0
AFAIK, the truncate function should just drop the fractional portion (at least that's what I need, anyway). A floor function might also do the trick AFAIK, but the floor implementations I've seen posted online use... you guessed it... trunc.
I'm not nitpicking this, I actually need this to be correct for a program I'm developing.
Any ideas on this?
Thanks.
Your problem is decimal numbers are represented as IEEE compliant binary representation (32, 64 or 128 bit).
If you really need precision you should use other numerical data structures as Binary Coded Decimal or fixed-point arithmetic.
Hope this helps!
if you want to make a TRUNC to float, maybe this one can help:
select substring (convert(varchar(14), CAST (20160303013458 as varchar(14))) , 1 , 8)