I am currently digging into optimising some highly used code in go. My Question boils down to the following code snippet (copied with memory allocations from pprof list command). As you can see all the allocations are done in the line where map is being filled (line 959).
ROUTINE ======================== cart.BenchmarkMapWithOutCapacityFixVal in /.../cart_test.go
3328966 3328966 (flat, cum) 15.50% of Total
. . 954:
. . 955:func BenchmarkMapWithOutCapacityFixVal(b *testing.B) {
. . 956: for i := 0; i < b.N; i++ {
. . 957: m := make(map[int]float32)
. . 958: for k := 0; k < 10; k++ {
3328966 3328966 959: m[k] = 0.434295723423
. . 960: }
. . 961: }
. . 962:}
Here what I am trying to do: I am trying to allocate the memory before the (inner) loop so there are no unnecessary allocations happening:
ROUTINE ======================== cart.BenchmarkMapWithCapacityFixVal in /.../cart_test.go
3214263 3214263 (flat, cum) 14.97% of Total
. . 963:
. . 964:func BenchmarkMapWithCapacityFixVal(b *testing.B) {
. . 965: for i := 0; i < b.N; i++ {
3048075 3048075 966: m := make(map[int]float32, 10)
. . 967: for k := 0; k < 10; k++ {
166188 166188 968: m[k] = 0.434295723423
. . 969: }
. . 970: }
. . 971:}
Why are there still allocations happening in line 968 (second sample) and how can I correctly allocate the map before the inner loop?
A map is not an array. You can't preallocate space in it because you can't know where in the map the elements will be inserted. In make(map..., X) X is just a capacity hint, it doesn't bound the map and definitely doesn't guarantee that the keys will hash perfectly into it. As such it will do a best effort of minimizing the number of future allocations, but there's no way to eliminate all of them.
In this particular example you should just use an array rather than a map if you want perfect control over allocations. With a proper array you'd have just one allocation.
Related
I'm trying to run a WIN32 PE executable from memory (not for malware just for software protection purposes). When I allocate at the desired image base address (0x00400000) it works perfectly. But this is not ideal since this address is not always available, sometimes even already in use by the current process depending on ASLR.
Instead I have to relocate the image with the new address obtained from VirtualAlloc() using this generic code.
while (pIBR->VirtualAddress)
{
if (pIBR->SizeOfBlock >= sizeof(IMAGE_BASE_RELOCATION))
{
count = (pIBR->SizeOfBlock - sizeof(IMAGE_BASE_RELOCATION)) / sizeof(WORD);
list = (PWORD)(pIBR + 1);
for (i = 0; i < count; i++)
{
if (list[i])
{
ptr = (PDWORD)((LPBYTE)image + (pIBR->VirtualAddress + (list[i] & 0xFFF)));
*ptr += delta;
}
}
}
pIBR = (PIMAGE_BASE_RELOCATION)((LPBYTE)pIBR + pIBR->SizeOfBlock);
}
which works fine for simple executable's, but more complex executable's with resources, TLS, and various other things, don't load correctly or at all.
My question, is there a better way of doing image relocation, or how can I always reserve the address 0x00400000 for my new PE image.
Thanks.
newbie question here, I'm trying to reuse a code, but since I'm not a programmer I couldn't identify it. I tried googling already some "programming language identifier by syntax" and googling some of the syntax in it and at first I thought it matched vb.net but upon downloading and opening a project there, it had too many syntax errors IIRC, so I don't think this is it.
Here's the snippet:
;==============================================================================================================================
; Function: _MemoryReadStdString($address, $handle, [, $offset=False])
;
; Description: Read memory for string. If str_length > 15 read pointer else read str.
;
; Parameter(s): $address - Address in memory to read.
; $handle - An array containing the Dll handle and the handle of the open
; process as returned by _MemoryOpen().
; $offset - If we wanna read pointer put offset in hex value example:
; $offset[2] = [0x20, 0x264]
;
; Return Value(s): On Success - Returns string value
; On Failure - Returns empty string ''
;
; Author(s): Ascer
;===============================================================================================================================
Func _MemoryReadStdString ($address, $handle, $offset)
; read memory for address of our name and return address to this name.
Local $addr_start = '0x' & Hex(_MemoryPointerRead($address, $handle, $offset)[0], 8) ;==> example 0x8A16308
; read memory for name length, just one byte
Local $str_length = _MemoryRead($addr_start + 0x10, $handle, 'byte') ;==> 0x8A16308 + 0x10 = 0x8A16318
; check if string < 16 then read name from $addr_start
If $str_length < 16 Then
Return BinaryToString(_MemoryRead($addr_start, $handle, 'char[15]')) ;==> 'King Medivius'
; string length is > 15 then we must read memory($addr_start) for new address
Else
$str_pointer = '0x' & Hex(_MemoryRead($addr_start, $handle), 8) ;==> example 0x8C95320
Return BinaryToString(_MemoryRead($str_pointer, $handle, 'char[32]')) ;==> read memory in $str_pointer region to get true name
EndIf
; return empty string if nothing found
Return ""
EndFunc
Sorry about the "EndFunc" there, it's inside the "pre code" but somehow ended up outside it. Any help is appreciated! Thanks!
Googling a portion of the code reveals a possible source at this post about what seems to be a Tibia private server named Medivia.
From the context it seems to be written in AutoIt - there is a screenshot near the bottom of the post showing the user testing code using a program named AutoIt3.exe Here is the download link.
I am working with a big shared memory matrix of 1.3e6x1.3e6 in a foreach loop. I create that matrix with FBM function of bigstatsr package.
I need the results of the loop in the FBM class object to not run out of RAM memory.
This is what I want to do without FBM class object.
library(doParallel)
library(foreach)
library("doFuture")
cl=makeCluster(2)
registerDoParallel(cl
)
registerDoFuture()
plan(multicore)
results=foreach(a=1:4,.combine='cbind') %dopar% {
a=a-1
foreach(b=1:2,.combine='c') %dopar% {
return(10*a + b)
}
}
And this is how I try it
library(bigstatsr)
results=FBM(4,4,init=0)
saveinFBM=function(x,j){results[,j]=x}
foreach(a=1:4,.combine='savinFBM') %dopar% {
a=a-1
foreach(b=1:2,.combine='c') %dopar% {
return(10*a + b)
}
}
Error in get(as.character(FUN), mode = "function", envir = envir) :
object 'savinFBM' of mode 'function' was not found
PS: Could anybody add the tag "dofuture"?
If I understand correctly what you want to do, a faster alternative is using outer(1:2, 1:4, function(b, a) 10 * (a - 1) + b).
If you want to fill an FBM with this function, you can do:
library(bigstatsr)
X <- FBM(200, 400)
big_apply(X, a.FUN = function(X, ind) {
X[, ind] <- outer(rows_along(X), ind, function(b, a) 10 * (a - 1) + b)
NULL
})
Usually, using parallelism won't help when you write data on disk (what you do when you fill X[, ind]), but it you really want to try, you can use ncores = nb_cores() as additional argument of big_apply().
Good afternoon people, i'm trying to code in Verilog a structure than can store up to 64 different 8bit numbers (64X8), which is only allowed to store numbers greater than 125 and bellow or equal to 250. When it is writing (or not), it can show the maximum current stored value (VAL_MAX) as well as it's position (POS_MAX). When not writing (EN_WR ==0) i simply put in POS_RD the position that i want, in order to see what number is stored there, and when the memory is full (NR_ST = 64) it replaces the oldest stored numbers with the new ones, one by one. I currently have the code but there are some issues:
1st - When the memory is full and for eg. i have 250 in the 2nd position, the output will be
VAL_MAX= 250 ; POS_MAX=1. When a bunch of new numbers come, that maximum should be replaced with the 2nd highest stored value and must show it's position, but the memory isn't showing a new Max Value.
2nd - When i want to see the number stored in the 1st position (POS_RD = 0) the output VAL_RD (used to read the stored numbers) is "X" and not the stored number, i don't know if it is saving or not.
The code is:
module Bloco(VAL_SENSOR, EN_WR, POS_RD, NR_ST, VAL_MAX, POS_MAX, VAL_RD, segundo, clk);
parameter MEM_SIZE = 64;
parameter MEM_WIDTH = 8;
parameter ADDR_SIZE = 5;
input[MEM_WIDTH - 1:0] VAL_SENSOR;
input[5:0] POS_RD;
input EN_WR,segundo,clk;
output[6:0] NR_ST;
output[MEM_WIDTH - 1:0] VAL_MAX;
output[ADDR_SIZE :0]POS_MAX;
output[MEM_WIDTH - 1:0]VAL_RD;
reg[MEM_WIDTH - 1:0] ram[MEM_SIZE - 1:0]; // C , L
reg[MEM_WIDTH - 1:0] VAL_RD;
reg[MEM_WIDTH - 1:0] val_max = 0; //necessita de variavel so por causa do valor inicial
reg[ADDR_SIZE :0] POS_MAX = 0;
reg[ADDR_SIZE :0] POS_MAX2 = 0;
reg[ADDR_SIZE + 1:0] NR_ST_COUNTER = 0; //addr_size + 1 because it needs to count from zero to the number of values
reg[ADDR_SIZE :0] POS_POINTER = 0;
assign VAL_MAX = val_max;
assign NR_ST = NR_ST_COUNTER;
always # (posedge clk)
begin
if(EN_WR) //Caso esteja habilitado o sistema de armazenamento
begin
if(segundo)
begin
if(VAL_SENSOR > 125 && VAL_SENSOR <= 250) //Se for um numero abaixo de 250 unid. luminosas e acima de 125
begin //Escrita
if(POS_POINTER == POS_MAX)
POS_MAX <= POS_MAX2;
else
ram[POS_POINTER] <= VAL_SENSOR;
if(NR_ST_COUNTER < 64) //atualizar Contador de Valores guardados
NR_ST_COUNTER = NR_ST_COUNTER + 1;
if(VAL_SENSOR > val_max) //atualizar MAX
begin
POS_MAX <= POS_POINTER;
val_max <= VAL_SENSOR;
end
else //ver se encaixa no segundo maior POS_MAX2
begin
if(VAL_SENSOR > ram[POS_MAX2]) //nao precisa guardar o valor
POS_MAX2 <= POS_POINTER;
end
POS_POINTER <= POS_POINTER + 1;
end
end
end
else
VAL_RD <= ram[POS_RD];
end
endmodule
.
NOTE = The input "segundo" is like EN_WR, but it's only used after 10 clock cycles (it will be linked to a Counter).
Thank you.
To answer your questions:
1) The reason you are likely not seeing the second highest value in your memory appear on VAL_MAX when the current max is overridden is because you never change val_max register to contain the value of ram[POS_MAX2], ie the second highest value. However, as you do not have an ordered data structure and do not store any more than the second highest value (and do not search the memory for the second highest value), you cannot reliably keep finding the next highest value when the current highest is removed/overridden. You might need to rethink alot of how you are doing this if you need to always have the current highest value in the memory being output on VAL_MAX.
2) Position 0 is not being written to the first time you write to the memory; thus you are getting the default value of the memory, ie 'x. Here way:
if(POS_POINTER == POS_MAX)
POS_MAX <= POS_MAX2;
else
ram[POS_POINTER] <= VAL_SENSOR;
In the above lines, you only put values in the memory if the current position is not equal to the position of the current highest value. The first time you write (ie, to position 0), POS_POINTER and POS_MAX are at their initial values of 0, thus equal. So, ram[0] never gets updated as it is only executed if POS_POINTER and POS_MAX are not equal (which in this case, they are equal). If you were to write to position 0 a second time, you might be able to write to it as the value of POS_MAX might have changed. Note also, that you sometimes update the write pointer POS_POINTER even if you dont write there, as in the example above (even through ram[0] wasnt written to, you will still update POS_PONTER to be 1).
As mentioned in #1, depending on your requirements, you might not have the right structures here to meet them. If you always need to have the highest value in the memory output on VAL_MAX and always have the old value ejected, you might need to make old of those operations do a search, or store and update all the needed meta data.
Is there a way to have the auto-dividers sort by last name?
I don't think it should have anything to do with the php code, but I thought I would include it for a reference below:
$result = mysql_query("SELECT * FROM `patients` WHERE `company_id` = " . $user_data['company_id'] . " ORDER BY `patient_lastname`");
while($row = mysql_fetch_array($result)) {
echo '<li>' . $row['patient_firstname'] . ' ' . $row['patient_lastname'] . '<span class="ui-li-count">DOB: ' . $row['patient_dob'] . '</span></li>';
}
Appreciate any help!
You can do the sorting in the front-end by selecting the list items and sorting them afterward. In the example below, instead of selecting the text content of the list items, you can select the last-name value.
var listContentArray = listViewInstance.find('li').not('.ui-li-divider').get();
listContentArray.sort(function (a, b) {
var first = $(a).text(),
second = $(b).text();
if (first < second) {
return -1;
}
if (first > second) {
return 1;
}
return 0;
});
Then you can destroy the content of the listViewInstance, re-append the elements in the listContentArray, and finally refresh the listView component.
You can download a fully functional example that does all of this at:
http://appcropolis.com/page-templates/autodividers/
Couldn't find a way to display Firstname then Lastname and autodivide by Lastname, so I just replaced the code with:
' . $row['patient_lastname'] . ', ' . $row['patient_firstname'] . '
which displays: "Lastname, Firstname" and autodivides by Lastname. It'll have to work for now.