GRUB memory map gives me weird values - memory

I am trying to use grub in order to get the memory map, instead of going through the bios route. The problem is that grub seems to be giving me very weird values for some reason. Can anyone help with this?
Relevant code:
This is how I parse the mmap
void mm_init(mmap_entry_t *mmap_addr, uint32_t length)
{
mmap = mmap_addr;
/* Loop through mmap */
printk("-- Scanning memory map --");
for (size_t i = 0; mmap < (mmap_addr + length); i++) {
/* RAM is available! */
if (mmap->type == 1) {
uint64_t starting_addr = (((uint64_t) mmap->base_addr_high) << 32) | ((uint64_t) mmap->base_addr_low);
uint64_t length = (((uint64_t) mmap->length_high) << 32) | ((uint64_t) mmap->length_low);
printk("Found segment starting from 0x%x, with a length of %i", starting_addr, length);
}
/* Next entry */
mmap = (mmap_entry_t *) ((uint32_t) mmap + mmap->size + sizeof(mmap->size));
}
}
This is my mmap_entry_t struct (not the one in multiboot.h):
struct mmap_entry {
uint32_t size;
uint32_t base_addr_low, base_addr_high;
uint32_t length_low, length_high;
uint8_t type;
} __attribute__((packed));
typedef struct mmap_entry mmap_entry_t;
And this is how I call mm_init()
/* Kernel main function */
void kmain(multiboot_info_t *info)
{
/* Check if grub can give us a memory map */
/* TODO: Detect manually */
if (!(info->flags & (1<<6))) {
panic("couldn't get memory map!");
}
/* Init mm */
mm_init((mmap_entry_t *) info->mmap_addr, info->mmap_length);
for(;;);
}
This is the output I get on qemu:
-- Scanning memory map --
Found segment starting from 0x0, with a length of 0
Found segment starting from 0x100000, with a length of 0
And yes, I am pushing eax and ebx before calling kmain. Any ideas on what is going wrong here?

It turns out that the bit masking stuff was the problem. If we drop that, we can still have 32-bit addresses and the memory map works just fine.

Related

Store the ADC stream on µSD Card without loss on STM32H743ZI

I am working on a project in which I have to store the datas of an ADC Stream on a µSD card. However even if I use a 16 bits buffer, I lose data from the ADC stream. My ADC is used with DMA and I use FATFS (WITHOUT DMA) and the SDMMC1 peripheral to fill a .bin file with the datas.
Do you have an idea to avoid this loss ?
Here is my project : https://github.com/mathieuchene/STM32H743ZI
I use a nucleo-h743zi2 Board, CubeIDE, and CubeMx in their last version.
EDIT 1
I tried to implement Colin's solution, it's better but I have a strange things in the middle of my acquisition. However when I increase the maximal count value or try to debug, the HardFault_Handler appears. I modified main.c file by creating 2 blocks (uint16_t blockX[BUFFERLENGTH/2]) and 2 flags for when adcBuffer is half filled or completely filled.
I also changed the while(1) part in main function like this
if (flagHlfCplt){
//flagCplt=0;
res = f_write(&SDFile, block1, strlen((char*)block1), (void *)&byteswritten);
memcpy(block2, adcBuffer, BUFFERLENGTH/2);
flagHlfCplt = 0;
count++;
}
if (flagCplt){
//flagHlfCplt=0;
res = f_write(&SDFile, block2, strlen((char*)block2), (void *)&byteswritten);
memcpy(block1, adcBuffer[(BUFFERLENGTH/2)-1], BUFFERLENGTH/2);
flagCplt = 0;
count++;
}
if (count == 10){
f_close(&SDFile);
HAL_ADC_Stop_DMA(&hadc1);
while(1){
HAL_GPIO_TogglePin(LD1_GPIO_Port, LD1_Pin);
HAL_Delay(1000);
}
}
}
EDIT 2
I modified my program. I set block 1 and block 2 with the length of BUFFERLENGTH and I added a pointer (*idx) to change the buffer which is filled. I don't have HardFault_Handler anymore but I still loose some datas from my adc's stream.
Here are the modification I made:
// my pointer and buffers
uint16_t block1[BUFFERLENGTH], block2[BUFFERLENGTH], *idx;
// init of pointer and adc start
idx=block1;
HAL_ADC_Start_DMA(&hadc1, (uint32_t*)idx, BUFFERLENGTH);
// while(1) part
while (1)
{
if (flagCplt){
if (flagToChangeBuffer) {
idx=block1;
res = f_write(&SDFile, block2, strlen((char*)block2), (void *)&byteswritten);
flagCplt = 0;
flagToChangeBuffer=0;
count++;
}
else {
idx=block2;
res = f_write(&SDFile, block1, strlen((char*)block1), (void *)&byteswritten);
flagCplt = 0;
flagToChangeBuffer=1;
count++;
}
}
if (count == 150){
f_close(&SDFile);
HAL_ADC_Stop_DMA(&hadc1);
while(1){
HAL_GPIO_TogglePin(LD1_GPIO_Port, LD1_Pin);
HAL_Delay(1000);
}
}
}
Does someone know how to solve my matter with these loss?
Best Regards
Mathieu

Stm32f4 dma m2m

I'm using STM32F407VG Discovery Board and I've issue with DMA memory to memory transfer. I want to copy 32 bytes of data from one place in memory to other using DMA by writing copy_dma() function. In while loop i'm checking Transfer Complete flag but DMA never returns it. I want to ask where i'm making mistake? Maybe something in configuration is wrong. I'm using Standart Peripheral Libraries. Here's my code.
#include "stm32f4xx.h"
#define BUFFER_SIZE 32
uint8_t src_buffer[BUFFER_SIZE];
uint8_t dst_buffer[BUFFER_SIZE];
void copy_dma(void);
int main(void)
{
RCC_AHB1PeriphClockCmd(RCC_AHB1Periph_DMA1, ENABLE);
int i;
DMA_InitTypeDef dma;
DMA_DeInit(DMA1_Stream4);
DMA_StructInit(&dma);
dma.DMA_Channel = DMA_Channel_1;
dma.DMA_PeripheralBaseAddr = (uint32_t)src_buffer;
dma.DMA_PeripheralInc = DMA_PeripheralInc_Enable;
dma.DMA_Memory0BaseAddr = (uint32_t)dst_buffer;
dma.DMA_MemoryInc = DMA_MemoryInc_Enable;
dma.DMA_BufferSize = BUFFER_SIZE;
dma.DMA_DIR = DMA_DIR_MemoryToMemory;
dma.DMA_FIFOMode = DMA_FIFOMode_Disable;
dma.DMA_MemoryBurst = DMA_MemoryBurst_Single;
dma.DMA_MemoryDataSize = DMA_MemoryDataSize_Byte;
dma.DMA_PeripheralBurst = DMA_PeripheralBurst_Single;
dma.DMA_PeripheralDataSize = DMA_PeripheralDataSize_Byte;
dma.DMA_Mode = DMA_Mode_Normal;
dma.DMA_Priority = DMA_Priority_High;
DMA_Init(DMA1_Stream4, &dma);
for (i = 0; i < BUFFER_SIZE; i++) {
src_buffer[i] = 100 + i;
}
copy_dma();
while(1) {
}
}
void copy_dma(void) {
DMA_Cmd(DMA1_Stream4, ENABLE);
while (DMA_GetFlagStatus(DMA1_Stream4, DMA_FLAG_TCIF4) == RESET);
}
In app note "Using the STM32F2 and STM32F4 DMA controller"(http://stm32.eefocus.com/download/index.php?act=down&id=6312)
is mentioned:
"Memory to memory (only DMA2 is able to do such transfer, in this mode, the circular and direct modes are not allowed.)"
So, try to use DMA2.
In addition to Mariusz Górka's awnser:
When using the DMA you need to know which memory region you are using. The stm32f4 has a memory section called Core Coupled Memory (CCM). The DMA does not have access to this region.
Check your map file and make sure your buffers are not in the region 0x10000000 - 0x1000FFFF.

Memory leakage in QLocalServer/QLocalSocket

I have some problem in using QLocalServer/QLocalSocket.
I'm sending raw pixel data from server to client, and there is a huge memory leakage while process. but I couldn't know what is the reason...
Memory increases about 20MB/1sec (when I checked with my eyes in system administrator.)
Followings are my codes.
Server
void qsharedServer::updateImageData(unsigned char* r_data, int r_width, int r_height, int r_step, int r_label_i){
QLocalSocket* connection = 0;
connection = clientSocket;
if (connection)
{
if (connection->isOpen())
{
QByteArray block;
QDataStream out(&block, QIODevice::WriteOnly);
out.setVersion(QDataStream::Qt_5_7);
const char* rc_data = reinterpret_cast<const char*>(r_data);
out <<r_step*r_height<< r_width << r_height << r_step;
out.writeBytes(rc_data, r_step*r_height);
connection->write(block);
connection->flush();
}
}
Client
void qsharedClient::readSocket(){
QByteArray block = connection->readAll();
QDataStream in(&block, QIODevice::ReadOnly);
in.setVersion(QDataStream::Qt_5_7);
/* Read Raw Data */
char* data;
uint size;
int width;
int height;
int step;
while (!in.atEnd())
{
in >> size >> width >> height >> step;
in.readBytes(data, size);
}
emit drawData((unsigned char*)data, width, height, step);
}
These two codes can communicate well, but memories are increasing very sharply and terminated when it over certain level.
I tried like connection->reset() or QByteArray::clear()..etc but it doesn't worked.
Is there any idea with my problem??
How about using QTcpServer/QTcpSocket?? This can solve my problem??
Please share your idea. Thanks!!
This should be happening because named pipe stores all data until you close it.
As a solution you should call disconnectFromServer() for socket (on the sender or receiver side - does not matter) once you have sent/received enough data for one packet.

Changing DCT coefficients

I decided to use libjpeg as the main library working with jpeg files.
I've read libjpg.txt file. And I was pleased that library allows DCT coefficients reading/writing in a convenient way. Since writing an own decoder will take a long time.
My work is related to the lossless embedding. Currently I need to read DCT coefficients from a file then modify some of them and write changed coefficients in the same file.
Well, I found jpeg_write_coefficients() function. And I naively thought that I could apply it to a decompression object (struct jpeg_decompress_struct). But it does not work and requires a compression object.
I can't believe that such the powerful library is not able to do this.
I think that most likely I'm missing something. Although I tried to be attentive.
Perhaps the writing coefficients can be done more sophisticated way.
But I don't know how to.
I will be very glad if you propose your ideas.
You can ue jpeg_write_coefficients to write your changed DCT.
The following information is avaliable in libjpeg.txt
To write the contents of a JPEG file as DCT coefficients, you must provide
the DCT coefficients stored in virtual block arrays. You can either pass
block arrays read from an input JPEG file by jpeg_read_coefficients(), or
allocate virtual arrays from the JPEG compression object and fill them
yourself. In either case, jpeg_write_coefficients() is substituted for
jpeg_start_compress() and jpeg_write_scanlines(). Thus the sequence is
* Create compression object
* Set all compression parameters as necessary
* Request virtual arrays if needed
* jpeg_write_coefficients()
* jpeg_finish_compress()
* Destroy or re-use compression object
jpeg_write_coefficients() is passed a pointer to an array of virtual block
array descriptors; the number of arrays is equal to cinfo.num_components.
The virtual arrays need only have been requested, not realized, before
jpeg_write_coefficients() is called. A side-effect of
jpeg_write_coefficients() is to realize any virtual arrays that have been
requested from the compression object's memory manager. Thus, when obtaining
the virtual arrays from the compression object, you should fill the arrays
after calling jpeg_write_coefficients(). The data is actually written out
when you call jpeg_finish_compress(); jpeg_write_coefficients() only writes
the file header.
When writing raw DCT coefficients, it is crucial that the JPEG quantization
tables and sampling factors match the way the data was encoded, or the
resulting file will be invalid. For transcoding from an existing JPEG file,
we recommend using jpeg_copy_critical_parameters(). This routine initializes
all the compression parameters to default values (like jpeg_set_defaults()),
then copies the critical information from a source decompression object.
The decompression object should have just been used to read the entire
JPEG input file --- that is, it should be awaiting jpeg_finish_decompress().
jpeg_write_coefficients() marks all tables stored in the compression object
as needing to be written to the output file (thus, it acts like
jpeg_start_compress(cinfo, TRUE)). This is for safety's sake, to avoid
emitting abbreviated JPEG files by accident. If you really want to emit an
abbreviated JPEG file, call jpeg_suppress_tables(), or set the tables'
individual sent_table flags, between calling jpeg_write_coefficients() and
jpeg_finish_compress().
So to change a single dct, you can use the following simple code:
To access any dct coeff, you need to change four index, cx, bx, by, bi.
In my code, I used blockptr_one[bi]++; to increase one dct Coeff
#include <stdio.h>
#include <jpeglib.h>
#include <stdlib.h>
#include <iostream>
#include <string>
int write_jpeg_file(std::string outname,jpeg_decompress_struct in_cinfo, jvirt_barray_ptr *coeffs_array ){
struct jpeg_compress_struct cinfo;
struct jpeg_error_mgr jerr;
FILE * infile;
if ((infile = fopen(outname.c_str(), "wb")) == NULL) {
fprintf(stderr, "can't open %s\n", outname.c_str());
return 0;
}
cinfo.err = jpeg_std_error(&jerr);
jpeg_create_compress(&cinfo);
jpeg_stdio_dest(&cinfo, infile);
j_compress_ptr cinfo_ptr = &cinfo;
jpeg_copy_critical_parameters((j_decompress_ptr)&in_cinfo,cinfo_ptr);
jpeg_write_coefficients(cinfo_ptr, coeffs_array);
jpeg_finish_compress( &cinfo );
jpeg_destroy_compress( &cinfo );
fclose( infile );
return 1;
}
int read_jpeg_file( std::string filename, std::string outname )
{
struct jpeg_decompress_struct cinfo;
struct jpeg_error_mgr jerr;
FILE * infile;
if ((infile = fopen(filename.c_str(), "rb")) == NULL) {
fprintf(stderr, "can't open %s\n", filename.c_str());
return 0;
}
cinfo.err = jpeg_std_error(&jerr);
jpeg_create_decompress(&cinfo);
jpeg_stdio_src(&cinfo, infile);
(void) jpeg_read_header(&cinfo, TRUE);
jvirt_barray_ptr *coeffs_array = jpeg_read_coefficients(&cinfo);
//change one dct:
int ci = 0; // between 0 and number of image component
int by = 0; // between 0 and compptr_one->height_in_blocks
int bx = 0; // between 0 and compptr_one->width_in_blocks
int bi = 0; // between 0 and 64 (8x8)
JBLOCKARRAY buffer_one;
JCOEFPTR blockptr_one;
jpeg_component_info* compptr_one;
compptr_one = cinfo.comp_info + ci;
buffer_one = (cinfo.mem->access_virt_barray)((j_common_ptr)&cinfo, coeffs_array[ci], by, (JDIMENSION)1, FALSE);
blockptr_one = buffer_one[0][bx];
blockptr_one[bi]++;
write_jpeg_file(outname, cinfo, coeffs_array);
jpeg_finish_decompress( &cinfo );
jpeg_destroy_decompress( &cinfo );
fclose( infile );
return 1;
}
int main()
{
std::string infilename = "you_image.jpg", outfilename = "out_image.jpg";
/* Try opening a jpeg*/
if( read_jpeg_file( infilename, outfilename ) > 0 )
{
std::cout << "It's Okay..." << std::endl;
}
else return -1;
return 0;
}
You should really take a look at transupp.h and sources for jpegtran that comes with the library.
Anyway, here is my dirty code with comments, assembled partially from jpegtran. It lets you manipulate DCT coefficients one by one.
#include "jpeglib.h" /* Common decls for cjpeg/djpeg applications */
#include "transupp.h" /* Support routines for jpegtran */
struct jpeg_decompress_struct srcinfo;
struct jpeg_compress_struct dstinfo;
struct jpeg_error_mgr jsrcerr, jdsterr;
static jpeg_transform_info transformoption; /* image transformation options */
transformoption.transform = JXFORM_NONE;
transformoption.trim = FALSE;
transformoption.force_grayscale = FALSE;
jvirt_barray_ptr * src_coef_arrays;
jvirt_barray_ptr * dst_coef_arrays;
/* Initialize the JPEG decompression object with default error handling. */
srcinfo.err = jpeg_std_error(&jsrcerr);
jpeg_create_decompress(&srcinfo);
/* Initialize the JPEG compression object with default error handling. */
dstinfo.err = jpeg_std_error(&jdsterr);
jpeg_create_compress(&dstinfo);
FILE *fp;
if((fp = fopen(filePath], "rb")) == NULL) {
//Throw an error
} else {
//Continue
}
/* Specify data source for decompression */
jpeg_stdio_src(&srcinfo, fp);
/* Enable saving of extra markers that we want to copy */
jcopy_markers_setup(&srcinfo, JCOPYOPT_ALL);
/* Read file header */
(void) jpeg_read_header(&srcinfo, TRUE);
jtransform_request_workspace(&srcinfo, &transformoption);
src_coef_arrays = jpeg_read_coefficients(&srcinfo);
jpeg_copy_critical_parameters(&srcinfo, &dstinfo);
/* Do your DCT shenanigans here on src_coef_arrays like this (I've moved it into a separate function): */
moveDCTAround(&srcinfo, &dstinfo, 0, src_coef_arrays);
/* ..when done with DCT, do this: */
dst_coef_arrays = jtransform_adjust_parameters(&srcinfo, &dstinfo, src_coef_arrays, &transformoption);
fclose(fp);
//And write everything back
fp = fopen(filePath, "wb");
/* Specify data destination for compression */
jpeg_stdio_dest(&dstinfo, fp);
/* Start compressor (note no image data is actually written here) */
jpeg_write_coefficients(&dstinfo, dst_coef_arrays);
/* Copy to the output file any extra markers that we want to preserve */
jcopy_markers_execute(&srcinfo, &dstinfo, JCOPYOPT_ALL);
jpeg_finish_compress(&dstinfo);
jpeg_destroy_compress(&dstinfo);
(void) jpeg_finish_decompress(&srcinfo);
jpeg_destroy_decompress(&srcinfo);
fclose(fp);
And the function itself:
void moveDCTAround (j_decompress_ptr srcinfo, j_compress_ptr dstinfo, JDIMENSION x_crop_offset, jvirt_barray_ptr *src_coef_arrays)
{
size_t block_row_size;
JBLOCKARRAY coef_buffers[MAX_COMPONENTS];
JBLOCKARRAY row_ptrs[MAX_COMPONENTS];
//Allocate DCT array buffers
for (JDIMENSION compnum=0; compnum<srcinfo->num_components; compnum++)
{
coef_buffers[compnum] = (dstinfo->mem->alloc_barray)((j_common_ptr) dstinfo, JPOOL_IMAGE, srcinfo->comp_info[compnum].width_in_blocks,
srcinfo->comp_info[compnum].height_in_blocks);
}
//For each component,
for (JDIMENSION compnum=0; compnum<srcinfo->num_components; compnum++)
{
block_row_size = (size_t) sizeof(JCOEF)*DCTSIZE2*srcinfo->comp_info[compnum].width_in_blocks;
//...iterate over rows,
for (JDIMENSION rownum=0; rownum<srcinfo->comp_info[compnum].height_in_blocks; rownum++)
{
row_ptrs[compnum] = ((dstinfo)->mem->access_virt_barray)((j_common_ptr) &dstinfo, src_coef_arrays[compnum], rownum, (JDIMENSION) 1, FALSE);
//...and for each block in a row,
for (JDIMENSION blocknum=0; blocknum<srcinfo->comp_info[compnum].width_in_blocks; blocknum++)
//...iterate over DCT coefficients
for (JDIMENSION i=0; i<DCTSIZE2; i++)
{
//Manipulate your DCT coefficients here. For instance, the code here inverts the image.
coef_buffers[compnum][rownum][blocknum][i] = -row_ptrs[compnum][0][blocknum][i];
}
}
}
//Save the changes
//For each component,
for (JDIMENSION compnum=0; compnum<srcinfo->num_components; compnum++)
{
block_row_size = (size_t) sizeof(JCOEF)*DCTSIZE2 * srcinfo->comp_info[compnum].width_in_blocks;
//...iterate over rows
for (JDIMENSION rownum=0; rownum < srcinfo->comp_info[compnum].height_in_blocks; rownum++)
{
//Copy the whole rows
row_ptrs[compnum] = (dstinfo->mem->access_virt_barray)((j_common_ptr) dstinfo, src_coef_arrays[compnum], rownum, (JDIMENSION) 1, TRUE);
memcpy(row_ptrs[compnum][0][0], coef_buffers[compnum][rownum][0], block_row_size);
}
}

Limiting a Lua script's memory usage?

I've seen it said multiple times that there is no way to limit a Lua script's memory usage, including people jumping through hoops to prevent Lua scripts from creating functions and tables. But given that lua_newstate allows you to pass a custom allocator, couldn't one just use that to limit memory consumption? At worst, one could use an arena-based allocator and put a hard limit even on the amount of memory that could be used by fragmentation.
Am I missing something here?
static void *l_alloc_restricted (void *ud, void *ptr, size_t osize, size_t nsize)
{
const int MAX_SIZE = 1024; /* set limit here */
int *used = (int *)ud;
if(ptr == NULL) {
/*
* <http://www.lua.org/manual/5.2/manual.html#lua_Alloc>:
* When ptr is NULL, osize encodes the kind of object that Lua is
* allocating.
*
* Since we don’t care about that, just mark it as 0.
*/
osize = 0;
}
if (nsize == 0)
{
free(ptr);
*used -= osize; /* substract old size from used memory */
return NULL;
}
else
{
if (*used + (nsize - osize) > MAX_SIZE) /* too much memory in use */
return NULL;
ptr = realloc(ptr, nsize);
if (ptr) /* reallocation successful? */
*used += (nsize - osize);
return ptr;
}
}
To make Lua use your allocator, you can use
int *ud = malloc(sizeof(int)); *ud = 0;
lua_State *L = lua_State *lua_newstate (l_alloc_restricted, ud);
Note: I haven't tested the source, but it should work.

Resources