I wrote a program which only purpose was to allocate a given amount of memory so that I could see its effect on the Ubuntu 12.04 System Monitor.
This is what I wrote
#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>
int main(int argc, char **argv)
{
if(argc<2)
exit(-1);
int *mem=0;
int kB = 1024*atoi(argv[1]);
mem = malloc(kB);
if(mem==NULL)
exit(-2);
sleep(3);
free(mem);
exit(0);
}
I used sleep() so that the program would not end immediately so that System Monitor would have time to show the change in memory.
What happens (and I'm puzzled about) is that even if I allocate 1GB, the System Monitor shows no memory change at all! Why is that so?
I thought the reason could be because the allocated memory was never accessed so I tried to insert the following before sleep()
int i;
for(i=0; i<kB/sizeof(int); i++)
mem[i]=i;
But this too had no effect on the graph in System Monitor.
Thanks.
Related
In OSTEP(Operating Systems: Three Easy Pieces), the author offers a simple c program code to show how OS Virtualizing it's Memory
#include <unistd.h>
#include <stdio.h>
#include <stdlib.h>
#include "common.h"
int
main(int argc, char *argv[])
{
int *p = malloc(sizeof(int));
//assert(p != NULL);
printf("(%d) address pointed to by p: %p\n", getpid(), p);
*p = 0;
while (1) {
sleep(1);
*p = *p + 1;
printf("(%d) p: %d\n", getpid(), *p);
}
return 0;
}
the book says that because of the virtualizing process the outcome shuld be:
prompt> ./mem &; ./mem &
[1]24113
[2]24114
(24113) address pointed to by p: 0x200000 (24114) address pointed to
(24114) address pointed to by p: 0x200000
(24113) p: 1
(24114) p: 1
(24114) p: 2
(24113) p: 2
(24113) p: 3
(24114) p: 3
(24113) p: 4
(24114) p: 4
the author explains why this happens:
Now, we again run multiple instances of this same program to see what
happens (Figure 2.4). We see from the example that each running
program has allocated memory at the same address (0x200000), and yet
each seems to be updating the value at 0x200000 independently! It is
as if each running program has its own private memory, instead of
sharing the same physical memory with other running programs 5 .
...
but in my computer(ubuntu) the outcome is:
it makes me feel really confuse...
In an attempt to frustrate malware, ubuntu is configured to load programs at pseudo-random addresses; so when your program is loaded into memory, the OS choose some base address, then loads the program relative to that. This is widely known as Address Space Layout Randomization (ASLR).
There is a kernel option: /proc/sys/kernel/randomize_va_space which you can set to zero to disable this feature. Note that if this offers any security, by disabling it you are losing that security feature.
as root:
$ oldval = $(cat /proc/sys/kernel/randomize_va_space)
$ echo 0 >> /proc/sys/kernel/randomize_va_space
then, after you have run your examples as somebody other than root, return to this session and:
$ echo "$oldval" >> /proc/sys/kernel/randomize_va_space
I am running an application on an embedded (PowerPC 32 bit) system where there is a stack size limitation of 64K. I am experiencing some occasional crashes because of stack overflow.
I can build the application also for a normal Linux system (with some minor little changes in the code), so I can run an emulation on my development environment.
I was wondering which is the best way to find the methods that exceed the stack size limitation and which is the stack frame when this happens (in order to perform some code refactoring).
I've already tried Callgrind (a Valgrind tool), but it seems not to be the right tool.
I'm looking more for a tool than changes in the code (since it's a 200K LOC and 100 files project).
The application is entirely written in C++03.
While it seems that there should be an existing tool for this, I would approach it by writing a small macro and adding it to the top of suspected functions:
char *__stack_root__;
#define GUARD_SIZE (64 * 1024 - 1024)
#define STACK_ROOT \
char __stack_frame__; \
__stack_root__ = &__stack_frame__;
#define STACK_GUARD \
char __stack_frame__; \
if (abs(&__stack_frame__ - __stack_root__) > GUARD_SIZE) { \
printf("stack is about to overflow in %s: at %d bytes\n", __FUNCTION__, abs(&__stack_frame__ - __stack_root__)); \
}
And here how to use it:
#include <stdio.h>
#include <stdlib.h>
void foo(int);
int main(int argc, char** argv) {
STACK_ROOT; // this macro records the top of the bottom of the thread's stack
foo(10000);
return 0;
}
void foo(int x) {
STACK_GUARD; // this macro checks if we're approaching the end of memory available for stack
if (x > 0) {
foo(x - 1);
}
}
couple notes here:
this code assumes single thread. If you have multiple threads, you need to keep track of individual __stack_frame__ variables, one per thread. Use thread local storage for this
using abs() to make sure the macro works both when PowerPC grows its stack up, as well as down (it can: depends on your setup)
adjust the GUARD_SIZE to your liking, but keep it smaller than the max size of your stack on the target
I am using MATLAB in my office and Octave when I am at home. Although they are very similar, I was trying to do something I would expected to be very easy and obvious, but found it really annoying. I can't find out how to import TIFF images in Octave. I know the MATLAB geotiffread function is not present, but I thought there would be another method.
I could also skip importing them, as I can work with the imread function in some cases, but then the second problem would be that I can't find a way to write a georeferenced TIFF file (in MATLAB I normally call geotiffwrite with geotiffinfo inputs inside). My TIFF files are usually 8 bit unsigned integer or 32 bit signed integer. I hope someone can suggest a way to solve this problem. I also saw this thread but did not understand if it is possible to use the code proposed by Ashish in Octave.
You may want to look at the mapping library in Octave.
You can also use the raster functions to work with GeoTiffs
Example:
pkg load mapping
filename=”C:\\sl\SDK\\DTED\\n45_w122_1arc_v2.tif”
rasterinfo (filename)
rasterdraw (filename)
The short answer is you can't do it in Octave out of the box. But this is not because it is impossible to do it. It is simply because no one has yet bothered to implement it. As a piece of free software, Octave has the features that its users are willing to spend time or money implementing.
About writing of signed 32-bit images
As of version 3.8.1, Octave uses either GraphicsMagick or ImageMagick to handle the reading and writing of images. This introduces some problems. The number 1 is that your precision is limited to how you built GraphicsMagick (its quantum-depth option). In addition, you can only write unsigned integers. Hopefully this will change in the future but since not many users require it, it's been this way until now.
Dealing with geotiff
Provided you know C++, you can write this functions yourself. This shouldn't be too hard since there is already libgeotiff, a C library for it. You would only need to write a wrapper as an Octave oct function (of course, if you don't know C or C++, then this "only" becomes a lot of work).
Here is the example oct file code which needs to be compiled. I have taken reference of https://gerasimosmichalitsianos.wordpress.com/2018/01/08/178/
#include <octave/oct.h>
#include "iostream"
#include "fstream"
#include "string"
#include "cstdlib"
#include <cstdio>
#include "gdal_priv.h"
#include "cpl_conv.h"
#include "limits.h"
#include "stdlib.h"
using namespace std;
typedef std::string String;
DEFUN_DLD (test1, args, , "write geotiff")
{
NDArray maindata = args(0).array_value ();
const dim_vector dims = maindata.dims ();
int i,j,nrows,ncols;
nrows=dims(0);
ncols=dims(1);
//octave_stdout << maindata(i,0);
NDArray transform1 = args(1).array_value ();
double* transform = (double*) CPLMalloc(sizeof(double)*6);
float* rowBuff = (float*) CPLMalloc(sizeof(float)*ncols);
//GDT_Float32 *rowBuff = CPLMalloc(sizeof(GDT_Float32)*ncols);
String tiffname;
tiffname = "nameoftiff2.tif";
cout<<"The transformation matrix is";
for (i=0; i<6; i++)
{
transform[i]=transform1(i);
cout<<transform[i]<<" ";
}
GDALAllRegister();
CPLPushErrorHandler(CPLQuietErrorHandler);
GDALDataset *geotiffDataset;
GDALDriver *driverGeotiff;
GDALRasterBand *geotiffBand;
OGRSpatialReference oSRS;
char **papszOptions = NULL;
char *pszWKT = NULL;
oSRS.SetWellKnownGeogCS( "WGS84" );
oSRS.exportToWkt( &pszWKT );
driverGeotiff = GetGDALDriverManager()->GetDriverByName("GTiff");
geotiffDataset = (GDALDataset *) driverGeotiff->Create(tiffname.c_str(),ncols,nrows,1,GDT_Float32,NULL);
geotiffDataset->SetGeoTransform(transform);
geotiffDataset->SetProjection(pszWKT);
//CPLFree( pszSRS_WKT );
cout<<" \n Number of rows and columns in array are: \n";
cout<<nrows<<" "<<ncols<<"\n";
for (i=0; i<nrows; i++)
{
for (j=0; j <ncols; j++)
rowBuff[j]=maindata(i,j);
//cout<<rowBuff[0]<<"\n";
geotiffDataset->GetRasterBand(1)->RasterIO(GF_Write,0,i,ncols,1,rowBuff,ncols,1,GDT_Float32,0,0);
}
GDALClose(geotiffDataset) ;
CPLFree(transform);
CPLFree(rowBuff);
CPLFree(pszWKT);
GDALDestroyDriverManager();
return octave_value_list();
}
it can be compiled and run using following
mkoctfile -lgdal test1.cc
aa=rand(50,53);
b=[60,1,0,40,0,-1];
test1(aa,b);
Hi I am trying to read and write data from the i2c bus on the beagle bone black. But I keep reading 0x00 whenever I try to access the Who Am I register on a MMA84152 (or any other register for that matter), which is a constant register which means its value does not change. I am trying to read i2c-1 character driver located in /dev and I connect the sda and scl of the MMA852 lines to pins 19 and 20 on the p9 header. The sda and scl lines are both pulled high with 10 k resistors. Both pin 19 and pin 20 show 00000073 for their pin mux which means it is set for i2c functionality and slew control is slow, reciever is active, the pin is using a pull up resistor, and pull up is enabled. I ran i2cdetect -r 1 and my device shows up as 0x1d which is its correct address. I also ran i2cdump 1 0x1d and 0x2a shows up under 0x0d which is the register I am trying to read from my device and contains the correct value according to the datasheet. But when I read it it returns 0x00 for me. I am also running the latest angstrom distribution and I am logged in under root so no need for sudo.I'm lost now. Here is the code:
#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>
#include <linux/i2c-dev.h>
#include <linux/i2c.h>
#include <sys/ioctl.h>
#include <sys/types.h>
#include <sys/stat.h>
#include <fcntl.h>
#include <string>
using namespace std;
int main(int argc, char **argv){
int X_orientation=0;
char buffer1[256];
string i2cDeviceDriver="/dev/i2c-1";
int fileHandler;
if((fileHandler=open(i2cDeviceDriver.c_str(),O_RDWR))<0){
perror("Failed To Open i2c-1 Bus");
exit(1);
}
if(ioctl(fileHandler,I2C_SLAVE,0x1d)<0){
perror("Failed to acquire i2c bus access and talk to slave");
exit(1);
}
char buffer[1]={0x0D};
if(write(fileHandler,buffer,1)!=1){
perror("Failed to write byte to accelerometer");
exit(1);
}
if(read(fileHandler,buffer1,1)!=1){
perror("Failed to read byte from accelerometer");
exit(1);
}
printf("Contents of WHO AM I is 0x%02X\n",buffer1[0]);
}
It is likely that your I2C device does not support using separate write() and read() commands to query registers (or the equivalent i2c_smbus_write_byte() and i2c_smbus_read_byte() commands). The kernel adds a 'stop bit' to separate your messages on the wire, and some devices do not support this mode.
To confirm:
Try using the Linux i2cget command with the -c ('write byte/read byte' mode, uses separate read and write messages with a stop bit between) flag:
$ i2cget -c 1 0x1d 0x0d
Expected result: 0x00 (Incorrect response)
Then try using i2cget with the -b ('read byte data' mode, which combines the read and write messages without a stop bit) flag:
$ i2cget -b 1 0x1d 0x0d
Expected result: 0x2a (Correct response)
To resolve:
Replace your read() and write() commands with a combined i2c_smbus_read_byte_data() command if available on your system:
const char REGISTER_ID = 0x0d;
char result = i2c_smbus_read_byte_data(fileHandler, REGISTER_ID);
Alternatively (if the above is not available), you can use the the ioctl I2C_RDWR option:
const char SLAVE_ID = 0x1d;
const char REGISTER_ID = 0x0d;
struct i2c_rdwr_ioctl_data readByteData;
struct i2c_msg messages[2];
readByteData.nmsgs = 2;
readByteData.msgs = messages;
// Write portion (send the register we wish to read)
char request = REGISTER_ID;
i2c_msg& message = messages[0];
message.addr = SLAVE_ID;
message.flags = 0; // 0 = Write
message.len = 1;
message.buf = &request;
// Read portion (read in the value of the register)
char response;
message = messages[1];
message.addr = SLAVE_ID;
message.flags = I2C_M_RD;
message.len = 1; // Number of bytes to read
message.buf = &response; // Where to place the result
// Submit the combined read+write message
ioctl(fileHandler, I2C_RDWR, &readByteData);
// Output the result
printf("Contents of register 0x%02x is 0x%02x\n", REGISTER_ID, response);
More information:
https://www.kernel.org/doc/Documentation/i2c/smbus-protocol
https://www.kernel.org/doc/Documentation/i2c/dev-interface
I am trying to use tbb::parallel_for on a machine with 160 parallel threads (8 Intel E7-8870) and 0.5 TBytes of memory. It is a current Ubuntu system with kernel 3.2.0-35-generic #55-Ubuntu SMP. TBB is from the package libtbb2 Version 4.0+r233-1
Even with a very simple task, I tend to run out of resources, either "bad_alloc" or "thread_monitor Resource temporarily unavailable". I boiled it down to this very simple test:
#include <vector>
#include <cstdlib>
#include <cmath>
#include <iostream>
#include "tbb/tbb.h"
#include "tbb/task_scheduler_init.h"
using namespace tbb;
class Worker
{
std::vector<double>& dst;
public:
Worker(std::vector<double>& dst)
: dst(dst)
{}
void operator()(const blocked_range<size_t>& r ) const
{
for (size_t i=r.begin(); i!=r.end(); ++i)
dst[i] = std::sin(i);
}
};
int main(int argc, char** argv)
{
unsigned int n = 10000000;
unsigned int p = task_scheduler_init::default_num_threads();
std::cout << "Vector length: " << n << std::endl
<< "Processes : " << p << std::endl;
const size_t grain_size = n/p;
std::vector<double> src(n);
std::cerr << "Starting loop" << std::endl;
parallel_for(blocked_range<size_t>(0, n, grain_size), RandWorker(src));
std::cerr << "Loop finished" << std::endl;
}
Typical output is
Vector length: 10000000
Processes : 160
Starting loop
thread_monitor Resource temporarily unavailable
thread_monitor Resource temporarily unavailable
thread_monitor Resource temporarily unavailable
The errors appear randomly, and more frequent with greater n. The value of 10 million here is a point where they happen quite regularly. Nevertheless, given the machine characteristics, this should by far not exhaust the memory (I am using it alone for these tests).
The grain size was introduced after tbb created too many instances of the Worker, which made it fail for even smaller n.
Can anybody advise on how to set up tbb to handle large numbers of threads?
Summarizing the discussion in comments in an answer:
The message "thread_monitor Resource temporarily unavailable in pthread_create" basically tells that TBB cannot create enough threads; the "Resource temporarily unavailable" is what strerror() reports for the error code returned by pthread_create(). One possible reason for this error is insufficient memory to allocate stack for a new thread. By default, TBB requests 4M of stack for a worker thread; this value can be adjusted with a parameter to tbb::task_scheduler_init constructor if necessary.
In this particular case, as Guido Kanschat reported, the problem was caused by ulimit accidentally set which limited the memory available for the process.