I have a simple question regarding file descriptors passage from processes into their threads. I'm almost sure but need to a confirmation, if the files descriptors are treated as normal integers and thus can be passed through an array of integers for example to the process thread through the pthread_create() thread argument. Thanks
The rough definition of the term "process" could be "a memory space with at least one thread". In other words, all threads within the same process share a memory space.
Now, file descriptors are basically indices that reference objects within a table that belongs to the process. Since the objects belong to the process, and the threads operate inside the process, the threads can refer to these objects via their index ("file descriptor").
Yes, file descriptors are just integers and so can be passed as function arguments like any other variable. They will still refer to the same files, because the open files are shared by all the threads in a process.
#include <pthread.h>
struct files {
int count;
int* descriptors;
};
void* worker(void* p)
{
struct files *f = (struct files*)p;
// ...
}
int main(void)
{
struct files f;
f.count = 4;
f.descriptors = (int*)malloc(sizeof(int) * f.count);
f.descriptors[0] = open("...", O_RDONLY);
// ...
pthread_t t;
pthread_create(&t, NULL, worker, &f);
// ...
pthread_join(t);
}
Related
In most managed languages (that is, the ones with a GC), local variables that go out of scope are inaccessible and have a higher GC-priority (hence, they'll be freed first).
Now, C is not a managed language, what happens to variables that go out of scope here?
I created a small test-case in C:
#include <stdio.h>
int main(void){
int *ptr;
{
// New scope
int tmp = 17;
ptr = &tmp; // Just to see if the memory is cleared
}
//printf("tmp = %d", tmp); // Compile-time error (as expected)
printf("ptr = %d\n", *ptr);
return 0;
}
I'm using GCC 4.7.3 to compile and the program above prints 17, why? And when/under what circumstances will the local variables be freed?
The actual behavior of your code sample is determined by two primary factors: 1) the behavior is undefined by the language, 2) an optimizing compiler will generate machine code that does not physically match your C code.
For example, despite the fact that the behavior is undefined, GCC can (and will) easily optimize your code to a mere
printf("ptr = %d\n", 17);
which means that the output you see has very little to do with what happens to any variables in your code.
If you want the behavior of your code to better reflect what happens physically, you should declare your pointers volatile. The behavior will still be undefined, but at least it will restrict some optimizations.
Now, as to what happens to local variables when they go out of scope. Nothing physical happens. A typical implementation will allocate enough space in the program stack to store all variables at the deepest level of block nesting in the current function. This space is typically allocated in the stack in one shot at the function startup and released back at the function exit.
That means that the memory formerly occupied by tmp continues to remain reserved in the stack until the function exits. That also means that the same stack space can (and will) be reused by different variables having approximately the same level of "locality depth" in sibling blocks. The space will hold the value of the last variable until some other variable declared in some sibling block variable overrides it. In your example nobody overrides the space formerly occupied by tmp, so you will typically see the value 17 survive intact in that memory.
However, if you do this
int main(void) {
volatile int *ptr;
volatile int *ptrd;
{ // Block
int tmp = 17;
ptr = &tmp; // Just to see if the memory is cleared
}
{ // Sibling block
int d = 5;
ptrd = &d;
}
printf("ptr = %d %d\n", *ptr, *ptrd);
printf("%p %p\n", ptr, ptrd);
}
you will see that the space formerly occupied by tmp has been reused for d and its former value has been overriden. The second printf will typically output the same pointer value for both pointers.
The lifetime of an automatic object ends at the end of the block where it is declared.
Accessing an object outside of its lifetime is undefined behavior in C.
(C99, 6.2.4p2) "If an object is referred to outside of its lifetime, the behavior is undefined. The value of a pointer becomes indeterminate when the object it points to reaches the end of its lifetime."
Local variables are allocated on the stack. They are not "freed" in the sense you think about GC languages, or memory allocated on the heap. They simply go out of scope, and for builtin types the code won't do anything - and for objects, the destructor is called.
Accessing them beyond their scope is Undefined Behaviour. You were just lucky, as no other code has overwritten that memory area...yet.
I am trying to use cli/c++ system/multithreading for a opencv detection software I have. I have got several haarcascades which I want to use them in several threads in order to run simultaneously. I am trying to following the instructions from here:. I have noticed that when I create ref class I cant defined as a class member opencv objects. For example when I am trying to define private variables for my ref class Detection:
private:
Mat *image;
CascadeClassifier *cascade;
double scale;
int neighbors;
public:
Detection(cv::Mat &img, cv::CascadeClassifier &cas, double sc, int neigh)
{
image = new cv::Mat(img);
cascade = new cv::CascadeClassifier(cas);
scale = sc;
neighbors = neigh;
}
void detect_faces(){
Mat gray_image;
cv::cvtColor((*image), gray_image, CV_BGR2GRAY);
cv::equalizeHist(gray_image, gray_image);
std::vector<cv::Rect> faces1;
(*cascade).detectMultiScale( gray_image, faces1, scale, neighbors, 0| CASCADE_SCALE_IMAGE, Size(3, 3), Size(190,190));
faces.insert(faces.end(), faces1.begin(), faces1.end());
}
Main function:
int main()
{
Mat image = imread(...);
cv::CascadeClassifier face_cascade1;
face_cascade1.load("cascades/lbpcascade_profileface.xml");
Detection^ obj1 = gcnew Detection(image, face_cascade1, 1.01,5);
ThreadStart^ myThreadDelegate1 = gcnew ThreadStart(obj1, &Detection::detect_faces );
Thread^ Thread1 = gcnew Thread( myThreadDelegate1 );
Thread1->start();
...//the rest threads
}
This code it seems to work. However as Berak mention I shouldnt make copy of cascadeClassifier. Is there something else I could do? Is there an issue of loosing time with that implementation? Moreover is there a chance to move detectMultiScale in main function?
C++/CLI ref classes can only hold .Net objects or primitive types as members.
If you want to have a C++ member, you'll have to hold a pointer to it. (Which can be compiled to an integer type big enough to hold the pointer).
Now if you want to check if the threads are running concurrently, you can do 2 things:
Pause the program, and look at the threads window. (Debug -> Windows -> Threads)
Here you can see all the working threads, but you might be unlucky and "miss" the timing that shows both threads working.
Add a long sleep() in each thread. just before starting its work. Then Pause the program like in (1) and if you see the threads waiting on the sleep() then it means they're working concurrently.
I'm trying to write data structures defines in C to my Arduino Uno board's non-volatile memory, so the values of the struct will be retained after the power goes off or it is reset.
To my understanding, the only way to do this (while the sketch is running) would be to write to arduino's EEPROM. Although I can write individual bytes (sets a byte with value 1 at address 0):
eeprom_write_byte(0,1);
I am stuck trying to write a whole struct:
typedef struct NewProject_Sequence {
NewProject_SequenceId sequenceId;
NewProject_SequenceLength maxRange;
NewProject_SequenceLength minRange;
NewProject_SequenceLength seqLength;
NewProject_SceneId sceneList[5];
} NewProject_Sequence;
Because of the EEPROM's limit of 100,000 writes, I don't want to write to the Arduino in a loop going through each byte, for this will probably use it up pretty fast. Does anyone know a more efficient way of doing this, either with EEPROM or if there's a way to write to PROGMEM while the sketch is running? (without using the Arduino Library, just C).
RESOLVED
I ended up writing two custom functions -- eepromWrite and eepromRead. They are listed below:
void eepromRead(uint16_t addr, void* output, uint16_t length) {
uint8_t* src;
uint8_t* dst;
src = (uint8_t*)addr;
dst = (uint8_t*)output;
for (uint16_t i = 0; i < length; i++) {
*dst++ = eeprom_read_byte(src++);
}
}
void eepromWrite(uint16_t addr, void* input, uint16_t length) {
uint8_t* src;
uint8_t* dst;
src = (uint8_t*)input;
dst = (uint8_t*)addr;
for (uint16_t i = 0; i < length; i++) {
eeprom_write_byte(dst++, *src++);
}
}
The would be implemented like this:
uint16_t currentAddress;
struct {
uint16_t x;
uint16_t y;
} data;
struct {
} output;
uint16_t input
eepromWrite(currentAddress, data, sizeof(data);
eepromRead(currentAddress, output, sizeof(data));
Several solutions and or combinations.
setup a timer event to store the values periodically, rather then
back to back.
use a checksum, then increment the initial offset,
when writing. Where when reading you attempt each increment until
you have a valid checksum. this spreads your data across the entire
range increasing your life. modern flash drives do this.
Catch the unit turning off, by using an external Brown Out Detector to trigger an INT to then quickly write the EEPROM. Where you can then also use the internal BOD to prevent corruption, before it falls below safe writing voltages. By having the external significantly higher than the internal thresholds. The time to write before complete shutdown can be increased by increasing the VCC capacitance. Where the external BOD is compared before the VCC and not directly the VCC itself.
Here is a video explaining how to enable the internal BOD, for a ATtiny, where it is nearly identical for the other ATmega's. Video
The Arduino EEPROM library provides get/put functions that are able to read and write structs...
Link to EEPROM.put(...)
The write is made only when a byte has changed.
So, using put/get is the solution to your problem.
I'm using these in a wide (25k) project without any problem.
And as already said I've used a timer to write not each time but some time to times.
Turning off detection is also a very good way to do this.
I have a weird problem, so I thought I would ask and see if someone more experienced than me could see a solution.
I am writing a program with CUDA C/C++, and I have some constant integers that specify various things, like coordinates of the bounds of the calculation, etc.. Currently I just have those things in global device memory. They are accessed by every thread in every kernel call, and so I figured that if they are in global memory, then they never are being cached or broadcast (right?). And so these little integers are taking up a lot (relatively) of overhead, and have a lot of 'read redundancy.'
So I declare in a header:
__constant__ int* number;
I include that header, and, when I do memory stuff, I do:
cutilSafeCall( cudaMemcpyToSymbol(number, &(some_host_int), sizeof(int) );
I pass number into all my kernel's then:
__global__ void magical_kernel(int* number, ...){
//and I access 'number' like this
int data_thingy = big_array[ *number ];
}
My code crashes. With number in global memory, it is just fine. I have determined that it crashes sometime upon accessing number within the kernel. This means that either I am accessing or allocating it wrong. If it holds the wrong value, it will also cause a crash, because it is used to index into arrays.
To conclude, I will ask a few questions. First, what am I doing wrong? As a bonus: is there a better way than constant memory to accomplish this task - I don't know the value of number at compile time, so a simple #define won't work. Will constant memory even speed the code up at all, or has it been cached and broadcasted all along? Could I somehow put the data in shared memory for each threadblock and have it remain in shared memory through multiple kernel calls?
There are several problems here:
You have declared number as a pointer, but never assigned it a value which is valid address in GPU memory
You have a variable scope onflict: the argument variable int * number defined in magic_kernel is not the same variable as the __constant__ int * variable defined as compilation unit scope.
The first argument of the cudaMemcpyToSymbol call is almost certainly incorrect.
If you don't understand why either of the first two point are true, you have some revision to do on pointers and scope in C++.
Based on your response to a now deleted answer, I suspect what you are actually trying to do is this:
__constant__ int number;
__global__ void magical_kernel(...){
int data_thingy = big_array[ number ];
}
cudaMemcpyToSymbol("number", &(some_host_int), sizeof(int));
i.e. number is intended to be an integer in constant memory, not a pointer, and not a kernel argument.
EDIT: here is an exmaple which shows this in action:
#include <cstdio>
__constant__ int number;
__global__ void magical_kernel(int * out)
{
out[threadIdx.x] = number;
}
int main()
{
const int value = 314159;
const size_t sz = size_t(32) * sizeof(int);
cudaMemcpyToSymbol("number", &value, sizeof(int));
int * _out, * out;
out = (int *)malloc(sz);
cudaMalloc((void **)&_out, sz);
magical_kernel<<<1,32>>>(_out);
cudaMemcpy(out, _out, sz, cudaMemcpyDeviceToHost);
for(int i=0; i<32; i++)
fprintf(stdout, "%d %d\n", i, out[i]);
return 0;
}
You should be able to run this yourself and confirm it works as advertised.
I've written a complicated lua script which uses the lua sockets library. It reads a list of files from disk, sorts them by date and sends them to a HTTP process. The number of files on disk is around 65K.The memory usage in taskmanager doesn't exceed 200Mb.
After quite a while the script returns:
lua: not enough memory
I print out the current GC count at points and it never goes above 110Mb
local freeMem = collectgarbage('count');
print("GC Count : " .. freeMem/1024 .. " MB");
This is on a 32 bit windows machine.
What's the best way to diagnose this?
All memory goes through the single lua_Alloc function. This takes the form of:
typedef void* (*lua_Alloc) (void* ud, void* ptr, size_t oszie, size_t nsize);
All allocations, reallocations and frees go through this. The documentation for this can be found at this web page. You can easily write your own to track all memory operations. For example,
void* MyAlloc (void* ud, void* ptr, size_t osize, size_t nsize)
{
(void)ud; (void)osize; // Not used
if (nsize == 0)
{
free(ptr)
TrackSubtract(osize);
return NULL;
}
else
{
void* p = realloc(ptr,nsize);
TrackSubtract(osize);
if (p) TrackAdd(nsize);
return p;
}
}
You can write the TrackAdd() and TrackSubtract() functions to whatever you want: output to a log; adjust a counter and so on.
To use your new function you pass a pointer to it when you create the Lua state:
lua_State* L = lua_newstate(&MyAlloc,0);
The documentation to lua_newstate is found here.
Good luck.
Use perfmon to monitor your process and add counters for private bytes and virtual bytes.
When your script ends with 'not enough memory' see the value of each counter. If you see sudden peaks in your memory usage, try to add more points in which you print the memory usage.