Gradle does not work correctly in a docker environment, it is destined to use too much memory and be killed for using too much memory.
The memory manager gets its snapshots using the following class
https://github.com/gradle/gradle/blob/master/subprojects/process-services/src/main/java/org/gradle/process/internal/health/memory/MemInfoOsMemoryInfo.java
and in particular Gradle determines how much free memory is left by reading /proc/meminfo, which provides an inaccurate reading in a container.
Gradle only kills off Worker Daemons when a request comes in to make a new Worker Daemon with a larger min heap size then is available according to this reading.
Thus, Gradle will keep making workers until it uses up the alotted amount for the container and be killed.
Does anyone have a workaround for this? Don't really understand how this hasn't been a problem for more people. I suppose it only really becomes an issue if your worker daemons can't be reused and so new ones get created, which is the case for me as I have a large number of modules.
I have a temporary workaround wherein I give every jvm spawned a huge -Xms and so it always triggers the min heap size > available and so always removes prior worker daemons, but this is not satisfactory.
-- edit
To preempt some things, --max-workers does not affect the number of Worker Daemons allowed to exist, it merely affects the number which are allowed to be active. Even with --max-workers = 1, it is allowed to have arbitrary many idle Worker Daemons.
Edit - Ignore the below, it somewhat works but I have since patched Gradle by overwriting the MemInfoOsMemoryInfo class and it works a lot better. Will provide a link to the MR onto Gradle soon.
Found a reasonable work around, we listen for the os memory updates, and every time a task is done we request more memory than is determined to be free, ensuring a daemon is stopped.
import org.gradle.process.internal.health.memory.OsMemoryStatus
import org.gradle.process.internal.health.memory.OsMemoryStatusListener
import org.gradle.process.internal.health.memory.MemoryManagertask
task expireWorkers {
doFirst {
long freeMemory = 0
def memoryManager = services.get(MemoryManager.class)
gradle.addListener(new TaskExecutionListener() {
void beforeExecute(Task task) {
}
void afterExecute(Task task, TaskState state) {
println "Freeing up memory"
memoryManager.requestFreeMemory(freeMemory * 2)
}
})
memoryManager.addListener(new OsMemoryStatusListener() {
void onOsMemoryStatus(OsMemoryStatus osMemoryStatus) {
freeMemory = osMemoryStatus.freePhysicalMemory
}
})
}
}
Related
I am using heap_1 memory allocation. There is an initialization task Task_ini, from which 2 tasks Task_1 and Task_2 are launched. Then I delete Task_ini. At some point in time from Task_1 I need to create a new task Task_3. How can I create Task_3 in the FreeRTOS heap in place of Task_ini which has already been deleted by that time, knowing only its TaskHandle_t?
int main(void){
xTaskCreate(Task_ini, "Task_ini", configMINIMAL_STACK_SIZE, NULL, 1, &htask_ini);
vTaskStartScheduler();
for(;;);
}
void Task_ini(void *pParams){
xTaskCreate(Task_function, "Task_1", configMINIMAL_STACK_SIZE, ¶m1, 1, &htask1);
xTaskCreate(Task_function, "Task_2", configMINIMAL_STACK_SIZE, ¶m2, 1, &htask2);
vTaskDelete(NULL);
}
void Task_function(void *pParams){
for(;;){
//task code
//...
//end task code
if(create == true){
create = false;
//Here I need to create a task at the address where the "Task_ini" task was.
//My code creates a task in a new heap section, and if there is no space it will cause a memory allocation error.
xTaskCreate(Task_function, "Task_3", configMINIMAL_STACK_SIZE, ¶m3, 1, &htask3);
}
}
}
The main idea of heap_1 is that you can't free memory. It is simply not capable of doing so. If you want to delete tasks, you need to use other heap_n methods. Even in that case, you should let the kernel to do its job: It's kernels job to manage memory for FreeRTOS objects, not yours.
Actually, deleting tasks isn't considered as a good practice in general. Unless you are really low on heap space, you can simply suspend the task. In this way, you can wake it up again without any cost in case its services are required again.
It's true that an init task will become useless after the system initialization. But there is a well known solution for your init task problem: It can evolve into another task after it completes the initialization sequence. For example, Task_ini can create only Task_2, and instead of creating a Task_1, it can do the Task_1's job itself.
Update:
It's kernels job to manage memory for FreeRTOS objects, not yours.
Actually, FreeRTOS allows you to manage the memory manually, if you prefer to do so. There are static versions of object creation functions, like xTaskCreateStatic(). When using these static versions, you pass two statically allocated buffers to the function for the task stack and the task control block (TCB). Then you will literally be able to place one task onto another (provided that it's deleted). To be able to use these functions, configSUPPORT_STATIC_ALLOCATION must be defined as 1.
But I suggest you to avoid manual memory management unless you have a specific reason to do so.
From Pro Asynchrnous Programming with .Net:
for (int nTry = 0; nTry < 3; nTry++)
{
try
{
AttemptOperation();
break;
}
catch (OperationFailedException) { }
Thread.Sleep(2000);
}
While sleeping, the thread doesn’t consume any CPU-based resources,
but the fact that the thread is alive means that it is still consuming
memory resources. On a desktop application this is probably no big
deal, but on a server application, having lots of threads sleeping is
not ideal because if more work arrives on the server, it may have to
spin up more threads, increasing memory pressure and adding additional
resources for the OS to manage.
Ideally, instead of putting the thread to sleep, you would like to
simply give it up, allowing the thread to be free to serve other
requests. When you are ready to continue using CPU resources again,
you can obtain a thread ( not necessarily the same one ) and continue
processing. You can solve this problem by not putting the thread to
sleep, but rather using await on a Task that is deemed to be completed
in a given period.
for (int nTry = 0; nTry < 3; nTry++)
{
try
{
AttemptOperation();
break;
}
catch (OperationFailedException) { }
await Task.Delay(2000);
}
I don't follow author's reasoning. While it's true that calling await Task.Delay will release this thread ( which is processing a request ), but it's also true that task created by Task.Delay will occupy some other thread to run on. So does this code really enable server to process more simultaneous requests or is the text wrong?!
Task.Delay does not occupy some other thread. It gives you a task without blocking. It starts a timer that completes that task in its callback. The timer does not use any thread while waiting.
It is a common myth that async actions like delays or IO just push work to a different thread. They do not. They use OS facilities to truly use zero threads while the operation is in progress. (They obviously need to use some thread to initiate and complete the operation.)
If async was just pushing work to a different thread it would be mostly useless. It's value would be just to keep the UI responsive in client apps. On the server it would only cause harm. It is not so.
The value of async IO is to reduce memory usag (less thread stacks), context switching and thread-pool utilization.
The async version of the code you posted would scale to literally tens of thousands of concurrent requests (if you increase the ASP.NET limits appropriately, which is a simple web.config change) with small memory usage.
I am trying to understand how I shall port my Java chess engine to dart.
So I have understood that I should use an Isolates to run my engine in parallell with the GUI but how can I force the engine to terminate the search.
In java I just set some boolean that where shared between the engine thread and the gui thread.
Answer I got:
You should send a message to the isolate, telling it to stop. You can simply do something like:
port.send('STOP');
My request
Thanks for the clarification. What I don't understand is that if the chess engine isolate is busy due to a port.send('THINK') command how can it respond to a port.send('STOP') command
Each isolate is single-threaded. As long as your program is running nobody else will have the means to interfere with your execution.
If you want to be able to react to outside events (including messages from other isolates) you need to split your long running execution into smaller parts. A chess-engine probably has already some state to know where to look for the next move (assuming it's built with something like A*). In this case you could just periodically interrupt your execution and resume after a minimal timeout.
Example:
var state;
var stopwatch = new Stopwatch()..run();
void longRunning() {
while (true) {
doSomeWorkThatUpdatesTheState();
if (stopwatch.elapsedMilliseconds > 200) {
stopwatch.reset();
Timer.run(longRunning);
return;
}
}
}
The new API will contain a
isolate.kill(loopForever ? Isolate.IMMEDIATE : Isolate.AS_EVENT);
See https://code.google.com/p/dart/issues/detail?id=21189#c4 for a full example.
I use the heart program to restart an Erlang node when it becomes unresponsive. However, I am finding it hard to understand why the node freezes. SASL logs don't show any errors, and my own logs don't seem to show anything remarkable happening at those times. Can anybody give advice on debugging this sort of thing?
By default the heart program issues a SIGKILL to kill off the unresponsive VM so it can quickly start a new one. This makes getting any useful information about the VM pretty much impossible. Something I've tried in the past is to patch the heart program to avoid the hard kill and instead get the VM to create a crash dump and a coredump. I used a patch like this (this one is for Erlang/OTP R14B02):
--- erts/etc/common/heart.c.orig 2011-04-17 12:11:24.000000000 -0400
+++ erts/etc/common/heart.c 2011-04-17 12:12:36.000000000 -0400
## -559,10 +559,11 ##
int res;
if(heart_beat_kill_pid != 0){
pid = (pid_t) heart_beat_kill_pid;
- res = kill(pid,SIGKILL);
+ res = kill(pid,SIGUSR1);
+ sleep(4);
for(i=0; i < 5 && res == 0; ++i){
sleep(1);
- res = kill(pid,SIGKILL);
+ res = kill(pid,i < 2 ? SIGQUIT : SIGKILL);
}
if(errno != ESRCH){
print_error("Unable to kill old process, "
As you can see, with this patch heart will first issue a SIGUSR1 to try to get the VM to create a crash dump. Since this can take awhile, heart then sleeps for 4 seconds. You might have to increase this sleep time if you're not getting full crash dumps. After that, heart then tries twice to issue a SIGQUIT with the hope of getting a coredump, and if that fails, issues a SIGKILL.
Note that this patch will slow down heart's VM restart due to the time required to wait for the crash dumps and coredumps. If you use it in production, be aware of this limitation.
You could try to call erlang:halt/1 from your HEART_COMMAND thus creating a crash dump from the unresponsive node.
You can try using the erl_call tool with e.g. -a erlang halt 123.
If the erlang node can't respond to this is also interesting information.
Did you try increasing `HEART_BEAT_TIMEOUT? Maybe the node is just bogged down a bit an misses the timeout but doesn't freeze.
If you have any idea of why it is freezing you could try to trace the module using dbg.
http://www.erlang.org/doc/man/dbg.html
In short try
dbg:tracer(), dbg:p(all,c), dbg:tpl(Module, Function, x).
If you want to stop this tracing issue
dbg:ctpl()
See documentation for more info.
Note: Change Module and Function to whatever you want to trace, leave x as it is. You can also skip Function and only give Module, x.
Warning: Running this on a live system can be dangerous as the amount of information that is going to be printed to the shell can be enormous.
OpenCL doesn't have a global barrier that will stop all threads, so I'm trying to create a work around with the following code:
void barrier(__global uint* scratch) {
uint nThreads = get_global_size(0);
atom_inc(scratch);
/* this loop never terminates */
while(scratch[0] < nThreads) {
continue;
}
}
The idea is that each thread loops until all of them increment that one piece of memory.
However, the value read from scratch[0] never changes for the threads once it's been read, and it loops forever. I know it's being incremented because it's the correct value when I read it back to the host.
Is the global memory being locally cached? What's going on here?
Found the problem: the order in which work groups are executed is implementation defined. This means that some threads might start only after others have finished.
In the code I gave, the work groups that are started first will loop forever waiting on the the others to hit the 'barrier'. And the work groups that would be started later won't ever start because they're waiting for the first ones to finish.
If the implementation (I'm on a Radeon 5750, using Stream SDK 2.2) executes all work groups concurrently, then it probably wouldn't be an issue. But that's not the case for my setup.