I am using FreeRTOS v8.2.3 on a PIC32 micro controller. I have a case where I need to post the following 3 events to 3 corresponding queues from an ISR, in order to unblock a task awaiting one of these events at a time -
a) SETUP packet arrival
b) Transfer completed event 1
c) Transfer completed event 2
My exection sequence and requirement are as follows:
Case 1 (execution is blocked for an event at point_1):
As SETUP arrives while waiting at point_1 of execution -
i) the waiting task should be unblocked
ii)Setup received from queue and processed
Some code is processed and reaches point_2
Case 2 (execution is blocked for an event at point_2):
If any one of SETUP or transfer complete events occur at point_2 -
i) unblock the wait
ii) receive transfer_complete_1 or transfer_complete_2 event from queue to carry out some additional transfers and loop at point_2
iii)if it was a Setup queue event, do not receive, but go to point_1
The code does not seem to work when I try to use xQueueReceive and xQueueSelectFromSet on the Setup queue even when one of them is used at point_1 and the other used at point_2.
But seems to work fine if I use xQueueSelectFromSet at both the places and verify the queuset member handle that caused the event to proceed further.
Given the requirement above, the problem with using xQueueSelectFromSet at both the places is that
- the xQueueSelectFromSet call will be placed back to back, first on a Setup event at point_2 and then immediately on point_1 which is not intentional
- the xQueueSelectFromSet call at point_1 is also not desired
Hence can anyone please explain whether and how to use both a queueset and queuereceive on the same queue? If not possible how do we typically implement the above requirement in FreeRTOS?
This is a duplicate of a question asked on the FreeRTOS support forum, so below is a duplicate of the answer I gave there:
I don't fully understand your usage scenario, but some points which may help.
1) If a queue is a member of a queue set, then the queue can only be read after its handle has been returned from the queue set. Further, if a queue's handle is returned from a queue set then the item must be read from the queue. If either of these requirements are not met then the state of the queue set will not match that of the queues in the set.
2) If the same task is reading from the multiple queues then it is probably not necessary to use a queue set at all. See the "alternatives to using queue sets" section on the following page: http://www.freertos.org/Pend-on-multiple-rtos-objects.html
Related
Can I wake up task before vTaskDelay expires?
I have code like this:
In the task (hnd_uart_task) code:
transmit_frame();
vTaskDelay(100); // task should wait 100 ticks or be woken up by uart ISR
parse_response();
UART Interrupt:
// if byte was received
BaseType_t xYieldRequired = xTaskResumeFromISR(hnd_uart_task);
portYIELD_FROM_ISR(xYieldRequired);
Instead of using vTaskDelay(), you can use task notifications with timeout.
USART Interrupt:
// if byte was received
BaseType_t xHigherPriorityTaskWoken = pdFALSE;
vTaskNotifyGiveFromISR(hnd_uart_task, &xHigherPriorityTaskWoken);
portYIELD_FROM_ISR(xHigherPriorityTaskWoken);
Task Code:
transmit_frame();
ulTaskNotifyTake(pdTRUE, 100);
parse_response();
ulTaskNotifyTake(pdTRUE, 100) returns when a task notification is received from the ISR, or when 100 tick timeout period elapses.
But as #Jacek Ślimok pointed out, a byte-by-byte parsing may not be good idea. The exact method depends on the protocol used. But in general, you set up your DMA or interrupts to fill a reception buffer with incoming bytes. For example, when parsing Modbus frames, you can use idle line detection hardware interrupt and give notification to a task which parses the RX buffer.
No, because that's not what vTaskDelay was meant to be used for.
The closest solution to yours would be to create a semaphore that you attempt to take inside the task with a 100ms delay and that you give from ISR. This way the task will block for a maximum of 100ms waiting for semaphore to be given, after which it'll unblock and resume execution anyway. If it's given earlier, it'll unblock earlier, which is what I assume you want.
However, from what you've written I assume you want to achieve something like the following:
Send some data over UART
Wait for response
As soon as response is received, do something with the response
In this case, doing both blocking and parsing in the same task is going to be hard (you reaaaalllly don't want to do any sort of parsing inside of your ISR). I'd therefore recommend the following "layout" of two tasks, an interrupt and one shared smepahore between the two tasks:
"High level" task (let's call it ApplicationTask) that can do the following:
Construct whole frames and request them to be sent over UART (add them to some kind of queue). This "construction of whole frames" and sending them over to the other tasks would usually be wrapped into some functions.
Will block waiting for response
Will receive already parsed data (full frame or object/structure holding that parsed data)
"Byte level" task (let's call it ByteTask) that can do the following:
Has a queue for transmitted data (queue of frames or queue of raw bytes)
Has a queue for received data
"Pushes" data from "data to be transmitted" queue into UART
Parses UART data that appears in "received data" queue and gives semaphore to unblock the ApplicationTask
UART Interrupt:
Only transmits data that it's told to transmit by ByteTask
Pushes received data into ByteTask receive queue
Shared semaphore between ApplicationTask and ByteTask:
Whenever ApplicationTask wants to "wait to receive response", it attempts to take this semaphore. Maximum blocking time can be used as a "response receiving timeout".
Whenever ByteTask receives and parses enough data to decide that "yes, this is a complete response", it gives this semaphore.
This above is a super simple example of something that should be easy enough to scale to more tasks as you develop your application. You can get a lot more fancy than that (e.g. ByteTask handling multiple UARTs at the same time, have a pool of semaphores used for blocking for multiple tasks, do some more sophisticated message dispatching etc.), but the example above should hopefully give you a better idea of how something like this can be approached.
I am still confused about the NumberOfConcurrentThreads parameter within CreateIoCompletionPort(). I have read and re-read the MSDN dox, but the quote
This value limits the number of runnable threads associated with the
completion port.
still puzzles me.
Question
Let's assume that I specify this value as 4. In this case, does this mean that:
1) a thread can call GetQueuedCompletionStatus() (at which point I can allow a further 3 threads to make this call), then as soon as that call returns (i.e. we have a completion packet) I can then have 4 threads again call this function,
or
2) a thread can call GetQueuedCompletionStatus() (at which point I can allow a further 3 threads to make this call), then as soon as that call returns (i.e. we have a completion packet) I then go on to process that packet. Only when I have finished processing the packet do I then call GetQueuedCompletionStatus(), at which point I can then have 4 threads again call this function.
See my confusion? Its the use of the phrase 'runnable threads'.
I think it might be the latter, because the link above also quotes
If your transaction required a lengthy computation, a larger
concurrency value will allow more threads to run. Each completion
packet may take longer to finish, but more completion packets will be
processed at the same time.
This will ultimately affect how we design servers. Consider a server that receives data from clients, then echoes that data to logging servers. Here is what our thread routine could look like:
DWORD WINAPI ServerWorkerThread(HANDLE hCompletionPort)
{
DWORD BytesTransferred;
CPerHandleData* PerHandleData = nullptr;
CPerOperationData* PerIoData = nullptr;
while (TRUE)
{
if (GetQueuedCompletionStatus(hCompletionPort, &BytesTransferred,
(PULONG_PTR)&PerHandleData, (LPOVERLAPPED*)&PerIoData, INFINITE))
{
// OK, we have 'BytesTransferred' of data in 'PerIoData', process it:
// send the data onto our logging servers, then loop back around
send(...);
}
}
return 0;
}
Now assume I have a four core machine; if I leave NumberOfConcurrentThreads as zero within my call to CreateIoCompletionPort() I will have four threads running ServerWorkerThread(). Fine.
My concern is that the send() call may take a long time due to network traffic. Hence, I could be receiving a load of data from clients that cannot be dequeued because all four threads are taking a long time sending the data on?!
Have I missed the point here?
Update 07.03.2018 (This has now been resolved: see this comment.)
I have 8 threads running on my machine, each one runs the ServerWorkerThread():
DWORD WINAPI ServerWorkerThread(HANDLE hCompletionPort)
{
DWORD BytesTransferred;
CPerHandleData* PerHandleData = nullptr;
CPerOperationData* PerIoData = nullptr;
while (TRUE)
{
if (GetQueuedCompletionStatus(hCompletionPort, &BytesTransferred,
(PULONG_PTR)&PerHandleData, (LPOVERLAPPED*)&PerIoData, INFINITE))
{
switch (PerIoData->Operation)
{
case CPerOperationData::ACCEPT_COMPLETED:
{
// This case is fired when a new connection is made
while (1) {}
}
}
}
I only have one outstanding AcceptEx() call; when that gets filled by a new connection I post another one. I don't wait for data to be received in AcceptEx().
I create my completion port as follows:
CreateIoCompletionPort(INVALID_HANDLE_VALUE, NULL, 0, 4)
Now, because I only allow 4 threads in the completion port, I thought that because I keep the threads busy (i.e. they do not enter a wait state), when I try and make a fifth connection, the completion packet would not be dequeued hence would hang! However this is not the case; I can make 5 or even 6 connections to my server! This shows that I can still dequeue packets even though my maximum allowed number of threads (4) are already running? This is why I am confused!
the completion port - is really KQUEUE object. the NumberOfConcurrentThreads is corresponded to MaximumCount
Maximum number of concurrent threads the queue can satisfy waits for.
from I/O Completion Ports
When the total number of runnable threads associated with the
completion port reaches the concurrency value, the system blocks the
execution of any subsequent threads associated with that completion
port until the number of runnable threads drops below the concurrency
value.
it's bad and not exactly said. when thread call KeRemoveQueue ( GetQueuedCompletionStatus internal call it) system return packet to thread only if Queue->CurrentCount < Queue->MaximumCount even if exist packets in queue. system not blocks any threads of course. from another side look for KiInsertQueue - even if some threads wait on packets - it activated only in case Queue->CurrentCount < Queue->MaximumCount.
also look how and when Queue->CurrentCount is changed. look for KiActivateWaiterQueue (This function is called when the current thread is about to enter a wait state) and KiUnlinkThread. in general - when thread begin wait for any object (or another queue) system call KiActivateWaiterQueue - it decrement CurrentCount and possible (if exist packets in queue and became Queue->CurrentCount < Queue->MaximumCount and threads waited for packets) return packet to wait thread. from another side, when thread stop wait - KiUnlinkThread is called. it increment CurrentCount.
your both variant is wrong. any count of threads can call GetQueuedCompletionStatus(). and system of course not blocks the execution of any subsequent threads. for example - you have queue with MaximumCount = 4. you can queue 10 packets to queue. and call GetQueuedCompletionStatus() from 7 threads in concurrent. but only 4 from it got packets. another will be wait (despite yet 6 packets in queue). if some of threads, which remove packet from queue begin wait - system just unwait and return packet to another thread wait on queue. or if thread (which already previous remove packet from this queue (Thread->Queue == Queue) - so active thread) again call KeRemoveQueue will be Queue->CurrentCount -= 1;
I am working on the project where I communicate with another device over COM port.
For incoming data I am using VaComm1RXchar event, there I store the message into the array and increment msgIndex which represents number of messages.
Then I call the function where I work with this message.
Inside this function is this timeout cycle where I wait for this message:
while MsgIndex < 1 do
begin
stop := GetTickCount;
if (stop - start)> timeout then
begin
MessageBox(0, 'Timeout komunikace !', 'Komunikace', MB_OK);
exit(false);
end;
sleep(10);
end;
The strange thing for me is that, when have it like this above then it always end with timeout. But when I put on there before this while cycle a ShowMessage('Waiting') then It works correcly.
Does anyone know what can caused this and how can I solve it? Thanks in advance!
We can deduce that the VaComm1RXchar event is a synchronous event and that by blocking your program in a loop you are preventing the normal message processing that would allow that event to execute.
Showing a modal dialog box, on the other hand, passes message handling to that dialog so the message queue is properly serviced and your Rx events and handled normally.
You can be certain this is the case if this also works (please never write code like this - it's just to prove the point):
while MsgIndex < 1 do begin
stop := GetTickCount;
if (stop - start)> timeout then begin
MessageBox(0, 'Timeout komunikace !', 'Komunikace', MB_OK);
exit(false);
end;
Application.ProcessMessages; // service the message queue so that
sleep(10); // your Rx event can be handled
end;
If there is a lesson here it is that RS-232 communication really needs to be done on a background thread. Most all implementations of "some chars have been received" events lead to dreadful code for the very reason you are discovering. Your main thread needs to be free to process the message that characters have been received but, at the same time, you must have some parallel process that is waiting for those received characters to complete a cogent instruction. There does not exist a sensible solution in an event-driven program to both manage the user interface and the communication port at the same time on one thread.
A set of components like AsyncPro**, for example, wrap this functionality into data packets where synchronous events are used but where the components manage start and end string (or bytes) detection for you on a worker thread. This removes one level of polling from the main thread (ie: you always get an event when a complete data packet has arrived, not a partial one). Alternatively, you can move the communication work to a custom thread and manage this yourself.
In either case, this is only partly a solution, of course, since you still cannot stick to writing long procedural methods in synchronous event handlers that require waiting for com traffic. The second level of polling, which is managing a sequence of complete instructions, will still have you needing to pump the message queue if your single procedure needs to react to a sequence of more than one comport instruction. What you also need to think about is breaking up your long methods into shorter pieces, each in response to specific device messages.
Alternatively, for heavily procedural process automation, it is also often a good idea to move that work to a background thread as well. This way your worker threads can block on synchronization objects (or poll in busy loops for status updates) while they wait for events from hardware. One thread can manage low level comport traffic, parsing and relaying those commands or data packets, while a second thread can manage the higher level process which is handling the sequence of complete comport instructions that make up your larger process. The main thread should primarily only be responsible for marshalling these messages between the workers, not doing any of the waiting itself.
See also : I do not understand what Application.ProcessMessages in Delphi is doing
** VAComm may also support something like this, I don't know. The API and documentation are not publicly available from TMS for ASync32 so you'll need to consult your local documentation.
What happens is that a modal message loop executes when the dialog is showing. That this message loop changes behaviour for you indicates that your communications with the device require the presence of a message loop.
So the solution for you is to service the message queue. A call to Application.ProcessMessages in your loop will do that, but also creates other problems. Like making your UI become re-entrant.
Without knowing more about your program, I cannot offer more detailed advice as to how you should solve this problem.
When pthread_exit(PTHREAD_CANCELED) is called I have expected behavior (stack unwinding, destructors calls) but the call to pthread_cancel(pthread_self()) just terminated the thread.
Why pthread_exit(PTHREAD_CANCELED) and pthread_cancel(pthread_self()) differ significantly and the thread memory is not released in the later case?
The background is as follows:
The calls are made from a signal handler and reasoning behind this strange approach is to cancel a thread waiting for the external library semop() to complete (looping around on EINTR I suppose)
I have noticed that calling pthread_cancel from other thread does not work (as if semop was not a cancellation point) but signalling the thread and then calling pthread_exit works but calls the destructor within a signal handler.
pthread_cancel could postpone the action to the next cancellation point.
In terms of thread specific clean-up behaviour there should be no difference between cancelling a thread via pthread_cancel() and exiting a thread via pthread_exit().
POSIX says:
[...] When the cancellation is acted on, the cancellation clean-up handlers for thread shall be called. When the last cancellation clean-up handler returns, the thread-specific data destructor functions shall be called for thread. When the last destructor function returns, thread shall be terminated.
From Linux's man pthread_cancel:
When a cancellation requested is acted on, the following steps occur for thread (in this order):
Cancellation clean-up handlers are popped (in the reverse of the order in which they were pushed) and called. (See pthread_cleanup_push(3).)
Thread-specific data destructors are called, in an unspecified order. (See pthread_key_create(3).)
The thread is terminated. (See pthread_exit(3).)
Referring the strategy to introduce a cancellation point by signalling a thread, I have my doubts this were the cleanest way.
As many system calls return on receiving a signal while setting errno to EINTR, it would be easy to catch this case and simply let the thread end itself cleanly under this condition via pthread_exit().
Some pseudo code:
while (some condition)
{
if (-1 == semop(...))
{ /* getting here on error or signal reception */
if (EINTR == errno)
{ /* getting here on signal reception */
pthread_exit(...);
}
}
}
Turned out that there is no difference.
However some interesting side effects took place.
Operations on std::iostream especially cerr/cout include cancellation points. When the underlying operation is canceled the stream is marked as not good. So you will get no output from any other thread if only one has discovered cancellation on an attempt to print.
So play with pthread_setcancelstate() and pthread_testcancel() or just call cerr.clear() when needed.
Applies to C++ streams only, stderr,stdin seems not be affected.
First of all, there are two things associated to thread which will tell what to do when you call pthread_cancel().
1. pthread_setcancelstate
2. pthread_setcanceltype
first function will tell whether that particular thread can be cancelled or not, and the second function tells when and how that thread should be cancelled, for example, should that thread be terminated as soon as you send cancellation request or it need to wait till that thread reaches some milestone before getting terminated.
when you call pthread_cancel(), thread wont be terminated directly, above two actions will be performed, i.e., checking whether that thread can be cancelled or not, and if yes, when to cancel.
if you disable cancel state, then pthread_cancel() can't terminate that thread, but the cancellation request will stay in a queue waiting for that thread to become cancellable, i.e., at some point of time if you are enabling cancel state, then your cancel request will start working on terminating that thread
whereas if you use pthread_exit(), then the thread will be terminated irrespective to the cancel state and cancel type of that particular thread.
*this is one of the differences between pthread_exit() and pthread_cancel(), there can be few more.
I have been doing overlapped serial port communication in Delphi lately and there is one problem I'm not sure how to solve.
I communicate with a modem. I write a request frame (an AT command) to the modem's COM port and then wait for the modem to respond. The event mask of the port is set to EV_RXCHAR, so when I write a request, I call WaitCommEvent() and start waiting for data to appear in the input queue. When overlapped waiting for event finishes, I immediately start reading data from the queue and read all that the device sends at once:
1) write a request
2) call WaitCommEvent() and wait until waiting finishes
3) read all the data that the device sends (not only the data being in the input queue at that moment)
4) do something and then goto 1
Waiting for event finishes after first byte appears in the input queue. During my read operation, however, more bytes appear in the queue and each of them causes an internal event flag to be set. This means that when I read all the data from the queue and then call WaitCommEvent() for the second time, it will immediately return with EV_RXCHAR mask, even though there is no data to be read.
How should I handle reading and waiting for event to be sure that the event mask returned by WaitCommEvent() is always valid? Is it possible to reset the flags of the serial port so that when I read all data from the queue and call WaitCommEvent() after then, it will not return immediately with a mask that was valid before I read the data?
The only solution that comes to my mind is this:
1) write a request
2) call WaitCommEvent() and wait until waiting finishes
3) read all the data that the device sends (not only the data being in the input queue at that moment)
4) call WaitCommEvent() which should return true immediately at the same time resetting the event flag set internally
5) do something and goto 1
Is it a good idea or is it stupid? Of course I know that the modem almost always finishes its answers with CRLF characters so I could set the comm mask to EV_RXFLAG and wait for the #10 character to appear, but there are many other devices with which I communicate and they do not always send frame end characters.
Your help will be appreciated. Thanks in advance!
Mariusz.
Your solution does sound workable. I just use a state machine to handle the transitions.
(psuedocode)
ioState := ioIdle;
while (ioState <> ioFinished) and (not aborted) do
Case ioState of
ioIdle : if there is data to read then set state to ioMidFrame
ioMidframe : if data to read then read, if end of frame set to ioEndFrame
ioEndFrame : process the data and set to ioFinished
ioFinished : // don't do anything, for doc purposes only.
end;