We have to develop a Message Store using a IMAP interface that should have the functionality of Persistent UIDs so that later those messages can be synchronized across multiple devices (like mobile, PC, Laptop) and the device can delete / copy the messages. RFC 3501 mentions UIDs are unique within the session.
So my question is: does any IMAP RFC talk about the persistent UID for a mailbox?
Every IMAP message is assigned a UID that is specific to the user, as in two users may receive two different messages, but be assigned the same UID. [aka UID != UUID] If your software is issuing a simple FETCH 1:* (FLAGS) the server will respond with a sequentially numbered list disregarding the message's UID. Any command in which you want to deal specifically with a message's UID you must be sure that you're issuing it properly, as in FETCH UID 1:* (FLAGS).
eg:
a1 fetch 1:* (flags)
* 1 FETCH (FLAGS (\Seen))
* 2 FETCH (FLAGS (\Seen))
* 3 FETCH (FLAGS (\Seen))
* 4 FETCH (FLAGS (\Seen))
* 5 FETCH (FLAGS (\Answered \Seen))
* 6 FETCH (FLAGS (\Seen))
* 7 FETCH (FLAGS (\Seen))
* 8 FETCH (FLAGS (\Seen))
* 9 FETCH (FLAGS (\Seen))
* 10 FETCH (FLAGS (\Seen))
versus:
a8 uid fetch 1:* (flags)
* 1 FETCH (UID 1 FLAGS (\Seen))
* 2 FETCH (UID 2 FLAGS (\Seen))
* 3 FETCH (UID 3 FLAGS (\Seen))
* 4 FETCH (UID 4 FLAGS (\Seen))
* 5 FETCH (UID 5 FLAGS (\Answered \Seen))
* 6 FETCH (UID 6 FLAGS (\Seen))
* 7 FETCH (UID 8 FLAGS (\Seen))
* 8 FETCH (UID 9 FLAGS (\Seen))
* 9 FETCH (UID 10 FLAGS (\Seen))
* 10 FETCH (UID 11 FLAGS (\Seen))
That said, I don't understand why you need to track the UIDs separately to synchronize across multiple devices. So long as each device is gathering its information from the IMAP server they will be in sync by default. You're essentially re-implementing functionality that already exists in any IMAP server.
What RFC3501 mandates is that the UIDs must remain constant within a session. This does not mean that they should not be perisstent across sessions -- on the opposite, unless they are persistent, the IMAP clients will have to download them all the time.
I would suggest to re-read the relevant portions of RFC3501 several times here -- this is a crucial piece of the IMAP mailbox synchronization and it is important to get this right -- including the relations with UIDVALIDITY and UIDNEXT, as well as the CONDSTORE and QRESYNC extensions.
Also please keep in mind that there are pretty strict guarantees on how the UIDs are assigned and what the UID of a newly arriving message must look like. Having a per-server unique identifier is not enough.
Related
I have a query that does two queries. The first query looks at future events and what user is in that event from one table. The second query looks at the historical events and creates stats for the user in that event based off all the previous events that user was in from a second table.
I am having troubles joining the two queries. The goal of the join would be to join the second query to the first query based off the most recent stats for each user. The stat can't be newer than the date that the future event occured on. If that user hasn't been in any previous event it would just return null for the query 2 in the join.
Below is also an example table for the future query, a table for the historic query, and an ideal output of the join.
Future:
user
date
event code
event info
User 1
1/26/2023
5596
info_5596
User 2
1/26/2023
5586
info_5586
User 3
1/26/2023
5582
info_5582
User 1
1/20/2023
5492
info_5492
User 1
1/2/2023
5341
info_5341
User 2
1/2/2023
5333
info_5333
Historical:
user
date
stat 1
stat2
event code
event info
User 1
1/25/2023
10
52
4352
info_4352
User 2
1/25/2023
11
22
4332
info_4332
User 2
1/12/2023
2
45
4298
info_4298
User 3
1/12/2023
8
88
4111
info_4111
User 1
1/12/2023
7
67
4050
info_4050
User 3
1/2/2023
3
91
4000
info_4000
User 1
1/1/2023
6
15
3558
info_3558
Output of the JOIN:
user
date future
stat 1
stat2
event code future
event info future
User 1
1/26/2023
10
52
5596
info_5596
User 2
1/26/2023
11
22
5586
info_5586
User 3
1/26/2023
8
88
5582
info_5582
User 1
1/20/2023
7
67
5492
info_5492
User 1
1/2/2023
3
91
5341
info_5341
User 2
1/2/2023
null
null
5333
info_5333
I tried using a subquery in the join, but bigquery was saying that it is unsupported. Below is my code attempt. I have also tried using MAX() but it was not liking that to be used in the join as well
Another option I am thinking of is to join the two datasets before ever calculating the query 2 stats. Then filtering. I have a large query already written for both though, so I would prefer not to start over.
Select Distict
A.*
B.Stat1, B.Stat2
from future as A
Left Join historic as B
ON (
A.Date = (Select MAX(B.date) FROM historic as recent_historic WHERE recent_historic.user = A.user)
AND
A.user = B.user
)
ORDER BY A.date
What should I do to properly remove 'event' entries from Thingsboard?
As far as I know, the current API does not provide a way to remove events. It seems like the only way is to directly delete the records in DB.
By the way, I'm using PostgreSQL as DB.
After two hours of research in Thingsboard source code, I found the solution.
The date is contained in the uid_event field in V1 UUID format.
So first, you need to write a function uuid_timestampin order to convert the UUID to a timestamp. I found the solution, here: https://stackoverflow.com/a/24191574/5300212
CREATE FUNCTION uuid_timestamp(id uuid) RETURNS timestamptz AS $$
select TIMESTAMP WITH TIME ZONE 'epoch' +
(((('x' || lpad(split_part(id::text, '-', 1), 16, '0'))::bit(64)::bigint) +
(('x' || lpad(split_part(id::text, '-', 2), 16, '0'))::bit(64)::bigint << 32) +
((('x' || lpad(split_part(id::text, '-', 3), 16, '0'))::bit(64)::bigint&4095) << 48) - 122192928000000000) / 10000000 ) * INTERVAL '1 second';
$$ LANGUAGE SQL
IMMUTABLE
RETURNS NULL ON NULL INPUT;
After that, to delete all events older than 30 days ago, you can run a query like:
DELETE FROM public.event WHERE uuid_timestamp(event_uid::uuid) < now() - '30 days'::interval;
Your assumption is correct. You will need to execute a SQL script to cleanup the "events" table. I must note that for Cassandra DB we already have "cassandra.query. ts_key_value_ttl" and "cassandra.query.events_ttl" configuration parameter to automate this process.
I'm new to Freeradius. My NAS is StrongSwan and seems to be supporting CoA.
I also have the sql accounting mod enabled, which allows me to count the octets.
There are many tutorials about how to count the session time of a user and action a session-timeout once the time is over. But virtually there is nothing I could find that explains how to count the user's data usage and disconnect him after let say 100 KB of monthly usage.
vim /etc/freeradius/mods-enabled/sqlcounter
sqlcounter totaldatacounter {
sql_module_instance = sql
dialect = ${modules.sql.dialect}
counter_name = Max-Capacity
check_name = Acct-Output-Octets
reply_name = Session-Timeout
key = User-Name
reset = monthly
query = "SELECT ((SUM(`acctinputoctets`)+SUM(`acctoutputoctets`))) FROM radacct WHERE `username`='%{${key}}' AND Month(acctstoptime)=(Month(NOW())) AND Year(acctstoptime)=Year(NOW())"
}
When I run the query above for the given user I get the following data usage: 76827648
This is more than my limit of 100000 set in radcheck table:
INSERT INTO `radcheck` (`id`, `username`, `attribute`, `op`, `value`)
VALUES
(3, '0799a559-1426-478a-b46a-a33f1198cd24', 'Acct-Output-Octets', ':=', '100000');
So why am still able to connect?
I have a pastebin here of the freeradius -X logfile.
How to find the last email ( message UID ) using IMAP commands.
I am using putty on windows pc to connect to imap server .
When you select the inbox, it will tell you how many messages are in the message via the EXISTS response. This number is also the highest message sequence numbers run the mailbox. You can then get the highest UID in the mailbox by fetching the UID for this message.
That is:
a SELECT INBOX
...
* 23 EXISTS
....
b FETCH 23 (UID)
* 23 FETCH (UID 207)
b OK
Therfore, 207 is the highest UID in the mailbox.
You could also do UID SEARCH ALL and take the largest number returned.
The easiest way is to ask for the UID of message identified by * which represents the largest number in use:
>>> a SELECT INBOX
...
<<< * 3 EXISTS
<<< a OK [READ-WRITE] Select completed.
>>> b FETCH * (UID)
<<< * 3 FETCH (UID 283)
<<< b OK Fetch completed.
That said, I'm curious on why you want to know the highest UID in the mailbox. That value is often meaningless; you do not need it for any IMAP command and it does not represent any upper bound on the UID which was ever in that mailbox -- you want to track the UIDNEXT for that.
where do I begin?
I can't seem to find any definitive documentation. (I am probably looking in the wrong places....).
I want to be able to edit IMAP keywords (for tagging purposes) for email messages.
I have some really noobie questions;
how are they added?
are keywords applied directly to message headers (if so, what is the syntax), or is there some other kind of imap voodoo at work...
thanks in advance.
IMAP custom keywords are applied to messages via the STORE command. Once added, they'll be returned when you do a FETCH for FLAGS and they'll be searchable via SEARCH KEYWORD:
A001 FETCH 5 (UID RFC822.SIZE FLAGS)
* 5 FETCH (UID 292 RFC822.SIZE 2554 FLAGS (\Seen))
A001 OK FETCH completed
A002 STORE 5 +FLAGS (pending ignored uninteresting)
* 5 FETCH (FLAGS (\Seen pending ignored uninteresting))
A002 OK STORE completed
A003 FETCH 5 (UID RFC822.SIZE FLAGS)
* 5 FETCH (UID 292 RFC822.SIZE 2554 FLAGS (\Seen pending ignored uninteresting))
A003 OK FETCH completed
A004 SEARCH KEYWORD pending
* SEARCH 5
A004 OK SEARCH completed
Note that when you create a new keyword via STORE, the server should respond with an updated list of all the system and user flags defined on the mailbox:
A002 STORE 5 +FLAGS (pending ignored uninteresting)
* FLAGS (\Answered \Deleted \Draft \Flagged \Seen pending ignored uninteresting)
* 5 FETCH (FLAGS (pending ignored uninteresting))
A002 OK STORE completed
Some servers won't allow you to create used-defined keywords. They'll let you know this by not including \* at the end of the PERMANENTFLAGS list when you SELECT the mailbox.
The STORE command allows you to add keywords to a message's existing set (via +FLAGS), remove them (-FLAGS), or replace the set with an entirely new set (FLAGS).