I am trying to develop a web application that allows users to post a short job description and set a time limit as to when the message should cease to show on a time line. (NB: The post is not deleted it only ceases to show up on time lines) The least time is 4hrs other times are in multiples of 4 up to 24hrs. I don't know the best way to approach this, I am thinking of doing some multi-threading but I am not sure if that is the right approach. In essence I am trying to build something like snapchat but a text based one.
I would to know if:
I need a special hosting package to host such an application.
If multi threading is a viable option
What would you do if you were building an app like this.
NB: I am using ASP.NET with C#
You don't need any threading or special processes, just a better database design.
Also, deleting items from a database generally isn't a good idea, instead just modify your design to be like this:
JobPostings( JobPostingId bigint, Title nvarchar, Description nvarchar, VisibleUntil datetime )
then just exclude old job postings from your queries:
SELECT * FROM JobPostings WHERE VisibleUntil >= NOW()
Related
I have a topic derived from a MySQL users table and now want to create a table in ksqldb that always contains all users registered within the last 50 days. After spending some time in the docs I still can't find a solution for this. Windowing doesn't seem to work because it only applies to stream-stream joins as far as I can see. Is this sort of thing even possible using purely ksql, or do I need to look for other solutions?
Thanks!
I don't think this is currently possible with ksqlDB. This because a filter, such as WHERE DATEDIFF('days', user.registerDate, now()) < 50 would only be evaluated at the time the source row was received.
This isn't too different to standard SQL dbs, where a materialized view built with a similar filter would also not update automagically as time advanced.
You could likely build a system using Kafka Streams that could use punctuation to evict old entries.
Some enhancement requests on ksqlDB may also be of use:
Table over topic with time based retention
View updates
Essentially each time a visitor reaches the application, the controller performs a database query to check what are the most relevant items to show.
Although the items shown vary with time, they are not personally selected for each user.
This means that instead of being calculated each time a visitor comes, it would be better to be system performing a single query every like 10 minutes and store it, to apply on each visit.
What is the best way to apply this idea? I was thinking on cronjobs and maybe store on redis but IDK, some help is appreciated!
There are a number of ways to do this. One way that I've used in the past with success is to have a table in your database that represents the most relevant items and then have a cron job that updates that table.
Fragment caching like #wesley6j recommended isn't a bad way to go either and you can combine the 2 techniques as well if you want.
If you want more detailed suggestions, you can provide some more details about what you are trying to achieve.
This is quite a broad question but ill try and summarise it as best I can.
I have an MVC front end which displays/allows processing of records which are classed as outstanding. I also have a scheduled console app which runs nightly and attempts to resolve each of these records using some logic I wrote.
I have a new requirement, which is to have an email sent every time the total number of outstanding records exceeds a certain amount, this amount needs to be configurable.
The table will contain every record with a flag to say if they have been resolved or not, so I will need to count the outstanding's then fire an email to notify if the threshold is broken.
I initially thought about adding a SQL Server trigger on insert however I soon realised that if no more records were added for a few days but the total number stayed above the threshold because nobody resolved them, then no further email would be sent.
I need the email to send every day on a schedule independently of insert/update.
So now I'm thinking possibly a SQL Server job, or an SSIS package or even a service which runs, but I'm aware this threshold number needs to be configurable.
So what would be the quickest simplest solution to my requirements, I'm open to any suggestion as long as it ticks all the boxes.
Given that the OP already has a console app running on a schedule, the most logical choice would be to simply add this check to the console app along with the email sending logic. It will be much easier to send emails that way, anyways, especially if you employ something like Postal, which will let you use MVC-style views to create your emails.
An SQL Server scheduled job seems to me to be the simplest way to go.
you can add a table to your database that will hold the threshold number and read it's value from there.
In many cases a GeneralParams table is a good thing to have anyway.
The other option you mentioned (windows service) is also configurable in many ways: you can use a GeneralParams table, or the App.Config file of the service (but you will have to restart it every time you change the app.config), or even a simple text file. anything goes. the downside is that it's outside of your sql server, but the upside is that it is probably easier to send emails from.
Is there an option in DynammoDB to store auto incremented ID as primary key in tables? I also need to store the server time in tables as the "created at" fields (eg., user create at). But I don't find any way to get server time from DynamoDB or any other AWS services.
Can you guys help me with,
Working with auto incremented IDs in DyanmoDB tables
Storing server time in tables for "created at" like fields.
Thanks.
Actually, there are very few features in DynamoDB and this is precisely its main strength. Simplicity.
There are no way automatically generate IDs nor UUIDs.
There are no way to auto-generate a date
For the "date" problem, it should be easy to generate it on the client side. May I suggest you to use the ISO 8601 date format ? It's both programmer and computer friendly.
Most of the time, there is a better way than using automatic IDs for Items. This is often a bad habit taken from the SQL or MongoDB world. For instance, an e-mail or a login will make a perfect ID for a user. But I know there are specific cases where IDs might be useful.
In these cases, you need to build your own system. In this SO answer and this article from DynamoDB-Mapper documentation, I explain how to do it. I hope it helps
Rather than working with auto-incremented IDs, consider working with GUIDs. You get higher theoretical throughput and better failure handling, and the only thing you lose is the natural time-order, which is better handled by dates.
Higher throughput because you don't need to ask Dynamo to generate the next available IDs (which would require some resource somewhere obtaining a lock, getting some numbers, and making sure nothing else gets those numbers). Better failure handling comes when you lose your connection to Dynamo (Dynamo goes down, or you are bursty and your application is doing more work than currently provisioned throughput). A write-only application can continue "working" and generating data complete with IDs, queueing it up to be written to dynamo, and never worry about ID collisions.
I've created a small web service just for this purpose. See this blog post, that explains how I'm using stateful.co with DynamoDB in order to simulate auto-increment functionality: http://www.yegor256.com/2014/05/18/cloud-autoincrement-counters.html
Basically, you register an atomic counter at stateful.co and increment it every time you need a new value, through RESTful API.
I would like to sync my db (tasks on my db, that have a decription, a date, a start time and an end time, and a user) with Google calendar.
For sync with google i plan to use these components (of course I could somehow write the whole stuff on my own but this is something I can plan for the future now I am short of time, or in alternative can you suggest some working code that connects to google calendar to send/recieve data?).
Now my main problem is not really linked to Delphi programming anyway I must ask a Delphi related questions because other questions get unviewd (like this one i asked).
So I wonder how to do the sync. Note: I do one way sync and the generated calendar will be a read only calendar.
I can set a max number in the past and future to be synced (like 10 days in past and 100 in the future for example). Then the idea I have is this:
as I start the sync app I comletely read the google calendar itmes in the range, I compare one by one with what I have in db and then I "merge" changes. Then on timer I check for differences in my db and i upload changes.
But I am not sure that these is the best solution.
A simplification of the real case is this: imagine it is a CRM with some task assigend to every user. Since beyond every task there is a logic i want to managea that logic only in my application, but the idea of pulishing the calendar to google is that it is then easily available from any mobile device. This is way there is a one way sync. Ic ould also let the calendar not be readonly anyway at every sync I wil "download" the newly inserted tasks but I will ignore the deleted ones and the edited ones. In this second case it is not enough to track changes in db, but I shuold also track changes on google, at least to "intercept" the newly added tasks.
I am aware this is gerneic question but I would like to trigger an answer that can be useful, etiher redirecting me to a sync algorithm or to Delphi sample code or anything that can help me progress on this issue. Thanks.
Google: "calendar sync algorithms"
https://wiki.mozilla.org/Calendar:Syncing_Algorithm
http://today.java.net/pub/a/today/2007/01/16/synchronizing-web-client-database.html
Synchronisation algorithms
The last one actually is funny because it leads right back to StackOverflow ;) Point is: I think there is no need to reinvent the wheel. Ps: The first link contains some useful thoughts similar to yours.