Is there an option in DynammoDB to store auto incremented ID as primary key in tables? I also need to store the server time in tables as the "created at" fields (eg., user create at). But I don't find any way to get server time from DynamoDB or any other AWS services.
Can you guys help me with,
Working with auto incremented IDs in DyanmoDB tables
Storing server time in tables for "created at" like fields.
Thanks.
Actually, there are very few features in DynamoDB and this is precisely its main strength. Simplicity.
There are no way automatically generate IDs nor UUIDs.
There are no way to auto-generate a date
For the "date" problem, it should be easy to generate it on the client side. May I suggest you to use the ISO 8601 date format ? It's both programmer and computer friendly.
Most of the time, there is a better way than using automatic IDs for Items. This is often a bad habit taken from the SQL or MongoDB world. For instance, an e-mail or a login will make a perfect ID for a user. But I know there are specific cases where IDs might be useful.
In these cases, you need to build your own system. In this SO answer and this article from DynamoDB-Mapper documentation, I explain how to do it. I hope it helps
Rather than working with auto-incremented IDs, consider working with GUIDs. You get higher theoretical throughput and better failure handling, and the only thing you lose is the natural time-order, which is better handled by dates.
Higher throughput because you don't need to ask Dynamo to generate the next available IDs (which would require some resource somewhere obtaining a lock, getting some numbers, and making sure nothing else gets those numbers). Better failure handling comes when you lose your connection to Dynamo (Dynamo goes down, or you are bursty and your application is doing more work than currently provisioned throughput). A write-only application can continue "working" and generating data complete with IDs, queueing it up to be written to dynamo, and never worry about ID collisions.
I've created a small web service just for this purpose. See this blog post, that explains how I'm using stateful.co with DynamoDB in order to simulate auto-increment functionality: http://www.yegor256.com/2014/05/18/cloud-autoincrement-counters.html
Basically, you register an atomic counter at stateful.co and increment it every time you need a new value, through RESTful API.
Related
I build an app that is able to store OData offline by using SAP Kapsel Plugins.
More or less it's the same as generated by WEB ID or similer to the apps in this example: https://blogs.sap.com/2017/01/24/getting-started-with-kapsel-part-10-offline-odatasp13/
Now I am at the point to check the error resolution potential. I created a sync conflict (chaning data on the server after the offline database was stored and changed something on the app and started a flush).
As mentioned in the documentation I can see the error in ErrorArchive and could also see some details. But what I am missing is the information of the "current" data on the database.
In the error details I can just see the data on the device but not the data changed on the server.
For example:
Device is loading some names into offline store
Device is offline
User A is changing some names
User B is changing one of this names directly online
User A is online again and starts a sync
User A is now informend about the entity that was changed BUT:
not the content user B entered
I just see the "offline" data.
Is there a solution to see the "current" and the "offline" one in a kind of compare view?
Please also note that the server communication is done by the Kapsel Plugin and not with normal AJAX calls. This could be an alternative but I am wondering if there is no smarter way supported by the API?
Meanwhile I figured out how to load the online data (manually).
This could be done by switching http handler back to normal one.
sap.OData.removeHttpClient();
sap.OData.applyHttpClient();
Anyhow this does not look like a proper solution and I also have the issue with the conflict log itself. It must be deleted before any refresh could be applied.
I could not find any proper documentation for that. Also ETag handling is hardly described in SAPUI5 and SAP Kapsel documentation.
This question is a really tricky one, due to its implications. I understand that you are simulating a synchronization error due to concurrent modification, and want to know if there is a way for the client to obtain the "current" server state in order to give the user a means to compare the local and server state.
First, let me give you the short answer: No, there is no way for the client to see the current server state "for reference" via the Offline APIs when there are synchronization errors. Doing an online query as outlined above might work, but it certainly is a bad idea.
Now for the longer answer, which explains why this is not necessarily a defect and why I said there are quite some implications to the answer.
Types of Synchronization Errors
We distinguish a number of synchronization errors, and in this context, we are clearly dealing with business-related issues. There are two subtypes here: Those that the user can correct, e.g. validation errors, and those that are issues in the business process itself.
If the user violates the input range, e.g. by putting a negative price for a product, the server would reply with the corresponding message: "-1 is not a valid input value for 'Price'". You, as a developer, can display such messages to the user from the error archive, and the ensuing fix is indeed a very easy one.
Now when we talk about concurrent modification, things get really, really nasty. In fact, I like to say that in this case there is an issue with the business process, because on one hand, we allow data to get out of sync. On the other hand, the process allows multiple users to manipulate the same piece of information. How all relevant users should now be notified and synchronize, is no longer just a technical detail, but in fact a new business process. There just is no way to generically device how to handle this case. In most cases, it would involve back-office experts who need to decide how the changes should be merged.
A Better Solution
Angstrom pointed out that there is no way to manipulate ETags on the client side, and you should in fact not even think about it. ETags work like version numbers in optimistic locking scenarios, and changing the ETag basically means "Just overwrite what's on the server". This is a no-go in serious scenarios.
An acceptable workaround would be the following:
Make sure the server returns verbose error messages so that the user can see what happened and what caused the conflict.
If that does not help, refresh the data. This will get you an updated ETag, and merge the local changes into the "current" server state, but only locally. "Merging" really means that local changes always overwrite remote changes.
The user now has another opportunity to review the data and can submit it again.
A Good Solution
Better is not necessarily good, so here is what you should really do: Never let concurrent modification happen because it is really expensive to handle. This implies that not the developer should address this issue, but the business needs to change the process.
The right question to ask is, "When you replicate data in a distributed system, why do you allow it to be modified concurrently at all?" Typically stakeholders will not like this kind of question, and the appropriate reaction is to work out a conflict resolution process together with them. Only then they will realize how expensive fixing that kind of desynchronization is, and more often than not they will see that adjusting the process is way cheaper than insisting in yet another back-office process to fix the issues it causes. Even if they insist that there is a need for this concurrent modification, they will now understand that it is not your task to sort this out and that they need to invest in a conflict resolution process.
TL;DR
There is no way to compare the server and client state to the server state on the client, but you can do a refresh to retain the local changes and get an updated ETag. The real solution, however, is to rework the business process, because this no longer is a purely technical issue.
The default solution is that SMP or HCPms is detecting errors by ETags. At client side there is no API to manipulate ETags in case of conflicts. A potential solution to implement a kind of diff view on the device would work like this:
Show errors
Cache errors (maybe only in memory?)
delete the errors
do a refresh of the database
build a diff view with current data and cached errors
The idea with
sap.OData.removeHttpClient();
sap.OData.applyHttpClient();
could also work but could be very tricky and may introduce side effects.
Maybe some requests are triggered against the "wrong" backend.
This is quite a broad question but ill try and summarise it as best I can.
I have an MVC front end which displays/allows processing of records which are classed as outstanding. I also have a scheduled console app which runs nightly and attempts to resolve each of these records using some logic I wrote.
I have a new requirement, which is to have an email sent every time the total number of outstanding records exceeds a certain amount, this amount needs to be configurable.
The table will contain every record with a flag to say if they have been resolved or not, so I will need to count the outstanding's then fire an email to notify if the threshold is broken.
I initially thought about adding a SQL Server trigger on insert however I soon realised that if no more records were added for a few days but the total number stayed above the threshold because nobody resolved them, then no further email would be sent.
I need the email to send every day on a schedule independently of insert/update.
So now I'm thinking possibly a SQL Server job, or an SSIS package or even a service which runs, but I'm aware this threshold number needs to be configurable.
So what would be the quickest simplest solution to my requirements, I'm open to any suggestion as long as it ticks all the boxes.
Given that the OP already has a console app running on a schedule, the most logical choice would be to simply add this check to the console app along with the email sending logic. It will be much easier to send emails that way, anyways, especially if you employ something like Postal, which will let you use MVC-style views to create your emails.
An SQL Server scheduled job seems to me to be the simplest way to go.
you can add a table to your database that will hold the threshold number and read it's value from there.
In many cases a GeneralParams table is a good thing to have anyway.
The other option you mentioned (windows service) is also configurable in many ways: you can use a GeneralParams table, or the App.Config file of the service (but you will have to restart it every time you change the app.config), or even a simple text file. anything goes. the downside is that it's outside of your sql server, but the upside is that it is probably easier to send emails from.
I am trying to develop a web application that allows users to post a short job description and set a time limit as to when the message should cease to show on a time line. (NB: The post is not deleted it only ceases to show up on time lines) The least time is 4hrs other times are in multiples of 4 up to 24hrs. I don't know the best way to approach this, I am thinking of doing some multi-threading but I am not sure if that is the right approach. In essence I am trying to build something like snapchat but a text based one.
I would to know if:
I need a special hosting package to host such an application.
If multi threading is a viable option
What would you do if you were building an app like this.
NB: I am using ASP.NET with C#
You don't need any threading or special processes, just a better database design.
Also, deleting items from a database generally isn't a good idea, instead just modify your design to be like this:
JobPostings( JobPostingId bigint, Title nvarchar, Description nvarchar, VisibleUntil datetime )
then just exclude old job postings from your queries:
SELECT * FROM JobPostings WHERE VisibleUntil >= NOW()
I have a rails app that tracks membership cardholders, and needs to report on a cardholder's status. The status is defined - by business rule - as being either "in good standing," "in arrears," or "canceled," depending on whether the cardholder's most recent invoice has been paid.
Invoices are sent 30 days in advance, so a customer who has just been invoiced is still in good standing, one who is 20 days past the payment due date is in arrears, and a member who fails to pay his invoice more than 30 days after it is due would be canceled.
I'm looking for advice on whether it would be better to store the cardholder's current status as a field at the customer level (and deal with the potential update anomalies resulting from potential updates of invoice records without updating the corresponding cardholder's record), or whether it makes more sense to simply calculate the current cardholder status based on data in the database every time the status is requested (which could place a lot of load on the database and slow down the app).
Recommendations? Or other ideas I haven't thought of?
One important constraint: while it's unlikely that anyone will modify the database directly, there's always that possibility, so I need to try to put some safeguards in place to prevent the various database records from becoming out of sync with each other.
The storage of calculated data in your database is generally an optimisation. I would suggest that you calculate the value on every request and then monitor the performance of your application. If the fact that this data is not stored becomes an issue for you then is the time to refactor and store the value within the database.
Storing calculated values, particularly those that can affect multiple tables are generally a bad idea for the reasons that you have mentioned.
When/if you do refactor and store the value in the DB then you probably want a batch job that checks the value for data integrity on a regular basis.
The simplest approach would be to calculate the current cardholder status based on data in the database every time the status is requested. That way you have no duplication of data, and therefore no potential problems with the duplicates becoming out of step.
If, and only if, your measurements show that this calculation is causing a significant slowdown, then you can think about caching the value.
Recently I had similar decision to take and I decided to store status as a field in database. This is because I wanted to reduce sql queries and it looks simpler. I choose to do it that way because I will very often need to get this status and calculating it is (at least in my case) a bit complicated.
Possible problem with it is that it get out of sync, so I added some after_save and after_destroy to child model, to keep it synchronized. And of course if somebody would modify database in different way, it would make some problems.
You can write simple rake task that will check all statuses and, if needed, correct them. You can run it in cron so you don't have to worry about it.
I'm looking to add a 14-Day trial limit to my software. The program has been written in Delphi 7.
Any help would be much appreciated.
You could try Turbopower OnGuard. This is now opensource.
http://sourceforge.net/projects/tponguard/
There are several tricks you can use, but none of them 100% fail save.
You can use some kind of licensing mechanism.
You can store the setup time somewhere hidden in the registry.
You can store the setup time in a file (possibly an executable file or dll).
You can store the IP address in a central database and check each time if the 14 days are passed (you need a internet connection for that).
You can create a file (for example a dll) dynamically on your server and have the installer retreive that file. (Be sure to log the IP so a second attempt will not be possible).
But I think the best way, is to give trial versions with limited functionality. For example: No printing, no save of project, or only small projects can be saved.
That way you avoid the hassle and possible clients can take the time to evaluate your project.
EDIT: If you build a mechanism to check against roling back the clock. Be sure to build in a margin, else the program will be locked if you travel back to an other timezone. Or put the clock back in wintertime. I think a margin of 25 hour will cover everything. (And to be at the save side, you can build in a limit else, the user can roll back the time each day.).
But the best way to keep paying customers, is giving good support. I discontinue products if the service is bad.
One of the things you need to guard against with a time-limited application is users' rolling their calendar back so the application still works. One way around this is to store in your hidden registry place (or wherever) a timestamp whenever the application is started up. If the current date/time is ever earlier than the last timestamp recorded by your app, that means the user has rolled the calendar back and you should disable the application.
Time-limitation is a real pain, though, both for the programmer and the user. It's also not a great marketing idea: why go to the trouble of distributing promotional material (which is what your trial version is) that has an expiration date? It would be like a company mailing out advertisements on paper designed to disintegrate after two weeks.
If your trial version is functionally crippled instead, you might still get sales out of it even months or years later.
You can find the similar question here.
On general note i find time restriction much more useful than functionality restriction. As i explained in the comment to Gamecat post
something to be aware of when performing any of these checks. That the date is never GREATER than 14 days from the date you entered in either direction. A common method around most of these types of limits is to set the date a few years in advance, install and run your software, then set the date back to the current time. If you are hard coded to die 14 days from the original start date, then the user has a few years to try your software. Checking the other direction also gives the user at most 28 days.
I have used Armadillo, Asprotect and Winlicense. Both Armadillo and Asprotect have had serious problems, such as being considered viruses/trojans by some AVs, incompatibility problems, etc.
I haven't used Winlicense enough to have much of an opinion, but support is pretty great.
Obviously both are more complete solutions than what you are asking for - they include protection, licensing, keys, etc.
As mentioned by others, sometimes limiting a feature or adding a watermark is the best option. I've added a watermark to one of my programs (STGThumb) and sales went up about 400%...
I would recommend making a trial serial number with timestamp and force user to enter it into software when its installed. You can even automate it by calling server side page after setup is done.
Timestamp in trial serial key allows you to extend their trial if needed.
In addition you can count backwards to avoid user from changing year when installing:
e.g. if you have 14 days trial generated at 15.11.2008 (server time), you can check that locate date must be greater than 1.11.2008 or less than 24.11.2008 always when serial is used or entered.
You can use a professional tool as SoftwareShield.
I use it in our apps and it provides several licence's models, including timelimited demo.
I created my own key generater (separate program for creating keys). The key values are stored in a binary file with the same name as my program, just a different ext. Example: myprogram.key
I store:
Name
Email
RegType (REG, TRIAL)
RegDate
FirstRun (0 OR 1)
The program looks for the file. If it is not there, it throws a message to the user and closes. The key file generator writes the values in encrypted strings which are then written using the built in stream routines.
I create a TRIAL Key that i distribute with the program. If someone registers, i then create them an official REG key.
Anway, if they are running my program, it first looks for the key file. if found, it checks the reg type, if its a regitered version, then the program loads, and the registration info is displayed. I also store a regdate, which i compare with the day the program runs and - if the regdate is greater than or equal to todays date, the user get sprompted to re-register.
If it finds that the key file stores a RegType of TRIAL, then the date they first ran it is stored in the keyfile, and the flag first run is set to 1. They can then use it for 14 days. Each time they run the program, the date stored is compared with the running date.
Very simple process to write. Is it fool proof? NO, nothing is! I have had great success with my app. Its not wide known, so there are no hackers lookijng to hack it.
The best would be to get the registration info from your server.
The big drawback: 1. The server must be ALWAYS online! 2. The user must be connected to internet (when it uses your app).
To get you started you can use a Delphi license management library to help you encrypt the license info and generate a string-based key that you can send to your customers upon registration. There are quite few libraries out there.
Anyway, whatever you send to your server needs to be based on the hardware fingerprint of that computer. Otherwise your license key will leak out on some warez website and everyone will be able to use that key. But if the key is hardware-based it would be useless if it is leaked on Internet.
And don't over do it! There is no such thing as unbreakable software protection. If Microsoft could not do it, you will not do it. Concentrate on adding nice features to your app instead of creating a bullet proof protection system (which is not possible).