I am going to write a simple credit system that user can "add", "deduct" credits in the system. Currently I am thinking of two approaches.
Simple one: Store the user' credit as balance field in the database, and all actions ("add", "deduct") are logged but not used to compute the latest balance.
History based: Don't store the balance in database. The balance is computed by looking at the history of transactions, e.g. ("add", "deduct")
Both case would works I think, but I am looking to see if any caveat when designing such a system, particularly I am favoring the History based system.
Or, are there any reference implementation or open source module I am use?
Update: Or are there any Ruby/Rail based module like AuthLogic so I can plug and play into my existing code without reinventing the wheel (e.g. transaction, rollback, security etc)?
Absolutely use both.
The balance-based way gives you fast access to the current amount.
The history-based way gives you auditing. The history table should store the transaction (as you describe), a timestamp, the balance before the transaction happened, and ideally a way to track the funds' source/destination.
See the Ruby Toolbox for bookkeeping and Plutus double-entry bookkeeping gem.
In addition, if your credit system may affect users, then I recommend also using logging, and ideally read about secure log verification and provable timestamp chaining.
For logging details see: techniques for ensuring verifiability of event log files.
For open source code that does credit, you may want to look into: http://www.gnucash.org/
Adding and deducting credits implies that you might also need to be aware of where these credits came from and where they went. Any time you get into a situation like this, whether it is with currency or some other numerical quantity that needs to be tracked and accounted for, you should consider using a double entry accounting pattern.
This pattern has worked for centuries and gives you all of the functionality you need to be able to see what your balances are and how they got to be that way:
Audit log of all transactions (including sources and sinks of "funds")
Running balance of all accounts over time (if you choose to record it)
Easy validation of the correctness of records
Ability to "write-once" - no updates means no tampering
If you aren't familiar with the details, start here: Double Entry Bookkeeping or ask anyone who has taken an introductory course in bookkeeping.
You asked for a Ruby on Rails open source solution that you could plug and play into your application. You can use Plutus. Here is an excerpt from the description of this project on Github:
The plutus plugin provides a complete double entry accounting system
for use in any Ruby on Rails application. The plugin follows general
Double Entry Bookkeeping practices. ... Plutus consists of tables that
maintain your accounts, entries and debits and credits. Each entry can
have many debits and credits. The entry table, which records your
business transactions is, essentially, your accounting Journal.
yes, use both.
On top of that, you'll sometime need to reverse a transaction/
transactions.When doing that, create a new reversed transaction to
notate the money transfer.
sometimes, You'll need to unify several transactions under one roof. I suggest to create a third table called 'tokens' that will be the payments manager and you'll unify those grouped transactions under that token.
token.transactions = (select * from transactions t where t.token = "123") for example
Related
I am building a master database to store all relevant information about our customers. I am using Neo4j.
Below is a sample of our model. We have Person, that can be registered in 3 of our mobile applications. (App.01, App. 02, App. 03 - We use CPF key, it is like a SSN). In those apps the user can be registered with an email. So it is represented by Email entity. Those user can have multiple address represented by Address entity.
The question is:
As I am building a Master Data, IMO, if someone query the mdm database asking for all "best" information about a person, I would return for example:
Name: John
Best email: email2 (because it has two apps using it)
Best address: addr1 (because it has tow apps using it)
So I am going to build some heuristis to define what is the "best" email and address.
For this purpose, I have some options:
I could create an edge from John to email2 and to addr1. So it's going to be easy for an user of MDM to get the "best" address/email from John.
I could build a rest API endpoint and create this heuristic in query time.
Does anyone have experience using graph database or design MDM database?
Is it a good approach?
This question is a complement for the question: Using Neo4j to build a Master Data Management
The graph data model is good to store your master data, however, your master data most likely will co-exist with operational and reference data in the form of dimensions.
if you decide to go with a graph model for your DMD, make sure that you have a well defined semantic model for the core dimension is MDM, usually:
products
customer
employees
Assets
Location
These core dimensions become attributes of your nodes.
Also, decide what DMD architecture style you are going to adopt, some popular ones are:
The Registry - Graph fits very well with this style because your master data remains in the SOS(system of record) and the references can be represented in the graph very nicely.
Master data Hub - Extra transformations ar4e required to transpose your system of record from tabular to the graph.
Master-Master. - this style fits well with your MDM in the graph if you do not have too many legacy apps that depend on your MDM.
Approach 1 would add a lot of essentially redundant information (about 2N extra relationships, where N is the number of people), and also require more complex coding to handle changes to a person's apps. And, as always when information is stored redundantly, you would have to be especially careful that inconsistencies do not creep in. But, it should be faster when querying for the "best" contact info.
Approach 2 keeps the DB the same size, but requires a more complex and slower query to get the "best" contact info. However, changing a person's apps and contact info is straightforward.
To decide which approach to use, you should consider whether DB size is an issue, and also look at your use cases and how frequently they will be performed.
Here is a simple heuristic if DB size is not an issue. Suppose G is the frequency at which you need to get a person's "best" contact info, and M is the frequency at which you need to modify a person's apps or contact info. You would pick approach 1 if the value of G/M exceeds some threshold value, K, that you would have to decide on, taking into consideration the above considerations.
I am about to embark on the creation of one of my first meaningful rails apps, and am a bit unsure of how to structure things.
Here's my situation. I am using a SAAS inventory software that keeps track of approximately 4000 products. I need an app that can perform routine maintenance functions on the products. For example, to give you an idea:
Every week, calculate and set the "low stock alert quantity" for each product based on historic sales
Assign products to product categories based on rules (i.e. if the word "t-shirt" is in the title, automatically add it to the t-shirts category)
etc etc
My questions are as follows:
I'm not sure exactly how to structure the data in this app. Should I query the API each time I need to retrieve the products? Or should I build a local copy of the inventory in a local database, for faster querying? If I am to build a local copy of the database, what would be an efficient way of keeping an up to date version of each product without consuming too much server resources? Obviously I can't pull 4000 products via the API at once...and I also don't want a cron job to be running every minute of the day.
Where should I place the code for my remote API functions? Do I create a class, module, or something else?
Thanks for any advice.
First of all, do you really need a web interface? I don't see so in your app description. If what you want to build is a maintenance set of tools that would be executed periodically, why would you use Rails?
The way of structuring data depends on your needs and resources. Does your SAAS provide instant notifications on products update, such as webhooks? Does your SAAS API let you fetch only products that were updated after a certain date? What's more important to you, speed or working with up to date data? Answering those questions to yourself should help you to decide what the best approach is.
Regarding the app design, it looks that you could write a gem where you could have different modules to manage data, execute operations with that data and connect to the third party service, and use that gem in scripts that would execute periodically. I would recommend you reading a book on Object Oriented design that I truly believe it'd help you, Practical Object-Oriented Design in Ruby.
Just one last word, I'm taking too many assumptions here, no one but you know better your requirements, and the design of your software depends on that.
If your API follows REST then it can very easily be integrated into rails using ActiveResource. If the API is more complicated, you could use ActiveModel and roll your own implementation. Just create a base class of your own that handles connection and authentication and extend your models from that. Given the limited description of your problem, there's not much more I can say.
I have a requirement (pending) that allows the user to buy "credits" and then to trade those credits for real goods or services.
This is the first time I have had to do this in an application and I am concerned it would paint a target on my database's back. A hacker could (in theory) change the amount of credits a user has and then "spend" those credits, or convert them to cash.
I'm building the solution in ruby/rails, but I am not limited to this tech. I can even use an outside provider if it's more practical.
Does anyone have any suggestions on how to do this? Would you encrypt your DB? Would that be enough?
There are a lot of different things to consider. When it comes to matters of security, there is never a silver bullet (anyone who suggests that is likely selling snake oil); rather, security often involves many different steps to mitigate and manage risks and also redundant layers of protection so that one is still protected if some subset of those layers fail for whatever reason.
In terms of storing monetary transactions, there are often a number of legal regulations that need to be followed, so I suggest consulting those in addition to other security measures. In terms of encrypting the database, there are many different ways to apply encryption... one can encrypt the database as a whole or individual rows of the database using different keys. If you just encrypt the database as a whole, it won't provide you much protection if someone has access to the key (which also begs the question of who has the key, where is it stored, etc.?). What you probably want, in addition to the database itself, is a write-only log of transactions with some sort of checksumming so that you can be assured of its authenticity that provides you with an audit trail that is independent of the database (and from which the database could be reconstructed in the event of some sort of breach). In addition, you'll want to ensure that only authorized applications (such as your production instance of your frontend server) can talk to and decrypt the database (and ensure that only a very limited number of people you trust can deploy new versions of those, so that no one can arbitrarily deploy malicious versions that abuse that access). If it's possible to independently encrypt individual rows with different keys (e.g. to encrypt each per-user row with key material derived from the user's login credential), then that is highly advisable (though this is not always possible to do, such as if you need to be able to process that row even when the user is not actively interacting with your application). I'm sure there are other things that I have not thought of, which is why you'll also want to regularly conduct penetration testing to check for any vulnerabilities (and not only fix anything you discover this way, but also use it to inform projects or processes that you can employ to prevent similar vulnerabilities in the future).
In addition to the security considerations, monetary transactions is one of the few cases where "eventual consistency" doesn't really work; you'll need to make sure that you are careful in your programming to make transactions appropriately atomic. That is, you wouldn't want the number of credits to decrease at a separate time step from the dispension of dollars, as that would allow the same credits to be spent twice... you'll want to be very careful in your coding that the decrease in credits and increase in dollars (or vice-versa) happen simultaneously. For this and other reasons thorough testing and good code review practice is a good idea.
So, I have a Rails application that tracks employees. We want to track the employment history over time. We'd like to be able to do the following sorts of queries:
Search employees by their current status (salary, manager, title etc.)
Search for remployees by status as of a particular date
Search for employees who ever matched a condition
We don't mind the historical queries being significantly slower, even maybe 2 orders of magnitude slower.
This history is different from an audit trail - an audit trail is the history of the data, so an audit trail will tell you "what salary was stored for employee X on date Y," but it won't let you correct that older data if it is wrong.
[We are also using audit trails for auditing purposes, but I think of the logic for audit trails as providing an almost orthogonal design requirement.]
Is there a known database design pattern for best implementing this kind of history? Or a site where I can find discussions of the various "obvious" designs and the trade-offs? Any Rails plugins?
Rails 2.3.5 with Ruby 1.8.7, if that matters.
I haven't used it, but paper_trail claims to support this type of version management. It would seem to handle your requirements (1) and (2) easily -- not sure about (3), you might need to do some heavy lifting to get a query that searched across models and versions at the same time.
This history is different from an audit trail
Meaning that you need the ability to make changes to old versions -- I doubt that is supported, and it seems problematic to implement if you also need this to act as an audit trail.
I have an application which will require a "dynamic business rules" engine. Some of the business rules changes very frequently. Some of then applies for a limited set of business accounts. For example: my customer have a process where they qualify stores, based on their size, number of the sales person, number of products, location, etc. But he manages different account, and each account give different "weights" to each attribute.
How do I implement this engine using Ruby? I know Java has drools, but I find drools annoying and complex. And I prefer not having to use JRuby...
Regards,
Rubem
If you're sure a rule engine is what you need, you will need to find one you can use in Ruby. A quick Google search brought up Rools (http://rools.rubyforge.org/) and Ruby Rules (http://xircles.codehaus.org/projects/ruby-rules). I'm not sure of the status of either project though. Using JRuby with Drools might be your best bet but then again, I'm a Java developer and a big Drools advocate. :)
Without knowing all the details, it's a little hard to say how that should be implemented. It also depends on how you want the rules to be updated. One approach is to write a collection of rules similar to this: "if a store exists with more than 50 sales people and the store hasn't had its weight updated to reflect that, then update the store's weight." However, in some way you could compare that to hardcoding.
A better approach might be to create Weight objects with criteria that need to be met for the weight to apply. Then you could write one rule that matches on both Weights and Stores: "if a Store exists that matches a Weight's criteria and the Store doesn't already have that Weight assigned to it, then add the Weigh to the Store." Then the business folks could just create and update Weights, possibly in a web front-ended database, instead of maintaining rules.