General Problem
How do retail establishments constrain activation for gift cards, or those pre-paid phone/debit cards?
They must have a system in place that only keeps you from calling in to activate cards that haven't scanned through the register, and I assume there must be a standard solution built into the retail ERP/accounting systems. It probably involves web services or EDI.
Specific Problem
I ask all this because one of my clients wants me to develop a product that you get into by purchasing a $30 card at a retail store. The card has a unique number on it. Once you've purchased a card and activated it via a web site, coupons for restaurants and so on are emailed to you periodically.
However, if someone were to steal a bunch of cards or figure out the numbering sequence, we don't want the cards to work.
Presumably, this is a solved problem because retailers are doing this with the products above (pre-paid phone cards, etc).
I can think of a number of ways to solve this problem, however I need to provide the "standard" solution that the retailers expect, so that the product will snap into their infrastructure in the normal way.
Thanks a lot!
I've worked on a few of these types of systems and they all basically work the same way. The card has a # encoded into the magnetic strip (it could also be a barcode). That's usually all that's on the card itself. Cards are then activated at time of purchase.
Here's the basic flow:
Customer comes in and purchases a card:
The card is swiped and/or scanned.
A call is made to an on-line system (usually via some type of webservice call). It includes the card #, the amount they are activating with, and maybe a bit of additional information (ex. invoice #), and possible something like the previous transaction #.
If the call is successful, you get back a transaction ID #.
If the call fails, there is usually some protocol you are supposed to follow (sometimes handled during the daily settlement process). Things like retrying the activation, or running a query to determine if the last transaction went through.
If it was successful, the card is now active.
So basically, the card is worthless until it's activated. At that point it becomes "live" and has money associated with it. That is, back on some server somewhere is a database that has this card #, when/where it was activated, amounts, etc.
There is usually some functionality to generate an "end of day" transaction report to help you reconcile your numbers (what your system says vs what they have recorded).
Since cards are centrally managed it becomes easy for them to flag cards if they were stolen (not that it matters since they have $0 value until they have been activated).
I found out through other sources that there are about eight card processing services that integrate with the various retail locations.
Each retail location uses one. When a card scans through the register, the retailer notifies the card processing service (unlocking the PIN so that it can be activated), and then presumably the card processing service notifies us via an API call.
Then, when the customer goes to activate their card, we can tell which ones have scanned through the register (because they are unlocked). In this way, we get around problems surrounding stolen cards or guessed pin numbers.
The names of a few of these networks are:
Blackhawk Networks
InCom
Coin Star
I had the joy of working on one of these systems right out of college. Depending on the way they handle their processing whether end of day batch or weekly report could cause quite a lot of problems. One of the things I saw was that if the person whom had the card, whether legit or not, if they managed to make a bunch of purchases that were < the day's starting balance but by the end of the day > greater than starting balance all the purchases would go through. Not very fun when the company had to swallow upwards of 100 dollars per user a day.
In terms of security, make the company that you interface with be held responsible for purchases. That is the best way of handling this in what I have seen, because that is what they are there for. Hope that helps in some roundabout way.
You have to be careful, I have seen this played out in retailers..a customer in front of you is asked if they have a loyalty card by the cashier. The customer says no, but the customer behind them offers their card and it gets swiped hence collecting points for something that they did not buy...
Thus gets points at the expense of the customer in not having it thus skewing/distorting the results of the customer (the one who has a loyalty card)'s personal shopping experience and what they did not buy...on the appropriate system's database
In short, there is no foolproof way of getting around that other than asking for a retina scan or finger print that identifies the customer. Some customers would be cautious in joining a club in regards to their privacy...that is another thing to be kept in mind...not all of them would have a loyalty card...
Hope this helps,
Best regards,
Tom.
I believe that the most secure solution possible is to have a server that generates and prints (or exports) the card numbers. When a customer is interested in purchasing a gift card, it is scanned at the register and the register notifies the server that the card has been approved (probably with the credentials of the cashier).
Then when input on your website, the website checks with the card server to see if the card number is valid and approved.
Then, stolen cards are not approved. If someone figures out the numbering scheme, then you may be screwed so it is recommended that the numbers be random with enough digits to make guessing numbers unreasonable (perhaps with something similar to a CV2 code).
This is similar to how debit cards work: card number/CV2 generated ("server") -> shipped to customer -> customer activates via phone ("register", with the "credentials" being their SSN or similar) -> customer then uses at a store and the store contacts the card company's server
I know that Intuit Quickbooks Point of Sale offers a service like this (complete with an API), you could look them up.
I like HalfBrain's solution. I'd also imagine they have certain pieces of security in mind, like a single IP address (or some other criterion) with more than some number of failed activation attempts getting flagged for apparently attempting to probe the system.
Related
I run a marketplace iOS app and from time to time we have "competitions", where we have an especially sought after item for sale for a good price, that drops as a specific time. Sometimes thousands of people will try to buy this item within 1-2 seconds and I therefore need to make sure that only 1 person will get the item. The solution I have for it now feels kind of clumsy, so I was wondering how a good solution would look like when I use Firebase as my database.
The process is as such:
User finds the item on his iOS app and clicks "Purchase".
A request is sent to our API (build on RoR) that processes the purchase (usually takes 10-20 seconds for the purchase to go through).
Right now, I set the buyers ID temporarily as an attribute on the item, I wait a second and check whether the buyer ID is still the same on the item. It works, but it doesn't feel optimal.
Any suggestions on how I can make sure 2 people can't purchase the same item?
To avoid something like this in your rails app, the keywords mutex and race condition should probably help you to find a bunch of appropriate gems.
I personally like to use redis for this kind of task, because in redis, transactions are atomic by default (https://en.wikipedia.org/wiki/Atomicity_(database_systems)).
So maybe this gem could suit your needs (untested): https://github.com/kenn/redis-mutex.
For the theory, refer to this articles:
https://en.wikipedia.org/wiki/Mutual_exclusion
https://en.wikipedia.org/wiki/Race_condition
Store in /items/foo
a record with the structure:
{id:<blah>, available: <timestamp>, (purchaser: null)}
let buyers write their user name to to buy:
/item/foo/purchaser
You want 3 things to happen.
Block someone writing before the servers timestamp of available
only allow 1 person to do the operation. Once the /item/foo/purchaser is set, you don't want it modifiable (i.e. write once)
only allow the authenticated user id to be used in the purchaser field
To enforce this logic you use security rules, on the subpath of "/items/$itemid/purchaser"
".write": "now > data.parent().child('available').val()" +// 1.
"&& data.val() == null" + // 2.
"&& newData.val() == auth.id" // 3.
My guess is tht you should use locks.
On a request coming in, check if you can acquire a lock. If yes, the the user is the first one. Then, the next requests won't be able to acquire the lock. This means the product as already been purchased.
Take a look at this redis doc part : http://redis.io/topics/distlock
At the application(RoR) level, you can set a flag(eg: lock_foo=true) that is shared across the cluster(can be in your cache store).
If this value is true, don't allow any other users to access the product/make the purchase.
You can definitely implement this with Firebase. As dvxam and Anshul Mengi mentioned, a lock system is the good way to go:
You could have on the document a property called lock:
{
"lock": {
"userId": "myUserId",
"expiresAt": "myTimestamp"
}
}
When a user clicks on the purchase button, you can use a Firebase transaction to make sure only one user can get the lock and that the first one gets it.
When another user clicks the purchase button, if a non-expired lock is present with a different userId, you can deny the purchase.
When the user completes the purchase you can then use another transaction to check if it is the same userId and if the lock is not expired.
Transactions are absolutely necessary here, and they are not available on the Firebase REST api (hence no more in the ruby wrapper), so you would need to run this code client-side using the iOS SDK, or to spin a nodeJS server for this task.
Hope it helps.
How about this for different:
When user clicks purchase, immediately create a purchase request record that contains product, user and timestamp, and then poll every few seconds to see if the purchase was successful
Run a background job that searches for un-purchased products that have at least one purchase request against them, and marks the product as purchased (selecting one purchase request / user as the "winner")
I'm not sure if there's a specific pattern I can apply in this case or how this is "normally" solved?
I can't speak to Firebase, but I can definitely speak to how this is "normally" solved in Rails and relational databases.
Before jumping in to code, note that it seems like you need linearizability, one of the hardest things to ask of a database, and some databases can't guarantee it even when they say they do. You might be able to hack around needing linearizability if all you need to know is whether it's been purchased or not, but I wouldn't take that hack lightly. Consistency in distributed systems is a really complex and edge-case-ridden topic, especially while under load (which it sounds like you'll be).
In Rails+RDB (postgres, mysql, sqlite) an atomic, linearized quantity update looks roughly like this (with some rails validation niceities thrown in):
class Product
validates :quantity, numericality: {greater_than: 0}, on: :purchase
def purchase
with_lock do # simultaneously aquires a lock and reloads the model
return false if !valid?(:purchase) # immediately release the lock if not valid
update_attribute(:quantity, quantity - 1) # saves without validation; YYMV
end
end
end
This general pattern of "lock+reload -> check -> update" is the gold standard for reliability, but it's "heavy." The first object to acquire the lock will win, but while it's doing its thing, all the other processes asking for a lock will be in queue. Somewhere there's a timeout and max connection pool defined, so if say 4000 locks are asked for within 1 second but it takes 10 seconds to determine success, you'll need 4000 connections and, even worse, the last lock asked for will be waiting for over 11 hours! That will make managing the connection pools and setting reasonable timeouts challenging.
The benefits, though, are that it will "just work" - if the first purchase fails, the next purchase will acquire a lock, and so on, until someone wins. Then, it will return helpful ActiveModel errors to everyone else in the queue. Additionally, it's simple enough code-wise that you know as long as your database provides linearizability, you're in the clear.
To mitigate the 11-hours issue hopefully you can very quickly deny everyone with outstanding locks to flush the queue.
I don't know exactly what you're doing while you try to make a purchase, but if it was just a credit card validation and a data update, I'd highly recommend the approach I've outlined with a database known to be linearizably consistent. Otherwise, you're going to need to consult a true distributed systems expert or run your users under the bus figuring this out.
I was wondering what the thinking is in regards to using User Stories to describe automated, scheduled, or reactive functionality. For example, what do you do when you have something like an order fulfillment process which involves pulling an order from a queue, preparing a "fill order form", sending the form to an order processing center, then waiting for some sort of confirmation from the processing center, such as "order fulfilled", or "order fulfillment error: reasons...", etc.. Keep in mind that the only user intervention during this entire process would be when the order was entered. One could always argue that the fulfillment process is something that can be implied from a order entry story, or that it is an implementation detail, but it seems to me that the fulfillment process is too large to simply take it as implied from an order entry user story or as an implementation detail. It feels like there should be a description of the fulfillment itself as a story as well.
Particularly, the aspects that interest me about writing a user story for automated, scheduled, or reactive functionality is from whose perspective should it be described? Given that we are using a story format like "As a [role], I want [functionality] so that [purpose]", what is the role in the "As a [role]" part of the story, and what is the purpose in the "so that [purpose]" part of the story? The functionality is usually clear enough but the role and the purpose seem a bit relative. For example, I could use the system as my point of reference and write something like "As the Order Fulfillment System/Agent, I want to be able to pull an order from the fulfillment queue, prepare a fill order form, and send it to the order processing center so that the order can be fulfilled". Or alternatively, I could look at things from the business's point of view and write something like "As an order taker, I want to be able to process the orders that are entered by customers so that I can fulfill my responsibility to my customers and give them what they want" (or something along those lines). However, I could also write this from the perspective of the customer and say something like "As a customer, I want to my order entry to be processed/fulfilled so that I can receive the things that I want".
I realize that there may not be one ultimate answer as to whose perspective is the valid one or more usefull one. I'm sure I will get a lot of "it depends" replies. Nonetheless, I would be very interested in hearing about what others have done in situations like these, or if anyone knows of any recommendations, guidance or practices specifically for these types of scenarios.
It might help to move away from the traditional user story template and towards the stakeholder-focused format of Feature Injection (BDD in the analysis space):
In order to <achieve a goal>
As <the stakeholder>
I want <someone to do something for me>.
You can work out who the stakeholder is by thinking about who would be willing to pay for the story to be delivered. For instance, CAPTCHA boxes - those annoying things users have to fill in - are done for a moderator's benefit, or to make the site more attractive to gain revenue, and not for the user's benefit at all! In fact, when you think of most sites, applications, etc., they're hardly ever done for a user. Most websites are about advertising revenue. Most enterprise applications involve one department entering data so that another department can use it, or so that money can be taken from customers.
When you do this, it becomes more obvious that there might be more than one user involved, and a user might be another system. In your case I'm guessing that some kind of sales head is the main stakeholder for this story.
In order to make sales
As the Sales Head
I want customers to be notified of any errors with their order.
In order to make sales
As the Sales Head
I want customers' orders to be fulfilled within 24 hours.
You can see from this that the goals become quite high-level, so if you have a piece of software which plays into those goals, you can break them down:
In order to fulfil customer's orders within 24 hours...
Now every story can be traced back to the project vision, and you can see all the systems in play. So your automated scenarios might read:
Given a valid order in the queue
When the order fulfilment system runs
Then it should send a fill order form to the processing centre
When the processing centre responds successfully
Then the successful fulfillment should be logged
And the customer should be notified by email.
Given an invalid order in the queue
When the order fulfilment system runs
Then it should send a fill order form to the processing centre
When the processing centre responds with an error
Then the error should be logged
And the customer should be notified of the problem by email.
For instance.
By the way, if you're thinking of moving to this format now, be aware that the transparency it creates can cause absolute havoc with people who are, say, developing because they've got the budget, and not a proper project vision. I think this is a good thing. Others find the politics less comfortable! Good luck, whatever you decide.
I'm currently developing an application where I need to implement some sort of way of handling credits which are purchased by users of my application. These credits will then be used to preform certain tasks inside my application such as purchasing hits in mechanical turk. The reason I need to do this is because in the case of mechanical turk there is a possibility that our orders won't be filled and instead of just keeping the extra money for hits they didn't get I want to credit them for future purchases.
The important parts I need help fleshing out is how do you accurately manage an ongoing total of credits. I can't obviously calculate it every time. Additionally I need to manage the adding and subtracting of the credits. Also I probably need to track the origin of these credits, ie money or free because it is possible we might give out free credits as a reward but we need to be careful how to handle turning free credits into cash because it opens an incentive for scammers to use the credits to purchase turk hits then do the turk hit themselves and keep the money.
I currently work on a system with something very similar.
Users have an Account which tracks all transactions - origin, type and amount. We have a finite number of transactions types (including obvious the ones like credit/debit). The Account table is therefore actually an archive of all of the transactions for a particular user account, rather than being a simple running total.
Account totals are therefore calculated every time we need them, although it would be pretty simple to add some aggregates to the database that increment/decrement a total value as transactions are processed. As with all performance-related issues - don't do it until you need to, summing values across some smartly indexed columns is fine for our current system.
When users make credit card transactions on my web app, I'd like to include the transaction fee on the confirmation page before the user makes the order. The thing is, there are different transaction fees for different cards. Is there a way to determine a transaction fee from the card number?
I'm using Rails and ActiveMerchant, but I figure this question was applicable to other languages as well.
Note that the transaction fee that your merchant account provider charges you is almost never a single amount per card type (Visa / MC / Amex). Rather, the amount varies depending on the style of card (Visa regular card vs Visa Corporate cards, vs Visa Signature Reward cards vs International Visa card, etc).
The different rates depend on how highly "qualified" the card is.
And there is no way to tell what kind of card / transaction fee you'll be charged for the card ahead of time. -- The banks don't want you to know the fee ahead of time since that would give merchants a financial incentive to refuse the more expensive types of cards.
I recommend: check your merchant account bank statement. Figure out the average transaction fee that your bank is charging you, then charge that to your customers. It's almost always a combination of a rate and a per transaction amount. Eg 2.2% / $.10
And remember that to get the best qualification, always send the bank the customer's street address and zip, and put values in the order number, description and po fields. If you don't have a po from the customer, make one up from the customer's name and telephone.
I've also heard of some merchants putting a fixed value in the tax field, apparently there are corp cards where you'll get a better qualification if the tax field has a value.
You can use the Bank Card Number as a heuristic for determining the card type before actually submitting the transaction. There is also a much more detailed list, but you probably just need Visa/MC/Discover/AmEx determination I'm guessing.
Our website uses a credit system to allow users to purchase inexpensive digital goods (eg. photos). We use credits, rather than asking the user to pay for items individually, because the items are cheap and we are trying to keep our credit-card/PayPal overhead low.
Because we aren't a bank, we have to expire credits after a certain amount of time. We expire deposit credits after a year, but other types of credits (bonuses, prizes, refunds) may have a different shelf-life. When a buyer buys an item, we spend the credit that is going to expire first.
Our current system keeps track of every deposit by storing the original value and the remainder to be spent. We keep a list of all purchases as well, of course.
I am currently moving to a system which is much more like a traditional double-entry accounting system. A deposit will create a ledger item, increasing the user's 'spending' account balance. Every purchase will also create a ledger item, decreasing the user's 'spending' account balance. The new system has running balances, while the old system does not, which greatly improves our ability to find problems and do reconciliations.
We do not want to use the old system of keeping a 'remainder' value attached to each deposit record because it is inefficient to replay a user's activities to calculate what the remainder of each deposit is over time (for the user's statement).
So, after all of this verbose introduction, my question is "Does anyone else out there have a similar system of expiring credits?" If you could describe how you calculate expired credits it would be a great help.
If all expired credits had the exact same shelf life, we would be able to calculate the expired amount using:
Total Deposits - Total Spending - Deposits Not Due To Expire = Amount to Expire
However, because deposits can have different shelf lives, this formula does not work because more than one deposit can be partially spent at any given time.
No, I don't have a similar system, but it sounds like you should probably add the expired credits to the ledger as debits.
These credits you describe don't sound to me like a currency; they sound more like a financial instrument with an expiration date, like an option or a futures contract. If you use conventional double-entry bookeeping, and account for each instrument individually the same way a financial brokerage would, then things should work out about right.