In our company, we want to develop a simple feature flag system to work beside our API Gateway. Since we expect not more than 100 req/s on the system we decided that third-party solutions will probably be costly(as they require time and effort to understand) and also it would be a fun effort to do it on our own. Here are the requirements:
We have the list of users and the latest app version they are using. We want to apply three kinds of policies on their usage of the some application feature:
Everybody can use the feature
Only users with a version higher than a given version can use the feature
Some random percentage of users can use the feature( this is for A/B testing in our future plans)
These policies are dynamic and can change (maybe we want to remove the policy or edit it in the future)
So, the first problem is how to store the policies ?
I think We can describe each policy with at most three fields: feature , type , and specification . So for instance something like A 2 1.5 means that feature A is only active for users who use version 1.5 or above. A 3 10 means activate feature A for 10 percent of users at random and finally A 1 means that feature A should be active for all uses.
Although in this way we can describe a policy in a meaningful matter, I can't see how I can use these in a system. For instance, how can I write these policies in an RDBMS or in Redis for instance?
The second problem is how to apply these policies in the real world. In our architecture, we want each request to first visit this service, and then(if authorized) it can proceed to the core services. Ideally, we want to call an endpoint of this service and the service returns a simple yes/no response(boolean). For simplicity lets assume we get a user id and a feature like 146 A (users 146 wants to use feature A), we want to determine whether 146 is allowed to use it or not.
I thought of two approaches, first is real-time processing: as a request comes, we do a database call to fetch enough data to determine the result. The second approach is to precompute these conditions before hand, for instance when a new policy is added we can apply that policy on all user records and store the results somewhere, in this way, when a request comes, we can simply get the list of all features that the uses has access to and check if the requested feature is in the list.
The first approach requires a lot of DB calls and computation per each request, and the second one is probably much more complicated to implement and needs to query all users when a new policy is added.
These are our two main problems, I tried to simplify things so that it could become a more generic problem. I would appreciate if you could share your thoughts on each of one them.
Related
I need to make requests to the Google Books API form my app which includes the API key in the URL.
I thought about just creating it as file private variable in my app, though this is a big problem because it would then be uploaded to Github.
Then I thought about environment variables but I heard they aren't included if the app isn't run by Xcode.
I'm aware that this way the key could be extracted, but should I worry?
Can't users anyway just use Wireshark or something similar and see the key in the URL?
And I can restrict the key so it is only valid when called from my Bundle ID.
What do you think would be the best option for making the calls? I mean other than that, the app barely gets 10 downloads a week so this can't be too big of an issue, right?
Whether it is an issue entirely depends on your usecase and threat model. Consider your api key public if you include or send it in any way in/from your app, and think about things like what can people do with it. What level of harm can they cause you? This gives yo the impact. Would they be motivated, for example is there a financial benefit for them somehow? This estimates the likelihood of this happening. This together, impact x likelihood = risk, which you can either accept (do nothing about it), mitigate (decrease the impact or likelihood), eliminate (fix it) or transfer (rg. buy some kind of an insurance).
As for mitigations, can you limit the api key scope, so that only necessary things can be done on the api with it? Can you set up rate limiting? Monitoring, alerting? I'm not familiar with the Books api, but these could be mitigating controls.
As for eliminating the risk, you should not put the api key in the app. You could set up your own server, which would hold the api key, and would pretty much forward requests to theBooks api, augmented with thr api key. Note though that you would still need some kind of authentication and access control in your server, otherwise it can just be used as an oracle by an attacker to perform anything in the actual Books api the same as if they had the key, only in this case they don't need it. This role could also be fulfilled by some kind of an api gateway, which can also add data to api queries.
Eliminating the risk is obviously more expensive. Defenses should be proportionate to risk, so you have to decide whether it is worth it.
I am trying to build an offline synchronization capability into my iOS App and would like to get some feedback/advice from the community on the strategy and best practice to be followed to do the same. The app details are as follows:
The app shows a digital catalog to users and allows them to perform actions like creating and placing orders, among others.
Currently the app only works when online, and we have APIs for all actions like viewing the catalog, creating/placing orders which return JSON data.
We would like to provide offline/synchronization capability to users, through which users can view the catalog and create/place orders while offline, and when they come online the order details will be synchronized and updated to our server.
We would also like to pull the latest data from the server, and have the app keep itself up to date in case of catalog changes or order changes that happened at the Server while the app was offline.
Can you guys help me to come with the best design and approach for handling this kind of functionality?
I have done something similar just in the beginning of this year. After I read about NSOperationQueue and NSOperation I did a straight forward approach:
Whenever an object is changed/added/... in my local database, I add a new "sync"-operation to the queue and I do not care about, if the app is online or offline (I added a reachability observer which either suspended the queue or takes it back working; of course, I do re-queueing if an error occurs (lost network during sync)). The operation itself reads/writes the database and does the networking stuff. My ViewController use a NSFetchedResultsController (with delegate=self) to get callbacks on changes. In some cases I needed some extra local data (it is about counting objects), where I have used NSManagedObjectContextObjectsDidChangeNotification.
Furthermore, I have used Multi-Context CoreData which sounded quite reasonable to use (I have only two contexts).
To get notified about changes from your server, I believe that iOS 7 has something new for you.
On the server side, you should read a little for the actual approach you want to go for: i.e. Data Synchronization by Dan Grover or Developing Android REST Client Applications (of course there are many more good articles out there).
Caution: you might be disappointed when you expect an easy solution. Your requirement is not unusual, but the solution might become more complex than you expect - depending on the "business rules" and other reasonable requirements. If you intelligently restrict your requirements you may find a solution which you can implement yourself, otherwise you may also consider to use a commercial product.
I could imagine, that if you design the business logic such that it takes an offline state into account and exposes this explicitly in the business logic, you may find a solution which you can implement yourself with moderate effort. What I mean by this is for example, when a user creates an order, it is initially in "not committed" stated. The order will only be committed when there is access to the server and if the server gives the "OK" that this order can actually be placed by this user. The server may also deny the order, sending corresponding messages to the user.
There are probably quite a few subtle issues that may arise due to the requirement of eventual consistency.
See also this question which contains pointers to solutions from commercial products, and if you visit their web sites give valuable information about the complexity of the problem and how this can be solved.
Valence allows to do this, but I wondered if there's limitations to trying to automate user account setup and enrollment in this way. We're a relatively small institution with hundreds of enrollments perhaps in a given term scattered over several weeks, so I don't think there'd be a problem with realtime events. But I wondered what the implication might be for a larger university that might have thousands of enrollments updating all the time. There'd be spikes of activity certainly as a term reached official start.
The issue of performance is a complex one, with many inter-dependant factors. I can't comment on the impact of hardware hosting your back-end LMS, but obviously, higher-performance hardware and network deployment will result in higher performance interaction between clients and the LMS.
If your needs are "hundreds of creations scattered over several weeks", that is certainly within comfortable expected performance of a back-end service.
At the moment, each user-creation request must be done separately, and if you want to pre-provide a password for a user, then it takes two calls (you can, in one call, nudge the LMS to send a "password set" email to the new user, but if you want to manually set the password for the user, you have to do that in a separate call after the user record gets created). The individual calls are pretty light-weight, being a simple HTTP POST with response back to provide the created user record's details.
There are roadmapped plans to expand support for batch operations, and batched user-creation is certainly a logical possibility for improvement.
Is there an option in DynammoDB to store auto incremented ID as primary key in tables? I also need to store the server time in tables as the "created at" fields (eg., user create at). But I don't find any way to get server time from DynamoDB or any other AWS services.
Can you guys help me with,
Working with auto incremented IDs in DyanmoDB tables
Storing server time in tables for "created at" like fields.
Thanks.
Actually, there are very few features in DynamoDB and this is precisely its main strength. Simplicity.
There are no way automatically generate IDs nor UUIDs.
There are no way to auto-generate a date
For the "date" problem, it should be easy to generate it on the client side. May I suggest you to use the ISO 8601 date format ? It's both programmer and computer friendly.
Most of the time, there is a better way than using automatic IDs for Items. This is often a bad habit taken from the SQL or MongoDB world. For instance, an e-mail or a login will make a perfect ID for a user. But I know there are specific cases where IDs might be useful.
In these cases, you need to build your own system. In this SO answer and this article from DynamoDB-Mapper documentation, I explain how to do it. I hope it helps
Rather than working with auto-incremented IDs, consider working with GUIDs. You get higher theoretical throughput and better failure handling, and the only thing you lose is the natural time-order, which is better handled by dates.
Higher throughput because you don't need to ask Dynamo to generate the next available IDs (which would require some resource somewhere obtaining a lock, getting some numbers, and making sure nothing else gets those numbers). Better failure handling comes when you lose your connection to Dynamo (Dynamo goes down, or you are bursty and your application is doing more work than currently provisioned throughput). A write-only application can continue "working" and generating data complete with IDs, queueing it up to be written to dynamo, and never worry about ID collisions.
I've created a small web service just for this purpose. See this blog post, that explains how I'm using stateful.co with DynamoDB in order to simulate auto-increment functionality: http://www.yegor256.com/2014/05/18/cloud-autoincrement-counters.html
Basically, you register an atomic counter at stateful.co and increment it every time you need a new value, through RESTful API.
I am using Rails 3.1.0 with Devise 2.1.0. I would like to limit the number of times a user can perform an action in a day. The main purpose of this limitation is to prevent spam.
I see many questions similar to this one but was wondering if there is a specific way to accomplish what I am trying to do through Devise.
For the actions that create model instances, the number of times an action has been performed in a day is easy to keep track of. However, at least one action that I would like to restrict does not create a model instance, so I'm not sure what to do about it.
I was also wondering if this is a legitimate/effective way of preventing spam (in addition to requiring users to register and sign in to perform the actions).
Personally, I find these sorts of systems to be over-complications. Unless spam is an existing, provable problem I'm not sure adding in a system that's likely to be rather extensive is a good use of time and energy.
Alternatives to this would be requiring registration through a third-party service (say Facebook) and using either captchas or exciting and new negative captchas.
That said, if you want to do this, I think the best place to keep track of it would be in an ephemeral data store. Redis would be really good for this since you can use queues. In the actions that you want to restrict, add a timestamp to the queue, and before you allow the user to perform said action, check the number of elements in the queue, purging ones that are too old while you do so.
That's sort of pseudo-codey, but should at least help you get started.