I've been integrating Firebase for a clients back-end dashboard system and I'm generating timestamps on activities which for FINRA/SEC purposes they would like to see set immutable.
Users have zero control over the data, all of it is handled by API calls to Node but through Node, the server is given full access permissions. Does Firebase provide a way to create a collection so that not even Node nor the Web UI can make changes to the info? Similar in the way an S3 vault-lock works.
Related
I'm using Firebase for my mobile app's entire backend and I was wondering which of the two is a more reliable way of performing the task of creating a user and then a batch of documents.
Create the Firebase user (using Firebase Auth) on the client and if it succeeds then perform a Firestore batch write also on the client to create the documents.
Call a Firebase cloud function from the client to perform the above task.
My reliability concern has to do with the network. If the user is created on the client but is unable to create the documents then, well, that's not good. But if the client is able to invoke a cloud function then I feel like network reliability is not an issue in the cloud environment. If the task is to fail in the cloud it will be because of violating an error that I have control over (i.e. bad email format, weak password). Which of the two is more reliable for my purposes?
If the user has a spotty connection, the call to the Cloud Functions is just as likely to fail as the call that writes them directly to Firestore. The main difference is that the Firestore SDKs will automatically retry the write operation in such cases, while with a Cloud Function you'd have to implement that yourself.
I definitely would recommend using option 1. You should create a user with Firebase Auth and then create a collection called "users" and add a document with the user's UID, which is auto generated. This should occur after you insure that there is no error in the Firebase Auth process. If there is you should just display the error. If you need more specific info, feel free to respond.
I'm developing a Mac app that uses CloudKit as its back-end. Some of my users are requesting the ability to ingest and extract data via an automation/integration service such as Zapier. For this, I need to introduce a web API.
I am planning to use CloudKit Web Services to access the app's data. This data is user-specific and hence, resides in a private database. As a result, CloudKit requires user authentication as described here.
Essentially the user needs to be redirected to an Apple-hosted authentication page. After successful authentication, an authentication token is provided that can be used for data operations. Similar to how OAuth2 works, but different enough to not work with Zapier's (or probably any other similar services) supported authentication schemes.
Who has done something similar? What are my options? I want to keep things as simple as possible and make my web API's implementation as thin as possible.
Thanks.
Niels
This is definitely doable and you are on-track with your thinking. Here's how I envision it working:
You could do all of this with a front-end web app (no server-side app needed). I personally prefer Vue.js but you probably have something in mind already.
Your app will need to authenticate the user to CloudKit using the flow you mentioned. I highly recommend you use the Web Services API and not try to wrestle Apple's neglected CloudKit JS API. For this, you are going to need to generate an API token in the CloudKit Dashboard.
You app would then prompt the user to authenticate to Zapier.
You should now have user credentials for both CloudKit and Zapier in place in the user's browser cache (you can save, for example, the CloudKit token to sessionStorage and likewise with Zapier).
Make API calls to Zapier, pull down the data, and then save it to CloudKit all within your JS app. It's all API transactions at this point. I'm a fan of Axios for making the HTTP requests.
If you are downloading files, transacting huge amounts of data, or doing processor-intensive stuff, you might consider using a server for that work. But if you just need a place to pull and push reasonable chunks of data, I see no reason why you can't do it all in a front-end app.
Alternatively, if you don't want a web app at all, and want to only have the user work in the Mac app, that can be done, too. Just make API calls directly to Zapier from within your Cocoa app. Whether or not this is feasible depends some on how you want it to work.
If you have more specific questions or need help with any of the implementation details, feel free to add a follow-up comment or ask a new question.
Good luck!
I think the other answer is mostly correct. I don't know much about CloudKit, but we can talk through what you'd need for it to work.
Let's say you had a simple iOS app that stored contacts. On the iOS side, Apple presumably abstracts the upload and download operations.
If you wanted to make a web viewer for synced contacts using CloudKit, you'd need an endpoint to fetch all rows belonging to the authenticated user (each of which would have a UUID, name, and a phone number). I believe that's possible with CloudKit code Apple provides (but let me know if I'm off base).
Now, we want to integrate with Zapier. Say, a "New Contact" trigger. You make some sort of authenticated HTTP request from Zapier to Apple on behalf of an authenticated user. It gives back a list of contacts and Zapier can trigger on the ones it hasn't seen before. To do that, Zapier needs some sort of user token.
That's where the little front-end page the other answerer mentioned comes into play. If you've got a web page that can surface a user's token to them, they can paste it into Zapier and all of the above becomes possible. I'm not sure what the lifespan of the token is, but hopefully it can be automatically refreshed as needed (rather than the user needing to take any manual action).
I'm not sure if what I've described is possible, but do let me know if it is. It would be huge if it were possible to integrate Zapier and the iOS ecosystem!
Edit to respond to comments:
Zapier won't be able to interact with CloudKit in a way sufficient for me (some minor business logic is needed)
I'm not sure what that entails for you, but it's common to put logic in the Zapier integration to structure data in a way Zapier expects. There's a full Node.js execution environment, so the sky's the limit here.
I don't think Zapier can authenticate against CloudKit as it uses a non-standard authentication scheme
Once you've got a user's token (described above, which is unusual), you will almost certainly be able to use it in requests to cloudkit. Zapier provides a "custom" authentication scheme which lets you do basically anything you want. So unless Apple uses something that fetch can't handle (unlikely), it should be fine (once you get the token).
I would like to push data directly from my app into Zapier and have it done whatever magic the user has configured
This is also probably possible. Zapier ingests data in two ways:
polling, where Zapier frequently makes a web request, store the IDs of items we've seen before, and act on the new ones. You can read more about that here. Assuming you can work your business logic into the integration, this is doesn't require an external server besides Apple's
webhooks, where Zapier registers a subscription with you and you send new data, on demand, to that address. This would probably require a webserver on your end to handle. It's optional though - you can do polling instead.
Hopefully this cleared it up a bit. Feel free to reach out to partners#zapier.com and reference this question to talk more about it.
I use IdentityServer4 (with Asp.Net Core Identity) as a centralized auth point for multiple applications.
In one of the applications I want to setup a scheduled job to send out e-mail notifications to multiple users. When the job will execute, it will have no access to user claims (as it will execute not in the context of a single user request) and thus there will be no place to read user's e-mail from. This means I will have to duplicate e-mails in the application DB.
But how to keep e-mails in sync between the app and the IdentityServer if some user wants to change it?
A good approach would be to implement integration events in your system. This is a mechanism that raises an event 'This special thing happend', and allows other parts of your system to be notified.
You can use RabbitMQ or Azure ServiceBus for example to send messages to. Every system being subscribed to that kind of message, will receive it.
So in your case, you would create an event called UserChangedEmailAddressIntegrationEvent for example. Then in your emailing system, you subscribe to exact this event. Once it's raised, your emailing system will receive the message and be able to handle it.
The UserChangedEmailAddressIntegrationEvent could in fact be a class, containing (for example) two properties, OldEmail and NewEmail so the emailing system knows what value to change.
See the eShopOnContainers example project, which has this exact technique implemented
https://github.com/dotnet-architecture/eShopOnContainers
We have faced a similar problem where we had Hangfire jobs running that would extract reporting data from our system and then email the reports to the set of users configured when creating the scheduled job.
We also used Identity Server 4 with ASP.Net Identity and ended up storing the user id's on the scheduled job info. We then also created an api endpoint on our ASP.Net Identity that would serve back the required user info given user id (or a list of user id's). Lastly, we used client_credentials and created a client to consume that api during the job execution that would retrieve appropriate user info at a given point in time from our ASP.Net Identity api.
Such approach has worked well for us so far and it removed the pain of having to think about how to ensure data syncing all together.
Here are a few details in my current setup to keep in mind for this discussion to be meaningful.
Client App: Xamarin Forms app, uses Azure offline Sync
Web Access: There is a Asp.Net based web front end that also accesses the Mobile Service to read/ write data to the underlying tables
Backend: Managed dot net based App Service hosted on Azure
My Question: As of now I pass a null value to Incremental Sync parameter to ignore any incremental sync (while using PullAsync method) & get all data from the server. This however is inefficient as it increases the load time as the volume of data increases.
If I use a constant value for Incremental Sync parameter for the table under consideration, this is what it does
Each time a pull operation returns a set of results, the latest updatedAt timestamp from that result set is stored in the SDK local system tables. Subsequent pull operations will only retrieve records after that timestamp.
as read here
The problem is, my web access also updates the data from web & it also uses Azure App service framework to do that. In this case, if I use the incremental sync parameter do a Pull of data, then there is a change on the data from the web app & if the mobile app again does a Push, Pull.. will it get the latest changes from the web too?
My understanding is that incremental sync only works with the data the context is aware of, but in this case because the web app is also writing the data, which the context is not aware of, then it misses some of the data & the mobile app never downloads it.
Is there a way to fix this behavior? So I can start using incremental sync & still be able to ensure that the app always gets all the data from server without missing any bits?
Update:
#Adrian it's not working for me. May be because the way my Tables are structured, or the way i am dealing with them is not correct.
I have a User Table & a Department Table, I also have a UserDepartment Table.
When I sync the app, the data in this UserDepartment Table gets duplicated on the app.
The reason could be because, when I edit the user on web to assign departments to them, I delete all previous User Department entries from UD table & recreate new ones based on current selection. Do you think this could be causing issues with azure client SDK & thus duplicating some data for me?
If your web app and mobile app update through the same Mobile App API (i.e. using the .NET SDK and JS SDK respectively), then the updatedAt is set appropriately for both pushes and the other side should pick up the change. I've done this myself several times and it works.
One the web side (using the JS sdk), you just use an online table. On the mobile side (with the .NET SDK), you use incremental sync.
With CloudKit, you can focus on your client-side app development and let iCloud eliminate the need to write server-side application logic. CloudKit provides you with Authentication, private and public database, structured and asset storage services — all for free with very high limits.
You cannot upload any code to run on Apple's servers?
I've heard it being compared to Google App Engine and other cloud computing platforms, but without the ability to run your own code, isn't the whole thing pretty limited and not really comparable?
For example, if I want to build a news app which periodically pushes stories on topics that the user is interested, then this can't be done just using CloudKit because I would need scheduled jobs and data processing on the server.
Any thoughts?
Server-side
As you said CloudKit doesn't allow server-side code.
But there are possibilities.
Crons
You don't want to connect to the iCloud Dashboard everyday in order to perform the push by adding a record. One solution here is to code an app on a mac server (I guess mac mini as server will become more popular with CloudKit) that add a new Daily CKRecord every day.
Subscriptions
Subscriptions concept is that the client registers for specific updates. You can create a record type called Daily for instance and make users register to it. You should check the Apple documentation and WWDC14 videos (even if Subscriptions are not detailed, it's a good start point).
The good thing is push notifications are linked with the subscription concept. So basically you say: Send my a notification for each new CKRecord of type Daily added.
BaaS party
What is the point for using CloudKit (vs Parse and other?)
Price: CloudKit has a really nice pricing
Ready to go: 2 clicks inside XCode and you are ready to go
User consistency: you get free user login for all his devices through their iCloud account. With a very good privacy system. And you can get relationships with a smart system.
But:
You are stick on Apple platform. We don't even know if we could export the data..
Only data-centered for now (no server-side code)
The CloudKit dashboard is too limited
The future
CloudKit is still pretty new. At the WWDC some guys behind it made me understand that they are still heavily working on it. My bets are they are working on 2 important points :
Server side code execution through remote scheduled tasks
CloudKit for Analytics (Visualization side)
Edit: Apple guys are fully aware and concerned about the lack of web access for the data. It means that one day it may be accessible from other platforms. I read in a comment that Apple probably would have bought Parse if CloudKit wasn't better, AFAIK they tried to buy Parse (skills buy it's said, but we don't really know).
Update WWDC15
CloudKit is now available in JS and some dashboard are available now. Wait and see.
Update February 2016
CloudKit Now Supports Server-to-Server Web Service Requests
Web Services Reference
In some cases, we do not need server-side logic, and just storing static data can cover all the usage scenario.
In this case, it would be very helpful if there's a free accessible storage that you can store something. CloudKit provides such stuffs rather then full service platform.
Yes it is limited. Anyway can be useful for some people. For example, your case actually can be supported CloudKit. Though CloudKit is just a static storage, it support subscription. Which monitors a set of conditions and pushes the event notification to client. It's fortunate that the only background job feature supported by CloudKit is just what you need.
Anyway, if you need more, then you might need to consider full fledged servers. Usually simple web services with simple server-side code execution support are also limited.
You cannot upload any code to run on Apple's servers?
You can and you can't. You can't upload code / SOAP based web services to the server, instead of it you can upload / store observers on the server, called subscription.
whole thing pretty limited and not really comparable?
I would say in CloudKit and in MBaas client communicates with server though a more narrower more robust interface: you can not upload exotic web service to do XML parsing, database manipulations and based on it trigger push notifications, but RestFull architecture allows you to perform the 4 basic operation on the data store, and with subscription client can get notified about INSERT / UPDATE / DELETE operations performed on tables.
I think MBaas is just the next step in evolution of server - client architecture. First it seems it is limiting, but you can do all as in SOAP based web services world. Development is extremely fast / scalable / comfortable to use and easier to control things like permissions / setup, maintain server, security needs almost no effort.
Believe it or not, you can actually get REALLY far with this approach.
I've not used CloudKit, but I can describe for you my application stack:
AngularJS (or your favorite client side HTML rendering framework): A single page will host a series of templates/controllers selected by the router and driven by users changing the anchor to select which page they're on.
Firebase.io (or your favorite cloud storage): Any dynamic data goes into the cloud document store. The controller needs to load the data and render the template on the client, and when the data changes, send the data back. This also provides the authentication and authorization as well, since you can limit access to the data.
Now you need a place to serve the HTML/CSS/JS/images... which requires no 'server side code execution', just a web server where you can put the assets.
Using this technique you could store all the user's topics in the database for that user, and when the page loads, go and aggregate all the sources for those topics (also stored in the database) completely client side. There's nothing in your example application which actually requires server side execution that I can see, so long as you have cloud storage which will provide you with authentication and authorization services, and a 'dumb' web server for serving up static assets.
CloudKit isn't a full-fledged web hosting service. Instead, it's an SDK for iCloud. You shouldn't be putting a web site up there, just storing user data that you may want to use in multiple applications or platforms.
iCloud APIs enable your apps to store app data in iCloud, keeping your apps up to date automatically. Use iCloud to give your users a consistent and seamless experience across iCloud-enabled devices.