I'm developing a Mac app that uses CloudKit as its back-end. Some of my users are requesting the ability to ingest and extract data via an automation/integration service such as Zapier. For this, I need to introduce a web API.
I am planning to use CloudKit Web Services to access the app's data. This data is user-specific and hence, resides in a private database. As a result, CloudKit requires user authentication as described here.
Essentially the user needs to be redirected to an Apple-hosted authentication page. After successful authentication, an authentication token is provided that can be used for data operations. Similar to how OAuth2 works, but different enough to not work with Zapier's (or probably any other similar services) supported authentication schemes.
Who has done something similar? What are my options? I want to keep things as simple as possible and make my web API's implementation as thin as possible.
Thanks.
Niels
This is definitely doable and you are on-track with your thinking. Here's how I envision it working:
You could do all of this with a front-end web app (no server-side app needed). I personally prefer Vue.js but you probably have something in mind already.
Your app will need to authenticate the user to CloudKit using the flow you mentioned. I highly recommend you use the Web Services API and not try to wrestle Apple's neglected CloudKit JS API. For this, you are going to need to generate an API token in the CloudKit Dashboard.
You app would then prompt the user to authenticate to Zapier.
You should now have user credentials for both CloudKit and Zapier in place in the user's browser cache (you can save, for example, the CloudKit token to sessionStorage and likewise with Zapier).
Make API calls to Zapier, pull down the data, and then save it to CloudKit all within your JS app. It's all API transactions at this point. I'm a fan of Axios for making the HTTP requests.
If you are downloading files, transacting huge amounts of data, or doing processor-intensive stuff, you might consider using a server for that work. But if you just need a place to pull and push reasonable chunks of data, I see no reason why you can't do it all in a front-end app.
Alternatively, if you don't want a web app at all, and want to only have the user work in the Mac app, that can be done, too. Just make API calls directly to Zapier from within your Cocoa app. Whether or not this is feasible depends some on how you want it to work.
If you have more specific questions or need help with any of the implementation details, feel free to add a follow-up comment or ask a new question.
Good luck!
I think the other answer is mostly correct. I don't know much about CloudKit, but we can talk through what you'd need for it to work.
Let's say you had a simple iOS app that stored contacts. On the iOS side, Apple presumably abstracts the upload and download operations.
If you wanted to make a web viewer for synced contacts using CloudKit, you'd need an endpoint to fetch all rows belonging to the authenticated user (each of which would have a UUID, name, and a phone number). I believe that's possible with CloudKit code Apple provides (but let me know if I'm off base).
Now, we want to integrate with Zapier. Say, a "New Contact" trigger. You make some sort of authenticated HTTP request from Zapier to Apple on behalf of an authenticated user. It gives back a list of contacts and Zapier can trigger on the ones it hasn't seen before. To do that, Zapier needs some sort of user token.
That's where the little front-end page the other answerer mentioned comes into play. If you've got a web page that can surface a user's token to them, they can paste it into Zapier and all of the above becomes possible. I'm not sure what the lifespan of the token is, but hopefully it can be automatically refreshed as needed (rather than the user needing to take any manual action).
I'm not sure if what I've described is possible, but do let me know if it is. It would be huge if it were possible to integrate Zapier and the iOS ecosystem!
Edit to respond to comments:
Zapier won't be able to interact with CloudKit in a way sufficient for me (some minor business logic is needed)
I'm not sure what that entails for you, but it's common to put logic in the Zapier integration to structure data in a way Zapier expects. There's a full Node.js execution environment, so the sky's the limit here.
I don't think Zapier can authenticate against CloudKit as it uses a non-standard authentication scheme
Once you've got a user's token (described above, which is unusual), you will almost certainly be able to use it in requests to cloudkit. Zapier provides a "custom" authentication scheme which lets you do basically anything you want. So unless Apple uses something that fetch can't handle (unlikely), it should be fine (once you get the token).
I would like to push data directly from my app into Zapier and have it done whatever magic the user has configured
This is also probably possible. Zapier ingests data in two ways:
polling, where Zapier frequently makes a web request, store the IDs of items we've seen before, and act on the new ones. You can read more about that here. Assuming you can work your business logic into the integration, this is doesn't require an external server besides Apple's
webhooks, where Zapier registers a subscription with you and you send new data, on demand, to that address. This would probably require a webserver on your end to handle. It's optional though - you can do polling instead.
Hopefully this cleared it up a bit. Feel free to reach out to partners#zapier.com and reference this question to talk more about it.
Related
I am developing an iOS application (to be deployed on the App Store) that requires content updates on a weekly basis.
I understand that the best way to achieve this would be to use a server, where the app would query for new data and download responses in JSON. However I am not knowledgeable when it comes to HTML, PHP or MySQL and therefore am endeavoring to find an alternative.
Here's an idea: using Dropbox to substitute for a server backend. My app connects to one central Dropbox account, checks for new files, and downloads them if present.
Is this idea feasible? If not, are there any alternatives?
Dropbox cannot be a dependable substitute for your server/backend for following reasons:
Dropbox uses OAuth for authentication, which needs user interaction. You do not want your app users to go through Dropbox authentication with your 'common' credentials.
Users who have a Dropbox account or the app installed, will most likely use their own credentials to login which completely breaks your flow.
Drobox, although a good way of sharing and syncing files is not meant for more meaningful data like web services etc or user/database interaction etc. Just syncing JSON file may suffice your app's needs for now but from a long term perspective you want a proper back end.
As suggested in the comment by #tkanzakic you can use one of the substitution services if you don't want to get too technical on the backend.
I am pretty late to the party, but this is possible and not necessarily a stupid idea (though this depends on what you need). You might want to have a look at remote storage for example, which allows you to use Dropbox among other providers as backend.
For sure, you can use the Dropbox Sync API to achieve this (https://www.dropbox.com/developers/sync).
I'm developing an application which lets users upload pictures. I'd like to use Google cloud services to store these pictures. I am creating a unique GUID for each image in database and would like to store the images in the cloud with that name. It makes sense for me to make an ajax request for a GUID and then upload the image from the same page directly to google cloud services.
https://github.com/GoogleCloudPlatform/storage-getting-started-javascript/blob/master/index.html
Like shown in this example.
My first question is, should I be sending this to my back-end(C# code) and uploading it from there? Or is this the correct approach?
And my second question if this is the correct approach is, wouldn't exposing my details like that in javascript allow other people to upload from outside my application as well?
An API key, by itself, identifies a call as being associated with a certain project for purposes of billing. It's only necessary for anonymous calls. An API key does not grant any sort of authorizations. If there's an object in a bucket in your project that only your project members can see, the API key won't give anyone permission to read it.
That said, it's not a great idea to share your API key if you can help it, and if you need to share it, you should lock it down as much as possible. API keys can be limited to use with only certain IP addresses, only with certain web referrers (for instance, it will only work with JavaScript clients on www.yoursite.com), or only when run from a particular iPhone or Android app. These precautions aren't cryptographically fool-proof (there's no reason a hacker couldn't spoof a referer header), but they do make them pretty much useless for someone else who just wants to paste an API key somewhere to enable a web app and doesn't want to pay for it themselves.
The problem with using the javascript client's approach for your application is that individual users would either end up uploading objects completely anonymously or with their own Google accounts. Neither is super great, since the anonymous option would basically require you to create a bucket with anonymous writes enabled, and you don't want to do that.
There is a great approach to letting users upload pictures, though: signed URLs. Signed URLs allow your server to securely sign, in advance, a request to upload an object with your credentials. This is your best option for letting anonymous end users securely upload objects to your buckets.
Documentation on signed URLs: https://cloud.google.com/storage/docs/accesscontrol#Signed-URLs
With CloudKit, you can focus on your client-side app development and let iCloud eliminate the need to write server-side application logic. CloudKit provides you with Authentication, private and public database, structured and asset storage services — all for free with very high limits.
You cannot upload any code to run on Apple's servers?
I've heard it being compared to Google App Engine and other cloud computing platforms, but without the ability to run your own code, isn't the whole thing pretty limited and not really comparable?
For example, if I want to build a news app which periodically pushes stories on topics that the user is interested, then this can't be done just using CloudKit because I would need scheduled jobs and data processing on the server.
Any thoughts?
Server-side
As you said CloudKit doesn't allow server-side code.
But there are possibilities.
Crons
You don't want to connect to the iCloud Dashboard everyday in order to perform the push by adding a record. One solution here is to code an app on a mac server (I guess mac mini as server will become more popular with CloudKit) that add a new Daily CKRecord every day.
Subscriptions
Subscriptions concept is that the client registers for specific updates. You can create a record type called Daily for instance and make users register to it. You should check the Apple documentation and WWDC14 videos (even if Subscriptions are not detailed, it's a good start point).
The good thing is push notifications are linked with the subscription concept. So basically you say: Send my a notification for each new CKRecord of type Daily added.
BaaS party
What is the point for using CloudKit (vs Parse and other?)
Price: CloudKit has a really nice pricing
Ready to go: 2 clicks inside XCode and you are ready to go
User consistency: you get free user login for all his devices through their iCloud account. With a very good privacy system. And you can get relationships with a smart system.
But:
You are stick on Apple platform. We don't even know if we could export the data..
Only data-centered for now (no server-side code)
The CloudKit dashboard is too limited
The future
CloudKit is still pretty new. At the WWDC some guys behind it made me understand that they are still heavily working on it. My bets are they are working on 2 important points :
Server side code execution through remote scheduled tasks
CloudKit for Analytics (Visualization side)
Edit: Apple guys are fully aware and concerned about the lack of web access for the data. It means that one day it may be accessible from other platforms. I read in a comment that Apple probably would have bought Parse if CloudKit wasn't better, AFAIK they tried to buy Parse (skills buy it's said, but we don't really know).
Update WWDC15
CloudKit is now available in JS and some dashboard are available now. Wait and see.
Update February 2016
CloudKit Now Supports Server-to-Server Web Service Requests
Web Services Reference
In some cases, we do not need server-side logic, and just storing static data can cover all the usage scenario.
In this case, it would be very helpful if there's a free accessible storage that you can store something. CloudKit provides such stuffs rather then full service platform.
Yes it is limited. Anyway can be useful for some people. For example, your case actually can be supported CloudKit. Though CloudKit is just a static storage, it support subscription. Which monitors a set of conditions and pushes the event notification to client. It's fortunate that the only background job feature supported by CloudKit is just what you need.
Anyway, if you need more, then you might need to consider full fledged servers. Usually simple web services with simple server-side code execution support are also limited.
You cannot upload any code to run on Apple's servers?
You can and you can't. You can't upload code / SOAP based web services to the server, instead of it you can upload / store observers on the server, called subscription.
whole thing pretty limited and not really comparable?
I would say in CloudKit and in MBaas client communicates with server though a more narrower more robust interface: you can not upload exotic web service to do XML parsing, database manipulations and based on it trigger push notifications, but RestFull architecture allows you to perform the 4 basic operation on the data store, and with subscription client can get notified about INSERT / UPDATE / DELETE operations performed on tables.
I think MBaas is just the next step in evolution of server - client architecture. First it seems it is limiting, but you can do all as in SOAP based web services world. Development is extremely fast / scalable / comfortable to use and easier to control things like permissions / setup, maintain server, security needs almost no effort.
Believe it or not, you can actually get REALLY far with this approach.
I've not used CloudKit, but I can describe for you my application stack:
AngularJS (or your favorite client side HTML rendering framework): A single page will host a series of templates/controllers selected by the router and driven by users changing the anchor to select which page they're on.
Firebase.io (or your favorite cloud storage): Any dynamic data goes into the cloud document store. The controller needs to load the data and render the template on the client, and when the data changes, send the data back. This also provides the authentication and authorization as well, since you can limit access to the data.
Now you need a place to serve the HTML/CSS/JS/images... which requires no 'server side code execution', just a web server where you can put the assets.
Using this technique you could store all the user's topics in the database for that user, and when the page loads, go and aggregate all the sources for those topics (also stored in the database) completely client side. There's nothing in your example application which actually requires server side execution that I can see, so long as you have cloud storage which will provide you with authentication and authorization services, and a 'dumb' web server for serving up static assets.
CloudKit isn't a full-fledged web hosting service. Instead, it's an SDK for iCloud. You shouldn't be putting a web site up there, just storing user data that you may want to use in multiple applications or platforms.
iCloud APIs enable your apps to store app data in iCloud, keeping your apps up to date automatically. Use iCloud to give your users a consistent and seamless experience across iCloud-enabled devices.
Recently, YouTube decided to make video tags unavailable publicly. So to get the tags for a given video, I need to make an authenticated request to the API as the owner of the video. This is not a problem in my case as I'm fetching my own videos.
However, I'm confused about the authentication flow since YouTube strongly recommends to use OAuth2. Since I'm always going to authenticate as the same user (the owner of the video, aka myself), I definitely don't need to have any browser page for the actual user of the app to do anything. I see how I could have done it using ClientLogin (hardcoding login and password into the app) but I'm not sure how to approach this using OAuth2.
One last detail - that is not necessarily relevant since a high-level answer would be enough - is that I'm developing on iOS. Also I looked at this https://developers.google.com/accounts/docs/OAuth2 and particularly the web server case which seems closest to mine but was not able to get a clear idea from it.
Thanks in advance for your help and don't hesitate if you need me to be more specific.
There is no OAuth flow that supports your use case.
In general, you should not be distributing your YouTube login as part of your application. Even if this were available via ClientLogin, after a certain number of logins, you would likely be presented with a challenge because the authentication servers would detect a strange usage pattern.
OAuth is not for distribution a single user's login to a large N, where N is the number of users of your application. OAuth is meant for your application to act on behalf of an end user, and because tags are no longer exposed to end users through the UI, it does not make sense to expose them to users via the API either. More details can be found here:
http://apiblog.youtube.com/2012/08/video-tags-just-for-uploaders.html
How many videos do you have? What is the purpose for needing the tag metadata? From a pragmatic perspective, here are a few alternative implementations that would be easier and would not require users to log in as you:
Store a single file mapping video IDs to tags on a server somewhere and fetch this periodically. Google App Engine is a good place to do this.
Put the tag data in the description in a predictable format (you host the videos), and generate the metadata from this.
My question is not about FourSquare API and its functions, but about more simpler details that are not well explained on Foursquare API explanations. Thank you very much in advance if someone wants to help me with this doubts:
Foursquare API is a framework you can use to build applications for mobile devices, above of IOS and Android, so i can imagine that they have API for IOS (Objective-C) and Android (Java), right?
From API Doc: "Be sure to note that although API requests are against api.foursquare.com, OAuth token and authorization requests are against foursquare.com.". Does it mean that if i want to use FourSquare app, the users has to have an account on FourSquare?
From API Doc: "For example, if you write an iPhone application, every user who logs in with their foursquare account will be able to make up to 500 users/* requests and up to 500 venues/* requests, etc." I dont understand this sentence. Does it mean that for example, if you use an API method request like "checkins.add()", this method create two methods? one against api.foursquare.com to monitor the API limit requests, and another to your Web Application Server?
So as a question related to the third one, where do you have to store your database? is it stored on Foursquare cloud database because you are loggin there, or you have to create your own Web Service application with its own SQL database?
From API Doc: "All requests are simple GET or POST requests that return JSON or JSONP respones", so i can imagine that the Web Application Service should understand JSON. Well, my main question is, can i use Ruby on Rails to build the Web Application Service and Web Page frontend? I am seeing that there are some wrapps for RoR designed from third companies, but are not official and doesnt cover all the 2.0 API, just the ones they needed for their services.
If i want to create an app using FourSquare API, what do you advice me to use as a programming language/framework for the Web Service Application? the WSA that has to process the JSON requests and later store them on the database, interaction with users on the WebPage, etc.
i am so sorry if my questions are so simple, but i dont have any other place of this level of expertise.
thank you very very much in advance.
The API is REST/JSON based, which means that any language that can do an HTTP request and parse a string can be used. There are Java and iOS libraries available. But you could use just about anything - curl with bash would be a bit extreme but if that floats your boat...
For some of the APIs (search a venue, for example) you do not necessarily need a FourSquar OAuth user token. For others (like checkin) a FourSquare token is required. For any API calls that require a userid, your users will have to be FourSquare users and "trust" your application with their FourSquare data.
Only requests to FourSquare is counted. So if you do a single call to checkins.add() it counts as one call for the user that is doing the checkin. I wouldn't worry about the limits. As long as you're usage of the API is sensible they will not be a problem. And if they do become a problem and you're doing something extraordinarily cool, the folks at FourSquare might be sympathetic.
You have to create your own web server with your own database to store some information. The OAuth token is one. You probably want to cache venue information here for short periods as well.
Yes, your webapp will need to be able to understand JSON. Ruby has excellent JSON support - look for the json gem.
It is really difficult to suggest a language or framework without knowing what it is that you're trying to do. I wouldn't choose a framework based on the fact that you want to use FourSquare (anything will do) but rather on your experience and the unique features of your application. You mentioned RoR before - that would definitely work.