I don't think I have seen related threads on the forum.
I have a JSON database stored on AWS S3 and I would like to query it from iOS. I don't think Athena can be used on iOS and the only tutorials available between ios and s3 are uploading, or downloading files.
Another solution I think may be possible is to setup an API gateway with a Lambda function, then networking the browser links with iOS (I have not been able to do so, it is probably more complex and doable).
So, how to query a S3 database from ios ?
Related
I would like to use some publicly available data from a government website as a source of data in an iOS app. But I am not sure what is the best / most polite / scalable way have a large number of users request data from this website with the least impact on their servers and best reliability for me.
It is 1-50kb of static XML with a fixed URL scheme
It updates with a new XML once a day
New users would need to download past data
It has a Last-Modified header but no caching headers
It does not use compression or a CDN
It's a government website, so if someone even replies to my email I doubt they are going to change how they host it for me...
I'm thinking I could run a script on a server to download this file once a day and re-host for my app however my heart desires. But I don't currently run a server which I could use for this and it seems like a lot just for this. My knowledge of web development is not great, so am I perhaps missing something obvious and I just don't know what search terms I should be using to find the answer.
Can I point a CDN at this static data somehow and use that?
Is there something in CloudKit I could use?
Should I run a script on AWS somehow to do the rehosting without needing a full server?
Should I just not worry about it and access the data directly??
You can use the AWS S3 service (Simple Storage Service).
The flow is somewhat like this:
If the file doesn't exist on S3 yet, or, if the creation date of the file on S3 is yesterday, the iOS app downloads the XML from the gov site and stores it in S3.
If the file exists on S3 and is up to date, download it from S3.
After that, the data can be presented by the app without overloading to the site.
I think the best way for you is to create an intermediary database where you can store your data in a secure manner.
Create a pipeline that does some data transformation and store in you newly created database.
Create an api with pagination and you desired filters
Also make sure you are not violating any data policies in the process.
I hope this helps.
As we all know, I'm one of the thousands of devs who relied on Parse and now forced to find Parse alternative. While transferring Parse-Server to AWS+MongoDB, I've discovered DynamoDB. I'm thinking of just tranferring my whole server side logic to DynamoDB. What are some of the problems that Parse doesn't have that might exist for DynamoDB?
Since Parse includes a web server, you can interact with it via simple HTTP requests. DynamoDB is just a database, so you would need to connect directly through the AWS SDK, or build an API in front of it, possibly using API Gateway and Lambda.
In addition, since Parse is a full-featured Backend as a Service, and DynamoDB is only a database, there are some features in Parse that won't be available if you just use DynamoDB directly from your iOS application. For example user password resets require sending an email to the user. DynamoDB has no "password reset" functionality and can't send emails directly. You would have to build that feature yourself using something like Lambda and SES.
Parse also handles file upload and file hosting, which are features you would no longer have if you just used DynamoDB directly from iOS. You would have to build those features yourself, possibly using S3.
If you are only using Parse as a data store then using DynamoDB directly could certainly work for you, but then again so could MongoDB or any other NoSQL database. You should definitely explore how your database schema would look in DynamoDB before committing to it, because there are certain restrictions on index types and query types that might make it difficult to transition your current schema.
AWS + DynamoDB would be your way to go.
I worked extensively in both, DynamoDB and MongoDB systems and can give you a short summary of an advise.
MongoDB is very easy to work with and has unmatched flexibility in query structure, requires very little thinking ahead of setting up the system.
DynamoDB will provide unmatched scalability, much stricter (very strict) set of rules for creating schemas and requires a lot of planning before you do the setup. However, you don't need to worry about setting up or managing database environment, no worry about master/slave architecture and no concerns of scaling your database.
I go with DynamoDB these days and it's been great.
Just completed a migration from Parse to AWS Dynamo (a few thoughts were posted here: https://www.linkedin.com/pulse/parse-aws-migration-server-less-mobile-backend-mike-kirkwood?trk=prof-post
My experience was that DynamoDD was an acceptable replacement for much of Parse. However, it required some data model changes as DynamoDB doesn't support Pointers or Relationships like Parse did. So, in the app had to adjust some of the writes to add more data to the record in DynamoDB. This did offer some nice benefits in the queries.
DynamoDB also allows you to add indexes to match specific queries.
And, for my use, DynamoDB has proven to be much faster queries than Parse was.
DynamoDB is just a database service, so you can use it to store Parse data but you'll still need a server to process the data and host APIs, etc... On AWS, you could spin up an EC2 instance to run the server, or try to make it run on Lambda.
Parse Server does not natively support either Lambda as a hosting environment or DynamoDB as a storage backend, but fortunately members of the community have recently developed integration for both of these:
https://www.npmjs.com/package/parse-server-dynamodb-adapter
https://github.com/parse-community/parse-server/issues/483
I need a backend to store location updates and messages, I was thinking of using JSON to connect to the Amazon S3 server and to fetch and store data.
How many clients could be connected to this server? Is there a way to link a MYSQL server to Amazon S3 for login and users accounts?
S3 is not a database store; you write/delete/replace an entire object.
You want AWS RDS. Amazon manages the DB (MySQL supported). Skim the reference architectures for something applicable to your needs. Scale them down; they're designed to make use of as many AWS services as possible.
http://aws.amazon.com/rds/
http://aws.amazon.com/architecture/
Other option is Amazon Dynamo DB. This is an infinite-scale nosql db with a fully managed REST API. You dont worry about the data size growth, speed etc. AWS take care of all these.
http://aws.amazon.com/dynamodb/.
Even in this case, you need to have some code running in the backend, which receives your REST calls from the iOS and writes to the Dynamo.
Other even easier solutions are https://parse.com/ and https://www.firebase.com/
These are solutions specifically for your kind of needs - Make a mobile backend Datastore. They give client SDK, which has a very great value in terms of offline synch. You just invoke the SDK from the apps and will synch with the backend datasore when the connections are available - reduces your code complexity a lot !
I'm still new to GAE, and I would like to have more wisdom about couple of things.
I searched in documentation, but I think I'm just too stupid to understand some things from documentation.
How can I combine Google Cloud SQL with endpoints? Is there such possibility?
How can I use endpoints to upload videos to google platform?
You should be able to use anything you can do on a non cloud endpoints api to cloud endpoints like google cloud sql. But since it's on preview you might encounter bugs/changes when it goes out of preview. You should create a Cloud SQL tests models on regular app engine app then try to use it on cloud endpoints, so you can minimize debugging for errors.
https://developers.google.com/appengine/docs/python/cloud-sql/
You will need to use a blobstore api:
https://developers.google.com/appengine/docs/python/blobstore/
on your endpoints have a method that creates the upload url and use that to upload from your app then on the uploadHandler it will trigger once the whole file has been uploaded, process your blobInfo key store it appropriately.
For iOS developers, we have two options to user Amazon S3 service with
ASIHttpRequest S3
AWS SDK for iOS
Seem like they are doing the same thing to provide interface to use S3 service.
My question is what is the difference between them if have ?
Thanks
AWS SDK supports more options for IAM (Identity and Access Management), policies and STS (temporary or "federated" users). Additionally, it provides support for services other than S3.
If you just need to move data to/from S3 I believe ASI is the way to go. If you're doing anything that uses other AWS services, or requires anything beyond standard ID/SECRET authentication, the SDK is the way to go.
I have used ASI in a large application for access to S3 and SimpleDB and a few other things. I found that there are a few bugs, etc. Performance is good, though. So I am trying the AWS-sdk for next project. I think in the long run that the official SDK will be better, as Amazon appears to spend money on supporting (and writing?) it.