For my internship, I need to implement a blockchain based solution to manage a drug supply chain. The management of this supply chain implies to track-and-trace (geolocate) a drug on the chain, but also to monitor the storage temperature to see if the cold chain is respected. For that I also intend to use IOT, where a device will feed information on the blockchain solution. However, I have a few questions that I can't find easier.
The first one is that I don't know if I should use ethereum or not, since each time that a new block is added (the block representing the update on the information about the product on "real-time") I will to use money. Is there any solution for that? Or do I need to create a blockchain with javascript?
The second question is that I absolutely don't know from where to begin in order to implement IOT on the block chain. I searched on research site, but they only talk about it, without presenting any example...
The third one is more a confirmation than a question since I want to know if my idea to use an IOT to track and manage products on a supply chain can be done on a wide scale, since the bigger a blockchain the slower is the time to add a block because of the consensus mechanism. So it means that my "real time" tracking on truly be "on time" since there would be a waiting time before the block is added to the blockchain. If the time is just a few seconds to minutes, then there is no problem, but because the number of block will rapidly increase because of the real time tracking (1 block each minute for each storage or transport vehicles I was planning for) that this problem of scalability makes it impractible.
I'm thanking anybody in advance who will help me solve these questions.
#1) Whether you use Ethereum or another blockchain that may be better suited for this purpose is completely up to you. I expect you will get a lot of opinionated answers on this. Ethereum is certainly the most popular blockchain for a use like this, but that does not mean it is the best for your app. Over the last several years we have seen many new blockchains with lower/no fees, faster block times, and increased scalability. I would suggest doing some research into various "Supply Chain", "Enterprise", and "Business" blockchains, as these are likely the type you are looking for and will cost very little in blockchain fees due to them not being as widely used as Ethereum is.
#2) You will have to settle on a blockchain before you can start prototyping or looking for examples, as each one will be different. For storing "log" data for your application there are generally 2 options: Storing data in a smart contract on Ethereum (or an Ethereum-like blockchain), or storing data in the OP_RETURN field of a transaction on Bitcoin or a bitcoin-based blockchain. The latter is likely easier to get started with and is simpler to understand, you just put the data in a transaction and send it (to yourself, even).
#3) Yes, there a special purpose blockchains created exactly for this purpose that are meant to ingest large amounts of data and that can scale to meet the needs of an application like you describe. Some blockchains have block times of 1 minute or less meaning that on average, you could update the data every minute if you were willing and able to pay the blockchain fee to include the data in each new block (personally I would suggest a longer interval, such as every 5-10 minutes).
You can use Emercoin NVS technology and Emercoin (testnet) blockchain to upload you data into it. By creating some "name label" with name_new command, you can thereafter upload chain of modifications for values of your name. Blockchain has command name_show (shows recent uploaded value) and name_history (shows all chain of uploaded values). You can view/debug your uploaded values within Emercoin Testnet Explorer, tab NVS.
Regarding "use money".
I can give you (or anyone else) 100 test EMCs, for free. Just write your tEMC address in comments here. 100tEMCs will be enough for ~100,000 records within Testnet. Thus, I think, it is more than enough for your test tasks.
If you need to use your service for production, you need to use the "real blockchain" (with high trust), no testnet. Anyway, EMCs are very cheap right now, and you can buy 100EMSs for ~$5 only. I think, this is not big deal for your organization.
Ask more questions, and I will be happy to assist you here with this technology.
Because you have a permissioned / private environment that does not transfer value or store value, a blockchain specialised in value transfer, like Ethereum, is not a good choice.
Choosing a blockchain that is specialised in untrusted value writes simplies creating your product a lot. Some good choices include:
BigchainDB
HyperLedger SawTooth - comes with a stock supply chain example
Related
I'm having some difficulties thinking about a good way to assess the scalability of my school project. The assignment is simply put a twitter-service. Very bare bones. The main goal is to make it as scalable as possible. However, it's equally important to know how and why the assignment would scale. So the point is not really to create a very scalable project, but mainly to create a few different architectures for the server and see which one outperforms which one and why.
So so far I have the basic server architecture like this:
* 1 main process which holds the data
* 1 unique process server-side per user
A user sends messages to his own server-side process, which then simply delegates those messages to the central process.
To test this I would spawn 1 or more processes which would act as clients. I would spam the server with tweets and then assess how well it can withstand a certain load.
Now, to assess the scalability I came up with following metrics:
First off, a process is being a bottleneck if it's message queue is piling up. So, I would store the queuelength every time a tweet is processed (or every N tweets) and in the end calculate the average. If I run it on more cores and the average queue length goes down, it scales better.
Second, if I create N users to spam the server (on N processes or less) I simply time how long it takes for the server to process all these tweets.
Is there a better way to do this? I can't stop thinking that there should be better metrics..
Edt:
So far I have tried fprof and eprof. These tools however, show me how much time is spent in certain methods. While this is a good indicator where I can improve my code, it's not really a good indicator for scalability. It would be better if it would for example show the time spent per process.
Look at percept and percept2 if you are really interested in this.
I am trying to build an offline synchronization capability into my iOS App and would like to get some feedback/advice from the community on the strategy and best practice to be followed to do the same. The app details are as follows:
The app shows a digital catalog to users and allows them to perform actions like creating and placing orders, among others.
Currently the app only works when online, and we have APIs for all actions like viewing the catalog, creating/placing orders which return JSON data.
We would like to provide offline/synchronization capability to users, through which users can view the catalog and create/place orders while offline, and when they come online the order details will be synchronized and updated to our server.
We would also like to pull the latest data from the server, and have the app keep itself up to date in case of catalog changes or order changes that happened at the Server while the app was offline.
Can you guys help me to come with the best design and approach for handling this kind of functionality?
I have done something similar just in the beginning of this year. After I read about NSOperationQueue and NSOperation I did a straight forward approach:
Whenever an object is changed/added/... in my local database, I add a new "sync"-operation to the queue and I do not care about, if the app is online or offline (I added a reachability observer which either suspended the queue or takes it back working; of course, I do re-queueing if an error occurs (lost network during sync)). The operation itself reads/writes the database and does the networking stuff. My ViewController use a NSFetchedResultsController (with delegate=self) to get callbacks on changes. In some cases I needed some extra local data (it is about counting objects), where I have used NSManagedObjectContextObjectsDidChangeNotification.
Furthermore, I have used Multi-Context CoreData which sounded quite reasonable to use (I have only two contexts).
To get notified about changes from your server, I believe that iOS 7 has something new for you.
On the server side, you should read a little for the actual approach you want to go for: i.e. Data Synchronization by Dan Grover or Developing Android REST Client Applications (of course there are many more good articles out there).
Caution: you might be disappointed when you expect an easy solution. Your requirement is not unusual, but the solution might become more complex than you expect - depending on the "business rules" and other reasonable requirements. If you intelligently restrict your requirements you may find a solution which you can implement yourself, otherwise you may also consider to use a commercial product.
I could imagine, that if you design the business logic such that it takes an offline state into account and exposes this explicitly in the business logic, you may find a solution which you can implement yourself with moderate effort. What I mean by this is for example, when a user creates an order, it is initially in "not committed" stated. The order will only be committed when there is access to the server and if the server gives the "OK" that this order can actually be placed by this user. The server may also deny the order, sending corresponding messages to the user.
There are probably quite a few subtle issues that may arise due to the requirement of eventual consistency.
See also this question which contains pointers to solutions from commercial products, and if you visit their web sites give valuable information about the complexity of the problem and how this can be solved.
Valence allows to do this, but I wondered if there's limitations to trying to automate user account setup and enrollment in this way. We're a relatively small institution with hundreds of enrollments perhaps in a given term scattered over several weeks, so I don't think there'd be a problem with realtime events. But I wondered what the implication might be for a larger university that might have thousands of enrollments updating all the time. There'd be spikes of activity certainly as a term reached official start.
The issue of performance is a complex one, with many inter-dependant factors. I can't comment on the impact of hardware hosting your back-end LMS, but obviously, higher-performance hardware and network deployment will result in higher performance interaction between clients and the LMS.
If your needs are "hundreds of creations scattered over several weeks", that is certainly within comfortable expected performance of a back-end service.
At the moment, each user-creation request must be done separately, and if you want to pre-provide a password for a user, then it takes two calls (you can, in one call, nudge the LMS to send a "password set" email to the new user, but if you want to manually set the password for the user, you have to do that in a separate call after the user record gets created). The individual calls are pretty light-weight, being a simple HTTP POST with response back to provide the created user record's details.
There are roadmapped plans to expand support for batch operations, and batched user-creation is certainly a logical possibility for improvement.
Is there an option in DynammoDB to store auto incremented ID as primary key in tables? I also need to store the server time in tables as the "created at" fields (eg., user create at). But I don't find any way to get server time from DynamoDB or any other AWS services.
Can you guys help me with,
Working with auto incremented IDs in DyanmoDB tables
Storing server time in tables for "created at" like fields.
Thanks.
Actually, there are very few features in DynamoDB and this is precisely its main strength. Simplicity.
There are no way automatically generate IDs nor UUIDs.
There are no way to auto-generate a date
For the "date" problem, it should be easy to generate it on the client side. May I suggest you to use the ISO 8601 date format ? It's both programmer and computer friendly.
Most of the time, there is a better way than using automatic IDs for Items. This is often a bad habit taken from the SQL or MongoDB world. For instance, an e-mail or a login will make a perfect ID for a user. But I know there are specific cases where IDs might be useful.
In these cases, you need to build your own system. In this SO answer and this article from DynamoDB-Mapper documentation, I explain how to do it. I hope it helps
Rather than working with auto-incremented IDs, consider working with GUIDs. You get higher theoretical throughput and better failure handling, and the only thing you lose is the natural time-order, which is better handled by dates.
Higher throughput because you don't need to ask Dynamo to generate the next available IDs (which would require some resource somewhere obtaining a lock, getting some numbers, and making sure nothing else gets those numbers). Better failure handling comes when you lose your connection to Dynamo (Dynamo goes down, or you are bursty and your application is doing more work than currently provisioned throughput). A write-only application can continue "working" and generating data complete with IDs, queueing it up to be written to dynamo, and never worry about ID collisions.
I've created a small web service just for this purpose. See this blog post, that explains how I'm using stateful.co with DynamoDB in order to simulate auto-increment functionality: http://www.yegor256.com/2014/05/18/cloud-autoincrement-counters.html
Basically, you register an atomic counter at stateful.co and increment it every time you need a new value, through RESTful API.
I'm familiar with using Reachability to determine the type of internet connection (if any) being used on an iOS device. Unfortunately that's not a decent indicator of connection quality. Wifi with low signal strength is pretty sketchy and 3G with anything less than 3 bars is a disaster (not to mention networks that only allow EDGE connections).
How can I determine the quality of my connection so I can help my users decide if they should be downloading larger files on their current connection?
A pragmatic approach would be to download one moderately large-sized file hosted on a reliable, worldwide CDN, at the start of your application. You know the filesize beforehand, you just have to measure the time it takes, make a simple computation and then you've got your estimate of the quality of the connection.
For example, jQuery UI source code, unminified, gzipped weighs roughly 90kB. Downloading it from http://ajax.googleapis.com/ajax/libs/jqueryui/1.8.14/jquery-ui.js takes 327ms here on my Mac. So one can assume I have at least a decent connection that can handle approximately 300kB/s (and in fact, it can handle much more).
The trick is to find the good balance between the original file size and the latency of the network, as the full download speed is never reached on a small file like this. On the other hand, downloading 1MB right after launching your application will surely penalize most of your users, even if it will allow you to measure more precisely the speed of the connection.
Cyrille's answer is a good pragmatic answer, but is not really in the end a great solution in the mobile context for these reasons:
It involves doing a test "at the start of your application" by which I assume he means when your app launches. But your app may execute for a long while, may go background and then back into the foreground, and all the while the user is changing network contexts with changes in underlying network performance - so that initial test result may bear no relationship to the "current" performance of the network connection.
For the reason he rightly points out, that it is "penalizing" your user by making them download a test file over what may already be constrained network conditions.
You also suggest in your original post that you want your user to decide if they should download based on information you present to them. But I would suggest that this is not a good way to approach interacting with mobile users - that you should not be asking them to make complicated decisions. If absolutely necessary, only ask if they want to download the file if you think it may present a problem, but keep it that simple - "Do you want to download XYZ file (100 MB)?" I personally would even avoid even that.
Instead of downloading a test file, the better solution is to monitor and adapt. Measure the performance of the connection as you go along, keep track of the "freshness" of that information you have about how well the connection is performing, and only present your user with a decision to make if based on the on-going performance of the connection it seems necessary.
EDIT: For example, if you determine a patience threshold that in your opinion represents tolerable download performance, keep track of each download that the user does in order to determine if that threshold is being reached. That way, instead of clogging up the users connection with test downloads, you're using the real world activity as the determining factor for "quality of the connection", which is ultimately about the end-user experience of the quality of the connection. If you decide to provide the user with the ability to cancel downloads, then you have an excellent "input" about the user's actual patience threshold, and can adapt your functionality to that situation, by subsequently giving them the choice before they start the download. If you've flipped into this type of "confirmation" mode, but then find that files are starting to download faster, you could dynamically exit the confirmation mode.
Rob's answer is very good, but for a more specific implementation start with (https://developer.apple.com/library/archive/samplecode/SimplePing/Introduction/Intro.html#//apple_ref/doc/uid/DTS10000716)Apple's Simple Ping example source code
Target the domain for the server that you want to monitor connection quality to. Use the ping library to "ping" it on a regular basis (say 1 or 10 seconds depending upon your UI needs). Measure how long it takes to get a response to your ping (or if it never returns) to develop an estimate of the connection quality to communicate to your user.