How to efficiently store large amount of addresses in a smart contract using solidity? - storage

I am developing a smart contract where users need to deposit funds. Something similar to the DeFi projects. I am still new in solidity development and was wondering can you recommend to me an efficient way to store those user addresses? I found a similar question where I understood that storing them in an array is not very efficient, because when I iterate through it, it will be very costly. I saw other recommendations for using Maps, which are good alternatives, however, I don't know whether they will solve my issue.
My idea is to create a smart contract -> store user's addresses who interact/deposit funds -> transfer those addresses to another smart contract that is going to pay interest, etc.
I assume most of the DeFi projects should have resolved that, because they need to store the addresses of their users, so can you give me some tips on how it is done?

You could use the combinations of mapping. Mapping is like an object in javascript or a dictionary in python.
You want to keep track of addresses that deposits. for this, you might still need to keep them in an array for different purposes.
// keep private and then set a getter.
address[] private fundersAddresses;
also set a mapping to keep track of the index to address. for this, I can also set a variable to keep track of index of array
// you could call fundersAddresses.length and it's time complexity most likely O(1) but since I am not 100% sure, I set up a index variable
uint256 private index;
mapping(uint256=>address) private indexToAddress
Now if you write a function to store the funders
function storeFunders() public payable {
// add some logic
fundersAddresses.push(msg.sender);
index++;
// then also store in a mapping
indexToAddress[index]=msg.sender
}
Now if you want to get the i'th item in an array, it will take O(1) time because you will look up the mapping instead of calling array[i]. array[i]'s time complexity is O(n) because ethereum engine would iterate over array till the i'th index.
function getIndexedAddress (uint index) public returns(address){
require(index<fundersAddresses.length,"Index out of bounds")
return indexToAddress[index]
}
This is just a simple example. Based on the needs of your contract you might set up different structure.

If the addresses are final at the deployment, you can use a Merkle tree.

Related

Efficiently get state from Orleans Grain

I have a grain in Orleans for the players of a game. A player has several properties that I want to access directly in the client. Is it possible, is it efficient and does it make sense to have these as public properties on the grain? Or, should I have a GetAllState method that returns a DTO with the current value of these properties in the grain?
public interface IPlayerGrain : IGrainWithIntegerKey
{
// Individual public properties to access grain state?
string Name { get; }
int Score { get; }
int Health { get; }
// Or, get all the current grain state as DTO?
Task<PlayerState> GetAllState();
}
From my current understanding I think I will need to use GetAllState as I think any communication into the grain needs to be via a method and this may pass between silos. So, you probably want to minimise the number of messages passed and wouldnt want to pass three messages to get Name, Score and Health. Or, is message passing pretty cheap and not something I should worry about doing too much? In my example I've only included 3 properties, but in my real game there will be many more.
However, I don't really like the idea of having an anemic DTO model that is just a copy of the grain's internal properties.
So I was wondering if there was a better way, or a preferred pattern for this sort of thing in Orleans?
I think this depends a lot on the life cycle and access patterns of the properties. Do the properties tend to change independently or together? (At first glance, they seem to change independently and at quite different rates; I assume that Score and Health can change very frequently, but Name would almost never change.) And given that Name changes very infrequently, would it be a good fit for your access patterns to retrieve it every time you wanted an updated Score or Health value? Maybe Score and Health would be frequently accessed together, and Name along with other more static properties also belong together.
If you let these kinds of questions drive your API before you think about the cost of message passing, you will probably find some good sweet spots (probably not the whole state of the grain). You might also consider having a Player grain and a PlayerStats grain that have different life cycles and that correspond more closely to the change rate of various pieces of data.
Introduction of a complex return type to minimize roundtrips is a valid solution.
However, I wouldn't return the whole internal state, as I assume that not all the clients need all the data all the time. It may also be a sign that you have a business logic implemented outside of the grains and you should move it into the grain.
You might also consider that Health and Score, which are likely to change frequently, be exposed in a stream.

Parse iOS backend - Objects, Arrays, and User data advice

I hope this isn't an inappropriate post, but I wanted to make sure my first steps implementing parse as my backend are in the right direction to save some time. I'm new to both iOS programming and the parse sdk, so please bear with me!
In my app, users are able to create various polygon shape overlays on a Google Maps mapView, stored as a GMSMutablePath, which is basically a list of coordinates. Users will have at least one group of paths, each with at least one path. There will also be some information stored with each group, stored as strings or numbers. This information is specific to a single group of paths.
I'm trying to figure out the best way to store this data. My first basic question is 1) Can I store the GMSMutablePath as a whole in the Object data type? Or does the Object data type refer to a class that is created through parse? This link (https://www.parse.com/questions/what-is-data-type-of-object-in-data-browser) is the 'best' explanation I found of the Object data type, and it isn't very clear to me.
My gut instinct is no, I can't store the GMSMutablePath object, and that Object refers to a Parse object. Which leads me to 2) How should I store this data, then? I can get the individual lat/long values of the coordinates that make up each path, and I can store those as numbers, and use the numbers to recreate the paths elsewhere. None of the paths should use too many coordinates, and there shouldn't be too many paths in each group.
Playing around a little bit in the data browser, I see that I can store arrays, but I'm not sure how those are formatted, as I'd need an array (of groups) of arrays (of paths) of arrays (of lat/long values). A little bit of googling tells me it can be done, but doesn't show me how. Can any datatype be stored in any array, or is a datatype specified? I'm used to C++ programming, so I'm used to an array containing a single type of element. What I'm thinking is that I'd need an array of objects, which would be the groups of paths. Each one of those objects would have the string/number information associated with the group, as well as an array for the paths within the group. For each one of those paths, it would have to be either an array or an object. Since for the path I just need the coordinate lat/long values, I think that I can get away with an each path being an array of numbers, and I can write my program to use one array, with odd indexes being lat / even indexes being long values. That all being said, I'm not sure how to create all of that. I'm not looking for somebody to write my implementation for me, but all of the examples I can find are much more simple... if anybody could point me in the right direction to do this, or has a better idea of how to do it, I'd love some pointers.
Each user is going to have their own groups, but that data is going to be shared with others at some point. The data will be associated with the user it belongs to. With that in mind, my last question is 3) Should I store all of this information specific to a user and their groups on the User class, or make it all a separate class entirely? My guess it that I should add an Object to the User class, and store the groups within that Object. I just want to make sure I have that right, with future scalability in mind. Like, when I pull the group data, am I going to have to pull the entire User data from another user, and if so, is that going to slow things down significantly? I'm thinking that I do have to send entire user data, but I don't know if that poses any security risks. Would it be best to have a separate class for the groups, and store the user id associated with the groups? If I do this, should I also store the groups as an object on the User class?
Sorry for the wall of text, but thank you for any guidance you can provide!
If you need any clarification, let me know.
Thanks,
Jake
Creating a class to hold all the objects turned out to be unnecessary. It only had a few extra details that were just as convenient to add to the user object, and then have an array of objects on the user.
Some main things to note that I learned are: use addObject to add to an array, rather than setObject to add a single object to a PFObject/User.
Parse fetching/saving happens in background threads, so if you're loading the data to do something specific with it, make sure the code using the data occurs inside a block using the [PFObject fetchInBackgroundWithBlock] method.
Also, as changes are made to the structure of your data on a parse user/object, make sure you sign out of the current user and create a new one on your app, or you may run into lots of undefined behaviour that could crash your app.

What kind of sort does Cocoa use?

I'm always amazed by the abstractions our modern languages or frameworks create, even the ones considered relatively low level such as Objective-C/Cocoa.
Here I'm interested in the type of sort executed when one calls sortedArrayUsingComparator: on an NSArray. Is it dynamic, like analyzing the current constraints of the environment (particularly free memory) and the attributes of the array (length, unique values), and pick the best sort accordingly, or does it always use the same one, like Quick or Merge Sort?
It should be possible to test that by analyzing the running time of the method relatively to N, just wondering if anyone already bothered to.
This has been described at a developers conference. The sort doesn't need any memory. It checks if there is a sorted range of numbers at the start or the end or both and takes advantage of that. You can ask yourself how you would sort an 100,000 entry array if the first 50,000 are sorted in descending order.

Caching Consistency vs. Static Object with nHibernate/ASP.NET

I am a complete newbie to both caching, nhibernate, and everything involving the two, so this question may be excessively stupid.
I have certain instances of objects that are used by multiple other objects in my system. So for instance..
class Template {
// lots of data
}
class One {
IList<Template> Templates { get; set; }
}
class Two {
IList<Template> Templates { get; set; }
}
class Three {
IList<Template> Templates { get; set; }
}
Now, then, certain instances of Template are going to be used very, very frequently. (think like, every 20 seconds) and it includes a lot of things that need to be mathematically computed.
My question is basically which approach will yield the least stress on my database/server.
Am I best to just leave everything to Level 2 Caching in nHibernate? Or am I wiser to retrieve the Template object and store it in a static variable when my ASP.NET application starts up, and refresh this variable if it changes?
I've looked at some of the other similar questions around SO but I am still very much in the dark. Most of the documentation on caching assumes a good deal of knowledge on the subject, so I'm having a difficult time discerning what the optimal process is.
once every 20 second doesn't really sound very stressful. You need to weight the need for updated data vs the stress you can live with on your database.
2nd level cache won't necessarily help you in this case, since you use collections of objects. In order to know which object it needs, it still need to query the database, and if you do that it might even fetch the data anyway (unless it's a lot of raw data in the entities).
You basically have three different options:
1st level cache
For each connection/session that you make, NHibernate will always cache the unique entity that it has fetched. Every time you try to get a single entitity based in it's identifier (primary key), it will first check it's first level cache. This does not apply to collections of entities though, unless you can force NHibernate to only get "identifiers" for the collection and the get them one by one (usually very slow)
2nd level cache
This cache will available for each and every connection/session, and try to fetch the data from cache before it hits the database. Same rules apply as for the 1st level cache, that you can't get collections to an entity without querying the database unless it has already been loaded.
custom cache
You can always take care of caching your self, however, that way you need to model your classes accordingly (having Template objects stored, and the collections only keep track of the identifier instead of Template objects). If you refactor like this, 2nd and 1st level cache would still be equally useful though.
I will give you an example that shows you what I'm talking about:
if One contains templates with identifier [1,2,3,4]
Two contains templates with identifier [2,3]
Three contains templates with identifier [3,4,5]
In order for NHibernate to know that One needs templates 1,2,3,4, it needs to query the database. 1,2,3,4 will be cached individually here.
In order to actually know that Two needs entity 2 and 3, it still needs to query the database. It can't possibly know that 2,3 is also part of the collection in Two. Si it won't fetch them from cache, because it will select Template objects that belongs to class Two, hence full data. That is why caching won't help you here.
I think you need to give more details on what kind of data it is that you will be processing, and how it will be stored and updated in order to get an answer that is useful.
Static variables would be the less stress on your server, however that imposes some restrictions, specifically, it would be much harder to scale (web garden/farm), if you don't need to scale, that's the option you're looking for

Help me understand mnesia (NoSQL) modeling

In my Quest to understanding Mnesia, I still struggle with thinking in relational terms. So I will put my struggles up here and ask for the best way to solve them.
one-to-many-relations
Say I have a bunch of people,
-record(contact, {name, phone}).
Now, I know that I can define phone to always be saved as a list, so people can have multiple phone numbers, and I suppose that's the way to do it (is it? How would I then look this up the other way around, say, finding a name to a number?).
many-to-many-relations
now let's suppose I have multiple groups I can put people in. The group names don't have any significance, they are just names; the concept is "unix system groups" or "labels". Naively, I would model this membership as a proplist, like
{groups [{friends, bool()}, {family, bool()}, {work, bool()}]} %% and so on...
as a field within the "contact" record from above, for example. What is the best way to model this within mnesia if I want to be able to lookup all members of a group based on group name quickly, and also want to be able to lookup all group an individual is registered in? I also could just model this as a list containing just the group identifiers, of course. For use with mnesia, what is the best way to model this?
I apologize if this question is dumb. There's plenty of documentation on mnesia, but it's lacking (IMO) some good examples for the overall use.
For the first example, consider this record:
-record(contact, {name, [phonenumber, phonenumber, ...]}).
contact is a record with two fields, name and phone where phone is a list of phone numbers. As user425720 said it could make sense to store these as something else than strings, if you have extreme requirements for small storage footprint, for example.
Now here comes the part that is hard to "get" with key-value stores: you need to also store the inverse relationship. In other words, you need something similar to the following:
-record(phone, {phonenumber, contactname}).
If you have a layer in your application to abstract away database handling, you could make it always add/change the phone records when adding/changing a contact.
--
For the second example, consider these two records:
-record(contact, {uuid, name, [group_id, group_id]}).
-record(group, {uuid, name, [contact_id, contact_id]}).
The easiest way is to just store ids pointing to the related records. As Mnesia has no concept of referential integrity, this can become out of sync if you for example delete a group without removing that group from all users.
If you need to store the type of group on the contact record, you could use the following:
-record(contact, {name, [{family, [group_id, group_id]}, {work, [..]}]}).
--
Your second problem could also be solved by using a intermediate record, which you can think of as "membership".
-record(contact, {uuid, name, ...}).
-record(group, {uuid, name, ...}).
-record(membership, {contact_uuid, group_uuid}). # must use 'bag' table type
There can be any number of "membership" records. There will be one record for every users group.
First of all, you ask for key-value store design patters. Perfectly fine.
Before I will try to answer your question lets make it clear - what is Mnesia. It is k-v DB, which is included in OTP. Because it is native, it is very comfortable to use from Erlang. But be careful. This is old database with very ancient assumptions (e.g. data distribution with linear hashing). So go ahead, learn and play with it, but for production take your time and browse NoSQL shop to find the best for your needs.
#telephone example. Do not store stuff as strings (list()) - it is very heavy for GC. I would make couple fields like phone_1 :: < < binary > > , phone_2 :: < < binary > >, phone_extra :: [ < < binary > > ] and build index on the most frequent query-field. Also mnesia indicies are tricky - when node crashes and goes up, they need to rebuild themselves (it can take awfully lot of time).
#family example. It quite hard with flat namespace. You may play with more complex keys.. Maybe create separate table for TheGroup and keep identifiers of members? Or each member would have ids of groups he belongs (hard to maintain..). If you want to recognize friends I would implement some sort of contract before presenting data (A is B's friend iff B is A's friend) - this approach would cope with eventual consistency and conflicts in data.

Resources