I am using total count of devices as the "server attributes" at customer entity level that is in turn being used for Dashboard widgets like in "Doughnut charts". Hence to get the total count information, I have put a rule chain in place that handles "Device" Addition / Assignment event to increment the "totalDeviceCount" attribute at customer level. But when the device is getting deleted / UnAssigned , I am unable to get access to the Customer entity using "Enrichment" node as the relation is already removed at the trigger of this event. With this I have the challenge of maintaining the right count information for the widgets.
Has anyone come across similar requirement? How to handle this scenario?
Has anyone come across similar requirement? How to handle this scenario?
What you could do is to count your devices periodically, instead of tracking each individual addition/removal.
This you can achieve using the Aggregate Latest Node, where you can indicate a period (say, each minute), the entity or devices you want to count, and to which variable name you want to save it.
This node outputs a POST_TELEMETRY_REQUEST. If you are ok with that then just route that node to Save Timeseries. If you want an attribute, route that node to a Script Transformation Node and change the msgType to POST_ATTRIBUTE_REQUEST.
Related
regarding to asked question here :
suppose that we have ProductCreated and ProductRenamed events which both contain the title of the product.now we want to query EventStoreDB for all events of type ProductCreated and ProductRenamed with the given title.i want all these events to check whether there is any product in the system which has been created or renamed to the given title, so that i could throw the exception of repetitive title in the domain
i am using MongoDB for creating UI reports from all the published events and everything is fine there.but for checking some invariants, like checking for unique values, i have to either query the event store for some events along with their criteria and by iterating over them, decide whether there is a product created with the same title which has not renamed or a product renamed with the same title.
for such queries, the only way that event store provides is creating a one-time projection with the proper java script code which filters and emits required events to a new stream.and then all i have to do is to fetch events from the new generated stream which is filled by the projection
no the odd thing is, projections are great for subscriptions and generating new streams, but they seem to be odd for doing real time queries.immediately after i create a projection with the HTTP api, i check the new resulting stream for the query result, but it seems that the workers has not got the chance to elaborate on the result and i get 404 response.but after waiting for a bunch of seconds, the new streams pops out and gets filled with the result.
there are too many things wrong with this approach:
first, it seems that if the event store is filled with millions of events across many streams, it wont be able to process and filter all of them immediately to the resulting stream.it does not create the stream immediately, let alone the population.so i have to wait for some time and check for the result hoping the the projection is done
second, i have to fetch multiple times and issue multiple GET HTTP commands which seems to be slow.the new JVM client is not ready yet.
Third, i have to delete the resulting stream after i'm done with the result and failing to do so will leave event store with millions of orphan query result streams
i wish i could pass the java script to some api and get the result page by page like querying MongoDB without worrying about the projection, new streams and timing issues.
i have seen a query section in the Admin UI, but i dont know whats that for, and unfortunetly the documentation doesn't help much
am i expecting the event store to do something that is impossible?
do i have to create a bounded context inner read model for doing such checks?
i am using my events to dehyderate the aggregates and willing to use the same events for such simple queries without acquiring other techniques
I believe it would not be a separate bounded context since the check you want to perform belongs to the same bounded context where your Product aggregate lives. So, the projection that is solely used to prevent duplicate product names would be a part of the same context.
You can indeed use a custom projection to check it but I believe the complexity of such a solution would be higher than having a simple read model in MongoDB.
It is also fine to use an existing projection if you have one to do the check. It might be not what you would otherwise prefer if the aim of the existing projection is to show things in the UI.
For the collection that you could use for duplicates check, you can have the document schema limited to the id only (string), which would be the product title. Since collections are automatically indexed by the id, you won't need any additional indexes to support the duplicate check query. When the product gets renamed, you'd need to delete the document for the old title and add a new one.
Again, you will get a small time window when the duplicate can slip in. It's then up to the business to decide if the concern is real (it's not, most of the time) and what's the consequence of the situation if it happens one day. You'd be able to find a duplicate when projecting events quite easily and decide what to do when it happens.
Practically, when you have such a projection, all it takes is to build a simple domain service bool ProductTitleAlreadyExists.
I got a problem when designing a graph model with million users. I need to store information that user is registered or non-register.
As I see we have 2 options:
Store a property "register = true/false" in each user node. So with 1 million user, we have 1 million properties "register".
Store a Registered node then make relationship just for registered user to this node. So we have number of relationship equal exactly with the registered user.
Which option is better in performance searching also about minimum storage?
Thanks in advance,
Modeling your data as a graph is a difficult thing to pin down exactly. Typically, when it comes to NoSQL databases, the most important thing to consider is how you will be using your data, and to model it based on that.
Using the external node might run into performance problems, as Neo4J typically starts to run into issues during traversing as it approaches around 10,000 relationships in a single node. You will be well above that limit with an external "Registered" node; on the other hand as long as you are not anchoring your search to that node, it should be okay.
No matter which route you go, the query you described in the comments will likely anchor on (start with) the user, then traverse to who their friends are, and then for each friend, it will check whether it
A. has the "registered" property set to 'true'
B. has a relationship to the "Registered" node.
Each of these methods appears to have a similar execution time, and indexing on the "registered" property will have negligible impact because it is not being used as an anchor (presumably; you would have to PROFILE your query with both methods to find out for sure). So, like you mentioned, one might consider the space restraints.
Besides that, there is not much difference from a performance analysis perspective between the two methods that I can see.
A third option, mentioned by #InverseFalcon, is to use an additional label, ':Registered' on those nodes that are registered. This might well result in a faster comparison time than keeping it in a property, as labels will be inlined in the node store and can be checked there, whereas properties might have an additional level of indirection to the property store.
I build a social network with Neo4j, it includes:
Node labels: User, Post, Comment, Page, Group
Relationships: LIKE, WRITE, HAS, JOIN, FOLLOW,...
It is like Facebook.
example: A user follow B user: when B have a action such as like post, comment, follow another user, follow page, join group, etc. so that action will be sent to A. Similar, C, D, E users that follow B will receive the same notification.
I don't know how to design the data model for this problem and I have some solutions:
create Notification nodes for every user. If a action is executed, create n notification for n follower. Benefit: we can check that this user have seen notification, right? But, number of nodes quickly increase, power of n.
create a query for every call API notification (for client application), this query only get a action list of users are followed in special time (24 hours or a 2, 3 days). But Followers don't check this notification seen or yet, and this query may make server slowly.
create node with limited quantity such as 20, 30 nodes per user.
Create unlimited nodes (include time of action) on 24 hours and those nodes has time of action property > 24 hours will be deleted (expire time maybe is 2, 3 days).
Who can help me solve this problem? I should chose which solution or a new way?
I believe that the best approach is the option 1. As you said, you will be able to know if the follower has read or not the notification. About the number of notification nodes by follower: this problem is called "supernodes" or "dense nodes" - nodes that have too many connections.
The book Learning Neo4j (by Rik Van Bruggen, available for download in the Neo4j's web site) talk about "Dense node" or "Supernode" and says:
"[supernodes] becomes a real problem for graph traversals because the graph
database management system will have to evaluate all of the connected
relationships to that node in order to determine what the next step
will be in the graph traversal."
The book proposes a solution that consists in add meta nodes between the follower and the notification (in your case). This meta node should got at most a hundred of connections. If the current meta node reaches 100 connections a new meta node must be created and added to the hierarchy, according to the example of figure, showing a example with popular artists and your fans:
I think you do not worry about it right now. If in the future your followers node becomes a problem then you will be able to refactor your database schema. But at now keep things simple!
In the series of posts called "Building a Twitter clone with Neo4j" Max de Marzi describes the process of building the model. Maybe it can help you to make best decisions about your model!
Let's assume this Neo4j data construct.
(:Store)<-[:FROM]-(:Notification)-[:FOR]->(//users//:User)
Which should serve as, for example, a notification for users on a new sale in a store.
Such a notification may be addressed to a large number of users simultaneously, in this case I can see two approaches for this data modelling:
Create separate :Notification with relationships for each of the :User's; All connected to single :Store node. So when notification is received - relationships and :Notification node is removed.
Approach which i thought should be more effective performance wise and about which my question is: Create a single :Notification node connected to a store and multiple [:FOR]'s for different users. Notification received - :FOR for this user removed, if no :FOR's left - :Notification itself removed .
So my questions are:
Am I correct in assuming that the 2nd one is a better practice?
How can I generally, after deleting a relationship, check if there are more such relationships, and do it without making Neo4j match all connected ones of this type, i.e check if there is at least one relationship of this type left?
Definitely having fewer objects created in the first place would be more efficient in general. It will result in 2n fewer objects (1 node and relationship per user). The exception would be if you had so many users per notification that the node density was too high. To avoid that though you could simply create additional notifications at a particular user threshold.
The following query is adapted from #Luanne's answer to this question which i though was pretty slick.
It presumes that besides the User nodes, the only other connection to the Notification nodes is the store nodes. If the degree of nodes connected to the Notification node is one then it must be just a Store node remaining. Remove the relationship to the Store node and remove then remove the Notification node.
MATCH (n:Notification {name: 'Notification One'} )-[store_rel:FROM]->(s:Store)
WHERE size((n)--())=1
DELETE store_rel, n
I am working with the VSO REST API and have a question on how Iteration and Area ID's are assigned. Specifically, why is it when I assign a work item to the root Iteration or Area the ID that is returned for the WIT is not returned when I query the classification nodes?
For example, imagine I have this hierarchy when I query /DefaultCollection/my project/_apis/wit/classificationnodes?$depth=2
My Project: id=1234
Area 1: id=5678
Area 2: id= 9012
And I then query for a work item using /DefaultCollection/_apis/wit/workItems/1?$expand=all
If the work item is in Area 1 or Area 2, the System.AreaId field is as expected (5678 and 9012, respectively). However, if I assign the work item to My Project, the System.AreaID is some value that is not included when I query for all classification nodes. There appears to be some kind of relationship between the ID's as they are serial (e.g. the ID returned by the classification node query is 1232 for the area and 1233 for the iteration), but I can't seem to find a way to query to get the actual ID returned by the work item query.
In fact, not only is the ID returned for a work item not present when I query for all classification nodes, if I assign the work item to both the root iteration and area, the ID returned for both fields is the same value that is not included in the classification node query.
What I need is a way to look at a work item and figure out the area and iteration it belongs to. I could probably do something with the path field strings that are returned, but that seems error prone since users can change them.
****Edit****
The above appears to be a bug in the REST API, but for anyone who comes across this post there is a way to get a usable iteration ID by the path string. Structure your REST call like so:
/DefaultCollection/[Project Name]/_apis/wit/classificationnodes/iterations/Release 1/Sprint 1 (etc.)
I have never done this with ID. I only use the path. In the classification service you can get the node by path easily enough.
For example, using the REST API - you can access this url to get the data about a specific iteration:
/DefaultCollection/[Project Name]/_apis/wit/classificationnodes/iterations/[Release X]/[Sprint Y]
Note that trying to access the default iteration path (the project name instead of a specific iteration) will return an error:
/DefaultCollection/[Project Name]/_apis/wit/classificationnodes/iterations/[Project Name]
will give :
{"$id":"1","innerException":null,"message":"VS402485: The node name is not recognized: [Project Name]","typeName":"Microsoft.TeamFoundation.WorkItemTracking.Server.Metadata.WorkItemTrackingTreeNodeNotFoundException, Microsoft.TeamFoundation.WorkItemTracking.Server","typeKey":"WorkItemTrackingTreeNodeNotFoundException","errorCode":0,"eventId":3200}
So if you do batch work, you have to filter those before querying the api.
There are three ways of identifying an Area (everything I post equally applies to Iterations also). The Path (string), the ID (int) and a Guid. each of them are used in different ways, and have different ramifications.
For example renaming an Area, does NOT change it's identity, therefore does not update a work item (the path returned for in a workitem is dynamic).
It is also possible to delete and recreate and identical path, but it will have a different ID.
The GUID is used primarily for the Excel Reports (such as are parent of the SharePoint portal)
Depending on how you want things to react determines the appropriate element to use.
I have not seen any issues with the ID that you mention, and if you could create a simple repro, I would be glad to look at it.
david(dot)corbin(at)dynconcepts(dot)com