My problem is I put data into Neo4j from what was essentially a large spreadsheet essentially. Now I want to be able to get that data back out in a similar tabular format.
Lets say I have some notional spreadsheet of data that went in looked something like the following.
| Artist | Album | Song | Live | Filename | Genre | Year | Source | Label |
|--------|-------|------|------|----------|-------|------|--------|-------|
| .... | ..... | .... | .... | ........ | ..... | .... | ...... | ..... |
---------------------------------------------------------------------------
The spreadsheet was a listing of files with some metadata about each file. For analytic purposes it made more sense to not have the file be at the center of the graph but rather the Albums. So that every record in the table above would map to a handful of nodes and relationships. The data model for this might look something like this:
(Song)-[_IS_ON_]->(Album)
(Artist)-[_SINGS_]->(Song)
(Album)-[IS_IN_]->(Genre)
(Song)-[_IS_IN_]->(Genre)
(Album)-[_IS_]->(Live)
(Album)-[_FROM_]-(Year)
(Album)-[_IS_ON_]->(Source)
(Label)-[_PRODUCED_]->(Album)
I am able to query a single record from my spreadsheet above using a query similar to this.
MATCH (a:Album {name: "Hells Bells"})-[r]-(b)
OPTIONAL MATCH (s:Song)<-[_SINGS_]-(aa:Artist)
RETURN *
I have two questions here.
How do I make the above query return a table that looks similar to the original normalized table? If I did RETURN b.filename, b.genre ... I get a table that has a lot of null values. It would seem I need to do a DISTINCT on each of the fields. But I am still really new to Neo4j and am not positive I understand how to do this.
It would be great if there was a way to get all the fields in all the nodes without having to type them out in the query like this RETURN b.filename, b.genre .... I think I figured this out once but I stupidly didn't save it.
I hope this was clear enough. I can't share my graph model or data so I had to make this up on the fly.
TIA
Try the following (but, since you did not state how to get the filename, that value might be missing):
MATCH
(artist:Artist)-[:_SINGS_]->(song:Song)-[:_IS_ON_]->(album:Album {name: "Hells Bells"})-[:_FROM_]-(year:Year),
(album)-[:_IS_IN_]->(genre:Genre),
(album)-[:_IS_]->(live:Live),
(album)-[:_IS_ON_]->(source:Source),
(label:Label)-[:_PRODUCED_]->(album)
RETURN *
In a RETURN clause, if you specified a node/relationship (without a property name), that would generate a map of all its properties. The above query, for example, would return a map for each matched node.
If you actually want to have a single merged map, you can use the APOC function apoc.map.mergeList. For example:
MATCH
(artist:Artist)-[:_SINGS_]->(song:Song)-[:_IS_ON_]->(album:Album {name: "Hells Bells"})-[:_FROM_]-(year:Year),
(album)-[:_IS_IN_]->(genre:Genre),
(album)-[:_IS_]->(live:Live),
(album)-[:_IS_ON_]->(source:Source),
(label:Label)-[:_PRODUCED_]->(album)
RETURN apoc.map.mergeList([artist,song,year,genre,live,source,label,album]) AS result
Let's assume I have an app about cooking recipes with two fundamental features:
The first one involves the CURRENT recipe that I'm preparing
The second one stores the recipes that I've decided to save
STANDARD SCENARIO
My current recipe is "Cheese Cake" and in RecipeDetailViewController I can see the current ingredients I've added for this recipe:
Sugar
Milk
Butter
etc.
Well, let's say that I'm satisfied from the final result and I decide to save (to log) the recipe I've just prepared.
* click save *
The recipe is now saved (is now logged) and in RecipesHistoryViewController I can see something like this:
Nov 15, 2013 - Cheese Cake
Nov 11, 2013 - Brownie
etc.
Now if I want I can edit the recipe in the history and change Milk to Soy Milk, for example.
The issue it's that editing the recipe in the history SHOULDN'T edit the recipe (and its ingredients) in my current recipe and vice versa. If I edit the current recipe and replace Butter with Peanut Butter it must not edit anyone of the recipe stored in history. Hope I explained myself.
CONSEQUENCES
What this scenario implies? Implies that currently, for satisfing the function of this features, I'm duplicating the recipe and every sub-relationship (ingredients) everytime the user click on "Save Recipe" button. Well it works but I feel it can be something else more clean. With this implemention it turns out that I have TONS of different duplicates Core Data object (sqlite rows) like these:
Object #1, name: Butter, recipe: 1
Object #2, name: Butter, recipe: 4
Object #3, name: Butter, recipe: 3
etc.
Ideas? How can I optimize this model structure?
EDIT 1
I've already thought of creating any RecipeHistory object with an attribute NSString where I could store a json dictionary but I don't know if it's better or not.
EDIT 2
Currently a RecipeHistory object contains this:
+-- RecipeHistory --+
| |
| attributes: |
| - date |
+-------------------+
| relationships: |
| - recipes |
+-------------------+
+----- Recipe ------+
| relationships: |
| - recipeInfo |
| - recipeshistory |
| - ingredients |
+-------------------+
+-- RecipeInfo ----+
| |
| attributes: |
| - name |
+-------------------+
+--- Ingredient ----+
| |
| attributes: |
| - name |
+-------------------+
| relationships: |
| - recipe |
+-------------------+
paulrehkugler is true when he says that duplicating every Recipe object (and its relationships RecipeInfo and Ingredients) when I create a RecipeHistory is going to fill the database with a tons of data but I don't find another solution that allows me flexibility for the future. Maybe in the future I would to create stats about recipes and history and having Core Data objects could prove to be useful. What do you think? I think this is a common scenario in many apps that store history and allow to edit history item.
BIG UPDATE
I have read the answers from some users and I want to explain better the situation.
The example I stated above is just an example, I mean that my app doesn't involve cook/recipe argument but I have used recipes because I think it's pretty okay for my real scenario.
Said this I want to explain that the app NEEDS two sections:
- First: where I can see the CURRENT recipe with related ingredients
- Second: where I can see the recipe I decided to save by tapping a button 'Save Recipe' in the first section
The current recipe found in the first section and a X recipe found in the 'history' section doesn't have NOTHING in common. However the user can edit whatever recipes saved in 'history' section (he can edit name, ingredients, whatever he wants, he can completely edit all things about a recipe found in history section).
This is the reason why I came up duplicating all NSManagedObjects. However, in this way, the database will grow as mad because everytime the user saves the current recipe the object representing the recipe (Recipe) is duplicated and also the relationships the recipes had (ingredients). So there will be TONS of ingredients named 'Butter' for example. You can say me: why the hell you need to have TONS of 'Butter' objects? Well, I need it because ingredients has for example the 'quantity' attribute, so every recipe have ingredients with different quantities.
Anyhow I don't like this approach, even it seems to be the only one. Ask me whatever you want and I'll try to explain every detail.
PS: Sorry for my basic English.
EDIT
Since you must deal with history, and because the events are generated manually by end users, consider changing the approach: rather than storing the current view of the model entities (i.e. recipes, ingredients, and the connections among them) store the individual events initiated by the user. This is called Event Sourcing.
The idea is to record what user does, rather than recording the new state after the user's action. When you need to get the current state, "replay" the events, applying the changes to in-memory structures. In addition to letting you implement the immediate requirements, this would let you restore the state as of a specific date by "replaying" the events up to a certain date. This helps with audits.
You can do it by defining events like this:
CreateIngredient - Adds new ingredient, and gives it a unique ID.
UpdateIngredient - Changes an attribute of an existing ingredient.
DeleteIngredient - Deletes an ingredient from the current state. Deleting an ingredient deletes it from all recipes and recipe histories.
CreateRecipe - Adds a new recipe, and gives it a unique ID.
UpdateRecipeAttribute - Changes an attribute of an existing recipe.
AddIngredientToRecipe - Adds an ingredient to an existing recipe.
DeleteIngredientFromRecipe - Deletes an ingredient from an existing recipe.
DeleteRecipe - Deletes a recipe.
CreateRecipeHistory - Creates a new recipe history from a specific recipe, and gives the history a new ID.
UpdateRecipeHistoryAttribute - Updates an attribute of a specific recipe history.
AddIngredientToRecipeHistory - Adds an ingredient to a recipe history.
DeleteIngredientFromRecipeHistory - Deletes an ingredient from a recipe history.
You can store the individual events in a single table using Core Data APIs. Add a class that processes events in order, and creates the current state of the model. The events will come from two places - the event store backed by Core Data, and from the user interface. This would let you keep a single event processor, and a single model with the details of the current state of recipes, ingredients, and recipe histories.
Replaying the events should happen only when the user consults the history, right?
No, that is not what happens: you read the whole history on start-up into the current "view", and then you send the new events both to the view and to the DB for persistence.
When users need to consult the history (specifically, when they need to find out how the model looked as of a specific date in the past) you need to replay the events partially, up until the date of interest.
Since the events are generated by hand, there wouldn't be too many of them: I would estimate the count in the thousands at the most - that's for a list of 100 recipes with 10 ingredients each. Processing an event on a modern hardware should be in microseconds, so reading and replaying the entire event log should be in the milliseconds.
Furthermore, do you know any link that shows an example of how to use Event Sourcing in a Core Data application? [...] For example, should I need to get rid of RecipeHistory NSManagedObject?
I do not know of a good reference implementation for event sourcing on iOS. That wouldn't be too different from implementing it on other systems. You would need to get rid of all tables that you currently have, replacing it with a single table that looks like this:
The attributes would be as follows:
EventId - Unique ID of this event. This is assigned automatically on insertion, and never changes.
EntityId - Unique ID of the entity created or modified by this event. This ID is assigned automatically by a Create... processor, and never changes.
EventType - A short string representing the name of this event type.
EventTime - The time the event has happened.
EventData - A serialized representation of the event - this can be binary or textual.
The last item can be replaced for a "denormalized" group of columns representing a superset of attributes used by the 12 event types above. This is entirely up to you - this table is merely one possible way of storing your events. It does not have to be Core Data - in fact, it does not even need to be in a database (although it makes things a little easier).
I think when a row in RecipesHistoryViewController is selected to modification, we can optimize the Save process with two options:
Let the user chooses if a new row must be saved or an update may happen. Having a Save New button to create a new row in Recipe and an Update button to update the current selected row.
To trace the changes have been made to a recipe (when update happens), I will try to log only changes of the recipe. Using EAV pattern will be an option.
As a hint: Comma separated values of ingredient name could be used as old and new values, when
inserting a row in RecipeHistory table, the sample may helps.
About the BIG UPDATE:
Assuming that the real application have a database for persistent operation, some suggestions may be helpful.
The current recipe found in the first section and a X recipe found in
the 'history' section doesn't have NOTHING in common
Leads the natural way of having no relation between Current and In-History recipe, so
trying to create a relation will be vain. With no relation the design will not be in normal form, redundancy will be inevitable.Flowing the approach there will be many records, in the case
We can limit any user's saved recipes in a predefined number.
Another solution to optimize performance of recipe table would be range
partitioning the table based on creation date field (let a data
base administrator be involved).
Another suggestion is to have a separate table for ingredient
concept. Having ingredient, recipe, recipe-ingredient
tables will reduce redundancy.
Using NoSql
If relations are not trivial part of the applications logic, I mean if your are not going to be ended in complex queries like "Which ingredients have been used more than X times in recipes that have less than total Y ingredients and Milk is not one of them" or analytical procedures then,have a look at NoSql databases and comparison of them.
They offer being non-relational, distributed, open-source, schema-free, easy replication support, simple API, huge amount of data and horizontally scalable.
For a basic example of a document based database: Having couchdb installed on my local machine(port number 5984) creating recipe database(table) on couchdb will be done by sending an standard HTTP request (using curl) like:
curl -X PUT http://127.0.0.1:5984/recipe
Dropping recipe table:
curl -X DELETE http://127.0.0.1:5984/recipe
Adding a recipe:
curl -X PUT http://127.0.0.1:5984/recipe/myFirstRecipe -d
'{"name":"Cheese Cake","description":"i am using couchDB for my recipes",
"ingredients": [
"Milk",
"Sugar"
],}'
Getting myFirstRecipe record(document)
curl -X GET http://127.0.0.1:5984/recipe/myFirstRecipe
No need of classical server side process like object relation mapping, data base driver, etc
BTW using Nosql will have short comings you need to consider, like here and here.
As I see it, your problem is more conceptual than model structure related.
My idea for your model is:
+*******+
Recipe
-----------------
-----------------
properties:
-----------------
- isDraft - BOOL
- name - NSString
- creationDate - NSDate
-----------------
-----------------
relationships:
-----------------
- ingredients - to-many with Ingredient
-----------------
+*******+
+*******+
Ingredient
-----------------
-----------------
properties:
-----------------
- name - NSString
-----------------
-----------------
relationships:
-----------------
- recipes - to-many with Recipe
-----------------
+*******+
Now, Lets call your "current" recipe a draft (a user may have many drafts).
As you can see, you can now display your recipes with a single fetched results controller (FRC)
The fetch request will look like this:
NSFetchRequest* r = [NSFetchRequest fetchRequestWithEntityName:#"Recipe"];
[r setFetchBatchSize:25];
NSSortDescriptor* sortCreationDate = [NSSortDescriptor sortDescriptorWithKey:#"creationDate" ascending:NO];
[r setSortDescriptors:#[sortCreationDate]];
you can section your data on the isDraft property:
NSFetchedResultsController* frc = [[NSFetchedResultsController alloc] initWithFetchRequest:r
managedObjectContext:context
sectionNameKeyPath:#"isDraft"
cacheName:nil];
Remember to give appropriate titles to your sections as to not confuse the user.
Now, all you have left is add some specific functionality like:
create new recipe
save
save draft
edit recipe (draft or not)
if draft offer to save as complete recipe
else, save the actual recipe
if you like, you might add a "save as" option
create copy (the user is aware that he might introduce redundant data if he saves the same recipe more than once)
In any case the user experience should be consistent.
Meaning:
While the user is editing/adding an object, this object should not change "under his feet".
If a user is adding a new recipe, he then might wish to save it as draft, or as a complete recipe.
When he save, in either case, he might still wish to continue editing it. and so, no new object need be created.
If you like to add versioning for your recipes, you will need to add an entity like RecipeHistory related to a single recipe. this entity will record changes on each committed change in a complete recipe object (use changedValues of NSManagedObject or check against the existing/committed values).
You may serialise and store the data as you see fit.
So you can see, its more of a conceptual issue (how you access your data) than it is a modelling issue.
There are a few questions that need to be answered:
Is there a limit to the number of "history items" for a recipe or is it really necessary to keep all the versions of a recipe around?
When is a modification just a change of an existing recipe and when does the change result in a new recipe? For example, should the user be allowed to change a "cheese cake" recipe into a "meat loaf" recipe by completely replacing every ingredient and the title?
The answers to these questions are important when planing your data model. For example, ask yourself if this would be a valid use case for your app: The user creates a "Basic Cake" recipe that contains sugar, flour and eggs. The user now wants to take this "Basic Cake" recipe as a template to create a "Cheese Cake", a "Pound Cake" and a "Carrot Cake" recipe. Is that a valid use case?
If so, every time you save a recipe, it basically creates a completely new, independent recipe because the user is allowed to change everything and thus turn a cheese cake into a meat loaf.
However, I think that would be unexpected behavior for the user. In my opinion the user creates a "Cheese Cake" recipe and then might want to trace the changes to that one recipe and not turn it into something completely different.
This is what I would suggest:
Instead of a RecipeHistory owning Recipes, change your data model so that Recipes have multiple RecipeVersions. That way, users can explicitly create new recipes and then track the changes to that one recipe. Also, users would not be allowed to edit a RecipeVersion directly, but instead could "revert" their recipe to a specific version and then edit that.
Make Ingredients unique: "Butter", "Milk" and "Flour" exist exactly once in the database and are only references by the different recipes. That way, you will not have duplicates in your database and saving just the reference will take up less disk space than saving the name of the ingredient again and again.
Allow your users to create a new recipe based on an existing Recipe(Version). That way you give your users the ability to "base" a new recipe on an existing one without complicating your app and your data model.
This is my suggested data model:
+----- Recipe ------+
| attributes: |
| - name |
| relationships: |
| - recipeVersions |
+-------------------+
+-- RecipeVersion ----+
| attributes: |
| - timestamp |
+----------------------+
| relationships: |
| - recipe |
| - ingredients |
+----------------------+
+--- Ingredient ----+
| attributes: |
| - name |
+-------------------+
| relationships: |
| - recipeVersions |
+-------------------+
Enjoy.
You don't need to duplicate all of the ingredient objects. Instead, just change the relationships so that recipes have many ingredients and ingredients can be in many recipes. Then when you create a duplicate recipe you just connect to the existing ingredients.
This would also make it easier to list the recipes that use an (or some combination of) ingredients.
You should also consider your UI/UX - should it be a full duplicate? Or should you allow the user to create 'alternatives' within each recipe (which just list a set of replacement ingredients).
It's a tradeoff between storage size and retrieval time.
If you duplicate each recipe every time the user clicks the "Save Recipe" button, you duplicate a lot of data in the database.
If you create a RecipeHistory object that has a Recipe and a list of changes, it takes longer to retrieve the data and populate your View Controllers, because you have to reconstruct a full Recipe in memory.
I'm not sure which is easier - whichever suits your use case is probably best.
Not sure I am clear on the problem you are trying to solve but I would start by modelling the Recipe and Ingredients and keep them separate from the actual mix and method which may change as the cook experiments. With some smart application logic you could only track the changes in each version rather than make a new copy. For example if the user decides to try a new version of a recipe then by default show the previous versions (or allow the user to select a version) Method and RecipeIngredients and if any changes are made save these changes as new Method and RecipeIngredient associated with the RecipeVersion.
This approach will use less storage but requires much more complicated application logic, for example swapping an ingredient would setting the quantity to 0 for the ones being replaced and adding new records for the new ones. Simply duplicating the previous (or user selected) version is not going to use much space, these are small records, and will be much much simpler to implement.
I believe it would be better to define ingredient table to have ingredientID and ingredientDisplayName, and in recipie history table store RecipieID, HistoryDate, IngredientArray.
if in ingredient table,
id:1 is Butter
id:2 is Milk
id:3 is cheese
id:4 is Sugar
id 5 is Soymilk
then in history table
for recipe 1: Cheese Cake, data Nov 15, IngredientArray: {1,2,3,4}
if on Nov 16 Cheese cake changes to have soy milk instead of milk then on that date IngredientArray is {1,2,3,5} . Many database has array column option or alternately could be a comma separated string or a Json document.
Its better to keep the ingredient list in-memory to do fast lookup to get ingredient names from list.
maybe I did not understand your question, but do you need to change the name of butter by editing? Why not just delete butter from that one recipe and add peanut butter to it. That way you do not change butter to peanut butter for al your other recipes that are linked to it? And with new recipes you can select peanut butter or butter.
Just to be clear, we are talking about frontend?
First, like suggest by Mohsen Heydari, on SQL rdbms, you should create a table between many-to-many connections to make two one to many for performance.
So you want a historic
+-- RecipeHistory --+
| |
| attributes: |
| - id |
| - date |
| - new name? |
| - notes ?? |
| - recipe-id |
+-------------------+
| relationships: |
| - recipes |
+-------------------+
+----- Recipe ------+
| attributes: |
| - id |
| - name |
| - discription |
| - date |
| - notes | #may be useful?
| - Modifiable | #this field is false if in history, else true,
+-------------------+
| relationships: |
| recipe-ingredient |
+-------------------+
+-Recipe-ingridient-+
| attributes: |
| id |
| recipe-id |
| ingridient-id |
| quantity |
+-------------------+
+--- Ingredient ----+
| |
| attributes: |
| - id |
| - name |
+-------------------+
| relationships: |
| -recipe-ingredient|
+-------------------+
Now if modifiable field on Recipe = True it belongs on the MainPage
If its false, it belongs on the historic page
After finding a recipe you want, you can query the ingredients by its recipe-id using the Recipe-Ingredient table, or Recipe by Ingredients the same way.
Another option less space hungry would be create a Recipe history, and create a Modified recipe table -> which takes a base recipe ID,
And map it to -> Main Recipe ID, Discarded Ingredients and New Ingredients, if you want this solution explained just ask
I have a feed that is displayed to the user, which includes 4 different model types.
It simulates grouping the entries by day, by inserting a day object into the feed.
The feed is sorted chronologically and paginated.
This is how I currently build the feed.
def get_feed initial_feed_position=0, number_of_entries_to_get=30
# display is called on each model to get them all into a standard format
feed = (first_models + second_models + third_models + forth_models).map { |feed_entry| feed_entry.display }
feed += day_entries_for_feed(feed)
end_feed_position = initial_feed_position + number_of_entries_to_get
(feed.sort_by { |feed_entry| -feed_entry[:comparison_time].to_i })[initial_feed_position...end_feed_position]
end
def day_entries_for_feed feed
# iterate over a set of dates that contain feed entries
feed.map{ |feed_entry| feed_entry[:date] }.uniq.map do |day|
# building the day object in the standard feed entry format. fields that are not relevant to this question have been left out.
{
type: 'day',
comparison_time: (day + 24.hours - 1.second).time # to ensure that the day appears above it's corresponding entries in the feed, the comparison time is set to 1 second before the day ends
}
end
end
Over time, the number of objects in the system has built up, and now the feed takes a long time to build using this method. Is there a better way to do it?
I'm using Rails 3.2.13, Ruby 1.9.3 & PostgreSQL 9.1.9.
Because you're getting all the entries in the database a lot of models are loaded into memory, to solve this problem you would have to look in to UNION (which is a pain to maintain and you will have to have literal SQL in your codebase). A good example of it is here: PosgreSQL: How to union 3 tables sorted by date
Another option would be to derive a base class and do the querying on this. Which would result in something like this:
+-------------+
| BASE FEED |
| |
+------^------+
|
+-------------------+--------+---------+----------------+
| | | |
+-----+-------+ +------+------+ +-------+-----+ +------+-----+
| MODEL ONE | | MODEL TWO | | MODEL THREE | | MODEL FOUR |
| | | | | | | |
+-------------+ +-------------+ +-------------+ +------------+
Once you have your models set up like this it's a simple matter of querying this base table. Which could look something like this:
def get_feed(initial_feed_position = 0, number_of_entries_to_get = 30)
feeds = BaseFeed.
limit(number_of_entries_to_get).
offset(initial_feed_position).
order("DATE(date_field) DESC")
end
The above example is not the exact solution but if you elaborate a bit more on what you're trying to get as a result set I can adjust it but it's more about the approach to take.
Hope this helps.
Solution without changing the DB:
The reason your code is getting slower is that you query all the objects, and then taking
only the top 30 (number_of_entries_to_get)
Because it's a feed, we can assume that most of the times, users will look at the first few pages.
Instead of taking all the first_models/second_models etc, you can take the newest end_feed_position straight from the db (order by date)
Something like:
models = FirstModel.order("date DESC").limit(end_feed_position)
models += SecondModel.order("created_at DESC").limit(end_feed_position)
So if, for example, you are on the page 2, and searching for a feed of 30:
You only query 240 objects from the db (first_models + second_models + third_models + forth_models) * 60 and it is places 30..60 in these 240 are the the 30..60 of all the objects (so it wont get slower as the db is growing)
Some time ago I asked a question about nested loops on SO and as it was, there were queries inside the loops of my example and I got a clear answer:
NEVER EVER NEVER put an SQL query inside a loop
I've tried ever since and mostly it works. Just need to make an effort and write a query that retrieves all you need at once.
BUT what do you do when you have a dataset from a JOIN query which contains nested data which you need to output in a nested way?
Example join from table A and B:
A.a | B.a | B.b
--------|----------|-------------
fruits | banana | yellow
fruits | apple | red
animals | zebra | black&white
animals | elefant | gray
animals | fox | red
planets | earth | blue
planets | mars | red
ok, now I got that all in an array or rowset and now I need to display something like that:
fruits
yellow banana
red apple
animals
black&white zebra
gray elefant
red fox
planets
blue earth
red mars
it seems obvious that it should work but I've tried to wrap my mind around it several times now and I just can't come up with a solution.
At the moment I do it my old way:
query groups
foreach groups
{
query animals in group
foreach animal
}
but hey, NEVER EVER NEVER put sql inside a loop. so what shold I do? I do PHP but I think this is a meta question.
Use the control break algorithm.
I'd return a result set exactly as you show in the question:
A.a | B.a | B.b
--------|----------|-------------
fruits | banana | yellow
fruits | apple | red
animals | zebra | black&white
animals | elefant | gray
animals | fox | red
planets | earth | blue
planets | mars | red
loop over all the rows:
when A.a changes, output the title
then always output the B.b + B.a value
pseudo code for application calling SQL:
set last_A = null
exec query
loop over result set {
if last_A == null or fetch_A!=last_A {
last_A=fetch_A
display fetch_a
}
display fetch_Bb + fetch_Ba
}
}//loop
If what you have is a hierarchy, a "directed acyclic graph". SQL does not do these.
There are other graph-theory things SQL does not do.
Since SQL does not do this, the "never put SQL in a loop" rule goes out the window.
You must put the SQL in a loop for hierarchies and other graph-connection problems involving lattices and networks.
Indeed, for hierarchies, you must use recursive loops to connect all elements of the hierarchy to arbitrary depth.
If, on the other hand, you're just reformatting the query result to look like a nested hierarchy, then you're just reformatting a single SQL result set into what appears to be nested lists.
This will be one select with complex loops around the result set. One select -- not in a loop -- and a complex loop to process one result set.