Does TypeORM have a way to search for all documents with an array containing a value? - typeorm

My goal is to use an input array of strings (fake emails) as a search query for documents in my MongoDB database, which I am powering using TypeORM. This way if I want to search for documents using more than one email at a time, I can do that. Meaning I want to be able to feed in:
query = ["kim#gmail.com", "jim#gmail.com", "sarah#gmail.com"] and get 3 different documents where document one has kim#gmail.com as the attendee, jim#gmail.com is another document's attendee field, and sarah#gmail.com is the third document's attendee (or is among them).
I want to use an email as a query to search for and return all documents where the array field has the email in the array.
So as an example here is the results for the "get all documents" endpoint right now:
[
{
"_id": "6283d7ad706445dc33319bcb",
"hostUsername": "jack",
"hostEmail": "jack#outlook.com",
"meetingName": "nervous-fish-hautily-vetting",
"startTime": "2022-12-12T08:00:00.000Z",
"attendees": [
"kate#gmail.com",
"sawyer#gmail.com"
]
},
{
"_id": "6284235e662f7dfb073e2cbc",
"hostUsername": "jacob",
"hostEmail": "jacob#gmail.com",
"meetingName": "eager-fish-hautily-vetting",
"startTime": "2022-12-12T08:00:00.000Z",
"attendees": [
"kate#gmail.com",
"benjaminlinus#gmail.com"
]
},
{
"_id": "6283d7c3706445dc33319bcc",
"hostUsername": "richard",
"hostEmail": "richard#outlook.com",
"meetingName": "eager-cat-hautily-subtracting",
"startTime": "2022-12-12T08:00:00.000Z",
"attendees": [
"johnlocke#gmail.com",
"hurley#gmail.com"
]
},
{
"_id": "6283d82b706445dc33319bcd",
"hostUsername": null,
"hostEmail": "richard#outlook.com",
"meetingName": "nervous-cat-hautily-jumping",
"startTime": "1970-01-01T00:00:00.000Z",
"attendees": null
},
{
"_id": "6283d8af706445dc33319bce",
"hostUsername": null,
"hostEmail": "richard#outlook.com",
"meetingName": "eager-plant-ignorantly-jumping",
"startTime": "1970-01-01T00:00:00.000Z",
"attendees": null
}
]
I want to query the database with ["kate#gmail.com"] and get back the two results that have "kate#gmail.com" in the attendees field.
The closest solution (that doesn't work) is the one I found in this GitHub issue and also another close solution (that doesn't work) in this StackOverflow question
Here is me implementing those two suggestions:
import { In } from "typeorm";
async searchMeetingsByDetails(
attendees?: string[]
): Promise<IMeeting[]> {
console.log(attendees, 39);
const meetingsByAttendees = attendees
? await this.meetingRepository.find({
where: {
attendees: In([...attendees]),
},
})
: [];
return [
meetingsByAttendees,
].flat();
}
This gives me an empty array [] when the input is ["kate#gmail.com"] so if the In() thing worked, it would give results.
const meetings = await this.meetingRepository
.createQueryBuilder("meeting")
.where("meeting.attendees IN (:attendees)", {
attendees: [...attendees],
});
This one gives ERROR [ExceptionsHandler] Query Builder is not supported by MongoDB. TypeORMError: Query Builder is not supported by MongoDB.

Related

How to create dynamic node relation in neo4j for dynamic data?

I was able to create author nodes directly from the json file . But the challenge is on what basis or how we have to link the data. Linking "Author" to "organization". since the data is dynamic we cannot generalize it. I have tried with using csv file but, it fails the conditions when dynamic data is coming. For example one json record contain 2 organization and 3 authors, next record will be different. Different json record have different author and organization to link. organization/1 represent organization1 and organization/2 represents organization 2. Any help or hint will be great. Thank you. Please find the json file below.
"Author": [
{
"seq": "3",
"type": "abc",
"identifier": [
{
"idtype:auid": "10000000"
}
],
"familyName": "xyz",
"indexedName": "MI",
"givenName": "T",
"preferredName": {
"familyName": "xyz1",
"givenName": "a",
"initials": "T.",
"indexedName": "bT."
},
"emailAddressList": [],
"degrees": [],
"#id": "https:abc/2009127993/author/person/3",
"hasAffiliation": [
"https:abc/author/organization/1"
],
"organization": [
[
{
"identifier": [
{
"#type": "idtype:uuid",
"#subtype": "idsubtype:affiliationInstanceId",
"#value": "aff2"
},
{
"#type": "idtype:OrgDB",
"#subtype": "idsubtype:afid",
"#value": "12345"
},
{
"#type": "idtype:OrgDB",
"#subtype": "idsubtype:dptid"
}
],
"organizations": [],
"addressParts": [],
"sourceText": "",
"text": " Medical University School of Medicine",
"#id": "https:abc/author/organization/1"
}
],
[
{
"identifier": [
{
"#type": "idtype:uuid",
"#subtype": "idsubtype:affiliationInstanceId",
"#value": "aff1"
},
{
"#type": "idtype:OrgDB",
"#subtype": "idsubtype:afid",
"#value": "7890"
},
{
"#type": "idtype:OrgDB",
"#subtype": "idsubtype:dptid"
}
],
"organizations": [],
"addressParts": [],
"sourceText": "",
"text": "K University",
"#id": "https:efg/author/organization/2"
}
]
Hi I see that Organisation is part of the Author data, so you have to model it like wise. So for instance (Author)-[:AFFILIATED_WITH]->(Organisation)
When you use apoc.load.json which supports a stream of author objects you can load the data.
I did some checks on your JSON structure with this cypher query:
call apoc.load.json("file:///Users/keesv/work/check.json") yield value
unwind value as record
WITH record.Author as author
WITH author.identifier[0].`idtype:auid` as authorId,author, author.organization[0] as organizations
return authorId, author, organizations
To get this working you will need to create include apoc in the plugins directory, and add the following two lines in the apoc.conf file (create one if it is not there) in the 'conf' directory.
apoc.import.file.enabled=true
apoc.import.file.use_neo4j_config=false
I also see a nested array for the organisations in the output why is that and what is the meaning of that?
And finally I see also in the JSON that an organisation can have a reference to other organisations.
explanation
In my query I use UNWIND to unwind the base Author array. This means you get for every author a 'record' to work with.
With a MERGE or CREATE statement you can now create an Author Node with the correct properties. With the FOREACH construct you can walk over all the Organization entry and create/merge an Organization node and create the relation between the Author and the Organization.
here an 'psuedo' example
call apoc.load.json("file:///Users/keesv/work/check.json") yield value
unwind value as record
WITH record.Author as author
WITH author.identifier[0].`idtype:auid` as authorId,author, author.organization[0] as organizations
// creating the Author node
MERGE (a:Author { id: authorId })
SET a.familyName = author.familyName
...
// walk over the organizations
// determine
FOREACH (org in organizations |
MERGE (o:Organization { id: ... })
SET o.name = org.text
...
MERGE (a)-[:AFFILIATED_WITH]->(o)
// if needed you can also do a nested FOREACH here to process the Org Org relationship
)
Here is the JSON file I used I had to change something at the start and the end
[
{
"Author":{
"seq":"3",
"type":"abc",
"identifier":[
{
"idtype:auid":"10000000"
}
],
"familyName":"xyz",
"indexedName":"MI",
"givenName":"T",
"preferredName":{
"familyName":"xyz1",
"givenName":"a",
"initials":"T.",
"indexedName":"bT."
},
"emailAddressList":[
],
"degrees":[
],
"#id":"https:abc/2009127993/author/person/3",
"hasAffiliation":[
"https:abc/author/organization/1"
],
"organization":[
[
{
"identifier":[
{
"#type":"idtype:uuid",
"#subtype":"idsubtype:affiliationInstanceId",
"#value":"aff2"
},
{
"#type":"idtype:OrgDB",
"#subtype":"idsubtype:afid",
"#value":"12345"
},
{
"#type":"idtype:OrgDB",
"#subtype":"idsubtype:dptid"
}
],
"organizations":[
],
"addressParts":[
],
"sourceText":"",
"text":" Medical University School of Medicine",
"#id":"https:abc/author/organization/1"
}
],
[
{
"identifier":[
{
"#type":"idtype:uuid",
"#subtype":"idsubtype:affiliationInstanceId",
"#value":"aff1"
},
{
"#type":"idtype:OrgDB",
"#subtype":"idsubtype:afid",
"#value":"7890"
},
{
"#type":"idtype:OrgDB",
"#subtype":"idsubtype:dptid"
}
],
"organizations":[
],
"addressParts":[
],
"sourceText":"",
"text":"K University",
"#id":"https:efg/author/organization/2"
}
]
]
}
}
]
IMPORTANT create unique constraints for Author.id and Organization.id!!
In this way you can process any json file with an unknown number of author elements and an unknown number of affiliated organisations

Filtering chloropleth data using a slider for annual data in vega-lite

I have some geojson grids that map through to some annualised sales data over a period of 25 years. I am really struggling to filter this sales data by year to show the trends in a chloropleth map.
d3 = require('d3-dsv');
map_json = FileAttachment("Time_line#2.geojson").json()
sales_data = FileAttachment("Timeline_test#1.csv").csv()
vegalite = require('#observablehq/vega-lite')
vegalite ({
"$schema": "https://vega.github.io/schema/vega-lite/v5.json",
"width": 600,
"height": 350,
"data": {
"name": "mapdata",
"values": map_json,
"format": {"property": "features"},
},
"params": [{
"name" : "AnnualPeriod",
"value": 1995,
"bind" : {"input": "range", "min":1995, "max":2020,"step":1 }
}],
"transform" : [{
"lookup": "properties.id",
"from": {
"data": { "values": sales_data,},
"format":"csv",
"key": "derived_boundary_id",
"fields": ["sales_volume"],
},
},],
"layer": [
{
"mark": "geoshape",
"encoding": {
"color": {
"field": "sales_volume",
"type": "quantitative",
"scale": {"scheme": "Oranges"},
},
"stroke": { "value": "#ff75"},
},
},
]
})
I have tried to add transform.filter and cannot get it to work. At the moment it appears to be taking the first sales_data record for each of the boundary_ids.
I would like the data to be filtered according to the setting of the AnnualPeriod slider.
I think I need to include something like
"transform" :[{"filter": "datum.year == AnnualPeriod"}]
I have tried it in the transform section, with the lookup between the sales_data and the geojosn objects.
I have also tried to filter in and around the geoshape mark but neither work.
Does anyone have any ideas?
The is a sample of the sales_data:
sales_volume,year,derived_boundary_id
5,2015,602212
2,2016,602212
2,2019,602212
5,1995,602213
7,1996,602213
6,1997,602213
7,1998,602213
9,1999,602213
10,2000,602213
7,2001,602213
5,2002,602213
5,2003,602213
9,2004,602213
5,2005,602213
...
where the last column maps to an "id" in the geojson data.
and this is the 'map' that I get. Always the same, irrespective of the slider setting.
I have eventually worked out how to get this to work.
Create the main data source as the sales data and attach the maps/geojson to this via a transform/lookup.
It seems so simple now, but I thought I would post the result so others can see how it can be achieved.
These observations may help others:
The transform can take the "year" filter as well as the map_json lookup.
The map/json lookup is referenced as "geo" to make it easier to understand
The geoshape mark then references, via the encoding, this geojson "geo" object.
Thanks to Mike Bostock and the Observable and Vega-Lite team for their excellent work.
vegalite ({
"$schema": "https://vega.github.io/schema/vega-lite/v5.json",
"width": 600,
"height": 600,
"data": {
"values": sales_data,
},
"params": [{
"name" : "AnnualPeriod",
"value": 1995,
"bind" : {"input": "range", "min":1995, "max":2020,"step":1 }
}],
"transform" : [
{"filter" : "year(datum.year) === AnnualPeriod"},
{
"lookup": "derived_boundary_id",
"from" : {
"data": {
"values": map_json,
"format": {"property": "features"},
},
"key":"properties.id",
},
"as":"geo",
}
],
"mark":"geoshape",
"encoding": {
"shape": {
"field":"geo",
"type":"geojson",
},
"color":{
"field":"sales_volume",
"type":"quantitative",
"scale": {
"scheme":"Oranges",
"domain": [0,15],
},
}
}
})
How's your progress? I am not familiar with Choropleths, but by comparing with the Vega Examples, I spot 2 differences you may wanna take a look:
Choropleth needs a projection
Map data should be placed in lookup transform, and sale data in normal data. Ref 1 Ref 2
Do feel free to correct me if I am wrong. If you need more help, please share an editor with dummy data because I found it hard to make good use of the sale data you provided :'(

How to fetch "#microsoft.graph.downloadUrl" from Microsoft Search API?

I am using Microsoft Search API to search for files in SharePoint, objective is to fetch temporary url ("#microsoft.graph.downloadUrl") amoung other fields. I have used below search request,
Url - POST request: https://graph.microsoft.com/v1.0/search/query
Body:
{
"requests": [
{
"entityTypes": [
"driveItem"
],
"query": {
"queryString": "search-value",
"query": "path:\"https://{company}.sharepoint.com/sites/{site}/shared\""
},
"fields": [
"name",
"webUrl",
"#microsoft.graph.downloadUrl"
]
}
]
}
The result returned is below, and it is missing "#microsoft.graph.downloadUrl" field.
"moreResultsAvailable": false,
"hits": [
{
"hitId": "01CDLPULQVGFRNVHYTJVFL3NPPXIGOUDUS",
"rank": 1,
"summary": "string;#<c0>search-value</c0> <c0>search-value</c0> Internal 1 0 search-value Graph explorer {DA623115<ddd/>",
"resource": {
"#odata.type": "#microsoft.graph.driveItem",
"name": "word.docx",
"webUrl": "https://company.sharepoint.com/sites/{site}/Cases/search-value/word.docx"
}
},
Search API can retrieve only properties but not instance attributes like "#microsoft.graph.downloadUrl".
Properties
Instance attributes
I think there's no need to use the REST API. To get the temporary #microsoft.graph.downloadUrl download link I used the following to get the value:
var selectedFile = files.value[0];
var downloadLink = selectedFile["#microsoft.graph.downloadUrl"];
Found at this link: https://github.com/OneDrive/onedrive-api-docs/issues/547
I hope this helps

Openlayers bbox strategy

I have bbox strategy for one source of data. Code looks like this:
bbox: function newBboxFeatureSource(url, typename) {
return new ol.source.Vector({
loader: function (extent) {
let u = `${url}&TYPENAME=${typename}&bbox=${extent.join(",")}`;
$.ajax(u).then((response) => {
this.addFeatures(
geoJsonFormat.readFeatures(response)
);
});
},
strategy: ol.loadingstrategy.bbox
});
},
I works fine but... When I pan/move the map then this loader is calling again and add another features which fit to new box. But there is a lot of duplicates then because some of new features are just the same as old.
So I wanted first clear all features using this.clear() before add new features but when I add this command then loader is running all the time and I have "infinitive loop". Do you know why? How can I disable loading new features after calling this.clear()?
edit:
my response with features looks like this:
{ "type": "FeatureCollection", "crs": { "type": "name", "properties":
{ "name": "urn:ogc:def:crs:EPSG::3857" } },
"features": [ { "type": "Feature", "properties": { "ogc_fid": "2",
"name": "AL" }, "geometry": { "type": "MultiPolygon" , "coordinates":
[ [ [ ... ] ] ] } }, { "type": "Feature", "properties": { "ogc_fid":
"3", "name": "B" }, "geometry": { "type": "MultiPolygon" ,
"coordinates": [ [ [ ...] ] ] } } ..... and so on
I've removed coordinates because there was too many of them.
My features are generated by mapserver and are configured in .map file which looks like this:
LAYER
NAME "postcode_area_boundaries"
METADATA
"wfs_title" "Postcode area boundaries"
"wfs_srs" "EPSG:3857"
"wfs_enable_request" "*"
"wfs_getfeature_formatlist" "json"
"wfs_geomtype" "multipolygon"
"wfs_typename" "postcode_area_boundaries"
"wms_context_fid" "id"
"wfs_featureid" "id"
"gml_featureid" "id"
"gml_include_items" "id,postarea,wkb_geometry"
"gml_postarea_alias" "name"
"ows_featureid" "id"
"tinyows_table" "postcode_area_boundaries"
"tinyows_retrievable" "1"
"tinyows_include_items" "id,postarea,wkb_geometry"
END
TYPE POLYGON
STATUS ON
CONNECTIONTYPE POSTGIS
CONNECTION "..."
DATA "wkb_geometry FROM postcode_area_boundaries USING UNIQUE id"
DUMP TRUE
END
To summarize the discussion and answer the initial question:
The features sent by the server need an attribute called id, which must be unique and the same for the feature on every request.
{type: "Feature", id: "some-wfs.1234", properties: { "ogc_fid": 2, ...
See this GitHub Issue for the original comment of ahocevar.
In GeoServer this can be achieved if you set an identifier in your layer.
I guess there is something similar to set in MapServer.
Following up on #bennos comment: you can expose the fid in Mapserver using the FORMATOPTION USE_FEATUREID=true as described in: https://mapserver.org/output/ogr_output.html
USE_FEATUREID=true/false
Starting from MapServer v7.0.2. Defaults to false. Include feature ids in the generated output, if the ows_featureid metadata key is set at the layer level. The featureid column to use should be an integer column. Useful if you need to include an “id” attribute to your geojson output. Use with caution as some OGR output drivers may behave strangely when fed with random FIDs.
OUTPUTFORMAT
NAME "application/json"
DRIVER "OGR/GEOJSON"
MIMETYPE "application/json"
FORMATOPTION "FORM=SIMPLE"
FORMATOPTION "FILENAME=ol-query.json"
FORMATOPTION "STORAGE=memory"
FORMATOPTION "USE_FEATUREID=true"
END

Jstree - Add nodes to tree by user input (button click)

I have a ASP.NET MVC 4.0 project using JsTree. The JsTree is being populated thought a model that returns a JSON.
Now my problem is this, i have a huge tree that makes the user experience pretty bad. What i need is to load a few items (let'say 20), and have a button that when clicked by the user add's the next 20 records to the tree.
I have been searching in Google JsTree documentation and SO, but i haven't found a example that works for me.
Can anyone help me?
Thanks in advanced.
Ok, some break trought. I kind of got this working. In my view the user input call this function:
function getNextRecords() {
var new_data = { "attr": { "ID": "999999999999", "NodeText": "999999999999" },
"data": "999999999999",
"children": [{
"attr": { "ID": "ACT99", "NodeText": "969994222" },
"data": "969994222 - PPP",
"children": [{
"attr": { "ID": "TRFps800", "rel": "disabled" },
"data": "Voz com unlimited 800 fidelização até 01/11/2019",
"state": "open"
}],
"state": "open"
}], "state": "open"
};
var retDom = $.jstree._reference("#demoTree")._parse_json(new_data, -1, true);
$("#demoTree").jstree("move_node", retDom, -1, "inside", false, false, true);
}
This is working fine, expect that the parse json creates a "ul" instead of a "li" any ideas in how to change that?
OK, i finally got this working, this is the final version of my Function i hope this helps someone.
function getNextRecords() {
var new_data = { "attr": { "ID": "999999999999", "NodeText": "999999999999" },
"data": "999999999999",
"children": [{
"attr": { "ID": "ACT99", "NodeText": "969994222" },
"data": "969994222 - PPP",
"children": [{
"attr": { "ID": "TRFps800", "rel": "disabled" },
"data": "Voz com unlimited 800 fidelização até 01/11/2019",
"state": "open"
}], "state": "open"
}], "state": "open"
};
var ref = $("li ul:first")
var retDom = $.jstree._reference("#demoTree")._parse_json(new_data, ref, true);
$("#demoTree").jstree("move_node", retDom, ref, "first", false, false, true);
}
This add a new child after the first UL.
The second part "load part of the branch and with another load of the same branch
load more" is exactly what i'm trying to do but i have not yet succeeded, is it
possible to have an example?
Such task must be taken care of on server side. Take is this way: jsTree only displays whatever is passed to it. So you make a decision what to load in a first go and you can have a button (to reload the whole tree or a branch only) to load more or replace whatever was loaded. Don't forget the logic has (almost) nothing to do with jsTree.

Resources