Filtering chloropleth data using a slider for annual data in vega-lite - geojson

I have some geojson grids that map through to some annualised sales data over a period of 25 years. I am really struggling to filter this sales data by year to show the trends in a chloropleth map.
d3 = require('d3-dsv');
map_json = FileAttachment("Time_line#2.geojson").json()
sales_data = FileAttachment("Timeline_test#1.csv").csv()
vegalite = require('#observablehq/vega-lite')
vegalite ({
"$schema": "https://vega.github.io/schema/vega-lite/v5.json",
"width": 600,
"height": 350,
"data": {
"name": "mapdata",
"values": map_json,
"format": {"property": "features"},
},
"params": [{
"name" : "AnnualPeriod",
"value": 1995,
"bind" : {"input": "range", "min":1995, "max":2020,"step":1 }
}],
"transform" : [{
"lookup": "properties.id",
"from": {
"data": { "values": sales_data,},
"format":"csv",
"key": "derived_boundary_id",
"fields": ["sales_volume"],
},
},],
"layer": [
{
"mark": "geoshape",
"encoding": {
"color": {
"field": "sales_volume",
"type": "quantitative",
"scale": {"scheme": "Oranges"},
},
"stroke": { "value": "#ff75"},
},
},
]
})
I have tried to add transform.filter and cannot get it to work. At the moment it appears to be taking the first sales_data record for each of the boundary_ids.
I would like the data to be filtered according to the setting of the AnnualPeriod slider.
I think I need to include something like
"transform" :[{"filter": "datum.year == AnnualPeriod"}]
I have tried it in the transform section, with the lookup between the sales_data and the geojosn objects.
I have also tried to filter in and around the geoshape mark but neither work.
Does anyone have any ideas?
The is a sample of the sales_data:
sales_volume,year,derived_boundary_id
5,2015,602212
2,2016,602212
2,2019,602212
5,1995,602213
7,1996,602213
6,1997,602213
7,1998,602213
9,1999,602213
10,2000,602213
7,2001,602213
5,2002,602213
5,2003,602213
9,2004,602213
5,2005,602213
...
where the last column maps to an "id" in the geojson data.
and this is the 'map' that I get. Always the same, irrespective of the slider setting.

I have eventually worked out how to get this to work.
Create the main data source as the sales data and attach the maps/geojson to this via a transform/lookup.
It seems so simple now, but I thought I would post the result so others can see how it can be achieved.
These observations may help others:
The transform can take the "year" filter as well as the map_json lookup.
The map/json lookup is referenced as "geo" to make it easier to understand
The geoshape mark then references, via the encoding, this geojson "geo" object.
Thanks to Mike Bostock and the Observable and Vega-Lite team for their excellent work.
vegalite ({
"$schema": "https://vega.github.io/schema/vega-lite/v5.json",
"width": 600,
"height": 600,
"data": {
"values": sales_data,
},
"params": [{
"name" : "AnnualPeriod",
"value": 1995,
"bind" : {"input": "range", "min":1995, "max":2020,"step":1 }
}],
"transform" : [
{"filter" : "year(datum.year) === AnnualPeriod"},
{
"lookup": "derived_boundary_id",
"from" : {
"data": {
"values": map_json,
"format": {"property": "features"},
},
"key":"properties.id",
},
"as":"geo",
}
],
"mark":"geoshape",
"encoding": {
"shape": {
"field":"geo",
"type":"geojson",
},
"color":{
"field":"sales_volume",
"type":"quantitative",
"scale": {
"scheme":"Oranges",
"domain": [0,15],
},
}
}
})

How's your progress? I am not familiar with Choropleths, but by comparing with the Vega Examples, I spot 2 differences you may wanna take a look:
Choropleth needs a projection
Map data should be placed in lookup transform, and sale data in normal data. Ref 1 Ref 2
Do feel free to correct me if I am wrong. If you need more help, please share an editor with dummy data because I found it hard to make good use of the sale data you provided :'(

Related

Does TypeORM have a way to search for all documents with an array containing a value?

My goal is to use an input array of strings (fake emails) as a search query for documents in my MongoDB database, which I am powering using TypeORM. This way if I want to search for documents using more than one email at a time, I can do that. Meaning I want to be able to feed in:
query = ["kim#gmail.com", "jim#gmail.com", "sarah#gmail.com"] and get 3 different documents where document one has kim#gmail.com as the attendee, jim#gmail.com is another document's attendee field, and sarah#gmail.com is the third document's attendee (or is among them).
I want to use an email as a query to search for and return all documents where the array field has the email in the array.
So as an example here is the results for the "get all documents" endpoint right now:
[
{
"_id": "6283d7ad706445dc33319bcb",
"hostUsername": "jack",
"hostEmail": "jack#outlook.com",
"meetingName": "nervous-fish-hautily-vetting",
"startTime": "2022-12-12T08:00:00.000Z",
"attendees": [
"kate#gmail.com",
"sawyer#gmail.com"
]
},
{
"_id": "6284235e662f7dfb073e2cbc",
"hostUsername": "jacob",
"hostEmail": "jacob#gmail.com",
"meetingName": "eager-fish-hautily-vetting",
"startTime": "2022-12-12T08:00:00.000Z",
"attendees": [
"kate#gmail.com",
"benjaminlinus#gmail.com"
]
},
{
"_id": "6283d7c3706445dc33319bcc",
"hostUsername": "richard",
"hostEmail": "richard#outlook.com",
"meetingName": "eager-cat-hautily-subtracting",
"startTime": "2022-12-12T08:00:00.000Z",
"attendees": [
"johnlocke#gmail.com",
"hurley#gmail.com"
]
},
{
"_id": "6283d82b706445dc33319bcd",
"hostUsername": null,
"hostEmail": "richard#outlook.com",
"meetingName": "nervous-cat-hautily-jumping",
"startTime": "1970-01-01T00:00:00.000Z",
"attendees": null
},
{
"_id": "6283d8af706445dc33319bce",
"hostUsername": null,
"hostEmail": "richard#outlook.com",
"meetingName": "eager-plant-ignorantly-jumping",
"startTime": "1970-01-01T00:00:00.000Z",
"attendees": null
}
]
I want to query the database with ["kate#gmail.com"] and get back the two results that have "kate#gmail.com" in the attendees field.
The closest solution (that doesn't work) is the one I found in this GitHub issue and also another close solution (that doesn't work) in this StackOverflow question
Here is me implementing those two suggestions:
import { In } from "typeorm";
async searchMeetingsByDetails(
attendees?: string[]
): Promise<IMeeting[]> {
console.log(attendees, 39);
const meetingsByAttendees = attendees
? await this.meetingRepository.find({
where: {
attendees: In([...attendees]),
},
})
: [];
return [
meetingsByAttendees,
].flat();
}
This gives me an empty array [] when the input is ["kate#gmail.com"] so if the In() thing worked, it would give results.
const meetings = await this.meetingRepository
.createQueryBuilder("meeting")
.where("meeting.attendees IN (:attendees)", {
attendees: [...attendees],
});
This one gives ERROR [ExceptionsHandler] Query Builder is not supported by MongoDB. TypeORMError: Query Builder is not supported by MongoDB.

Application Insights API $select not returning all results when values share part of path

I'm not sure if this is an OData issue or an Application Insights issue, but the App Insights API is not giving me all of the values I selected. It works normally most of the time, but when I ask for two values that share the beginning of their path, it only gives me the second value I asked for.
Here's an example of my issue:
data:
{
"count": 1,
"type": "customEvent",
"customDimensions": {
"success": "true",
"version": "ver-1"
},
"other": {
"key": "val-1"
}
},
{
"count": 2,
"type": "customEvent",
"customDimensions": {
"success": "false",
"version": "ver-2"
},
"other": {
"key": "val-2"
}
}
These all return the results that I'm expecting:
Query: $select=count,type
{
"count": 1,
"type": "customEvent"
},
{
"count": 2,
"type": "customEvent"
}
Query: select=customDimensions/success,other/key
{
"customDimensions": {
"success":"true"
},
"other": {
"key":"ver-1"
}
},
{
"customDimensions": {
"success":"false"
},
"other": {
"key":"ver-2"
}
}
However, if I try to get two values that start with the same path, it only shows me the second one.
Query: select=customDimensions/success,customDimensions/version
{
"customDimensions": {
"version":"ver-1"
}
},
{
"customDimensions": {
"version":"ver-2"
}
}
Is this an issue with either OData or Application Insights, or is there some other way I can format my query to give me the information I want? Thanks!
Update:
You can use the query api as following to fetch the data:
https://api.applicationinsights.io/v1/apps/Your_application_id/query?query=requests
| where timestamp >ago(5h)
| project customDimensions.UsersNamed, customDimensions.TenantsCoded
I test it in postman, see screenshot below:
Seems that your App Insights query is ok, I tested it using this .
I fetch the operation/name and operation/id(which starts with same path), original like this:
Then input some necessary condition, as screenshot below:
After click "Fetch" button, you can see the operation/name and operation/id are both returned.

Openlayers bbox strategy

I have bbox strategy for one source of data. Code looks like this:
bbox: function newBboxFeatureSource(url, typename) {
return new ol.source.Vector({
loader: function (extent) {
let u = `${url}&TYPENAME=${typename}&bbox=${extent.join(",")}`;
$.ajax(u).then((response) => {
this.addFeatures(
geoJsonFormat.readFeatures(response)
);
});
},
strategy: ol.loadingstrategy.bbox
});
},
I works fine but... When I pan/move the map then this loader is calling again and add another features which fit to new box. But there is a lot of duplicates then because some of new features are just the same as old.
So I wanted first clear all features using this.clear() before add new features but when I add this command then loader is running all the time and I have "infinitive loop". Do you know why? How can I disable loading new features after calling this.clear()?
edit:
my response with features looks like this:
{ "type": "FeatureCollection", "crs": { "type": "name", "properties":
{ "name": "urn:ogc:def:crs:EPSG::3857" } },
"features": [ { "type": "Feature", "properties": { "ogc_fid": "2",
"name": "AL" }, "geometry": { "type": "MultiPolygon" , "coordinates":
[ [ [ ... ] ] ] } }, { "type": "Feature", "properties": { "ogc_fid":
"3", "name": "B" }, "geometry": { "type": "MultiPolygon" ,
"coordinates": [ [ [ ...] ] ] } } ..... and so on
I've removed coordinates because there was too many of them.
My features are generated by mapserver and are configured in .map file which looks like this:
LAYER
NAME "postcode_area_boundaries"
METADATA
"wfs_title" "Postcode area boundaries"
"wfs_srs" "EPSG:3857"
"wfs_enable_request" "*"
"wfs_getfeature_formatlist" "json"
"wfs_geomtype" "multipolygon"
"wfs_typename" "postcode_area_boundaries"
"wms_context_fid" "id"
"wfs_featureid" "id"
"gml_featureid" "id"
"gml_include_items" "id,postarea,wkb_geometry"
"gml_postarea_alias" "name"
"ows_featureid" "id"
"tinyows_table" "postcode_area_boundaries"
"tinyows_retrievable" "1"
"tinyows_include_items" "id,postarea,wkb_geometry"
END
TYPE POLYGON
STATUS ON
CONNECTIONTYPE POSTGIS
CONNECTION "..."
DATA "wkb_geometry FROM postcode_area_boundaries USING UNIQUE id"
DUMP TRUE
END
To summarize the discussion and answer the initial question:
The features sent by the server need an attribute called id, which must be unique and the same for the feature on every request.
{type: "Feature", id: "some-wfs.1234", properties: { "ogc_fid": 2, ...
See this GitHub Issue for the original comment of ahocevar.
In GeoServer this can be achieved if you set an identifier in your layer.
I guess there is something similar to set in MapServer.
Following up on #bennos comment: you can expose the fid in Mapserver using the FORMATOPTION USE_FEATUREID=true as described in: https://mapserver.org/output/ogr_output.html
USE_FEATUREID=true/false
Starting from MapServer v7.0.2. Defaults to false. Include feature ids in the generated output, if the ows_featureid metadata key is set at the layer level. The featureid column to use should be an integer column. Useful if you need to include an “id” attribute to your geojson output. Use with caution as some OGR output drivers may behave strangely when fed with random FIDs.
OUTPUTFORMAT
NAME "application/json"
DRIVER "OGR/GEOJSON"
MIMETYPE "application/json"
FORMATOPTION "FORM=SIMPLE"
FORMATOPTION "FILENAME=ol-query.json"
FORMATOPTION "STORAGE=memory"
FORMATOPTION "USE_FEATUREID=true"
END

SurveyMonkey API multiple choice with "other" option

So I've been looking into the surveymonkey v3 api documentation for formatting questions types. What I want to do is create a multiple choice question that has an "other" option, which if selected has a text field that a user can fill in to be more specific. Is there a way to accomplish this with the api?
You should be able to do that when creating/updating a question.
Example:
POST /v3/surveys/<id>/pages/<id>/questions
{
"family": "single_choice",
"subtype": "vertical",
"answers": {
"other": [{
"text": "Other (please specify)",
"is_answer_choice": true,
"num_lines": 1,
"num_chars": 50
}],
"choices": [
{
"text": "Apples"
},
{
"text": "Oranges"
},
{
"text": "Bananas"
}
]
},
"headings": [
{
"heading": "What is your favourite fruit?"
}
]
}
This is_answer_choice field seems to not be accepted currently. That is a bug, you can watch the docs to potentially get notified on updates, or try it again later.
Edit: This method should work now, give it a try and let me know if it solves your problem!

Jstree - Add nodes to tree by user input (button click)

I have a ASP.NET MVC 4.0 project using JsTree. The JsTree is being populated thought a model that returns a JSON.
Now my problem is this, i have a huge tree that makes the user experience pretty bad. What i need is to load a few items (let'say 20), and have a button that when clicked by the user add's the next 20 records to the tree.
I have been searching in Google JsTree documentation and SO, but i haven't found a example that works for me.
Can anyone help me?
Thanks in advanced.
Ok, some break trought. I kind of got this working. In my view the user input call this function:
function getNextRecords() {
var new_data = { "attr": { "ID": "999999999999", "NodeText": "999999999999" },
"data": "999999999999",
"children": [{
"attr": { "ID": "ACT99", "NodeText": "969994222" },
"data": "969994222 - PPP",
"children": [{
"attr": { "ID": "TRFps800", "rel": "disabled" },
"data": "Voz com unlimited 800 fidelização até 01/11/2019",
"state": "open"
}],
"state": "open"
}], "state": "open"
};
var retDom = $.jstree._reference("#demoTree")._parse_json(new_data, -1, true);
$("#demoTree").jstree("move_node", retDom, -1, "inside", false, false, true);
}
This is working fine, expect that the parse json creates a "ul" instead of a "li" any ideas in how to change that?
OK, i finally got this working, this is the final version of my Function i hope this helps someone.
function getNextRecords() {
var new_data = { "attr": { "ID": "999999999999", "NodeText": "999999999999" },
"data": "999999999999",
"children": [{
"attr": { "ID": "ACT99", "NodeText": "969994222" },
"data": "969994222 - PPP",
"children": [{
"attr": { "ID": "TRFps800", "rel": "disabled" },
"data": "Voz com unlimited 800 fidelização até 01/11/2019",
"state": "open"
}], "state": "open"
}], "state": "open"
};
var ref = $("li ul:first")
var retDom = $.jstree._reference("#demoTree")._parse_json(new_data, ref, true);
$("#demoTree").jstree("move_node", retDom, ref, "first", false, false, true);
}
This add a new child after the first UL.
The second part "load part of the branch and with another load of the same branch
load more" is exactly what i'm trying to do but i have not yet succeeded, is it
possible to have an example?
Such task must be taken care of on server side. Take is this way: jsTree only displays whatever is passed to it. So you make a decision what to load in a first go and you can have a button (to reload the whole tree or a branch only) to load more or replace whatever was loaded. Don't forget the logic has (almost) nothing to do with jsTree.

Resources