I'm trying to use Spearmint, the Bayesian optimization library, to tune hyperparameters for a machine learning classifier. My question is how does one express parameter search spaces that does not follow a uniform distribution?
From the project's github page, here is an example of how to set two uniformly distributed parameter search spaces:
"variables": {
"X": {
"type": "FLOAT",
"size": 1,
"min": -5,
"max": 10
},
"Y": {
"type": "FLOAT",
"size": 1,
"min": 0,
"max": 15
}
}
How would we define a search space like the one below in Spearmint?
SVC_PARAMS = [
{
"bounds": {
"max": 10.0,
"min": 0.01,
},
"name": "C",
"type": "double",
"transformation": "log",
},
{
"bounds": {
"max": 1.0,
"min": 0.0001,
},
"name": "gamma",
"type": "double",
"transformation": "log",
},
{
"type": "categorical",
"name": "kernel",
"categorical_values": [
{"name": "rbf"},
{"name": "poly"},
{"name": "sigmoid"},
],
},
]
Is there a place to look up all of the stochastic expressions (ie uniform, normal, log etc) currently being supported by Spearmint?
Spearmint learns these kinds of transformations automatically from the data. If you take a look here: https://github.com/HIPS/Spearmint/tree/master/spearmint/transformations
you can see the implementation of the beta warping that is applied (detailed in this paper: http://arxiv.org/abs/1402.0929). Spearmint doesn't have a way to specify these a-priori, but you could have Spearmint operate on e.g. the log of the parameters (by giving the log of the parameter ranges and exponentiating on your end).
Related
I have done some digging, but I could not find an answer. Is there a way to have plotly convert date times to the clients time zone (also respecting daylight savings)
I am combining data from different sources in my graph and some is in UTC format (e.g., 2021-01-06T06:00:00Z or 2021-01-06T06:00:00+00:00) and some is already in the client’s time zone (e.g., 2021-01-06 08:00:00).
As a result, the two are not aligned.
Here is an example of what I am sending to plotly (I have shortened it quite a bit)
[
{
"x": [
"2021-06-16T09:38:00Z",
"2021-06-16T09:40:00Z",
"2021-06-16T09:42:00Z",
"2021-06-16T09:44:00Z",
],
"y": [
11.0725,
9.1375,
3.8775,
16.98625,
],
"type": "scatter"
},
{
"x": [
"2021-06-16 11:00:07",
"2021-06-16 11:00:07"
],
"y": [
0.32065714285714289,
20.704228571428574
],
"type": "scatter",
"mode": "lines",
"line": {
"dash": "dot",
"color": "#f62447"
}
},
{
"x": [
"2021-06-16 11:00:07"
],
"y": [
22.334914285714289
],
"type": "scatter",
"mode": "text",
"text": [
"⚠"
],
"textfont": {
"size": "25",
"color": "#f62447"
}
}
]
I am trying to modify the quick sample provided here.
I tried to add a few custom sensor data type but it is failing. Then I tried a few data types mentioned in the documentation which also failed.
I am getting below error
Creating Sensor: {
"DataType": "Noise",
"DeviceId": "some-device-id",
"HardwareId": "SAMPLE_SENSOR_NOISE"
}
Request: POST
https://******.*******.azuresmartspaces.net/management/api/v1.0/sensors
Response Status: 404, NotFound , {"error":
{"code":"404.600.000.001","message":"There is no SensorDataType of the
given name."}}
Can we add custom sensor datatype?
If no then what are the inbuilt data types? or if yes then what went wrong here?
You need to post the DataType when creating the Sensor object. Use “None” if you want to change it later. Swagger DOCs show the “Model” you can expand and see required fields.
If the DataType is not in the api/v1/system/types you will need to enable it or create a new DataType. Create a new DataType POST to the Types with the required information. The minimum is the TypeName and SpaceID to neat the type under. My typical pattern is to create a root space and append any custom twin objects like types to this space.
I believe these are case sensitive names as well.
https://{servicename}.{region}.azuresmartspaces.net/management/swagger/ui/index#/Types
EDIT:
Check your Ontologies with:
https://{servicename}.{region}.azuresmartspaces.net/management/api/v1.0/ontologies
Select these by ID and POST to set them to true to get all available built-in types:
[
{
"id": 1,
"name": "Required",
"loaded": true
},
{
"id": 2,
"name": "Default",
"loaded": true
},
{
"id": 3,
"name": "BACnet",
"loaded": true
},
{
"id": 4,
"name": "Advanced",
"loaded": true
}
]
Then you can query all the given types:
https://{servicename}.{region}.azuresmartspaces.net/management/api/v1.0/types?includes=Description,FullPath,Ontologies,Space
You should receive something like:
[
{
"id": 1,
"category": "DeviceSubtype",
"name": "None",
"disabled": false,
"logicalOrder": 0,
"fullName": "None",
"spacePaths": [
"/system"
],
"ontologies": [
{
"id": 1,
"name": "Required",
"loaded": true
}
]
},
{
"id": 2,
"category": "DeviceType",
"name": "None",
"disabled": false,
"logicalOrder": 0,
"fullName": "None",
"spacePaths": [
"/system"
],
"ontologies": [
{
"id": 1,
"name": "Required",
"loaded": true
}
]
},
{
"id": 3,
"category": "DeviceBlobSubtype",
"name": "None",
"disabled": false,
"logicalOrder": 0,
"fullName": "None",
"spacePaths": [
"/system"
],
"ontologies": [
{
"id": 1,
"name": "Required",
"loaded": true
}
]
},
...Objects,
]
I am currently trying to export 3D geometry to GlTF and I come across an error I don't understand. In a file describing a simple grey cube, I get this on the normals accessor:
[glTF Validator] Accessor element at index 2 is not of unit length: 0.
[glTF Validator] Accessor element at index 5 is not of unit length: 0.
[glTF Validator] Accessor element at index 8 is not of unit length: 0.
[glTF Validator] Accessor element at index 11 is not of unit length: 0.
[glTF Validator] Accessor element at index 14 is not of unit length: 0.
[glTF Validator] Accessor element at index 17 is not of unit length: 0.
[glTF Validator] Accessor element at index 20 is not of unit length: 0.
[glTF Validator] Accessor element at index 23 is not of unit length: 0.
Here is the json:
{
"accessors": [
{
"bufferView": 0,
"byteOffset": 0,
"componentType": 5123,
"normalized": false,
"count": 36,
"type": "SCALAR",
"name": "31546_indices"
},
{
"bufferView": 1,
"byteOffset": 0,
"componentType": 5126,
"normalized": false,
"count": 8,
"type": "VEC3",
"max": [
32.808,
32.808,
32.808
],
"min": [
0.0,
0.0,
0.0
],
"name": "31546_vertices"
},
{
"bufferView": 2,
"byteOffset": 0,
"componentType": 5126,
"normalized": false,
"count": 8,
"type": "VEC3",
"name": "31546_normals"
},
{
"bufferView": 3,
"byteOffset": 0,
"componentType": 5126,
"normalized": false,
"count": 8,
"type": "VEC3",
"name": "31546_color"
}
],
"asset": {
"version": "2.0"
},
"buffers": [
{
"uri": "31546.bin",
"byteLength": 360,
"name": "31546"
}
],
"bufferViews": [
{
"buffer": 0,
"byteOffset": 0,
"byteLength": 72,
"name": "31546_indices"
},
{
"buffer": 0,
"byteOffset": 72,
"byteLength": 96,
"name": "31546_vertices"
},
{
"buffer": 0,
"byteOffset": 168,
"byteLength": 96,
"name": "31546_normals"
},
{
"buffer": 0,
"byteOffset": 264,
"byteLength": 96,
"name": "31546_color"
}
],
"meshes": [
{
"primitives": [
{
"attributes": {
"POSITION": 1,
"NORMAL": 2,
"COLOR_0": 3
},
"indices": 0,
"mode": 4
}
],
"name": "31546"
}
],
"nodes": [
{
"mesh": 0
}
],
"scene": 0,
"scenes": [
{
"nodes": [
0
],
"name": "RNT_Viewport"
}
]
}
I don't understand what the Validator is talking about accessors above 3 as there are only 4 accessors... To get this I used the GlTF plugin for Visual code. For the Khronos online validator, the JSON looks correct (https://github.khronos.org/glTF-Validator/), so at this point, I don't really know where is my mistake...
Thank you in advance for the insight :)
It's complaining here about the binary data, in the 31546.bin file referenced by your JSON. If you click one of the messages in the document problems window, it should focus the cursor on the accessor in question (and I'll go on a limb and guess that would be the accessor at index 2, named 31546_normals, as that looks like the only one that should be normalized in this model).
The actual index values reported in these messages are indexes into the data contained within this accessor. In VSCode, with the correct accessor selected, press ALT + d to decode the binary data into a text buffer, to examine it as text.
My guess as to what's happened here is that you have some zero-length normal vectors in your model. This isn't a big problem if the zero-length vectors are applied to degenerate triangles, but, that sort of thing is at best a waste of space in the bin file that could be removed, so the validator flags it with a warning.
If you're editing this model in some other tool like Blender or Maya, you might have an option to find and remove degenerate triangles, and recalculate normal vectors. This might get rid of your zero-length normals.
I have a list of words and phrases together with as score and a definition for each. I would like to present this as an interactive wordcloud where the text sizes are determined by the scores and the definitions appear as tooltips on hover. I would prefer to do this in Jupyter.
I know a number libraries that offer nice ways to generate wordclouds and/or tooltips. How I attach the tooltips to the words in the wordcloud?. The wordcloud needs to have a way of knowing what text you are hovering over and trigger the corresponding tooltip. I have not found a way to do that so far.
I am fairly agnostic regarding the linraries used to do this.
I mainly want the result to be fairly high-level and mostly declarative.
I have looked at Vega, bqplot and Andreas Mueller's wordcloud package.
Vega has both wordcloud and tooltip functionality and is designed to compose piplines nicely, but I am not sure how to connect them the right way. I would also prefer to write actual Python code rather than code using JSON though, but that is a minor concern.
Bqplot does tootips very nicely but does not have a wordcloud component.
The wordcloud package generates nice wordclouds but I do not know how to make them interactive.
I have done this using both ipyvega and brunel brunel is much simpler but I do not like its wordcloud layout.
Brunel
df = pd.DataFrame(data, columns=['word', 'size', 'text'])
%brunel cloud size(size) label(word) tooltip(text)
ipyvega
spec = {
"$schema": "https://vega.github.io/schema/vega/v3.json",
"name": "wordcloud",
"width": width,
"height": height,
"padding": 0,
"data" : [
{
'name' : 'table',
'values' : [{'word': word, 'text': text, 'size': size}
for word, text size in data]
}
],
"scales": [
{
"name": "color",
"type": "ordinal",
"range": ["#d5a928", "#652c90", "#939597"]
}
],
"marks": [
{
"type": "text",
"from": {"data": "table"},
"encode": {
"enter": {
"text": {"field": "word"},
"align": {"value": "center"},
"baseline": {"value": "alphabetic"},
"fill": {"scale": "color", "field": "word"},
"tooltip": {"field": "text", "type": "nominal"}
},
"update": {
"fillOpacity": {"value": 1}
},
},
"transform": [
{
"type": "wordcloud",
"size": [width, height],
"text": {"field": "text"},
"font": "Helvetica Neue, Arial",
"fontSize": {"field": "datum.size"},
}
]
}
],
}
Vega(spec)
The drawGeometry method will not apply a point style to a polygon. That makes sense but I have a requirement that a polygon feature should be represented with both a stroke+fill style suitable for the feature as well as a point style, IMO, less-suitable for the feature.
For example:
Using a combination of polygon and point feature styles:
[
{
"fill": {
"pattern": {
"orientation": "diagonal",
"color": "rgba(230,113,26,1)",
"spacing": 3,
"repitition": "repeat"
}
}
},
{
"circle": {
"fill": { "color": "blue" },
"opacity": 1,
"stroke": {
"color": "rgba(0,255,0,1)",
"width": 1
},
"radius": 20
}
},
{
"image": {
"anchor": [
16,
48
],
"imgSize": [
32,
48
],
"anchorXUnits": "pixels",
"anchorYUnits": "pixels",
"src": "http://openlayers.org/en/v3.17.1/examples/data/icon.png"
}
}
]
One solution I've come up with is to replace the drawGeometry to call both drawPolygon and drawPoint:
But it seems like support for rendering a polygon using a point styling should be supported some other way. Maybe in the drawPolygon implementation it should detect a point style and react accordingly?
Use the geometry option of a style object and then return the interior point with geometry.getInteriorPoint(), see: http://openlayers.org/en/latest/examples/polygon-styles.html?q=geometry+style for an example