Flocker data migration - docker

I'm using flocker to persist and migrate docker containers data through ZfS Dataset backend.
docker run -v test:/data --volume-driver flocker busybox
sh -c "echo hello world > /data/file.txt"
I had this error for a week:
Unable to find image 'busybox:latest' locally latest: Pulling from
library/busybox 583635769552: Pull complete b175bcb79023: Pull
complete Digest:
sha256:c1bc9b4bffe665bf014a305cc6cf3bca0e6effeb69d681d7a208ce741dad58e0
Status: Downloaded newer image for busybox:latest Error response from
daemon: Cannot start container
128ddff1c0e9d6740c23b2f475b14206775a131878b4ed725a3280e22de79666:
Timed out waiting for dataset to mount...
Any help would be appreciated.
flocker-docker-plugin.log
{
"task_uuid": "4001196a-902c-4139-8b4f-e217490242ab",
"error": true,
"timestamp": 1459172727.570846,
"message": "Unhandled Error\nTraceback (most recent call last):\n File \"/opt/flocker/local/lib/python2.7/site-packages/flocker/dockerplugin/_script.py\", line 93, in docker_plugin_main\n options=DockerPluginOptions()).main()\n File \"/opt/flocker/local/lib/python2.7/site-packages/flocker/common/script.py\", line 294, in main\n self._react(run_and_log, [], _reactor=self._reactor)\n File \"/opt/flocker/local/lib/python2.7/site-packages/twisted/internet/task.py\", line 882, in react\n finished = main(_reactor, *argv)\n File \"/opt/flocker/local/lib/python2.7/site-packages/flocker/common/script.py\", line 282, in run_and_log\n d = maybeDeferred(self.script.main, reactor, options)\n--- <exception caught here> ---\n File \"/opt/flocker/local/lib/python2.7/site-packages/twisted/internet/defer.py\", line 150, in maybeDeferred\n result = f(*args, **kw)\n File \"/opt/flocker/local/lib/python2.7/site-packages/flocker/dockerplugin/_script.py\", line 71, in main\n certificates_path.child(b\"plugin.key\"))\n File \"/opt/flocker/local/lib/python2.7/site-packages/flocker/apiclient/_client.py\", line 592, in __init__\n cert_path, key_path)\n File \"/opt/flocker/local/lib/python2.7/site-packages/flocker/ca/_validation.py\", line 137, in treq_with_authentication\n user_credential = UserCredential.from_files(user_cert_path, user_key_path)\n File \"/opt/flocker/local/lib/python2.7/site-packages/flocker/ca/_ca.py\", line 371, in from_files\n certificate = load_certificate_file(certificate_path)\n File \"/opt/flocker/local/lib/python2.7/site-packages/flocker/ca/_ca.py\", line 232, in load_certificate_file\n e.filename, code, failure\nflocker.ca._ca.PathError: Certificate file could not be opened. No such file or directory /etc/flocker/plugin.crt\n",
"message_type": "twisted:log",
"task_level": [
1
]
}
{
"task_uuid": "846b9f01-f618-4723-bf9e-2ce7ac6b79c9",
"error": true,
"timestamp": 1459172727.57451,
"message": "main function encountered error\nTraceback (most recent call last):\n File \"/opt/flocker/local/lib/python2.7/site-packages/flocker/dockerplugin/_script.py\", line 93, in docker_plugin_main\n options=DockerPluginOptions()).main()\n File \"/opt/flocker/local/lib/python2.7/site-packages/flocker/common/script.py\", line 294, in main\n self._react(run_and_log, [], _reactor=self._reactor)\n File \"/opt/flocker/local/lib/python2.7/site-packages/twisted/internet/task.py\", line 882, in react\n finished = main(_reactor, *argv)\n File \"/opt/flocker/local/lib/python2.7/site-packages/flocker/common/script.py\", line 282, in run_and_log\n d = maybeDeferred(self.script.main, reactor, options)\n--- <exception caught here> ---\n File \"/opt/flocker/local/lib/python2.7/site-packages/twisted/internet/defer.py\", line 150, in maybeDeferred\n result = f(*args, **kw)\n File \"/opt/flocker/local/lib/python2.7/site-packages/flocker/dockerplugin/_script.py\", line 71, in main\n certificates_path.child(b\"plugin.key\"))\n File \"/opt/flocker/local/lib/python2.7/site-packages/flocker/apiclient/_client.py\", line 592, in __init__\n cert_path, key_path)\n File \"/opt/flocker/local/lib/python2.7/site-packages/flocker/ca/_validation.py\", line 137, in treq_with_authentication\n user_credential = UserCredential.from_files(user_cert_path, user_key_path)\n File \"/opt/flocker/local/lib/python2.7/site-packages/flocker/ca/_ca.py\", line 371, in from_files\n certificate = load_certificate_file(certificate_path)\n File \"/opt/flocker/local/lib/python2.7/site-packages/flocker/ca/_ca.py\", line 232, in load_certificate_file\n e.filename, code, e file could nfailure\nflocker.ca._ca.PathError: Certificatot be opened. No such file or directory /etc/flocker/plugin.crt\n",
"message_type": "twisted:log",
"task_level": [
1
]
}{
"task_uuid": "7aab456b-a754-4160-8bcb-0b618a63ecda",
"error": false,
"timestamp": 1459172727.575473,
"message": "Main loop terminated.",
"message_type": "twisted:log",
"task_level": [
1
]
}{
"task_uuid": "5b490165-8eb1-4755-9c87-1b49f00bc700",
"error": false,
"timestamp": 1459172728.921761,
"message": "Log opened.",
"message_type": "twisted:log",
"task_level": [
1
]
}
flocker-dataset-agent.log
{
"task_uuid": "e0a6549f-7515-427f-abb6-b31379895bde",
"cluster_state": {
"node_uuid_to_era": {
"values": [
[
{
"hex": "e4b23086-3d3e-44c4-acc4-8b5d31c8fc9b",
"$__class__$": "UUID"
},
{
"hex": "c1ccd75f-14c4-4e33-a9a9-b6ea876c5a05",
"$__class__$": "UUID"
}
],
[
{
"hex": "70f6f5dd-378c-4913-8b2f-0b0f1f55f0a8",
"$__class__$": "UUID"
},
{
"hex": "f824f593-8723-47f3-8605-a45b2262b268",
"$__class__$": "UUID"
}
]
],
"$__class__$": "PMap"
},
"nodes": [
{
"paths": {
"values": [
[
"7645c292-5329-4aa3-b606-981c2f4e4892",
{
"path": "/flocker/55ab515f-43eb-488b-b3a3-e65fa5c62249.default.7645c292-5329-4aa3-b606-981c2f4e4892",
"$__class__$": "FilePath"
}
]
],
"$__class__$": "PMap"
},
"uuid": {
"hex": "e4b23086-3d3e-44c4-acc4-8b5d31c8fc9b",
"$__class__$": "UUID"
},
"$__class__$": "NodeState",
"hostname": "192.168.224.7",
"devices": {
"values": [
],
"$__class__$": "PMap"
},
"applications": [
],
"manifestations": {
"values": [
[
"7645c292-5329-4aa3-b606-981c2f4e4892",
{
"dataset": {
"deleted": false,
"dataset_id": "7645c292-5329-4aa3-b606-981c2f4e4892",
"metadata": {
"values": [
],
"$__class__$": "PMap"
},
"maximum_size": 80530636800,
"$__class__$": "Dataset"
},
"primary": true,
"$__class__$": "Manifestation"
}
]
],
"$__class__$": "PMap"
}
},
{
"paths": {
"values": [
],
"$__class__$": "PMap"
},
"uuid": {
"hex": "70f6f5dd-378c-4913-8b2f-0b0f1f55f0a8",
"$__class__$": "UUID"
},
"$__class__$": "NodeState",
"hostname": "192.168.224.89",
"devices": {
"values": [
],
"$__class__$": "PMap"
},
"applications": [
],
"manifestations": {
"values": [
],
"$__class__$": "PMap"
}
}
],
"nonmanifest_datasets": {
"values": [
],
"$__class__$": "PMap"
},
"$__class__$": "DeploymentState"
},
"action_type": "flocker:agent:converge",
"desired_configuration": {
"persistent_state": {
"blockdevice_ownership": {
"values": [
],
"$__class__$": "PMap"
},
"$__class__$": "PersistentState"
},
"nodes": [
{
"applications": [
],
"manifestations": {
"values": [
[
"7645c292-5329-4aa3-b606-981c2f4e4892",
{
"dataset": {
"deleted": false,
"dataset_id": "7645c292-5329-4aa3-b606-981c2f4e4892",
"metadata": {
"values": [
[
"name",
"apples"
]
],
"$__class__$": "PMap"
},
"maximum_size": 80530636800,
"$__class__$": "Dataset"
},
"primary": true,
"$__class__$": "Manifestation"
}
]
],
"$__class__$": "PMap"
},
"uuid": {
"hex": "70f6f5dd-378c-4913-8b2f-0b0f1f55f0a8",
"$__class__$": "UUID"
},
"$__class__$": "Node"
},
{
"applications": [
],
"manifestations": {
"values": [
],
"$__class__$": "PMap"
},
"uuid": {
"hex": "e4b23086-3d3e-44c4-acc4-8b5d31c8fc9b",
"$__class__$": "UUID"
},
"$__class__$": "Node"
}
],
"leases": {
"values": [
],
"$__class__$": "PMap"
},
"$__class__$": "Deployment"
},
"timestamp": 1459176487.344602,
"action_status": "started",
"task_level": [
2,
1
]
}{
"timestamp": 1459176487.345864,
"task_uuid": "e0a6549f-7515-427f-abb6-b31379895bde",
"action_type": "flocker:agent:discovery",
"action_status": "started",
"task_level": [
2,
2,
1
]
}{
"fsm_next_state": "<ConvergenceLoopStates=CONVERGING>",
"task_level": [
3
],
"action_type": "fsm:transition",
"timestamp": 1459176487.348982,
"fsm_output": [
"<ConvergenceLoopOutputs=CLEAR_WAKEUP>",
"<ConvergenceLoopOutputs=CONVERGE>"
],
"task_uuid": "e0a6549f-7515-427f-abb6-b31379895bde",
"action_status": "succeeded"
}{
"task_uuid": "e0a6549f-7515-427f-abb6-b31379895bde",
"task_level": [
2,
2,
2
],
"action_type": "flocker:agent:discovery",
"timestamp": 1459176487.354977,
"state": "NodeLocalState(node_state=NodeState(applications=None, paths=UnicodeFilepathPMap({}), manifestations=UnicodeManifestationPMap({}), hostname=u'192.168.224.89', uuid=UUID('70f6f5dd-378c-4913-8b2f-0b0f1f55f0a8'), devices=UuidFilepathPMap({})))",
"action_status": "succeeded"
}{
"timestamp": 1459176487.356563,
"task_uuid": "e0a6549f-7515-427f-abb6-b31379895bde",
"message_type": "flocker:agent:converge:actions",
"task_level": [
2,
3
],
"calculated_actions": "NoOp(sleep=datetime.timedelta(0, 1))"
}{
"timestamp": 1459176487.356925,
"task_uuid": "e0a6549f-7515-427f-abb6-b31379895bde",
"action_type": "flocker:change:noop",
"action_status": "started",
"task_level": [
2,
4,
1
]
}{
"timestamp": 1459176487.357233,
"task_uuid": "e0a6549f-7515-427f-abb6-b31379895bde",
"action_type": "flocker:change:noop",
"action_status": "succeeded",
"task_level": [
2,
4,
2
]
}

The logs contain the following error:
_ca.PathError: Certificate file could not be opened. No such file or directory /etc/flocker/plugin.crt
This means you are missing the necessary certificates in order for the docker plugin to communicate correctly.
Please see the below link to create the necessary certificates.
https://docs.clusterhq.com/en/latest/docker-integration/generate-api-plugin.html

The problem is still there. The full discussion with Ryan can be found in this link:
https://groups.google.com/forum/#!topic/flocker-users/l5wCW-U1zKs

Related

How to get a bone's weight using a json file

I am trying to get the weight for every bone using a gltf json file but i dont know how
here is the json file i am trying to get the weight from:
{
"accessors": [
{
"bufferView": 0,
"componentType": 5126,
"count": 317,
"max": [
10.570027351379395,
15.918471336364746,
6.125584125518799
],
"min": [
-2.611180305480957,
-1.9711413383483887,
-7.055622577667236
],
"type": "VEC3"
},
{
"bufferView": 1,
"componentType": 5126,
"count": 317,
"type": "VEC3"
},
{
"bufferView": 2,
"componentType": 5126,
"count": 317,
"type": "VEC2"
},
{
"bufferView": 3,
"componentType": 5121,
"count": 317,
"type": "VEC4"
},
{
"bufferView": 4,
"componentType": 5126,
"count": 317,
"type": "VEC4"
},
{
"bufferView": 5,
"componentType": 5123,
"count": 768,
"type": "SCALAR"
},
{
"bufferView": 6,
"componentType": 5126,
"count": 5,
"type": "MAT4"
}
],
"asset": {
"generator": "Khronos glTF Blender I/O v1.5.17",
"version": "2.0"
},
"bufferViews": [
{
"buffer": 0,
"byteLength": 3804,
"byteOffset": 0
},
{
"buffer": 0,
"byteLength": 3804,
"byteOffset": 3804
},
{
"buffer": 0,
"byteLength": 2536,
"byteOffset": 7608
},
{
"buffer": 0,
"byteLength": 1268,
"byteOffset": 10144
},
{
"buffer": 0,
"byteLength": 5072,
"byteOffset": 11412
},
{
"buffer": 0,
"byteLength": 1536,
"byteOffset": 16484
},
{
"buffer": 0,
"byteLength": 320,
"byteOffset": 18020
}
],
"buffers": [
{
"byteLength": 18340,
"uri": "manoe.bin"
}
],
"images": [
{
"mimeType": "image/png",
"name": "Material_002_baseColor",
"uri": "Material_002_baseColor.png"
}
],
"materials": [
{
"doubleSided": true,
"name": "Material.001",
"pbrMetallicRoughness": {
"baseColorTexture": {
"index": 0
},
"metallicFactor": 0,
"roughnessFactor": 0.4000000059604645
}
}
],
"meshes": [
{
"name": "Cone.001",
"primitives": [
{
"attributes": {
"JOINTS_0": 3,
"NORMAL": 1,
"POSITION": 0,
"TEXCOORD_0": 2,
"WEIGHTS_0": 4
},
"indices": 5,
"material": 0
}
]
}
],
"nodes": [
{
"name": "Bone.004",
"rotation": [
0.17750653624534607,
-2.8648443617385055e-8,
-0.9102426171302795,
0.37409862875938416
],
"scale": [
1.000000238418579,
1,
1
],
"translation": [
-4.411018750261064e-8,
0.8281745314598083,
1.4894996525072202e-7
]
},
{
"children": [
0
],
"name": "Bone.002",
"rotation": [
0.2184765785932541,
-0.2617807984352112,
-0.7560504078865051,
0.558682918548584
],
"scale": [
1.0000003576278687,
1,
1
],
"translation": [
6.093483762015239e-8,
0.8281747698783875,
-1.283853023892334e-9
]
},
{
"name": "Bone.003",
"rotation": [
-0.4916436970233917,
7.804754176277129e-8,
-0.14189021289348602,
0.8591586947441101
],
"scale": [
1,
1.0000001192092896,
0.9999999403953552
],
"translation": [
-2.4224684125329077e-7,
0.8281750679016113,
-2.68131650216219e-9
]
},
{
"children": [
1,
2
],
"name": "Bone.001",
"rotation": [
-0.11594496667385101,
0.08448944985866547,
-0.5036448836326599,
0.8519155383110046
],
"scale": [
1.000000238418579,
0.9999995827674866,
1
],
"translation": [
-1.357730283757519e-8,
1.0000007152557373,
-3.471412068391244e-14
]
},
{
"children": [
3
],
"name": "Bone",
"rotation": [
2.3709270635663415e-7,
2.8494374859633353e-8,
6.9593789722201e-15,
1
],
"scale": null,
"translation": null
},
{
"mesh": 0,
"name": "Cone.001",
"skin": 0
},
{
"children": [
5,
4
],
"name": "ArmatureBase",
"scale": [
5.400000095367432,
5.400000095367432,
5.400000095367432
]
}
],
"samplers": [
{
"magFilter": 9729,
"minFilter": 9987
}
],
"scene": 0,
"scenes": [
{
"name": "Scene",
"nodes": [
6
]
}
],
"skins": [
{
"inverseBindMatrices": 6,
"joints": [
4,
3,
1,
0,
2
],
"name": "ArmatureBase"
}
],
"textures": [
{
"sampler": 0,
"source": 0
}
]
}
The glTF overview card summarizes it nicely:
Every vertex can be associated with up to 4 joints, each with a given weight.
These are encoded in the components of the JOINTS_0 and WEIGHTS_0 accessors. In your case those are accessors 3 and 4 and bufferviews 3 and 4 with a component type of byte and float, respectively.
One more indirection into the bufferViews component here tells you where in the binary file to look.

Issue with fetching a custom tileset from CARTO and displaying it on a MapBox map iOS Swift

I'm attempting to fetch a custom tileset from CARTO using the documentation I found that the way to fetch from the API doc is:
curl --location --request GET 'https://gcp-us-east1.api.carto.com/v3/maps/carto_dw/tileset?name=carto-demo-data.demo_tilesets.nasadem_glo_quadgrid15' --header 'Authorization: Bearer <your access token>'
I made that request on Swift as follows:
{
let urlString = "\(environment.apiBaseURL)/v3/maps/\(environment.connection)/tileset"
return Future<CartoTileJsonResponse, Error> { [weak self] promise in
self?.session
.request(urlString,
method: .get,
parameters: ["name": tileSetName]
)
.validate()
.responseData(completionHandler: { response in
if let error = response.error {
promise(.failure(error))
return
}
guard let data = response.value else {
promise(.failure(response.error ?? NSError(domain: "unknow error", code: 999, userInfo: nil)))
return
}
do {
let cartoTileJsonResponse = try JSONDecoder().decode(CartoTileJsonResponse.self, from: data)
promise(.success(cartoTileJsonResponse))
} catch(let error) {
debugPrint(error)
promise(.failure(error))
}
})
}
}
As the name of the function says this return a new URL which I also make a GET request which finally returns a tilejson as shown:
{
"name": "NiallsFarm_Tileset_1_NDVI",
"tilejson": "3.1.0",
"center": [
-2.637422918924969,
55.96538584577466,
15
],
"description": "",
"maxzoom": 15,
"tilestats": {
"layers": [
{
"layer": "default",
"attributeCount": 2,
"count": 22146,
"geometry": "Polygon",
"attributes": [
{
"type": "String",
"attribute": "fieldName",
"categories": [
{
"category": "South Tempriggs",
"frequency": 7587
},
{
"category": "Kirklands",
"frequency": 6603
},
{
"category": "East Lothian",
"frequency": 4677
},
{
"category": "Crumstanes",
"frequency": 3279
}
]
},
{
"quantiles": {
"3": [
0.5691,
0.658
],
"4": [
0.5328,
0.6307,
0.6627
],
"5": [
0.5112,
0.5942,
0.6515,
0.6658
],
"6": [
0.5004,
0.5699,
0.6307,
0.6578,
0.6675
],
"7": [
0.4899,
0.5509,
0.6055,
0.6464,
0.6604,
0.6689
],
"8": [
0.4824,
0.5344,
0.5841,
0.6303,
0.6545,
0.6627,
0.6697
],
"9": [
0.4768,
0.5212,
0.5705,
0.6123,
0.6436,
0.6576,
0.6646,
0.6705
],
"10": [
0.4708,
0.5112,
0.5574,
0.5937,
0.6303,
0.6516,
0.6596,
0.6657,
0.6712
],
"11": [
0.4674,
0.504,
0.5447,
0.5801,
0.6151,
0.6413,
0.6555,
0.6613,
0.6666,
0.6718
],
"13": [
0.4594,
0.4931,
0.5235,
0.5607,
0.5872,
0.617,
0.6401,
0.6534,
0.6591,
0.6641,
0.6682,
0.6726
],
"14": [
0.4542,
0.4899,
0.5164,
0.5502,
0.5779,
0.6048,
0.6303,
0.6468,
0.6559,
0.6606,
0.665,
0.6688,
0.6731
],
"15": [
0.4514,
0.4866,
0.5112,
0.541,
0.5705,
0.5937,
0.6192,
0.6387,
0.6516,
0.6576,
0.6617,
0.6657,
0.6693,
0.6732
],
"16": [
0.4496,
0.4824,
0.5052,
0.5344,
0.5626,
0.5841,
0.6089,
0.6303,
0.6448,
0.6545,
0.6588,
0.6627,
0.6663,
0.6697,
0.6736
],
"17": [
0.4451,
0.4792,
0.5026,
0.5263,
0.5538,
0.5767,
0.5977,
0.6207,
0.6381,
0.6504,
0.6562,
0.66,
0.6637,
0.6669,
0.6702,
0.6738
],
"18": [
0.4417,
0.476,
0.499,
0.521,
0.547,
0.571,
0.5885,
0.612,
0.6304,
0.6434,
0.6531,
0.6578,
0.6609,
0.6646,
0.6675,
0.6706,
0.674
],
"19": [
0.4394,
0.4729,
0.4952,
0.5154,
0.5407,
0.5643,
0.5821,
0.6018,
0.6226,
0.6372,
0.6483,
0.6551,
0.6587,
0.662,
0.6653,
0.668,
0.6709,
0.6742
],
"20": [
0.4366,
0.4708,
0.4921,
0.5112,
0.5341,
0.5575,
0.576,
0.5941,
0.6135,
0.6304,
0.6423,
0.6521,
0.6564,
0.6596,
0.6627,
0.6658,
0.6684,
0.6712,
0.6744
],
"12We ": [
0.4622,
0.4983,
0.5344,
0.5705,
0.6005,
0.6303,
0.649,
0.6576,
0.6627,
0.6675,
0.6722
]
},
"attribute": "NDVI",
"min": 0.2076,
"avg": 0.5944896234082888,
"type": "Number",
"max": 0.6885,
"sum": "13165.56719999999999590190036258263717172608"
}
]
}
],
"layerCount": 1
},
"tiles": [
"https://gcp-us-east1.api.carto.com/v3/maps/omegacrop-production/tileset/{z}/{x}/{y}?name={NAME}&partition=12_15_16142_16145_10208_10211_3999_1&formatTiles=mvt&cache=1653383237150&access_token={ACCES_TOKEN}"
],
"minzoom": 12,
"attribution": "",
"bounds": [
-2.6481308960412906,
55.9604080548856,
-2.62664091461468,
55.97227003506276
],
"vector_layers": [
{
"field": {
"fieldName": "String",
"NDVI": "Number"
},
"id": "default",
"minzoom": 12,
"geometry_type": "Polygon",
"maxzoom": 15
}
]
}
Which I'm not being able to display on Mapbox Map using either Raster or Vector Layers. Also when trying to do a GET request from the tiles url the tilejson return I'm receiving the following error:
{
"error": "invalid zoom value, must be an integer number greater than or equal to 0",
"status": 400,
"message": "invalid zoom value, must be an integer number greater than or equal to 0",
"z": null,
"x": null,
"y": null
}
If anyone have experienced this or may know what I may be doing wrong it would be highly appreciated.

500 Internal Server Error when uploading GeoJSON to Azure Maps Data Service

I always receive 500 Internal Service Error
POST: https://atlas.microsoft.com/mapData/upload?subscription-key=&api-version=1.0&dataFormat=geojson
Sample data:
{
"type": "FeatureCollection",
"features": [
{
"type": "Feature",
"geometry": {
"type": "Polygon",
"coordinates": [
[
[
-122.13393688201903,
47.63829579223815
],
[
-122.13389128446579,
47.63782047131512
],
[
-122.13240802288054,
47.63783312249837
],
[
-122.13238388299942,
47.63829037035086
],
[
-122.13393688201903,
47.63829579223815
]
]
]
},
"properties": {
"geometryId": "1"
}
},
{
"type": "Feature",
"geometry": {
"type": "Polygon",
"coordinates": [
[
[
-122.13374376296996,
47.63784758098976
],
[
-122.13277012109755,
47.63784577367854
],
[
-122.13314831256866,
47.6382813338708
],
[
-122.1334782242775,
47.63827591198201
],
[
-122.13374376296996,
47.63784758098976
]
]
]
},
"properties": {
"geometryId": "2",
"validityTime": {
"expiredTime": "2019-01-15T00:00:00",
"validityPeriod": [
{
"startTime": "2019-01-08T01:00:00",
"endTime": "2019-01-08T17:00:00",
"recurrenceType": "Daily",
"recurrenceFrequency": 1,
"businessDayOnly": true
}
]
}
}
}
]
}
Need help

Getting error object field starting or ending with a [.]

While inserting the below document in an Elasticsearch index:
{
"id": "122223334444",
"name": "Mei",
"url": "mei-2019-tamil",
"alternate_urls": [
"mei-2019-tamil",
"sa-baskaran-aishwarya-rajesh-untitled"
],
"type": "Movie",
"poster": "ed3e439b-1ac1-45fe-a915-a5dae60257df",
"poster_url": "//assets.appserver.com/ed3e439b-1ac1-45fe-a915-a5dae60257df",
"alternate_names": [
"Mei",
"SA Baskaran - Aishwarya Rajesh Untitled"
],
"popularity": 0.2,
"info": {
"running_time": 0,
"cpl_types": [
"teaser",
"feature"
],
"has_cpls": true,
"genres": [
"Drama",
"Thriller"
],
"international_release_date": null,
"country_specific_release_dates": {},
"international_film_status": "CS",
"country_specific_film_statuses": {
"IN": "CS",
"CN": "CS",
"": "CS",
"SG": "CS"
},
"country_specific_certifications": {},
"language": "Tamil",
"synopsis": "A thriller film directed by SA Baskaran, starring Aishwarya Rajesh in the lead role.",
"schedules": {
"cities": [],
"countries": []
},
"featured": 0,
"movie_rating": 0,
"cast": [
{
"id": "05ffe715-db60-4947-a45a-99722537571c",
"name": "Aishwarya Rajesh",
"url": "aishwarya-rajesh",
"role": "Actress",
"poster": "65ab15b6-d54a-4965-95d5-38a839cee17d",
"poster_url": "//assets.appserver.com/65ab15b6-d54a-4965-95d5-38a839cee17d",
"type": "Person"
}
],
"crew": [
{
"id": "d9354648-5f48-4bf0-9a00-3de8c4d7a8d0",
"name": "SA Baskaran",
"url": "sa-baskaran",
"role": "Director",
"poster": null,
"poster_url": null,
"type": "Person"
}
]
},
"published": true
}
I'm getting the following Message:
Error: object field starting or ending with a [.] makes object
resolution ambiguous:
However, there is no value that is starting with .
I am clueless as regards which key is causing this issue.
Please help me identify and fix this issue.
ElasticSearch Version: 5.6.14
I am trying to index ES from a rails app using chewy gem.
Values of the JSON could be empty. But if the keys are empty then ES throws an error while indexing. Remove "":{} from the JSON and index again.
Reference: https://discuss.elastic.co/t/object-field-starting-or-ending-with-a-makes-object-resolution-ambiguous/123351

How to add dashboard configuration json file in Grafana image?

I have a grafana docker image which have hawkular-datasource pre-configured using configuration files.
After after running grafana instance, I have a json given by teammate, which can be imported inside grafana and that json file creates dashboard when imported.
How do I make that dashboards appear by default in Grafana instance?
I tried copying the json file to /etc/grafana/provisioning/dashboards/ folder and created a new docker image. But when I run the image, the instance doesn't contain the dashboard at the homepage or anywhere in it.
How do I add this json file in docker image. Am I following the correct way?
I tried this http://docs.grafana.org/administration/provisioning/ But it didn't help out much. Any suggestion?
Here is the json file.
{
"id": null,
"title": "Openshift Metrics",
"tags": [],
"style": "dark",
"timezone": "browser",
"editable": true,
"hideControls": false,
"sharedCrosshair": false,
"rows": [
{
"collapse": false,
"editable": true,
"height": "322px",
"panels": [
{
"content": "<center><p style='font-size: 40pt'>$app</p></center>",
"editable": true,
"error": false,
"id": 23,
"isNew": true,
"links": [],
"mode": "html",
"repeatIteration": 1476706310439,
"scopedVars": {},
"span": 2,
"style": {
"font-size": "36pt"
},
"title": "",
"type": "text"
},
{
"aliasColors": {},
"bars": false,
"datasource": "Hawk-DS",
"editable": true,
"error": false,
"fill": 1,
"grid": {
"threshold1": null,
"threshold1Color": "rgba(216, 200, 27, 0.27)",
"threshold2": null,
"threshold2Color": "rgba(234, 112, 112, 0.22)"
},
"id": 9,
"isNew": true,
"legend": {
"avg": false,
"current": false,
"max": false,
"min": false,
"show": true,
"total": false,
"values": false
},
"lines": true,
"linewidth": 2,
"links": [],
"nullPointMode": "connected",
"percentage": false,
"pointradius": 5,
"points": false,
"renderer": "flot",
"repeatIteration": 1476706310439,
"scopedVars": {},
"seriesOverrides": [],
"span": 6,
"stack": false,
"steppedLine": false,
"targets": [
{
"queryBy": "tags",
"rate": false,
"refId": "A",
"seriesAggFn": "none",
"tags": [
{
"name": "container_name",
"value": "$app"
},
{
"name": "descriptor_name",
"value": "memory/usage"
}
],
"target": "select metric",
"tagsQL": "container_name IN [$app] AND descriptor_name='memory/usage'",
"timeAggFn": "avg",
"type": "gauge"
}
],
"timeFrom": null,
"timeShift": null,
"title": "Memory usage",
"tooltip": {
"msResolution": true,
"shared": true,
"sort": 0,
"value_type": "cumulative"
},
"type": "graph",
"xaxis": {
"show": true
},
"yaxes": [
{
"format": "bytes",
"label": null,
"logBase": 1,
"max": null,
"min": 0,
"show": true
},
{
"format": "short",
"label": null,
"logBase": 1,
"max": null,
"min": null,
"show": true
}
]
},
{
"cacheTimeout": null,
"colorBackground": true,
"colorValue": false,
"colors": [
"rgba(50, 172, 45, 0.97)",
"rgba(237, 129, 40, 0.89)",
"rgba(245, 54, 54, 0.9)"
],
"datasource": "Hawk-DS",
"editable": true,
"error": false,
"format": "bytes",
"gauge": {
"maxValue": 100,
"minValue": 0,
"show": false,
"thresholdLabels": false,
"thresholdMarkers": true
},
"height": "100px",
"id": 12,
"interval": null,
"isNew": true,
"links": [],
"mappingType": 1,
"mappingTypes": [
{
"name": "value to text",
"value": 1
},
{
"name": "range to text",
"value": 2
}
],
"maxDataPoints": 100,
"nullPointMode": "connected",
"nullText": null,
"postfix": "",
"postfixFontSize": "50%",
"prefix": "",
"prefixFontSize": "50%",
"rangeMaps": [
{
"from": "null",
"text": "N/A",
"to": "null"
}
],
"repeatIteration": 1476706310439,
"scopedVars": {},
"span": 2,
"sparkline": {
"fillColor": "rgba(31, 118, 189, 0.18)",
"full": false,
"lineColor": "rgb(31, 120, 193)",
"show": false
},
"targets": [
{
"queryBy": "tags",
"rate": false,
"refId": "A",
"seriesAggFn": "sum",
"tags": [
{
"name": "container_name",
"value": "$app"
},
{
"name": "descriptor_name",
"value": "memory/usage"
}
],
"target": "select metric",
"tagsQL": "container_name IN [$app] AND descriptor_name='memory/usage'",
"timeAggFn": "live",
"type": "gauge"
}
],
"thresholds": "140000000,180000000",
"title": "Live, all pods",
"type": "singlestat",
"valueFontSize": "80%",
"valueMaps": [
{
"op": "=",
"text": "N/A",
"value": "null"
}
],
"valueName": "avg"
},
{
"cacheTimeout": null,
"colorBackground": true,
"colorValue": false,
"colors": [
"rgba(50, 172, 45, 0.97)",
"rgba(237, 129, 40, 0.89)",
"rgba(245, 54, 54, 0.9)"
],
"datasource": "Hawk-DS",
"editable": true,
"error": false,
"format": "bytes",
"gauge": {
"maxValue": 100,
"minValue": 0,
"show": false,
"thresholdLabels": false,
"thresholdMarkers": true
},
"height": "100px",
"id": 15,
"interval": null,
"isNew": true,
"links": [],
"mappingType": 1,
"mappingTypes": [
{
"name": "value to text",
"value": 1
},
{
"name": "range to text",
"value": 2
}
],
"maxDataPoints": 100,
"nullPointMode": "connected",
"nullText": null,
"postfix": "",
"postfixFontSize": "50%",
"prefix": "",
"prefixFontSize": "50%",
"rangeMaps": [
{
"from": "null",
"text": "N/A",
"to": "null"
}
],
"repeatIteration": 1476706310439,
"scopedVars": {},
"span": 2,
"sparkline": {
"fillColor": "rgba(31, 118, 189, 0.18)",
"full": false,
"lineColor": "rgb(31, 120, 193)",
"show": false
},
"targets": [
{
"queryBy": "tags",
"rate": false,
"refId": "A",
"seriesAggFn": "avg",
"tags": [
{
"name": "container_name",
"value": "$app"
},
{
"name": "descriptor_name",
"value": "memory/usage"
}
],
"target": "select metric",
"tagsQL": "container_name IN [$app] AND descriptor_name='memory/usage'",
"timeAggFn": "live",
"type": "gauge"
}
],
"thresholds": "140000000,180000000",
"title": "Live per pod",
"type": "singlestat",
"valueFontSize": "80%",
"valueMaps": [
{
"op": "=",
"text": "N/A",
"value": "null"
}
],
"valueName": "avg"
},
{
"cacheTimeout": null,
"colorBackground": true,
"colorValue": false,
"colors": [
"rgba(50, 172, 45, 0.97)",
"rgba(237, 129, 40, 0.89)",
"rgba(245, 54, 54, 0.9)"
],
"datasource": "Hawk-DS",
"editable": true,
"error": false,
"format": "bytes",
"gauge": {
"maxValue": 100,
"minValue": 0,
"show": false,
"thresholdLabels": false,
"thresholdMarkers": true
},
"height": "100px",
"id": 10,
"interval": null,
"isNew": true,
"links": [],
"mappingType": 1,
"mappingTypes": [
{
"name": "value to text",
"value": 1
},
{
"name": "range to text",
"value": 2
}
],
"maxDataPoints": 100,
"nullPointMode": "connected",
"nullText": null,
"postfix": "",
"postfixFontSize": "50%",
"prefix": "",
"prefixFontSize": "50%",
"rangeMaps": [
{
"from": "null",
"text": "N/A",
"to": "null"
}
],
"repeatIteration": 1476706310439,
"scopedVars": {},
"span": 2,
"sparkline": {
"fillColor": "rgba(31, 118, 189, 0.18)",
"full": false,
"lineColor": "rgb(31, 120, 193)",
"show": false
},
"targets": [
{
"queryBy": "tags",
"rate": false,
"refId": "A",
"seriesAggFn": "sum",
"tags": [
{
"name": "container_name",
"value": "$app"
},
{
"name": "descriptor_name",
"value": "memory/usage"
}
],
"target": "select metric",
"tagsQL": "container_name IN [$app] AND descriptor_name='memory/usage'",
"timeAggFn": "avg",
"type": "gauge"
}
],
"thresholds": "140000000,180000000",
"title": "Average, all pods",
"type": "singlestat",
"valueFontSize": "80%",
"valueMaps": [
{
"op": "=",
"text": "N/A",
"value": "null"
}
],
"valueName": "avg"
},
{
"cacheTimeout": null,
"colorBackground": true,
"colorValue": false,
"colors": [
"rgba(50, 172, 45, 0.97)",
"rgba(237, 129, 40, 0.89)",
"rgba(245, 54, 54, 0.9)"
],
"datasource": "Hawk-DS",
"editable": true,
"error": false,
"format": "bytes",
"gauge": {
"maxValue": 100,
"minValue": 0,
"show": false,
"thresholdLabels": false,
"thresholdMarkers": true
},
"height": "100px",
"id": 13,
"interval": null,
"isNew": true,
"links": [],
"mappingType": 1,
"mappingTypes": [
{
"name": "value to text",
"value": 1
},
{
"name": "range to text",
"value": 2
}
],
"maxDataPoints": 100,
"nullPointMode": "connected",
"nullText": null,
"postfix": "",
"postfixFontSize": "50%",
"prefix": "",
"prefixFontSize": "50%",
"rangeMaps": [
{
"from": "null",
"text": "N/A",
"to": "null"
}
],
"repeatIteration": 1476706310439,
"scopedVars": {},
"span": 2,
"sparkline": {
"fillColor": "rgba(31, 118, 189, 0.18)",
"full": false,
"lineColor": "rgb(31, 120, 193)",
"show": false
},
"targets": [
{
"queryBy": "tags",
"rate": false,
"refId": "A",
"seriesAggFn": "avg",
"tags": [
{
"name": "container_name",
"value": "$app"
},
{
"name": "descriptor_name",
"value": "memory/usage"
}
],
"target": "select metric",
"tagsQL": "container_name IN [$app] AND descriptor_name='memory/usage'",
"timeAggFn": "avg",
"type": "gauge"
}
],
"thresholds": "140000000,180000000",
"title": "Average per pod",
"type": "singlestat",
"valueFontSize": "80%",
"valueMaps": [
{
"op": "=",
"text": "N/A",
"value": "null"
}
],
"valueName": "avg"
},
{
"cacheTimeout": null,
"colorBackground": true,
"colorValue": false,
"colors": [
"rgba(50, 172, 45, 0.97)",
"rgba(237, 129, 40, 0.89)",
"rgba(245, 54, 54, 0.9)"
],
"datasource": "Hawk-DS",
"editable": true,
"error": false,
"format": "bytes",
"gauge": {
"maxValue": 100,
"minValue": 0,
"show": false,
"thresholdLabels": false,
"thresholdMarkers": true
},
"height": "100px",
"id": 11,
"interval": null,
"isNew": true,
"links": [],
"mappingType": 1,
"mappingTypes": [
{
"name": "value to text",
"value": 1
},
{
"name": "range to text",
"value": 2
}
],
"maxDataPoints": 100,
"nullPointMode": "connected",
"nullText": null,
"postfix": "",
"postfixFontSize": "50%",
"prefix": "",
"prefixFontSize": "50%",
"rangeMaps": [
{
"from": "null",
"text": "N/A",
"to": "null"
}
],
"repeatIteration": 1476706310439,
"scopedVars": {},
"span": 2,
"sparkline": {
"fillColor": "rgba(31, 118, 189, 0.18)",
"full": false,
"lineColor": "rgb(31, 120, 193)",
"show": false
},
"targets": [
{
"queryBy": "tags",
"rate": false,
"refId": "A",
"seriesAggFn": "sum",
"tags": [
{
"name": "container_name",
"value": "$app"
},
{
"name": "descriptor_name",
"value": "memory/usage"
}
],
"target": "select metric",
"tagsQL": "container_name IN [$app] AND descriptor_name='memory/usage'",
"timeAggFn": "max",
"type": "gauge"
}
],
"thresholds": "140000000,180000000",
"title": "Max, all pods",
"type": "singlestat",
"valueFontSize": "80%",
"valueMaps": [
{
"op": "=",
"text": "N/A",
"value": "null"
}
],
"valueName": "avg"
},
{
"cacheTimeout": null,
"colorBackground": true,
"colorValue": false,
"colors": [
"rgba(50, 172, 45, 0.97)",
"rgba(237, 129, 40, 0.89)",
"rgba(245, 54, 54, 0.9)"
],
"datasource": "Hawk-DS",
"editable": true,
"error": false,
"format": "bytes",
"gauge": {
"maxValue": 100,
"minValue": 0,
"show": false,
"thresholdLabels": false,
"thresholdMarkers": true
},
"height": "100px",
"id": 14,
"interval": null,
"isNew": true,
"links": [],
"mappingType": 1,
"mappingTypes": [
{
"name": "value to text",
"value": 1
},
{
"name": "range to text",
"value": 2
}
],
"maxDataPoints": 100,
"nullPointMode": "connected",
"nullText": null,
"postfix": "",
"postfixFontSize": "50%",
"prefix": "",
"prefixFontSize": "50%",
"rangeMaps": [
{
"from": "null",
"text": "N/A",
"to": "null"
}
],
"repeatIteration": 1476706310439,
"scopedVars": {},
"span": 2,
"sparkline": {
"fillColor": "rgba(31, 118, 189, 0.18)",
"full": false,
"lineColor": "rgb(31, 120, 193)",
"show": false
},
"targets": [
{
"queryBy": "tags",
"rate": false,
"refId": "A",
"seriesAggFn": "avg",
"tags": [
{
"name": "container_name",
"value": "$app"
},
{
"name": "descriptor_name",
"value": "memory/usage"
}
],
"target": "select metric",
"tagsQL": "container_name IN [$app] AND descriptor_name='memory/usage'",
"timeAggFn": "max",
"type": "gauge"
}
],
"thresholds": "140000000,180000000",
"title": "Max per pod",
"type": "singlestat",
"valueFontSize": "80%",
"valueMaps": [
{
"op": "=",
"text": "N/A",
"value": "null"
}
],
"valueName": "avg"
}
],
"repeat": "app",
"scopedVars": {
"app": {
"text": "aloha",
"value": "aloha",
"selected": true
}
},
"title": "New row"
}
],
"time": {
"from": "now-30m",
"to": "now"
},
"timepicker": {
"refresh_intervals": [
"5s",
"10s",
"30s",
"1m",
"5m",
"15m",
"30m",
"1h",
"2h",
"1d"
],
"time_options": [
"5m",
"15m",
"1h",
"6h",
"12h",
"24h",
"2d",
"7d",
"30d"
]
},
"templating": {
"list": [
{
"current": {},
"datasource": "Hawk-DS",
"hide": 0,
"includeAll": true,
"label": "Application",
"multi": true,
"name": "app",
"options": [],
"query": "tags/container_name:*",
"refresh": 1,
"regex": "",
"type": "query"
}
]
},
"annotations": {
"list": []
},
"schemaVersion": 12,
"version": 32,
"links": [],
"gnetId": null
}
You should put a YAML file pointing to the JSON files in that folder. For example write /etc/grafana/provisioning/dashboards/local.yml:
apiVersion: 1
providers:
- name: 'default'
orgId: 1
folder: ''
type: file
disableDeletion: false
updateIntervalSeconds: 10 #how often Grafana will scan for changed dashboards
options:
path: /var/lib/grafana/dashboards
And then write your JSON file to /var/lib/grafana/dashboards/openshift.json.
Before Grafana 5, my previous solution was to wrap the whole Docker process in a script that uses the API to create the dashboard once the Docker container is up. You can use the GF_SECURITY_ADMIN_PASSWORD environment variable to set the password. You can also use GF_AUTH_ANONYMOUS_ENABLED, but you'll need to make sure it's not accessible to the outside world.
docker run -p 3000:3000 -e GF_AUTH_ANONYMOUS_ENABLED=true grafana/grafana ...
sleep 10 # wait for grafana to load (use a better method of waiting in production)
curl -skfS -XPOST --header "Content-Type: application/json" "http://localhost:3000/grafana/api/dashboards/db" --data-binary #dashboard.json
I've faced with the same problem and I hope the next solution could help somebody.
For example, you have the next working directory:
├── docker-compose.yml
├── grafana
│ ├── Dockerfile
│ ├── dashboards
│ │ └── exported_dashboard.json #dashboard you saved before
│ └── provisioning
│ │
│ ├── dashboards
│ │ └── all.yml
│ └── datasources
│ └── all.yml
In the Dockerfile you have to add provisioning and dashboards folders into grafana container:
FROM grafana/grafana:latest
ADD ./provisioning /etc/grafana/provisioning
ADD ./dashboards /var/lib/grafana/dashboards
In provisioning/dashboards/all.yml you have to set path to imported dashboards folder
where exported_dashboard.json is actually placed:
- name: 'default'
org_id: 1
folder: ''
type: 'file'
options:
folder: '/var/lib/grafana/dashboards'
[Optional] Then you also can configure default datasourse in provisioning/datasources/all.yml, prometheus, for example:
datasources:
- access: 'proxy'
editable: true
is_default: true
name: 'prom1'
org_id: 1
type: 'prometheus'
url: 'http://prometheus:9090'
version: 1
Finally, docker-compose.yml:
version: '3.8'
services:
grafana:
image: awesome-grafana:latest
build:
context: grafana
ports:
- "3000:3000"
environment:
GF_DASHBOARDS_DEFAULT_HOME_DASHBOARD_PATH: "/var/lib/grafana/dashboards/exported_dashboard.json"
As for HOME dashborad, you can use the GF_DASHBOARDS_DEFAULT_HOME_DASHBOARD_PATH evironment variable, as shown above.
More about GF_DASHBOARDS_DEFAULT_HOME_DASHBOARD_PATH evironment variable you can read in that github issue.
The full comprehensive code example you can find in the next github repository
Original article with explanation there.

Resources