I am trying to create an object, from which some are static values, and others are the outcomes of previous tasks. How do I access them?
Current CDK Code:
this.setupIteration = new Pass(this, "Setup Iteration", {
result: Result.fromObject({
"count": "$.skusLength",
"index": 0,
"step": 1
}),
resultPath: "$.iterator",
});
Error: I am just getting the string "$.skusLength" instead of the actual value that is available above. I have also tried JsonPath.numberAt and JsonPath.stringAt, with no success.
CDK Docs: https://docs.aws.amazon.com/cdk/api/v1/docs/#aws-cdk_aws-stepfunctions.Result.html
Change result to parameters:
this.setupIteration = new Pass(this, "Setup Iteration", {
parameters: {
"count": JsonPath.numberAt("$.skusLength"),
"index": 0,
"step": 1
},
resultPath: "$.iterator",
});
Related
I'm currently working on a Twitter bot that automatically reply messages, I'm doing this by using tweepy (the official python twitter library)
I need to filter messages based on the created time as I don't want to reply same message twice. Now the problem is that the API endpoint returns created_timestamp as string representation of positive integers.
Below is an example of data returned as per the doc
{
"next_cursor": "AB345dkfC",
"events": [
{ "id": "110", "created_timestamp": "1639919665615", ... },
{ "id": "109", "created_timestamp": "1639865141987", ... },
{ "id": "108", "created_timestamp": "1639827437833", ... },
{ "id": "107", "created_timestamp": "1639825389806", ... },
{ "id": "106", "created_timestamp": "1639825389796", ... },
{ "id": "105", "created_timestamp": "1639825389768", ... },
...
]
}
My question is "How do I convert the created_timestamp to a valid date using python" ?.
You might play with timestamps on this resource
And in your case could use methods like:
timestamp = int('timestamp_string')
datetime.fromtimestamp(timestamp, tz=None)
date.fromtimestamp(timestamp)
From the datetime standard library. But integers after the first line are already well comparable if the task is to distinguish differences between the timestamps.
Details
OS: Manjaro Linux
Node.js version: 10
npm version: 6
googleapis version: 51.0.0
Steps to reproduce
We are sending a google slide API request with updateShapeProperties:
"updateShapeProperties": {
"objectId": objectId,
"shapeProperties": {
"shapeBackgroundFill": {
"solidFill": {
"color": {
"rgbColor": {
"red": r,
"green": g,
"blue": b
}
},
"alpha": 1
}
},
"outline": {
"outlineFill": {
"solidFill": {
"color": {
"rgbColor": {
"red": 0,
"green": 0,
"blue": 0
}
},
"alpha": 0.1
}
},
},
"contentAlignment": "MIDDLE"
},
"fields": "*"
}
where objectId comes from one of any pageElement we get from slide.pageElements
We get a 400 response that states:
message: ""Invalid requests[2].updateShapeProperties: At least one field must be listed in 'fields'. (Use '*' to indicate all fields.)
This somehow used to work the past few months, and was wondering why it suddenly doesn't work right now. Did we have any changes or updates on the updateShapeProperties API?
I believe that [2] of Invalid requests[2].updateShapeProperties is the request body in your question. When I tested your request body, I could replicate your situation. The error message is At least one field must be listed in 'fields'. (Use '*' to indicate all fields.). When I tested "fields": "", I got the same error message. From this situation, I thought that it might be a bug.
When I searched this situation at the Google issue tracker, I found it. Ref
So as the current workaround, in order to avoid this error, in your request body, how about the following modification?
From:
"fields": "*"
To:
"fields": "shapeBackgroundFill,outline,contentAlignment"
Problem:
I am unable to POST a transaction via the RESTful API generated by the composer-rest-server. I am receiving statusCode 422; the transaction instance is not valid. However, the same example works in the Playground.
Scenario:
I've set up a transaction called Offer in my .cto file which posts an offer to buy a house:
// Offer - Specifies an offer that a bidder places on a house listing with an associated price
transaction Offer {
o Double bidPrice
--> HouseListing listing
--> Person bidder
}
The composer-rest-server has generated an API with the following JSON string to post a transaction of type Offer:
{
"$class": "org.acme.purchasing.Offer",
"bidPrice": 0,
"listing": "string",
"bidder": "string",
"transactionId": "string",
"timestamp": "2017-07-21T13:37:09.460Z"
}
I've since replaced this with a sample transaction using the following JSON code derived from the above example:
{
"$class": "org.acme.purchasing.Offer",
"bidPrice": 1000,
"listing": "001",
"bidder": "RJOHNSON",
"transactionId": "1b9aa63c-dfad-4aad-a610-dfc80f2796b2",
"timestamp": "2017-07-21T13:37:09.460Z"
}
The response returned is error code 422:
{
"error": {
"statusCode": 422,
"name": "ValidationError",
"message": "The `Offer` instance is not valid. Details: `transactionId` can't be set (value: \"1b9aa63c-dfad-4aad-a610-d...6b2\").",
"details": {
"context": "Offer",
"codes": {
"transactionId": [
"absence"
]
},
"messages": {
"transactionId": [
"can't be set"
]
}
},
"stack": "ValidationError: The `Offer` instance is not valid. Details: `transactionId` can't be set (value: \"1b9aa63c-dfad-4aad-a610-d...6b2\").\n at /usr/lib/node_modules/composer-rest-server/node_modules/loopback-datasource-juggler/lib/dao.js:355:12\n at ModelConstructor.<anonymous> (/usr/lib/node_modules/composer-rest-server/node_modules/loopback-datasource-juggler/lib/validations.js:566:11)\n at ModelConstructor.next (/usr/lib/node_modules/composer-rest-server/node_modules/loopback-datasource-juggler/lib/hooks.js:93:12)\n at ModelConstructor.<anonymous> (/usr/lib/node_modules/composer-rest-server/node_modules/loopback-datasource-juggler/lib/validations.js:563:23)\n at ModelConstructor.trigger (/usr/lib/node_modules/composer-rest-server/node_modules/loopback-datasource-juggler/lib/hooks.js:83:12)\n at ModelConstructor.Validatable.isValid (/usr/lib/node_modules/composer-rest-server/node_modules/loopback-datasource-juggler/lib/validations.js:529:8)\n at /usr/lib/node_modules/composer-rest-server/node_modules/loopback-datasource-juggler/lib/dao.js:351:9\n at doNotify (/usr/lib/node_modules/composer-rest-server/node_modules/loopback-datasource-juggler/lib/observer.js:155:49)\n at doNotify (/usr/lib/node_modules/composer-rest-server/node_modules/loopback-datasource-juggler/lib/observer.js:155:49)\n at doNotify (/usr/lib/node_modules/composer-rest-server/node_modules/loopback-datasource-juggler/lib/observer.js:155:49)\n at doNotify (/usr/lib/node_modules/composer-rest-server/node_modules/loopback-datasource-juggler/lib/observer.js:155:49)\n at Function.ObserverMixin._notifyBaseObservers (/usr/lib/node_modules/composer-rest-server/node_modules/loopback-datasource-juggler/lib/observer.js:178:5)\n at Function.ObserverMixin.notifyObserversOf (/usr/lib/node_modules/composer-rest-server/node_modules/loopback-datasource-juggler/lib/observer.js:153:8)\n at Function.ObserverMixin._notifyBaseObservers (/usr/lib/node_modules/composer-rest-server/node_modules/loopback-datasource-juggler/lib/observer.js:176:15)\n at Function.ObserverMixin.notifyObserversOf (/usr/lib/node_modules/composer-rest-server/node_modules/loopback-datasource-juggler/lib/observer.js:153:8)\n at Function.ObserverMixin._notifyBaseObservers (/usr/lib/node_modules/composer-rest-server/node_modules/loopback-datasource-juggler/lib/observer.js:176:15)"
}
}
Now the strange thing is that I've deployed the same BNA onto the Hyperledger Composer Playground and am able to execute transactions of type Offer successfully.
Note that in the Playground, "transactionId" and "timestamp" are not specified as the Playground appears to take care of these values. For example, this is what Playground proposes to me initially:
{
"$class": "org.acme.purchasing.Offer",
"bidPrice": 0,
"listing": "resource:org.acme.purchasing.HouseListing#id:7965",
"bidder": "resource:org.acme.purchasing.Person#id:4441"
}
Can anyone advise why it's saying the Offer instance is not valid? My first thought was that it's not liking the string I'm placing in "transactionId" but another Stack Overflow post points out that the transactionId is just an arbitrary UUIDv4 string which I've generated already.
Update #1: Failing even with default demo
In order to ensure by BNA is error-free, I've deployed the default carauction-demo (resembles my example closely) onto my local Hyperledger Fabric instance and deployed the composer-rest-server. I've also deployed the same BNA into the Playground. All assets and participants were created identical in both from the Explorer (local instance) and Playground. When it comes time to submit an Offer transaction:
{
"$class": "org.acme.vehicle.auction.Offer",
"bidPrice": 800,
"listing": "resource:org.acme.vehicle.auction.VehicleListing#L001",
"member": "resource:org.acme.vehicle.auction.Member#member3#acme.org"
}
This JSON was generated by the Playground and succeeds there. Copy/paste/executing into the Explorer yields a status 500 error.
{
"error": {
"statusCode": 500,
"name": "Error",
"message": "error trying invoke chaincode. Error: chaincode error (status: 500, message: Error: Object with ID 'string' in collection with ID 'Asset:org.acme.vehicle.auction.VehicleListing' does not exist)",
"stack": "Error: error trying invoke chaincode. Error: chaincode error (status: 500, message: Error: Object with ID 'string' in collection with ID 'Asset:org.acme.vehicle.auction.VehicleListing' does not exist)\n at _initializeChannel.then.then.then.then.catch (/usr/lib/node_modules/composer-rest-server/node_modules/composer-connector-hlfv1/lib/hlfconnection.js:806:34)"
}
}
I'm still at a lost as to what is wrong here.
After much experimentation and some searching, I've concluded that the problem was that NPM was installed using sudo (as root). I redid the installation as non-root and the problem has now been solved. Everything is working as expected.
Composer updates the transactionId itself as its generated, you cannot do this in your JSON and hence why you get the error. This is a Loopback -> Swagger conversion issue as it should not appear in a /POST REST operation - captured here https://github.com/hyperledger/composer/issues/663
So I have successfully executed Offer transactions (/POST). I think the '500' error is because there is a missing field ('string' type), or relationship in your Offer transaction via REST.
Using the example of the Car Auction network https://github.com/hyperledger/composer-sample-networks/blob/master/packages/carauction-network/models/auction.cto
Either of these Offer transactions using /POST were successful in Explorer:
{
"$class": "org.acme.vehicle.auction.Offer",
"bidPrice": 20,
"listing": "org.acme.vehicle.auction.VehicleListing#100",
"member": "org.acme.vehicle.auction.Member#a#b.com"
}
OR
{
"$class": "org.acme.vehicle.auction.Offer",
"bidPrice": 20,
"listing": "org.acme.vehicle.auction.VehicleListing#100",
"member": "org.acme.vehicle.auction.Member#a#b.com",
"timestamp": "2017-07-28T14:07:02.558Z"
}
responded with a 200 (SUCCESS) and a transactionId in Explorer:
{
"$class": "org.acme.vehicle.auction.Offer",
"bidPrice": 20,
"listing": "org.acme.vehicle.auction.VehicleListing#100",
"member": "org.acme.vehicle.auction.Member#a#b.com",
"transactionId": "e75b9934-1f08-4daf-90db-702bbe4b8fa1"
}
This is my VehicleListing asset JSON for #100 above
{
"$class": "org.acme.vehicle.auction.VehicleListing",
"listingId": "100",
"reservePrice": 50,
"description": "string",
"state": "FOR_SALE",
"offers": [
{
"$class": "org.acme.vehicle.auction.Offer",
"bidPrice": 50,
"listing": "100",
"member": "resource:org.acme.vehicle.auction.Member#a#b.com",
"transactionId": "string123",
"timestamp": "2017-07-28T14:07:02.825Z"
}
],
"vehicle": "resource:org.acme.vehicle.auction.Vehicle#123"
}
And I created asset for Vehicle 123 as well as Member a#b.com
I use Firebase Database, I have a list of cards, I need to monitor some cards if they are deleted. I tried to do this with ref.observe (.childRemoved), but this is not appropriate for my case, since some data can be deleted by the cardholder, and then this block will fire several times, I need to watch if the entire card was deleted. How can I do that?
My reference
let ref = FIRDatabase.database().reference().child(MainGateways.cards.rawValue).child(card.id)
My cards structure
"cards": {
"-Khv9rUVwErNHBzXcruO": {
"additionalInfo": {
"note": ""
},
"dateProperties": {
"day": "Flex",
"isFlexDate": true,
"isFlexTime": true,
"iso8601": "",
"time": ""
},
"interestedUsers": {
"NfM26A2YcPUFz8rfYa23Kr3mjCO2": {
"isApproved": false,
"isNew": true,
"isoDate": "2017-04-17T13:08:41+04:00",
"userID": "NfM26A2YcPUFz8rfYa23Kr3mjCO2",
"userItchableName": "Alexsander K."
}
},
"isNewPendingRequest": true,
"isPrivateStatus": false,
"isTest": false,
"ownerID": "1Cix149ThIOG1ULPVjyy0LyTxbe2",
"peopleProperties": {
"availableSeats": 1,
"numberOfPeople": 1,
"numberOfPeopleDescription": "1 person"
},
"title": "Genius Lounge and Sake Bar",
"userName": "Tim C.",
"version": 1
},
I also tried to make such a link, but then this block works if any card was removed from the list.
guard let ref = FIRDatabase.database().reference().child(MainGateways.cards.rawValue).child(card.id).parent else { return }
Look into Cloud Functions for Firebase. Using that, you can write JavaScript code that acts in response to database changes at the path you specify. This allows you to write logic independent of the client app to react to changes in the database.
Elasticsearch experts,
I have been unable to find a simple way to just tell ElasticSearch to insert the _timestamp field for all the documents that are added in all the indices (and all document types).
I see an example for specific types:
http://www.elasticsearch.org/guide/reference/mapping/timestamp-field/
and also see an example for all indices for a specific type (using _all):
http://www.elasticsearch.org/guide/reference/api/admin-indices-put-mapping/
but I am unable to find any documentation on adding it by default for all documents that get added irrespective of the index and type.
Elasticsearch used to support automatically adding timestamps to documents being indexed, but deprecated this feature in 2.0.0
From the version 5.5 documentation:
The _timestamp and _ttl fields were deprecated and are now removed. As a replacement for _timestamp, you should populate a regular date field with the current timestamp on application side.
You can do this by providing it when creating your index.
$curl -XPOST localhost:9200/test -d '{
"settings" : {
"number_of_shards" : 1
},
"mappings" : {
"_default_":{
"_timestamp" : {
"enabled" : true,
"store" : true
}
}
}
}'
That will then automatically create a _timestamp for all stuff that you put in the index.
Then after indexing something when requesting the _timestamp field it will be returned.
Adding another way to get indexing timestamp. Hope this may help someone.
Ingest pipeline can be used to add timestamp when document is indexed. Here, is a sample example:
PUT _ingest/pipeline/indexed_at
{
"description": "Adds indexed_at timestamp to documents",
"processors": [
{
"set": {
"field": "_source.indexed_at",
"value": "{{_ingest.timestamp}}"
}
}
]
}
Earlier, elastic search was using named-pipelines because of which 'pipeline' param needs to be specified in the elastic search endpoint which is used to write/index documents. (Ref: link) This was bit troublesome as you would need to make changes in endpoints on application side.
With Elastic search version >= 6.5, you can now specify a default pipeline for an index using index.default_pipeline settings. (Refer link for details)
Here is the to set default pipeline:
PUT ms-test/_settings
{
"index.default_pipeline": "indexed_at"
}
I haven't tried out yet, as didn't upgraded to ES 6.5, but above command should work.
You can make use of default index pipelines, leverage the script processor, and thus emulate the auto_now_add functionality you may know from Django and DEFAULT GETDATE() from SQL.
The process of adding a default yyyy-MM-dd HH:mm:ss date goes like this:
1. Create the pipeline and specify which indices it'll be allowed to run on:
PUT _ingest/pipeline/auto_now_add
{
"description": "Assigns the current date if not yet present and if the index name is whitelisted",
"processors": [
{
"script": {
"source": """
// skip if not whitelisted
if (![ "myindex",
"logs-index",
"..."
].contains(ctx['_index'])) { return; }
// don't overwrite if present
if (ctx['created_at'] != null) { return; }
ctx['created_at'] = new SimpleDateFormat("yyyy-MM-dd HH:mm:ss").format(new Date());
"""
}
}
]
}
Side note: the ingest processor's Painless script context is documented here.
2. Update the default_pipeline setting in all of your indices:
PUT _all/_settings
{
"index": {
"default_pipeline": "auto_now_add"
}
}
Side note: you can restrict the target indices using the multi-target syntax:
PUT myindex,logs-2021-*/_settings?allow_no_indices=true
{
"index": {
"default_pipeline": "auto_now_add"
}
}
3. Ingest a document to one of the configured indices:
PUT myindex/_doc/1
{
"abc": "def"
}
4. Verify that the date string has been added:
GET myindex/_search
An example for ElasticSearch 6.6.2 in Python 3:
from elasticsearch import Elasticsearch
es = Elasticsearch(hosts=["localhost"])
timestamp_pipeline_setting = {
"description": "insert timestamp field for all documents",
"processors": [
{
"set": {
"field": "ingest_timestamp",
"value": "{{_ingest.timestamp}}"
}
}
]
}
es.ingest.put_pipeline("timestamp_pipeline", timestamp_pipeline_setting)
conf = {
"settings": {
"number_of_shards": 2,
"number_of_replicas": 1,
"default_pipeline": "timestamp_pipeline"
},
"mappings": {
"articles":{
"dynamic": "false",
"_source" : {"enabled" : "true" },
"properties": {
"title": {
"type": "text",
},
"content": {
"type": "text",
},
}
}
}
}
response = es.indices.create(
index="articles_index",
body=conf,
ignore=400 # ignore 400 already exists code
)
print ('\nresponse:', response)
doc = {
'title': 'automatically adding a timestamp to documents',
'content': 'prior to version 5 of Elasticsearch, documents had a metadata field called _timestamp. When enabled, this _timestamp was automatically added to every document. It would tell you the exact time a document had been indexed.',
}
res = es.index(index="articles_index", doc_type="articles", id=100001, body=doc)
print(res)
res = es.get(index="articles_index", doc_type="articles", id=100001)
print(res)
About ES 7.x, the example should work after removing the doc_type related parameters as it's not supported any more.
first create index and properties of the index , such as field and datatype and then insert the data using the rest API.
below is the way to create index with the field properties.execute the following in kibana console
`PUT /vfq-jenkins
{
"mappings": {
"properties": {
"BUILD_NUMBER": { "type" : "double"},
"BUILD_ID" : { "type" : "double" },
"JOB_NAME" : { "type" : "text" },
"JOB_STATUS" : { "type" : "keyword" },
"time" : { "type" : "date" }
}}}`
the next step is to insert the data into that index:
curl -u elastic:changeme -X POST http://elasticsearch:9200/vfq-jenkins/_doc/?pretty
-H Content-Type: application/json -d '{
"BUILD_NUMBER":"83","BUILD_ID":"83","JOB_NAME":"OMS_LOG_ANA","JOB_STATUS":"SUCCESS" ,
"time" : "2019-09-08'T'12:39:00" }'