The log4j2 PatternLayout offers a %notEmpty conversion pattern that allows you to skip sections of the pattern that refer to empty variables.
Is there any way to do something similar for JsonTemplateLayout, specifically for thread context data (MDC)? It correctly (IMO) suppresses null fields, but it doesn't do the same with empty ones.
E.g., given the following in my JSON template:
"application": {
"name": { "key": "x-app", "$resolver": "mdc" },
"context": { "key": "x-app-context", "$resolver": "mdc" },
"instance": {
"name": { "key": "x-appinst", "$resolver": "mdc" },
"context": { "key": "x-appinst-context", "$resolver": "mdc" }
}
}
is there a way to prevent blocks like this from being logged, where the only data in the subtree is the empty string values for context?
"application":{"context":"","instance":{"context":""}}
(Yes, ideally I'd prevent those empty strings being put into the context in the first place, but this isn't my app, I'm just configuring it.)
JsonTemplateLayout author speaking here. Currently, JsonTemplateLayout doesn't support blank property exclusion for the following reasons:
The definition of empty/blank is ambiguous. One might have, null, {}, "\s*", [], [[]], [{}], etc. as valid JSON values. Which one of these are empty/blank? Let's assume we have agreed on a certain behavior. Will it apply to the rest of its users?
Checking if a value is empty/blank incurs an extra runtime cost.
Most of the time you don't care. You persist logs in a storage system, e.g., ELK stack, and there blank value elimination is provided out of the box by the storage engine in the most efficient way.
Would you mind sharing your use case, please? Why do you want to prevent the emission of "context": "" properties? If you deliver your logs to Elasticsearch, there you can easily exclude such fields via appropriate index mappings.
Near as I can tell, no. I would suggest you create a Jira issue to get that addressed.
Related
Starting with a Google Doc that looks like:
* Item
I'm hoping to make a series of API calls to turn the doc into:
* Item
- Subitem
But, I can't figure out how to do this with the API. A CreateParagraphBulletRequest doesn't have an indent level I can specify. The documentation suggests:
The nesting level of each paragraph will be determined by counting leading tabs in front of each paragraph. To avoid excess space between the bullet and the corresponding paragraph, these leading tabs are removed by this request. This may change the indices of parts of the text.
However, prepending tabs to the beginning of an InsertTextRequest will prepend the tab character, rather than changing the indent:
* Item
* Subitem
Does anyone have any ideas for what I may be doing wrong?
I believe your goal as follows.
You want to create a nested list using Google Docs API.
At first, a list, which has one item as 1st level, is existing in the Google Document. It's as follows.
- item1
Under this situation, you want to insert a nested item to the existing list as 2nd level. It's as follows.
- item1
- item2
Points for achieving your goal:
In this case, in order to insert the item to the existing list as the 2nd level, in my experience, I couldn't directly insert it. In my case, as a workaround, I'm using the following flow.
Insert a text \n\titem2\n for 2nd level using insertText request.
In this case, the 1st level is also inserted. It seems that in order to insert the deep level items, it is required to set from the 1st level and convert to the list with the bullets.
Using createParagraphBullets, it gives the bullets to the list. By this, \t is converted to the nested items.
Remove the bullet of the 1st level.
Remove the line break.
Sample request body:
When above flow is reflected to the request body of the method of batchUpdate in Docs API, it becomes as follows.
{
"requests": [
{
"insertText": {
"text": "\n\titem2\n",
"location": {
"index": 7
}
}
},
{
"createParagraphBullets": {
"range": {
"startIndex": 1,
"endIndex": 15
},
"bulletPreset": "BULLET_DISC_CIRCLE_SQUARE"
}
},
{
"deleteParagraphBullets": {
"range": {
"startIndex": 7,
"endIndex": 8
}
}
},
{
"deleteContentRange": {
"range": {
"startIndex": 7,
"endIndex": 8
}
}
}
]
}
Result:
When above request body is used, the following result is obtained.
Before:
After:
Note:
Although I had looked for other methods for using Docs API without changing the existing list, unfortunately, I cannot still find them. I thought that in order to insert the deep nested items to the existing list, in the current stage, the items might be required to be given from 1st level using \t. Unfortunately, I'm not sure whether this is the specification. So, for example, how about requesting this for the issue tracker as the future request? Ref
References:
Working with lists
documents.batchUpdate
I am using the slack API to get the full list of emoji, so that when I get a message, I will just replace :squirrel: with the icon.
The method https://slack.com/api/emoji.list works like a charm, but returns 30 icons only. I think this is correct since in the documentation page (https://api.slack.com/methods/emoji.list) they say:
This method lists the custom emoji for a team.
Fair enough, but how can I get the full list of the associations icon-name / icon URL ?
I finally managed to get all the icons and to use them and I post here the solution for anyone that would like to use do similar:
First of all, I got the Slack Custom Emoji through this slack URL
Since at step 1 we get only Custom Emojis, it is useful to know that slack uses standard emoji defined in unicode characters, mapped through custom handles like :smiley: or :horse:. The good thing is that we can find, linked through slack page a link to a JSON object with all the emoji mappings. This file is HUGE, but has everything we need.
In the file you'll find an array of javascript object like the one below:
{
"name":"SMILING FACE WITH OPEN MOUTH",
"unified":"1F603",
"variations":[],
"docomo":"E6F0",
"au":"E471",
"softbank":"E057",
"google":"FE330",
"image":"1f603.png",
"sheet_x":26,
"sheet_y":18,"
short_name":"smiley",
"short_names":["smiley"],
"text":":)",
"texts":["=)","=-)"],
"category":"People",
"sort_order":5,
"has_img_apple":true,
"has_img_google":true,
"has_img_twitter":true,
"has_img_emojione":true
}
I used the following information:
shortnames are the names that are used in slack (you'll need to turn smiley into :smiley: )
unified is the unicode character to use (to use it directly in an HTML page you'll need to add &#x so in this case you'll have to use 😃 which is rendered 😃
Using this information you will be able to create a slack-to-html function to decode emojis and display them wherever you want
Not entirely sure if this is what you are looking for, but if it's just about mapping images to slack-style names, this is a pretty good library:
https://github.com/iamcal/emoji-data
So, building on the example in their README:
The emoji with the Slack style short name point_uphas the hex value 261d, and can thus be found here: https://github.com/iamcal/emoji-data/blob/master/img-apple-160/261d.png
(Apple, because the default slack emoji are the apple emoji)
Just extending on #Luca's awesome solution, I've created a shortnames => html unicode javascript dictionary...
Download: Slack emoticons to unicode html mapping.
Generated - 17th August 2018 from the source https://raw.githubusercontent.com/iamcal/emoji-data/master/emoji.json
Example:
{
"+1": "👍",
"-1": "👎",
"100": "💯",
"1234": "🔢",
"8ball": "🎱",
"ab": "🆎",
"abc": "🔤",
"abcd": "🔡",
"accept": "🉑",
...
"zebra_face": "🦓",
"zipper_mouth_face": "🤐",
"zombie": "🧟",
"zzz": "💤"
}
Which becomes...
{ "+1": "👍", "-1": "👎", "100": "💯",
"1234": "🔢", "8ball": "🎱", "ab": "🆎",
"abc": "🔤", "abcd": "🔡", "accept": "🉑",
... "zebra_face": "🦓", "zipper_mouth_face": "🤐",
"zombie": "🧟", "zzz": "💤" }
As far as I know, there is no API endpoint or comprehensive list of supported emoji/keywords available. I was able to grab the full set (including custom emoji for the workspace) by inspecting the Slack emoji picker using React Developer Tools (Chrome extension).
Here's an example (JSON):
...
{
"name": "beers",
"unicode": "1f37b",
"id": "E1f37b",
"keywords": ["bar", "beer", "clink", "drink", "mug", "ale", "food"]
},
{
"name": "baby_bottle",
"unicode": "1f37c",
"id": "E1f37c",
"keywords": ["baby", "bottle", "drink", "milk", "infant"]
},
{
"name": "knife_fork_plate",
"unicode": "1f37d-fe0f",
"id": "E1f37d-fe0f",
"keywords": ["cooking", "fork", "knife", "plate"]
},
{
"name": "champagne",
"unicode": "1f37e",
"id": "E1f37e",
"keywords": ["bar", "bottle", "cork", "drink", "popping"]
},
{ "name": "popcorn", "unicode": "1f37f", "id": "E1f37f", "keywords": [] },
...
Full dump (as of 12/20/2020, excluding custom emoji): https://gist.github.com/impressiver/87b5b9682d935efba8936898fbfe1919
Since it seemed impossible to find a complete and up-to-date source for these emoji names I came up with this browser console script. Just let it run in Slack and when it finishes it will download a full emoji.json. At time of writing it contains 2485 entries.
https://gist.github.com/8461e125072c2806301403a4e1eca891
This looks like this:
{
"+1": "https://a.slack-edge.com/production-standard-emoji-assets/13.0/google-medium/1f44d#2x.png",
"+1::skin-tone-2": "https://a.slack-edge.com/production-standard-emoji-assets/13.0/google-medium/1f44d-1f3fb#2x.png",
"+1::skin-tone-3": "https://a.slack-edge.com/production-standard-emoji-assets/13.0/google-medium/1f44d-1f3fc#2x.png",
"+1::skin-tone-4": "https://a.slack-edge.com/production-standard-emoji-assets/13.0/google-medium/1f44d-1f3fd#2x.png",
If you want the actual emoji instead of the image you can parse that out of the image file name, e.g. 1f44d-1f3fd#2x.png is "\u1f44d\u1f3fd".
Disclaimer: I'm a novice.
I want to simulate a join for my mongodb embedded document. If I have an embedded list:
{
_id: ObjectId("5320f6c34b6576d373000000"),
user_id: "52f581096b657612fe020000",
list: "52f4fd9f52e39bc0c15674ea"
{
player_1: "52f4fd9f52e39bc0c15674ex",
player_2: "52f4fd9f52e39bc0c15674ey",
player_3: "52f4fd9f52e39bc0c15674ez"
}
}
And a player collection with each player being something like:
{
_id: ObjectId("52f4fd9f52e39bc0c15674ex"),
college: "Louisville",
headshot: "player.png",
height: "6'2",
name: "Wayne Brady",
position: "QB",
weight: 205
}
I want to end up with:
{
_id: ObjectId("5320f6c34b6576d373000000"),
user_id: "52f581096b657612fe020000",
list: "52f4fd9f52e39bc0c15674ea"
{
player_1:
{
_id: ObjectId("52f4fd9f52e39bc0c15674ex"),
college: "Louisville",
headshot: "player.png",
height: "6'2",
name: "Wayne Brady",
position: "QB",
weight: 205
},
etc...
}
}
So I can call User.lists.first.player_1.name.
This is what makes sense in my mind since I'm new to rails...and I don't want to embed players in each user's list because I'd have so many redundancies...
Advice? Is this possible, if so how? Is it a good idea, or is there a better way?
So have have a typical relational model, let's call it "one to many", which you have users or "user teams" and a whole pool of players. And in typical modelling form you want to "de-normalize" this to avoid duplication.
But here's the thing, MongoDB does not do joins. Joins are not "webscale" in the current parlance. So it leaves you thinking what to do. Or does MongoDB do joins?
db.eval(function() {
var user = db.user.findOne({ "user_id": "52f581096b657612fe020000" });
for ( k in user.list ) {
var player = db.player.findOne({ "_id": user.list[k] });
user.list[k] = player;
}
return user;
});
Which "arguably" is "kind of a join". And it was all done on the server, right?
But DO NOT DO THAT. While db.eval() has uses, something that you are going to query regularly is not one of the practical uses. Read the documentation, which shows the warnings. In particular, all JavaScript is running in a single thread so that will lock things up very quickly.
Now client side, you are more or less doing the same thing. And you ODM of choice is likely again doing "the same thing", though it is usually hiding it away in some manner so you don't see that part. Likewise the same could likely be said of your SQL ORM, which was also "sneaking off behind your back" and querying the database while you just accessed the objects in your code.
As for mapReduce. Well the problem with the data you present is that there is nothing to "reduce". There is a technique known as in "incremental mapReduce" but it would not be well suited to this type of data. A topic in itself, but you would basically need all the "users" associated to the "players" as well, stored in the "player data" to make that any kind of viability. And it's ultimately just another way of "cheating" joins.
This is the space in which MongoDB exists.
So rather than going and doing all this fetching or joining, it allows the concept of being able to "pre-join" your data as it were. And the point of this is to allow faster, and more atomic reads and writes. And this is known as embedding.
Looking at your data, there should not be a problem with embedding at all. Consider the points:
Presumably you are modelling "fantasy teams" for a given user. It would be fair to day that a "team" does not consist of an infinite number of players.
Aside from other things your "A1" usage is likely to be "displaying" the players associated with that "user team". And in so much as, you want to "display" as much information as possible, and keep that to a single read operation. You also want to easily add "players" to the "user team".
While a "player" may have "extended information", and possibly even some global statistics or scores, that information may well be not what you want to access, while associated to the "user team" that often. It can probably be written independently, and only read when looking at the "player detail".
Those are three good cases to support embedding. Sure you would be duplicating information stored against each user team, opposed to just a small "key" reference. And sure that information is likely to exist elsewhere in the full "player detail" and that would be duplication as well.
But the point of the "duplication" here is to optimize. So here it would seem valid to embed "some of the data", not all, but what you regularly use in your main operations. Considering the "player's" name, position, height and weight are not likely to change on a regular basis or not even at all in the context, then that seems a reasonable trade-off.
{
"_id": ObjectId("5320f6c34b6576d373000000"),
"user_id": ObjectId("52f581096b657612fe020000"),
"list": [
{
"_id": ObjectId("52f4fd9f52e39bc0c15674ex"),
"label": "Player1",
"college": "Louisville",
"headshot": "player.png",
"height": "6'2",
"name": "Wayne Brady",
"position": "QB",
"weight": 205
},
{
"label": "Player2",
(...)
}
]
},
That's not that bad. And it would take a lot to break the 16MB limit. And considering this seems to be a "user team" then it could probably do with some information from the "user" as well.
You also get a lot of power out of this when data is kept together like this, to find the top "player" picked by each user:
db.userteams.aggregate([
// Unwind the array
{ "$unwind": "$list" },
// Group and use the player name
{ "$group": {
"_id": {
"user_id": "$user_id",
"player": "$list.name",
},
"count": { "$sum": 1 }
}},
// Sort the results descending by popularity
{ "$sort": { "_id.user_id": 1, "count": -1 } },
// Group to limit the first one
{ "$group": {
"_id": "$_id.user_id",
"player": { "$first": "$_id.player" },
"picks": { "$first:" "$count" }
}}
])
Which admittedly is a reasonably trivial use of a name in this case, but it is an example of using information that has become available by the use of some embedding.
Of course you really believe that you need everything to be properly normalized, then do it that way, and live with the patterns you would need to access it. But this offers a perspective of doing this another way.
So don't over-concern yourself with embedding everything, and lose a little fear on embedding some things. There are no "get out of jail free cards" for using something not suited to relational modeling in a standard relational way. Choose something that suits your needs.
Is there any mechanism for controlling the order of properties?
I cannot reproduce this in http://www.neo4j.org/console
Using Neo4j 1.9.2 Community if I do the following:
CREATE (m1 {`$type`: {moduleTypeName}, Name: 'M1', ModelNumber: 'MN1'})
Then later I get this node back from a cypher query using the REST cypher endpoint I get back...
{
"extensions": {},
"paged_traverse": "http://localhost:7575/db/data/node/3777/paged/traverse/{returnType}{?pageSize,leaseTime}",
"outgoing_relationships": "http://localhost:7575/db/data/node/3777/relationships/out",
"traverse": "http://localhost:7575/db/data/node/3777/traverse/{returnType}",
"all_typed_relationships": "http://localhost:7575/db/data/node/3777/relationships/all/{-list|&|types}",
"property": "http://localhost:7575/db/data/node/3777/properties/{key}",
"all_relationships": "http://localhost:7575/db/data/node/3777/relationships/all",
"self": "http://localhost:7575/db/data/node/3777",
"properties": "http://localhost:7575/db/data/node/3777/properties",
"outgoing_typed_relationships": "http://localhost:7575/db/data/node/3777/relationships/out/{-list|&|types}",
"incoming_relationships": "http://localhost:7575/db/data/node/3777/relationships/in",
"incoming_typed_relationships": "http://localhost:7575/db/data/node/3777/relationships/in/{-list|&|types}",
"create_relationship": "http://localhost:7575/db/data/node/3777/relationships",
"data": {
"ModelNumber": "MN1",
"$type": "ModuleType",
"Name": "M1"
}
}
I'm using http://james.newtonking.com/pages/json-net.aspx to parse JSON and for it to automatically infer an object type, the $type property must be first. It makes sense when parsing the JSON in a stream when you don't want to load the entire thing into memory first.
It does not appear to be alphabetical, and it does not seem to be random either. It seems that the order is consistent for different object types, but inconsistent between them.
I have pulled the node in the Shell as well and so it seems that the order does not depend on how I get the node, but is not related to the order in which I create the node either.
Properties have no guaranteed order. Do not take any assumptions on a 'maybe' ordering. An upcoming version might change this assumed behaviour and break your code.
I guess it is simpler in Cypher to not return the node itself in favour of a list of properties, e.g.
START node=node(<myid>)
RETURN node.`$type`, node.ModelNumber, node.Name
This has defined columns.
Definitively it seems not to have that functionality.
My workaround for it it is to alias the properties with a prefix in the format aXX_, as in a01_, a02, a03_ and then remove it in the code.
Not pretty, not great, but it works as neo4j respects numerical order.
It needs a letter character at the beggining though, hence the "a" before the numbers.
I am trying to store network layout in Couch DB, but my solution provides rather randomized graph.
I store a nodes with a document:
{_id ,
nodeName,
group}
and storing links in traditional:
{_id, source_id, target_id, value}
Following multiple tutorials on handling joins and multiple relationship in Couch DB I created view:
function(doc) {
if(doc.type == 'connection') {
if (doc.source_id)
emit("source", {'_id': doc.source_id});
if(doc.target_id)
emit("target", {'_id': doc.target_id});
}
}
which should have emitted sequence of source and target id, then I pass it to the list function with include_docs=true, assumes that source and target come in pairs stitches everything back in a structure like this:
{
"nodes":[
{"nodeName":"Name 1","group":"1"},
{"nodeName":"Name 2","group":"1"},
],
"links": [
{"source":7,"target":0,"value":1},
{"source":7,"target":5,"value":1}
]
}
Although my list produce a proper JSON, view map returns number of rows of source docs and then target docs.
So far I don't have any ideas how to make this thing working properly - I am happy to fetch additional values from document _id in the list, but so far I havn't find any good examples.
Alternative ways of achieving the same goal are welcome. _id values are standard for CouchDB so far.
Update: while writing a question I came up with different view which sorted my immediate problem, but I still would like to see other options.
updated map:
function(doc) {
if(doc.type == 'connection') {
if (doc.source_id)
emit([doc._id,0,"source"], {'_id': doc.source_id});
if(doc.target_id)
emit([doc._id,1,"target"], {'_id': doc.target_id});
}
}
Your updated map function makes more sense. However, you don't need 0 and 1 in your key since you have already "source"and "target".