This answer clearly shows some examples of proto text parsing, but does not have an example for maps.
If a proto has:
map<int32, string> aToB
I would guess something like:
aToB {
123: "foo"
}
but it does not work. Does anyone know the exact syntax?
I initially tried extrapolating from an earlier answer, which led me astray, because I incorrectly thought multiple k/v pairs would look like this:
aToB { # (this example has a bug)
key: 123
value: "foo"
key: 876 # WRONG!
value: "bar" # NOPE!
}
That led to the following error:
libprotobuf ERROR: Non-repeated field "key" is specified multiple times.
Proper syntax for multiple key-value pairs:
(Note: I am using the "proto3" version of the protocol buffers language)
aToB {
key: 123
value: "foo"
}
aToB {
key: 876
value: "bar"
}
The pattern of repeating the name of the map variable makes more sense after re-reading this relevant portion of the proto3 Map documentation, which explains that maps are equivalent to defining your own "pair" message type and then marking it as "repeated".
A more complete example:
proto definition:
syntax = "proto3";
package myproject.testing;
message UserRecord {
string handle = 10;
bool paid_membership = 20;
}
message UserCollection {
string description = 20;
// HERE IS THE PROTOBUF MAP-TYPE FIELD:
map<string, UserRecord> users = 10;
}
message TestData {
UserCollection user_collection = 10;
}
text format ("pbtxt") in a config file:
user_collection {
description = "my default users"
users {
key: "user_1234"
value {
handle: "winniepoo"
paid_membership: true
}
}
users {
key: "user_9b27"
value {
handle: "smokeybear"
}
}
}
C++ that would generate the message content programmatically
myproject::testing::UserRecord user_1;
user_1.set_handle("winniepoo");
user_1.set_paid_membership(true);
myproject::testing::UserRecord user_2;
user_2.set_handle("smokeybear");
user_2.set_paid_membership(false);
using pair_type =
google::protobuf::MapPair<std::string, myproject::testing::UserRecord>;
myproject::testing::TestData data;
data.mutable_user_collection()->mutable_users()->insert(
pair_type(std::string("user_1234"), user_1));
data.mutable_user_collection()->mutable_users()->insert(
pair_type(std::string("user_9b27"), user_2));
The text format is:
aToB {
key: 123
value: "foo"
}
Related
I have a map of key value pairs in Dart. I want to convert it to YAML and write into a file.
I tried using YAML package from dart library but it only provides methods to load YAML data from a file. Nothing is mentioned on how to write it back to the YAML file.
Here is an example:
void main() {
var map = {
"name": "abc",
"type": "unknown",
"internal":{
"name": "xyz"
}
};
print(map);
}
Expected output:
example.yaml
name: abc
type: unknown
internal:
name: xyz
How to convert the dart map to YAML and write it to a file?
It's a bit late of a response but for anyone else looking at this question I have written this class. It may not be perfect but it works for what I'm doing and I haven't found anything wrong with it yet. Might make it a package eventually after writing tests.
class YamlWriter {
/// The amount of spaces for each level.
final int spaces;
/// Initialize the writer with the amount of [spaces] per level.
YamlWriter({
this.spaces = 2,
});
/// Write a dart structure to a YAML string. [yaml] should be a [Map] or [List].
String write(dynamic yaml) {
return _writeInternal(yaml).trim();
}
/// Write a dart structure to a YAML string. [yaml] should be a [Map] or [List].
String _writeInternal(dynamic yaml, { int indent = 0 }) {
String str = '';
if (yaml is List) {
str += _writeList(yaml, indent: indent);
} else if (yaml is Map) {
str += _writeMap(yaml, indent: indent);
} else if (yaml is String) {
str += "\"${yaml.replaceAll("\"", "\\\"")}\"";
} else {
str += yaml.toString();
}
return str;
}
/// Write a list to a YAML string.
/// Pass the list in as [yaml] and indent it to the [indent] level.
String _writeList(List yaml, { int indent = 0 }) {
String str = '\n';
for (var item in yaml) {
str += "${_indent(indent)}- ${_writeInternal(item, indent: indent + 1)}\n";
}
return str;
}
/// Write a map to a YAML string.
/// Pass the map in as [yaml] and indent it to the [indent] level.
String _writeMap(Map yaml, { int indent = 0 }) {
String str = '\n';
for (var key in yaml.keys) {
var value = yaml[key];
str += "${_indent(indent)}${key.toString()}: ${_writeInternal(value, indent: indent + 1)}\n";
}
return str;
}
/// Create an indented string for the level with the spaces config.
/// [indent] is the level of indent whereas [spaces] is the
/// amount of spaces that the string should be indented by.
String _indent(int indent) {
return ''.padLeft(indent * spaces, ' ');
}
}
Usage:
final writer = YamlWriter();
String yaml = writer.write({
'string': 'Foo',
'int': 1,
'double': 3.14,
'boolean': true,
'list': [
'Item One',
'Item Two',
true,
'Item Four',
],
'map': {
'foo': 'bar',
'list': ['Foo', 'Bar'],
},
});
File file = File('/path/to/file.yaml');
file.createSync();
file.writeAsStringSync(yaml);
Output:
string: "Foo"
int: 1
double: 3.14
boolean: true
list:
- "Item One"
- "Item Two"
- true
- "Item Four"
map:
foo: "bar"
list:
- "Foo"
- "Bar"
package:yaml does not have YAML writing features. You may have to look for another package that does that – or write your own.
As as stopgap, remember JSON is valid YAML, so you can always write out JSON to a .yaml file and it should work with any YAML parser.
I ran into the same issue and ended up hacking together a simple writer:
// Save the updated configuration settings to the config file
void saveConfig() {
var file = _configFile;
// truncate existing configuration
file.writeAsStringSync('');
// Write out new YAML document from JSON map
final config = configToJson();
config.forEach((key, value) {
if (value is Map) {
file.writeAsStringSync('\n$key:\n', mode: FileMode.writeOnlyAppend);
value.forEach((subkey, subvalue) {
file.writeAsStringSync(' $subkey: $subvalue\n',
mode: FileMode.writeOnlyAppend);
});
} else {
file.writeAsStringSync('$key: $value\n',
mode: FileMode.writeOnlyAppend);
}
});
}
I built an oauth2 url with query params using URLSearchParmas API. However, the output URL didn't return an expected url. Can anyone help me understand the difference between those two APIs and how can I get the same result as the output of encodeURIComponent, using URLSearchParams? Thanks!
const expected = encodeURIComponent('code id_token'); // code%20id_token
const search = new URLSearchParams();
search.set('response_type', 'code id_token');
search.toString(); // code+id_token
According to WHATWG, URLSearchParams uses application/x-www-form-urlencoded format. While it's suitable for decoding URL queries, for encoding it can lead to unexpected results such as spaces being encoded as + and extra characters such as ~ being percent-encoded. It's better to use encodeURIComponent instead:
Having an object:
const params = {
text1: 'aaa bbb',
text2: '-._*~()'
}
Instead of:
url.search = (new URLSearchParams(params)).toString()
Use:
url.search = Object.entries(params)
.map(([key, value]) => `${encodeURIComponent(key)}=${encodeURIComponent(value)}`)
.join('&')
Also, according to MDN even encodeURIComponent doesn't conform to newer RFC 3986 which defines more characters to escape, for example *. While it's probably safe not to escape these additional characters if you aren't using them as field separators, if you want to be strictly conformant to latest RFC, use this updated implementation from MDN:
function fixedEncodeURIComponent(str) {
return encodeURIComponent(str).replace(/[!'()*]/g, function(c) {
return '%' + c.charCodeAt(0).toString(16).toUpperCase();
});
}
A playground for experimenting:
const params = {
text1: 'aaa bbb',
text2: '-._*~()'
}
const url1 = new URL('http://example.com')
const search1 = new URLSearchParams(params)
url1.search = search1 // Incorrect
console.log('URLSearchParams', url1.toString())
function fixedEncodeURIComponent(str) {
return encodeURIComponent(str).replace(/[!'()*]/g, function(c) {
return '%' + c.charCodeAt(0).toString(16).toUpperCase()
})
}
const url2 = new URL('http://example.com')
const search2 = Object
.entries(params)
.map(([key, value]) => `${fixedEncodeURIComponent(key)}=${fixedEncodeURIComponent(value)}`)
.join('&')
url2.search = search2 // Correct
console.log('fixedEncodeURIComponent', url2.toString())
I have the following situation.
I want to count in Module 1, how many objects are having links in links from Module 3.
example:
Module 1 Obj1 <- Module 2 Obj1 <- Module 3.Obj1
Module 1 Obj2 <- Module 2 Obj1 <- Module 3.Obj1
Module 1 Obj3 <- Module 2 Obj1 <- Module 3.Obj1
Module 1 Obj4 <- Module 2 Obj1
Module 1 Obj5 <- Module 2 Obj1
The count should return 3, in the above case.
Is it possible via DXL to follow a link, and then follow another link?
(not using the Wizard or DXL attributes)
Most important for me: knowing if somebody else did this and it's possible to do.
Please try the following DXL from within the module that has the incoming links. Before running the code:
make sure that you open the 'Edit DXL' window from the relevant module
set the string values assigned to global constant STR_LINK_MOD_FULLNAME (line 17) to the full pathname of the link module whose links you are interested in
set the string values assigned to global constant STR_SRC_MOD_FULLNAME (line 18) to the full pathname of the source formal module (Module 3, in your example) whose links you are interested in
You shouldn't need to change anything else to make it work.
N.B. I have not considered the implications of analysing links in all link modules by using the string "*" in place of a specific link module name in line 17 (see point 2, above).
I also haven't gone out of my way to explain the code, though I have tried to be nice and tidy up after myself where DOORS and DXL require it. Please feel free to reply with any questions on what I have done.
Kind regards,
Richard
//<CheckObjectInNestedLink.dxl>
/*
*/
///////////////
// Sanity check
if (null (current Module))
{
print "ERROR: this script must be run from a Formal Module."
}
///////////////////
// Global Constants
const string STR_LINK_MOD_FULLNAME = "/New Family Car Project/Admin/Satisfies" // the fullName of a single link module - results of using "*" have not been considered/tested
const string STR_SRC_MOD_FULLNAME = "/New Family Car Project/Architecture/Architectural Design" // The fullName of the desired source Formal Module
///////////////////
// Global Variables
Module modSource = null
Object objTarget = null
Object objSource = null
Link lkIn = null
Skip skLinkedMods = create()
Skip skObjsWithDesiredSource = create()
int intNoOfLinked = 0
//////////////////////
// Auxiliary Functions
void closeSourceMods ()
{
Module srcMod
for srcMod in skLinkedMods do
{
close(srcMod)
}
}
void openSourceMods (Object objWithLinks)
{
ModName_ srcModRef
Module srcMod
for srcModRef in (objWithLinks <- STR_LINK_MOD_FULLNAME) do
{
srcMod = read(fullName(srcModRef), false)
put(skLinkedMods, srcMod, srcMod)
}
}
void recurseFollowInLinks (Object objWithLinks)
{
openSourceMods(objWithLinks)
for lkIn in objWithLinks <- STR_LINK_MOD_FULLNAME do
{
openSourceMods(objWithLinks)
objSource = source(lkIn)
string strSrcModName = fullName(module(objSource))
if (strSrcModName == STR_SRC_MOD_FULLNAME)
{
bool blNewEntry = put(skObjsWithDesiredSource, objTarget, objTarget)
if (blNewEntry)
{
intNoOfLinked++
}
//print "put(skObjsWithDesiredSource, " identifier(objTarget) ", " identifier(objTarget) ")\n"
}
recurseFollowInLinks(objSource)
}
}
void checkObjectInNestedLink (Module modThis, string strSourceModuleFullname, string strLinkModuleFullName)
{
intNoOfLinked = 0
for objTarget in modThis do
{
recurseFollowInLinks(objTarget)
}
print "The following " intNoOfLinked " objects have direct or indirect links of type " STR_LINK_MOD_FULLNAME " from formal module " STR_SRC_MOD_FULLNAME ":\n"
for objTarget in skObjsWithDesiredSource do
{
print identifier(objTarget)
print "\n"
}
}
///////////////
// Main Program
checkObjectInNestedLink ((current Module), STR_SRC_MOD_FULLNAME, STR_LINK_MOD_FULLNAME)
closeSourceMods()
delete(skLinkedMods)
delete(skObjsWithDesiredSource)
Is it possible to format a document as follows, using Xtext formatting? As you can see, Test children are indented with 4 spaces while External children are indented with 2 spaces only. I am using Xtext 2.12.0.
Test my_prog {
Device = "my_device";
Param = 0;
}
External {
Path = "my_path";
File = "my_file";
}
you could try to work with custom replacers, dont know if this will work with nested block though
def dispatch void format(External model, extension IFormattableDocument document) {
model.regionFor.keyword("}").prepend[newLine]
for (l : model.ids) {
val region = l.regionFor.feature(MyDslPackage.Literals.IDX__NAME)
region.prepend[newLine]
val r = new AbstractTextReplacer(document, region) {
override createReplacements(ITextReplacerContext it) {
val offset = region.offset
it.addReplacement(region.textRegionAccess.rewriter.createReplacement(offset, 0, " "))
it
}
}
addReplacer(r)
}
}
I have to parse a document containing groups of variable-value-pairs which is serialized to a string e.g. like this:
4^26^VAR1^6^VALUE1^VAR2^4^VAL2^^1^14^VAR1^6^VALUE1^^
Here are the different elements:
Group IDs:
4^26^VAR1^6^VALUE1^VAR2^4^VAL2^^1^14^VAR1^6^VALUE1^^
Length of string representation of each group:
4^26^VAR1^6^VALUE1^VAR2^4^VAL2^^1^14^VAR1^6^VALUE1^^
One of the groups:
4^26^VAR1^6^VALUE1^VAR2^4^VAL2^^1^14 ^VAR1^6^VALUE1^^
Variables:
4^26^VAR1^6^VALUE1^VAR2^4^VAL2^^1^14^VAR1^6^VALUE1^^
Length of string representation of the values:
4^26^VAR1^6^VALUE1^VAR2^4^VAL2^^1^14^VAR1^6^VALUE1^^
The values themselves:
4^26^VAR1^6^VALUE1^VAR2^4^VAL2^^1^14^VAR1^6^VALUE1^^
Variables consist only of alphanumeric characters.
No assumption is made about the values, i.e. they may contain any character, including ^.
Is there a name for this kind of grammar? Is there a parsing library that can handle this mess?
So far I am using my own parser, but due to the fact that I need to detect and handle corrupt serializations the code looks rather messy, thus my question for a parser library that could lift the burden.
The simplest way to approach it is to note that there are two nested levels that work the same way. The pattern is extremely simple:
id^length^content^
At the outer level, this produces a set of groups. Within each group, the content follows exactly the same pattern, only here the id is the variable name, and the content is the variable value.
So you only need to write that logic once and you can use it to parse both levels. Just write a function that breaks a string up into a list of id/content pairs. Call it once to get the groups, and then loop through them calling it again for each content to get the variables in that group.
Breaking it down into these steps, first we need a way to get "tokens" from the string. This function returns an object with three methods, to find out if we're at "end of file", and to grab the next delimited or counted substring:
var tokens = function(str) {
var pos = 0;
return {
eof: function() {
return pos == str.length;
},
delimited: function(d) {
var end = str.indexOf(d, pos);
if (end == -1) {
throw new Error('Expected delimiter');
}
var result = str.substr(pos, end - pos);
pos = end + d.length;
return result;
},
counted: function(c) {
var result = str.substr(pos, c);
pos += c;
return result;
}
};
};
Now we can conveniently write the reusable parse function:
var parse = function(str) {
var parts = {};
var t = tokens(str);
while (!t.eof()) {
var id = t.delimited('^');
var len = t.delimited('^');
var content = t.counted(parseInt(len, 10));
var end = t.counted(1);
if (end !== '^') {
throw new Error('Expected ^ after counted string, instead found: ' + end);
}
parts[id] = content;
}
return parts;
};
It builds an object where the keys are the IDs (or variable names). I'm asuming as they have names that the order isn't significant.
Then we can use that at both levels to create the function to do the whole job:
var parseGroups = function(str) {
var groups = parse(str);
Object.keys(groups).forEach(function(id) {
groups[id] = parse(groups[id]);
});
return groups;
}
For your example, it produces this object:
{
'1': {
VAR1: 'VALUE1'
},
'4': {
VAR1: 'VALUE1',
VAR2: 'VAL2'
}
}
I don't think it's a trivial task to create a grammar for this. But on the other hand, a simple straight forward approach is not that hard. You know the corresponding string length for every critical string. So you just chop your string according to those lengths apart..
where do you see problems?