I have a map of key value pairs in Dart. I want to convert it to YAML and write into a file.
I tried using YAML package from dart library but it only provides methods to load YAML data from a file. Nothing is mentioned on how to write it back to the YAML file.
Here is an example:
void main() {
var map = {
"name": "abc",
"type": "unknown",
"internal":{
"name": "xyz"
}
};
print(map);
}
Expected output:
example.yaml
name: abc
type: unknown
internal:
name: xyz
How to convert the dart map to YAML and write it to a file?
It's a bit late of a response but for anyone else looking at this question I have written this class. It may not be perfect but it works for what I'm doing and I haven't found anything wrong with it yet. Might make it a package eventually after writing tests.
class YamlWriter {
/// The amount of spaces for each level.
final int spaces;
/// Initialize the writer with the amount of [spaces] per level.
YamlWriter({
this.spaces = 2,
});
/// Write a dart structure to a YAML string. [yaml] should be a [Map] or [List].
String write(dynamic yaml) {
return _writeInternal(yaml).trim();
}
/// Write a dart structure to a YAML string. [yaml] should be a [Map] or [List].
String _writeInternal(dynamic yaml, { int indent = 0 }) {
String str = '';
if (yaml is List) {
str += _writeList(yaml, indent: indent);
} else if (yaml is Map) {
str += _writeMap(yaml, indent: indent);
} else if (yaml is String) {
str += "\"${yaml.replaceAll("\"", "\\\"")}\"";
} else {
str += yaml.toString();
}
return str;
}
/// Write a list to a YAML string.
/// Pass the list in as [yaml] and indent it to the [indent] level.
String _writeList(List yaml, { int indent = 0 }) {
String str = '\n';
for (var item in yaml) {
str += "${_indent(indent)}- ${_writeInternal(item, indent: indent + 1)}\n";
}
return str;
}
/// Write a map to a YAML string.
/// Pass the map in as [yaml] and indent it to the [indent] level.
String _writeMap(Map yaml, { int indent = 0 }) {
String str = '\n';
for (var key in yaml.keys) {
var value = yaml[key];
str += "${_indent(indent)}${key.toString()}: ${_writeInternal(value, indent: indent + 1)}\n";
}
return str;
}
/// Create an indented string for the level with the spaces config.
/// [indent] is the level of indent whereas [spaces] is the
/// amount of spaces that the string should be indented by.
String _indent(int indent) {
return ''.padLeft(indent * spaces, ' ');
}
}
Usage:
final writer = YamlWriter();
String yaml = writer.write({
'string': 'Foo',
'int': 1,
'double': 3.14,
'boolean': true,
'list': [
'Item One',
'Item Two',
true,
'Item Four',
],
'map': {
'foo': 'bar',
'list': ['Foo', 'Bar'],
},
});
File file = File('/path/to/file.yaml');
file.createSync();
file.writeAsStringSync(yaml);
Output:
string: "Foo"
int: 1
double: 3.14
boolean: true
list:
- "Item One"
- "Item Two"
- true
- "Item Four"
map:
foo: "bar"
list:
- "Foo"
- "Bar"
package:yaml does not have YAML writing features. You may have to look for another package that does that – or write your own.
As as stopgap, remember JSON is valid YAML, so you can always write out JSON to a .yaml file and it should work with any YAML parser.
I ran into the same issue and ended up hacking together a simple writer:
// Save the updated configuration settings to the config file
void saveConfig() {
var file = _configFile;
// truncate existing configuration
file.writeAsStringSync('');
// Write out new YAML document from JSON map
final config = configToJson();
config.forEach((key, value) {
if (value is Map) {
file.writeAsStringSync('\n$key:\n', mode: FileMode.writeOnlyAppend);
value.forEach((subkey, subvalue) {
file.writeAsStringSync(' $subkey: $subvalue\n',
mode: FileMode.writeOnlyAppend);
});
} else {
file.writeAsStringSync('$key: $value\n',
mode: FileMode.writeOnlyAppend);
}
});
}
Related
I am trying to generate some generic Groovy code for Jenkins but I seem to have trouble with multi line strings and extra white space. I've tried everything I could find by Googling but I can't seem to get it working.
My issue isn't related to simple multi line strings. I managed to trim white space by using the stripIndent() and stripMargin() methods for simple cases. My issue is caused by having interpolated methods inside my strings.
Groovy info: Groovy Version: 3.0.2 JVM: 13.0.2 Vendor: Oracle Corporation OS: Mac OS X
String method2(String tier, String jobName) {
return """
Map downstreamJobs = [:]
stage ("${jobName}-${tier}-\${region}_${jobName}") {
test
}
""".stripIndent().stripMargin()
}
static String simpleLog() {
return """
script {
def user = env.BUILD_USER_ID
}
""".stripIndent().stripMargin()
}
static String method1() {
return """\
import jenkins.model.Jenkins
currentBuild.displayName = "name"
${simpleLog()}
""".stripIndent().stripMargin()
}
String generateFullDeploymentPipelineCode() {
return """Text here
${method1()}
${method2("test1", "test2")}
""".stripIndent().stripMargin()
}
println(generateFullDeploymentPipelineCode())
This is what it prints(or writes to disk):
Text here
import jenkins.model.Jenkins
currentBuild.displayName = "name"
script {
def user = env.BUILD_USER_ID
}
Map downstreamJobs = [:]
stage ("test2-test1-${region}_test2") {
test
}
Why the extra space around the import lines? I know the indentation method is supposed to trim all white space according to the least number of leading spaces, so that's why we use backslash (example here https://stackoverflow.com/a/19882917/7569335).
That works for simple strings, but it breaks down once use start using interpolation. Not with regular variables, just when you interpolate an entire method.
as variant - use just stripMargin() and only once on a final string
String method2(String tier, String jobName) {
return """\
|Map downstreamJobs = [:]
|stage ("${jobName}-${tier}-\${region}_${jobName}") {
| test
|}
"""
}
static String simpleLog() {
return """\
|script {
| def user = env.BUILD_USER_ID
|}
"""
}
static String method1() {
return """\
|import jenkins.model.Jenkins
|currentBuild.displayName = "name"
${simpleLog()}
"""
}
String generateFullDeploymentPipelineCode() {
return """\
|Text here
${method1()}
${method2("test1", "test2")}
""".stripIndent().stripMargin()
}
println(generateFullDeploymentPipelineCode())
result:
Text here
import jenkins.model.Jenkins
currentBuild.displayName = "name"
script {
def user = env.BUILD_USER_ID
}
Map downstreamJobs = [:]
stage ("test2-test1-${region}_test2") {
test
}
another variant with trim() and stripIndent()
def method2(String tier, String jobName) {
return """
Map downstreamJobs = [:]
stage ("${jobName}-${tier}-\${region}_${jobName}") {
test
}
""".trim()
}
def simpleLog() {
return """
script {
def user = env.BUILD_USER_ID
}
""".trim()
}
def method1() {
return """
import jenkins.model.Jenkins
currentBuild.displayName = "name"
${simpleLog()}
""".trim()
}
def generateFullDeploymentPipelineCode() {
return """\
Text here
${method1()}
${method2("test1", "test2")}
""".stripIndent()
}
println(generateFullDeploymentPipelineCode())
When you insert a string through interpolation you only indent the first line of it. The following lines of the inserted string will be indented differently, which messes everything up.
Using some lesser-known members of GString (namely .strings[] and .values[]), we can align the indentation of all lines of each interpolated value.
String method2(String tier, String jobName) {
indented """
Map downstreamJobs = [:]
stage ("${jobName}-${tier}-\${region}_${jobName}") {
test
}
"""
}
String simpleLog() {
indented """\
script {
def user = env.BUILD_USER_ID
}
"""
}
String method1() {
indented """\
import jenkins.model.Jenkins
currentBuild.displayName = "name"
${simpleLog()}
"""
}
String generateFullDeploymentPipelineCode() {
indented """\
Text here
${method1()}
${method2("test1", "test2")}
"""
}
println generateFullDeploymentPipelineCode()
//---------- Move the following code into its own script ----------
// Function to adjust the indentation of interpolated values so that all lines
// of a value match the indentation of the first line.
// Finally stripIndent() will be called before returning the string.
String indented( GString templ ) {
// Iterate over the interpolated values of the GString template.
templ.values.eachWithIndex{ value, i ->
// Get the string preceding the current value. Always defined, even
// when the value is at the beginning of the template.
def beforeValue = templ.strings[ i ]
// RegEx to match any indent substring before the value.
// Special case for the first string, which doesn't necessarily contain '\n'.
def regexIndent = i == 0
? /(?:^|\n)([ \t]+)$/
: /\n([ \t]+)$/
def matchIndent = ( beforeValue =~ regexIndent )
if( matchIndent ) {
def indent = matchIndent[ 0 ][ 1 ]
def lines = value.readLines()
def linesNew = [ lines.head() ] // The 1st line is already indented.
// Insert the indentation from the 1st line into all subsequent lines.
linesNew += lines.tail().collect{ indent + it }
// Finally replace the value with the reformatted lines.
templ.values[ i ] = linesNew.join('\n')
}
}
return templ.stripIndent()
}
// Fallback in case the input string is not a GString (when it doesn't contain expressions)
String indented( String templ ) {
return templ.stripIndent()
}
Live Demo at codingground
Output:
Text here
import jenkins.model.Jenkins
currentBuild.displayName = "name"
script {
def user = env.BUILD_USER_ID
}
Map downstreamJobs = [:]
stage ("test2-test1-${region}_test2") {
test
}
Conclusion:
Using the indented function, a clean Groovy syntax for generating code from GString templates has been achieved.
This was quite a learning experience. I first tried to do it completely different using the evaluate function, which turned out to be too complicated and not so flexible. Then I randomly browsed through some posts from mrhaki blog (always a good read!) until I discovered "Groovy Goodness: Get to Know More About a GString". This was the key to implementing this solution.
Is there a way for me to include empty lines into the parsed YAML file using YamlDotNet? What I have currently when I parse a file like this:
node1: "1.0"
node2: "some text"
node3: "string"
I end up with this as the result:
node1: "1.0"
node2: "some text"
node3: "string"
Is there a way for me to configure the parser to not ignore blank lines?
Briefly, I'm using the YamlDotNet Parser class like so:
var input = File.OpenText(file);
var parser = new Parser(_input);
public bool Read()
{
Value = null;
Path = null;
var hasMore = _parser.MoveNext();
if (!hasMore)
{
return false;
}
parser.Current.Accept(this);
LineNumber = _parser.Current.Start.Line;
return true;
}
And in a separate class:
while (reader.Read())
{
}
EDIT:
This doesn't only happen with blank lines, it also happens when I have a line break after a dash:
Before:
-
name: Mark McGwire
hr: 65
avg: 0.278
After:
- name: Mark McGwire
hr: 65
avg: 0.278
This answer clearly shows some examples of proto text parsing, but does not have an example for maps.
If a proto has:
map<int32, string> aToB
I would guess something like:
aToB {
123: "foo"
}
but it does not work. Does anyone know the exact syntax?
I initially tried extrapolating from an earlier answer, which led me astray, because I incorrectly thought multiple k/v pairs would look like this:
aToB { # (this example has a bug)
key: 123
value: "foo"
key: 876 # WRONG!
value: "bar" # NOPE!
}
That led to the following error:
libprotobuf ERROR: Non-repeated field "key" is specified multiple times.
Proper syntax for multiple key-value pairs:
(Note: I am using the "proto3" version of the protocol buffers language)
aToB {
key: 123
value: "foo"
}
aToB {
key: 876
value: "bar"
}
The pattern of repeating the name of the map variable makes more sense after re-reading this relevant portion of the proto3 Map documentation, which explains that maps are equivalent to defining your own "pair" message type and then marking it as "repeated".
A more complete example:
proto definition:
syntax = "proto3";
package myproject.testing;
message UserRecord {
string handle = 10;
bool paid_membership = 20;
}
message UserCollection {
string description = 20;
// HERE IS THE PROTOBUF MAP-TYPE FIELD:
map<string, UserRecord> users = 10;
}
message TestData {
UserCollection user_collection = 10;
}
text format ("pbtxt") in a config file:
user_collection {
description = "my default users"
users {
key: "user_1234"
value {
handle: "winniepoo"
paid_membership: true
}
}
users {
key: "user_9b27"
value {
handle: "smokeybear"
}
}
}
C++ that would generate the message content programmatically
myproject::testing::UserRecord user_1;
user_1.set_handle("winniepoo");
user_1.set_paid_membership(true);
myproject::testing::UserRecord user_2;
user_2.set_handle("smokeybear");
user_2.set_paid_membership(false);
using pair_type =
google::protobuf::MapPair<std::string, myproject::testing::UserRecord>;
myproject::testing::TestData data;
data.mutable_user_collection()->mutable_users()->insert(
pair_type(std::string("user_1234"), user_1));
data.mutable_user_collection()->mutable_users()->insert(
pair_type(std::string("user_9b27"), user_2));
The text format is:
aToB {
key: 123
value: "foo"
}
I have a Groovy file containing a bunch of simple functions like so:
// useful functions
def myFunc1(String arg) {
println("Hello " + arg)
}
def myFunc2(String arg) {
println("Goodbye " + arg)
}
I'd like to obtain from this:
the method name
the arguments
the body code of the function
(All as simple strings, I don't need to run anything yet.)
I was about to resort to some Regexing, but since I'm using a JVM language (Scala) I figured I might be able to use some of the Groovy compiler's stuff to do this a "nicer" way.
There seems to be a fair bit of information on loading Groovy code dynamically and running it, but not so much on introspecting the source. Any ideas?
(Failing a "nice" way, I'll also accept some Scala-foo to parse the information in a succinct fashion.)
This works, and demonstrates the token types required to find each node of importance in the AST. Hope it makes sense... By using lots of Groovy dynamism, I hope I haven't made it too hard for a port to Scala :-(
import org.codehaus.groovy.antlr.*
import org.codehaus.groovy.antlr.parser.*
import static org.codehaus.groovy.antlr.parser.GroovyTokenTypes.*
def code = '''
// useful functions
def myFunc1(String arg) {
println("Hello " + arg)
}
def myFunc2(arg, int arg2) {
println("Goodbye " + arg)
}
public String stringify( int a ) {
"$a"
}
'''
def lines = code.split( '\n' )
// Generate a GroovyRecognizer, compile an AST and assign it to 'ast'
def ast = new SourceBuffer().with { buff ->
new UnicodeEscapingReader( new StringReader( code ), buff ).with { read ->
read.lexer = new GroovyLexer( read )
GroovyRecognizer.make( read.lexer ).with { parser ->
parser.sourceBuffer = buff
parser.compilationUnit()
parser.AST
}
}
}
// Walks the ast looking for types
def findByPath( ast, types, multiple=false ) {
[types.take( 1 )[ 0 ],types.drop(1)].with { head, tail ->
if( tail ) {
findByPath( ast*.childrenOfType( head ).flatten(), tail, multiple )
}
else {
ast*.childrenOfType( head ).with { ret ->
multiple ? ret[ 0 ] : ret.head()[0]
}
}
}
}
// Walk through the returned ast
while( ast ) {
def methodModifier = findByPath( ast, [ MODIFIERS ] ).firstChild?.toStringTree() ?: 'public'
def returnType = findByPath( ast, [ TYPE, IDENT ] ) ?: 'Object'
def methodName = findByPath( ast, [ IDENT ] )
def body = findByPath( ast, [ SLIST ] )
def parameters = findByPath( ast, [ PARAMETERS, PARAMETER_DEF ], true ).collect { param ->
[ type: findByPath( param, [ TYPE ] ).firstChild?.toStringTree() ?: 'Object',
name: findByPath( param, [ IDENT ] ) ]
}
def (y1,y2,x1,x2) = [ body.line - 1, body.lineLast - 1, body.column - 1, body.columnLast ]
// Grab the text from the original string
def snip = [ lines[ y1 ].drop( x1 ), // First line prefix stripped
*lines[ (y1+1)..<y2 ], // Mid lines
lines[ y2 ].take( x2 ) ].join( '\n' ) // End line suffix stripped
println '------------------------------'
println "modifier: $methodModifier"
println "returns: $returnType"
println "name: $methodName"
println "params: $parameters"
println "$snip\n"
// Step to next branch and repeat
ast = ast.nextSibling
}
It prints out:
------------------------------
modifier: public
returns: Object
name: myFunc1
params: [[type:String, name:arg]]
{
println("Hello " + arg)
}
------------------------------
modifier: public
returns: Object
name: myFunc2
params: [[type:Object, name:arg], [type:int, name:arg2]]
{
println("Goodbye " + arg)
}
------------------------------
modifier: public
returns: String
name: stringify
params: [[type:int, name:a]]
{
"$a"
}
Hope it helps, or points you in the right direction :-)
I have to parse a document containing groups of variable-value-pairs which is serialized to a string e.g. like this:
4^26^VAR1^6^VALUE1^VAR2^4^VAL2^^1^14^VAR1^6^VALUE1^^
Here are the different elements:
Group IDs:
4^26^VAR1^6^VALUE1^VAR2^4^VAL2^^1^14^VAR1^6^VALUE1^^
Length of string representation of each group:
4^26^VAR1^6^VALUE1^VAR2^4^VAL2^^1^14^VAR1^6^VALUE1^^
One of the groups:
4^26^VAR1^6^VALUE1^VAR2^4^VAL2^^1^14 ^VAR1^6^VALUE1^^
Variables:
4^26^VAR1^6^VALUE1^VAR2^4^VAL2^^1^14^VAR1^6^VALUE1^^
Length of string representation of the values:
4^26^VAR1^6^VALUE1^VAR2^4^VAL2^^1^14^VAR1^6^VALUE1^^
The values themselves:
4^26^VAR1^6^VALUE1^VAR2^4^VAL2^^1^14^VAR1^6^VALUE1^^
Variables consist only of alphanumeric characters.
No assumption is made about the values, i.e. they may contain any character, including ^.
Is there a name for this kind of grammar? Is there a parsing library that can handle this mess?
So far I am using my own parser, but due to the fact that I need to detect and handle corrupt serializations the code looks rather messy, thus my question for a parser library that could lift the burden.
The simplest way to approach it is to note that there are two nested levels that work the same way. The pattern is extremely simple:
id^length^content^
At the outer level, this produces a set of groups. Within each group, the content follows exactly the same pattern, only here the id is the variable name, and the content is the variable value.
So you only need to write that logic once and you can use it to parse both levels. Just write a function that breaks a string up into a list of id/content pairs. Call it once to get the groups, and then loop through them calling it again for each content to get the variables in that group.
Breaking it down into these steps, first we need a way to get "tokens" from the string. This function returns an object with three methods, to find out if we're at "end of file", and to grab the next delimited or counted substring:
var tokens = function(str) {
var pos = 0;
return {
eof: function() {
return pos == str.length;
},
delimited: function(d) {
var end = str.indexOf(d, pos);
if (end == -1) {
throw new Error('Expected delimiter');
}
var result = str.substr(pos, end - pos);
pos = end + d.length;
return result;
},
counted: function(c) {
var result = str.substr(pos, c);
pos += c;
return result;
}
};
};
Now we can conveniently write the reusable parse function:
var parse = function(str) {
var parts = {};
var t = tokens(str);
while (!t.eof()) {
var id = t.delimited('^');
var len = t.delimited('^');
var content = t.counted(parseInt(len, 10));
var end = t.counted(1);
if (end !== '^') {
throw new Error('Expected ^ after counted string, instead found: ' + end);
}
parts[id] = content;
}
return parts;
};
It builds an object where the keys are the IDs (or variable names). I'm asuming as they have names that the order isn't significant.
Then we can use that at both levels to create the function to do the whole job:
var parseGroups = function(str) {
var groups = parse(str);
Object.keys(groups).forEach(function(id) {
groups[id] = parse(groups[id]);
});
return groups;
}
For your example, it produces this object:
{
'1': {
VAR1: 'VALUE1'
},
'4': {
VAR1: 'VALUE1',
VAR2: 'VAL2'
}
}
I don't think it's a trivial task to create a grammar for this. But on the other hand, a simple straight forward approach is not that hard. You know the corresponding string length for every critical string. So you just chop your string according to those lengths apart..
where do you see problems?