I am trying to parse using nom with nom_locate, following this tutorial. I only want output in the format:
Error: line 5, column 1
But currently I get:
Error: Parsing Failure: ParseError { span: LocatedSpan { offset: 37, line: 5, fragment: "{{G]\nK(1)key [J]\n", extra: () }, message: "Error: line 5, column 1" }
Relevant code:
pub type Span<'a> = LocatedSpan<&'a str>;
pub type IResult<'a, O> = nom::IResult<Span<'a>, O, ParseError<'a>>;
#[derive(Debug, PartialEq)]
pub struct ParseError<'a> {
span: Span<'a>,
message: String,
}
impl<'a> ParseError<'a> {
pub fn new(message: String, span: Span<'a>) -> Self {
Self {
span,
message,
}
}
}
impl<'a> nom::error::ParseError<Span<'a>> for ParseError<'a> {
fn from_error_kind(input: Span<'a>, _: nom::error::ErrorKind) -> Self {
Self::new(format!("Error: line {}, column {}", input.location_line(), input.location_line()), input)
}
fn append(input: Span<'a>, _: nom::error::ErrorKind, _: Self) -> Self {
Self::new(format!("Error: line {}, column {}", input.location_line(), input.location_line()), input)
}
fn from_char(input: Span<'a>, _: char) -> Self {
Self::new(format!("Error: line {}, column {}", input.location_line(), input.location_line()), input)
}
}
pub fn job(s: Span) -> IResult<Vec<entities::Step>> {
let (s, steps) = many1(job_lines)(s)?;
Ok((s, steps))
}
fn job_lines(s: Span) -> IResult<entities::Step> {
let (s, name) = match step_name(s) {
Ok((s, name)) => (s, name),
Err(nom::Err::Error(_)) => {
let line = s.location_line();
let column = s.get_column();
return Err(nom::Err::Failure(ParseError::new(
format!("Error: line {}, column {}", line, column),
s,
)));
},
Err(e) => return Err(e),
};
Ok((s, name))
}
Relevant error code in main:
let steps = match input::parsers::job(input::parsers::Span::new(&input)) {
Ok((_, steps)) => steps,
Err(error) => {
eprintln!("{}", error);
std::process::exit(1)
}
};
What do I need to do to get the short error message instead of the extensive one?
EDIT:
I deleted the span property from the custom ParseError and then the Error output becomes:
Parsing Failure: ParseError { message: "Error: line 5, column 1" }
Much better, but question remains: can I get of everything except of the message itself?
As you surmised, error is a nom::Err::Err which is an enum, so you will need to match on that to get at the inner error:
let steps = match input::parsers::job(input::parsers::Span::new(&input)) {
Ok((_, steps)) => steps,
Err(nom::Err::Failure (error)) => {
eprintln!("{}", error.message);
std::process::exit(1)
},
Err(error) => {
eprintln!("Unknown error: {}", error);
std::process::exit(1)
},
};
Related
I am working on a command line program where I need to parse the cli arguments. My problem is that there is an error when I try to parse elements from a vector of Strings
I have a function called ìnto_num_vec() which takes a vector of Strings and I should parse it into a new vector of integers.
Code from lib.rs
pub fn affirm_args(input: Vec<String>) {
if input.len() < 2 {
panic!("To few arguments");
} else {
let numbers = into_num_vec(input);
print_numbers(numbers);
}
}
fn into_num_vec(input: Vec<String>) -> Vec<i32> {
let mut collection: Vec<i32> = Vec::new();
for i in input {
match i.trim().parse() {
Ok(n) => collection.push(n),
Err(_) => panic!("Error parsing")
}
}
collection
}
pub fn print_numbers(input: Vec<i32>) {
for i in input {
println!("{}", i);
}
}
The function is panicking and I'am getting the custom panic msg "Error parsing".
Code in main.rs
use sort_program::*;
use std::env;
fn main() {
let args: Vec<String> = env::args().collect();
affirm_args(args);
}
The first argument to a program traditionally is the executable name. You should skip it:
fn main() {
let args: Vec<String> = env::args().skip(1).collect();
affirm_args(args);
}
I'm writing my first proc macro and despite trying to read through the source for thiserror, structopt and derive_more I can't seem to find exactly what I'm looking for. I want to transform this:
#[derive(Attach)]
#[attach(foo(SomeType, OtherType))]
#[attach(bar(OtherType))]
struct Plugin {}
into this:
impl Attach for Plugin {
fn attach(self, srv: &mut Server) {
let this = Arc::new(self);
srv.foo_order(this.clone(), &[TypeId::of::<SomeType>(), TypeId::of::<OtherType>()]);
srv.bar_order(this, &[TypeId::of::<OtherType>()]);
}
}
I've started writing a proc macro but got bungled up trying to parse the attributes:
extern crate proc_macro;
use proc_macro::{Span, TokenStream};
use quote::quote;
use std::any::TypeId;
use syn::{
parse::ParseStream, parse_macro_input, Attribute, AttributeArgs, DeriveInput, Ident, Result,
};
#[proc_macro_derive(Attach, attributes(attach))]
pub fn register_macro(input: TokenStream) -> TokenStream {
let input = parse_macro_input!(input as DeriveInput);
impl_register(input)
}
fn impl_register(input: DeriveInput) -> TokenStream {
let name = &input.ident;
let attrs = &input.attrs;
for attr in attrs {
if attr.path.is_ident("attach") {
parse_attach_attribute(&attr);
}
}
println!("{:#?}", input);
TokenStream::from(quote! {
impl ::corten::Attach for #name {
fn attach(self, srv: &mut ::corten::Server) {
}
}
})
}
fn parse_attach_attribute(attr: &Attribute) -> Result<Dependency> {
let list: syn::MetaList = attr.parse_args()?;
// println!("{:#?}", list);
let ident = list.path.get_ident().expect("expected identifier");
let method = Ident::new(&format!("{}_order", ident), ident.span());
println!("{:#?}", method);
let dependencies = list
.nested
.into_pairs()
.map(|pair| pair.into_value())
.collect::<Vec<_>>();
// How does one get the identifiers out of a NestedMeta?
println!("{:#?}", dependencies);
// attr.parse_args_with(|input: ParseStream| {
// let ident = input.p
// let method = Ident::new(&format!("{}_order", ident), Span::call_site());
// let dep = Dependency {
// method,
// }
// })
unimplemented!()
}
struct Dependency {
method: Ident,
dependencies: Vec<Ident>,
}
the difficulty I'm having is how to actually get the attributes list into a usable form? As far as I can tell, I need to get the "foo" and "bar" parsed from &[Attribute] so I can construct the method identifier, and also the "SomeType" & "OtherType" identifiers, which I'll all eventually feed into quote!. If I print the TokenStream in the console, all of the information is there:
[
Attribute {
pound_token: Pound,
style: Outer,
bracket_token: Bracket,
path: Path {
leading_colon: None,
segments: [
PathSegment {
ident: Ident {
ident: "attach",
span: #0 bytes(103..109),
},
arguments: None,
},
],
},
tokens: TokenStream [
Group {
delimiter: Parenthesis,
stream: TokenStream [
Ident {
ident: "predecode",
span: #0 bytes(110..119),
},
Group {
delimiter: Parenthesis,
stream: TokenStream [
Ident {
ident: "SomeType",
span: #0 bytes(120..128),
},
Punct {
ch: ',',
spacing: Alone,
span: #0 bytes(128..129),
},
Ident {
ident: "OtherType",
span: #0 bytes(130..139),
},
],
span: #0 bytes(119..140),
},
],
span: #0 bytes(109..141),
},
],
},
Attribute {
pound_token: Pound,
style: Outer,
bracket_token: Bracket,
path: Path {
leading_colon: None,
segments: [
PathSegment {
ident: Ident {
ident: "attach",
span: #0 bytes(145..151),
},
arguments: None,
},
],
},
tokens: TokenStream [
Group {
delimiter: Parenthesis,
stream: TokenStream [
Ident {
ident: "preresolve",
span: #0 bytes(152..162),
},
Group {
delimiter: Parenthesis,
stream: TokenStream [
Ident {
ident: "OtherType",
span: #0 bytes(163..172),
},
],
span: #0 bytes(162..173),
},
],
span: #0 bytes(151..174),
},
],
},
]
But I don't have a way to actually get at it. How do I get to tokens[0].stream.ident?
After much messing around I think I have something that works, although I'm happy to accept other answers that are better as I feel this is a bit messy:
extern crate proc_macro;
use proc_macro2::{Span, TokenStream};
use quote::quote;
use syn::{parse_macro_input, Attribute, DeriveInput, Ident, Result};
#[proc_macro_derive(Attach, attributes(attach))]
pub fn register_macro(input: proc_macro::TokenStream) -> proc_macro::TokenStream {
let input = parse_macro_input!(input as DeriveInput);
proc_macro::TokenStream::from(impl_register(input))
}
fn impl_register(input: DeriveInput) -> TokenStream {
let name = &input.ident;
let attrs = &input.attrs;
// println!("{:#?}", input);
let attrs = attrs
.iter()
.filter(|attr| attr.path.is_ident("attach"))
.map(|attr| parse_attach_attribute(&attr).expect("parse failed"))
.map(|dep| {
let method: Ident = dep.method;
let dependencies = dep.dependencies.iter().map(|ident: &Ident| {
quote! {
std::any::TypeId::of::<#ident>()
}
});
quote! {
srv.#method::<#name, _>(Arc::clone(&this), &[ #(#dependencies),* ]);
}
});
quote! {
impl corten::Attach for #name {
fn attach(self, srv: &mut corten::Server) {
let this = std::sync::Arc::new(self);
#(#attrs)*
}
}
}
}
fn parse_attach_attribute(attr: &Attribute) -> Result<Dependency> {
let list: syn::MetaList = attr.parse_args()?;
// println!("{:#?}", list);
let ident = list.path.get_ident().expect("expected identifier");
let method = Ident::new(&format!("{}_order", ident), Span::call_site());
println!("{:#?}", method);
let dependencies = list
.nested
.into_pairs()
.map(|pair| pair.into_value())
.filter_map(|pair| match pair {
syn::NestedMeta::Meta(meta) => match meta {
syn::Meta::Path(path) => path.get_ident().cloned(),
_ => panic!("only path meta supported"),
},
_ => panic!("lit not supported"),
})
.collect();
println!("{:#?}", dependencies);
Ok(Dependency {
method,
dependencies,
})
}
struct Dependency {
method: Ident,
dependencies: Vec<Ident>,
}
My proto buf format is this:
package main;
message Test {
optional string id = 1;
optional string name = 2;
optional string age = 3;
}
Then I populate the protobuf files from the input in golang using the following code. str is already parsed.
test = &Test{
id: proto.String(str[0]),
name: proto.String(str[1]),
age: proto.String(str[2]),
},
One condition I have to handle is that one or more optional fields in the Test structure could be absent randomly and I do not know that in advance. How do I handle that in golang?
To give more context, the real data can look like these in the file:
id=1, name=peter, age=24
id=2, age=30
name=mary, age=31
id=100
name=bob
age=11
You could use a regular expression to change your input strings into valid JSON, the use the encoding/json package to parse it. This has the advantage of letting the json parser take care of everything for you. Here is the regex for your particular case.
Basically, the regex looks for field=value and replaces with "field" : "value" and wraps it in {} to create valid JSON. The commas are left as is.
https://play.golang.org/p/_EEdTB6sve
package main
import (
"encoding/json"
"errors"
"fmt"
"log"
"regexp"
)
var ins = []string{
`id=1, name=peter, age=24`,
`id=2, age=30`,
`name=mary, age=31`,
`id=100`,
`name=bob`,
`age=11`,
}
var ParseError = errors.New("bad parser input")
var Regex *regexp.Regexp
type Test struct {
ID string
Name string
Age string
}
func (t *Test) String() string {
return fmt.Sprintf("ID: %s, Name: %s, Age: %s", t.ID, t.Name, t.Age)
}
func main() {
var err error
Regex, err = regexp.Compile(`([^,\s]*)=([^,\s]*)`)
if err != nil {
log.Panic(err)
}
for _, v := range ins {
test, err := ParseLine(v)
if err != nil {
log.Panic(err)
}
fmt.Println(test)
}
}
func ParseLine(inp string) (*Test, error) {
JSON := fmt.Sprintf("{%s}", Regex.ReplaceAllString(inp, `"$1" : "$2"`))
test := &Test{}
err := json.Unmarshal([]byte(JSON), test)
if err != nil {
return nil, err
}
return test, nil
}
Here is what I believe to be a minimum working case for what you are after, though I am not familiar enough with protocol buffers to get the strings to print right... or even verify if they are correct. Note that this doesn't run in the playground.
package main
import (
"errors"
"fmt"
"log"
"regexp"
"github.com/golang/protobuf/jsonpb"
_ "github.com/golang/protobuf/proto"
)
var ins = []string{
`id=1, name=peter, age=24`,
`id=2, age=30`,
`name=mary, age=31`,
`id=100`,
`name=bob`,
`age=11`,
}
var ParseError = errors.New("bad parser input")
var Regex *regexp.Regexp
type Test struct {
ID *string `protobuf:"bytes,1,opt,name=id,json=id" json:"id,omitempty"`
Name *string `protobuf:"bytes,2,opt,name=name,json=name" json:"name,omitempty"`
Age *string `protobuf:"bytes,3,opt,name=age,json=age" json:"age,omitempty"`
}
func (t *Test) Reset() {
*t = Test{}
}
func (*Test) ProtoMessage() {}
func (*Test) Descriptor() ([]byte, []int) {return []byte{}, []int{0}}
func (t *Test) String() string {
return fmt.Sprintf("ID: %v, Name: %v, Age: %v", t.ID, t.Name, t.Age)
}
func main() {
var err error
Regex, err = regexp.Compile(`([^,\s]*)=([^,\s]*)`)
if err != nil {
log.Panic(err)
}
for _, v := range ins {
test, err := ParseLine(v)
if err != nil {
fmt.Println(err)
log.Panic(err)
}
fmt.Println(test)
}
}
func ParseLine(inp string) (*Test, error) {
JSON := fmt.Sprintf("{%s}", Regex.ReplaceAllString(inp, `"$1" : "$2"`))
test := &Test{}
err := jsonpb.UnmarshalString(JSON, test)
if err != nil {
return nil, err
}
return test, nil
}
Looks like you can write the parser for each line of your input something like the following.
NOTE: I didn't make the struct with proto values because as an external package, it can't be imported in the playground.
https://play.golang.org/p/hLZvbiMMlZ
package main
import (
"errors"
"fmt"
"strings"
)
var ins = []string{
`id=1, name=peter, age=24`,
`id=2, age=30`,
`name=mary, age=31`,
`id=100`,
`name=bob`,
`age=11`,
}
var ParseError = errors.New("bad parser input")
type Test struct {
ID string
Name string
Age string
}
func (t *Test) String() string {
return fmt.Sprintf("ID: %s, Name: %s, Age: %s", t.ID, t.Name, t.Age)
}
func main() {
for _, v := range ins {
t, err := ParseLine(v)
if err != nil {
fmt.Println(err)
} else {
fmt.Println(t)
}
}
}
func ParseLine(inp string) (*Test, error) {
splt := strings.Split(inp, ",")
test := &Test{}
for _, f := range splt {
fieldVal := strings.Split(strings.TrimSpace(f), "=")
switch strings.TrimSpace(fieldVal[0]) {
case "id":
test.ID = strings.TrimSpace(fieldVal[1])
case "name":
test.Name = strings.TrimSpace(fieldVal[1])
case "age":
test.Age = strings.TrimSpace(fieldVal[1])
default:
return nil, ParseError
}
}
return test, nil
}
I'm starting to use rust to create a little toy command prompt applicationusing the gnu readline library. I started using the c foreign function interface but then came across this in rust 0.8.
The crate extra has a module extra::rl which appears to be a readline with all the tools that are needed. My question is, is this present in rust 0.10 and if so where is it? if not, is it packaged as an external module, and if so where can I find it?
Thanks in advance.
use std::c_str;
#[link(name = "readline")]
extern {
fn readline (p: *std::libc::c_char) -> * std::libc::c_char;
}
fn rust_readline (prompt: &str) -> Option<~str> {
let cprmt = prompt.to_c_str();
cprmt.with_ref(|c_buf| {
unsafe {
let ret = c_str::CString::new (readline (c_buf), true);
ret.as_str().map(|ret| ret.to_owned())
}
})
}
fn eval(input: &str)
{
println! ("{}", input);
}
fn main()
{
loop {
let val = rust_readline (">>> ");
match val {
None => { break }
_ => {
let input = val.unwrap ();
eval(input);
}
}
}
}
This is a sample REPL style interpreter that just prints instead of eval'ing. Based on code from http://redbrain.co.uk/2013/11/09/rust-and-readline-c-ffi/. His example no longer compiles.
This code is now on github here.
Sorry I really should have said, I came up with an answer to my own question, it includes a history for readline. I have put it here in case anyone else wants some of this functionality.
//! Readline FFI
//extern crate libc;
use std::libc;
use libc::c_char;
use std::c_str;
#[link(name = "readline")]
extern {
fn readline(p: *c_char) -> * c_char;
fn add_history(l: *c_char);
}
pub fn rust_readline(prompt: &str) -> Option<~str> {
let c_prompt = prompt.to_c_str();
c_prompt.with_ref(|c_buf| {
unsafe {
let ret = c_str::CString::new(readline(c_buf), true);
ret.as_str().map(|ret| ret.to_owned())
}
})
}
pub fn rust_add_history(line: &str) -> int {
// if line.len() == 0 { return Err("Empty string!") }
let c_line = line.to_c_str();
c_line.with_ref(|my_c_line| {
unsafe {
// println!("{}", my_c_line)
add_history(my_c_line);
}
});
0
}
fn main() {
loop {
let val = rust_readline("mikes_lisp> ");
match val {
None => { break }
_ => {
let input = val.unwrap();
rust_add_history(input);
println!("you said: '{}'", input);
}
}
}
}
I run the code below and I got an error without any stack trace.
My code:
typedef Check<T>(T value, [onError(T value)]);
main () {
List<Check> checks = [
(str) => str != null,
(str) => !str.isEmpty
];
Check<String> doCheck = (String value, [onError(String)]) {
checks.forEach((Check check) {
if (?onError) {
check(value, onError);
} else {
check(value);
}
});
};
doCheck("10");
}
And, the error I got.
file:///..()../sample.dart': Error: line 11 pos 12: formal parameter name expected
if (?onError) {
I want to get onError as an optional parameter in doCheck function, and pass this parameter to other functions in checks.
I confirmed to forward an optional parameter to 'one' function...
Is this one of restrictions to optional parameters?
I would say it is a bug (see issue 8007). To work around it, you have to use a temporary variable :
typedef Check<T>(T value, [onError(T value)]);
main () {
List<Check> checks = [
(str) => str != null,
(str) => !str.isEmpty
];
Check<String> doCheck = (String value, [onError(String)]) {
final isOnErrorPresent = ?onError;
checks.forEach((Check check) {
if (isOnErrorPresent) {
check(value, onError);
} else {
check(value);
}
});
};
doCheck("10");
}