How to match any of a Vector of strings with nom? - parsing

Question

I'm trying to create a parser with nom that will parse some text that could be one of many options. Nom has alt! for when the values are known at compile-time, but my values won't be.
This has been my attempt to create my own parser that can take a Vec<String> to match against, and I'm running into a couple issues.
#[macro_use]
extern crate nom;
use nom::IResult;
fn alternative_wrapper<'a>(input: &'a [u8], alternatives: Vec<String>) -> IResult<&'a [u8], &'a [u8]> {
for alternative in alternatives {
// tag!("alternative");
println!("{}", alternative);
}
return IResult::Done(input, "test".as_bytes());
}
#[test]
fn test_date() {
let input = "May";
named!(alternative, call!(alternative_wrapper));
let months = vec!(
"January",
"February",
"March",
"April",
"May",
"June",
"July",
"August",
"September",
"October",
"November",
"December"
).iter().map(|s| s.to_string()).collect();
println!("{:?}", alternative("May".as_bytes(), months));
}
I'm aware that my alternative_wrapper function doesn't actually do anything useful, but that's not the problem. This is what Rust complains about for this snippet:
error[E0061]: this function takes 1 parameter but 2 parameters were supplied
--> src/parser.rs:32:34
|
17 | named!(alternative, call!(alternative_wrapper));
| ------------------------------------------------ defined here
...
32 | println!("{:?}", alternative("May".as_bytes(), months));
| ^^^^^^^^^^^^^^^^^^^^^^^^ expected 1 parameter
|
= note: this error originates in a macro outside of the current crate
error[E0061]: this function takes 2 parameters but 1 parameter was supplied
--> src/parser.rs:17:5
|
6 | / fn alternative_wrapper<'a>(input: &'a [u8], alternatives: Vec<String>) -> IResult<&'a [u8], &'a
[u8]> {
7 | | for alternative in alternatives {
8 | | // tag!("alternative");
9 | | println!("{}", alternative);
10 | | }
11 | | return IResult::Done(input, "test".as_bytes());
12 | | }
| |_- defined here
...
17 | named!(alternative, call!(alternative_wrapper));
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ expected 2 parameters
|
= note: this error originates in a macro outside of the current crate
How I can create a parser out of my function? And how I can use existing parsers such as tag! from within alternative_wrapper?

Answer 1

Starting with the errors, the first error is due to named! only taking a single argument, namely the input string. named! will declare a function for you, in this case with the signature fn(&[u8]) -> IResult<&[u8],&[u8]>. There's no magic regarding any other arguments, so trying to pass your months vector as a second argument will not work. There's a variant of named! called named_args! that can be used to declare functions with more arguments than just the input that should sort that out.
The second error is similar but reversed. You're calling the alternative_wrapper with only the input and no vector via call!. The call! macro can actually pass arguments, but you must do it explicitly, i.e call!(myparser, monts).
With the reasons to the errors sorted out, you're asking about how to create a parser. Well, actually, alternative_wrapper is already a nom parser by signature, but since you didn't declare it via a nom macro, none of the magic input passing happens, which is why tag! doesn't work in the function body when you've tried.
In order to use other combinators in a function that you have declared yourself, you have to pass the input manually to the outermost macro. In this case it's only tag!, but if you were to use, say, do_parse! and then multiple macros within that, you'd only need to pass the input to do_parse!. I'll provide a working version with a few additional tweaks here:
#[macro_use]
extern crate nom;
use std::str;
use nom::IResult;
fn alternative<'a>(input: &'a [u8], alternatives: &Vec<String>) -> IResult<&'a [u8], &'a [u8]> {
for alternative in alternatives {
match tag!(input, alternative.as_bytes()) {
done#IResult::Done(..) => return done,
_ => () // continue
}
}
IResult::Error(nom::ErrorKind::Tag) // nothing found.
}
fn main() {
let months: Vec<String> = vec![
"January", "February", "March", "April", "May", "June", "July",
"August", "September", "October", "November", "December"
].into_iter().map(String::from).collect();
fn print_res(r: IResult<&[u8],&[u8]>) {
println!("{:?}", r);
println!("{:?}\n", str::from_utf8(r.unwrap().1).unwrap());
}
print_res(alternative(b"May", &months));
print_res(alternative(b"August", &months));
print_res(alternative(b"NoGood", &months));
}
You can check it out in the rust playground.

Answer 2

I'm not really familiar with nom, and still learning Rust, but I have used parser combinators in the past.
Caveats aside, it looks like the named! macro generates a function that only takes one parameter, the string to parse.
To satisfy nom's expectations, I think I'd look at writing alternative_wrapper as a function that returns a function instead. The test would end up looking like this:
#[test]
fn test_date() {
let months = vec!(
"January",
"February",
"March",
"April",
"May",
"June",
"July",
"August",
"September",
"October",
"November",
"December"
).iter().map(|s| s.to_string()).collect();
let parser = generate_alternative_parser(months);
named!(alternative, call!(parser));
println!("{:?}", alternative("May".as_bytes()));
}
It looks like you'd need to construct an alt! expression from tag!s, but it's not immediately obvious to me from the docs how you'd do that.
Where does your list of options ultimately come from?
Depending on exactly what you are trying to accomplish, there might be some other ways to accomplish what you're trying to do as well. For example, you might be able to parse any word and then validate it against one of your options afterwards.

Answer 3

With Nom 4, here is a fully generic input that works no matter what your parser operates on:
/// Dynamic version of `alt` that takes a slice of strings
fn alternative<T>(input: T, alternatives: &[&'static str]) -> IResult<T, T>
where
T: InputTake,
T: Compare<&'static str>,
T: InputLength,
T: AtEof,
T: Clone,
{
let mut last_err = None;
for alternative in alternatives {
let inp = input.clone();
match tag!(inp, &**alternative) {
done # Ok(..) => return done,
err # Err(..) => last_err = Some(err), // continue
}
}
last_err.unwrap()
}
/// Usage
named!(test<Span, Span>,
call!(alternative, &["a", "b", "c"])
);