I am implementing client agent for Robocup Soccer simulator in Erlang. Simulator sends sensory information to client in form of S-expressions. Like this
(see 15 ((f c) 2 0 0 0) ((f r t) 64.1 -32) ((f r b) 64.1 32) ((f g r b) 55.1 7)
((g r) 54.6 0) ((b) 2 0 -0 0) ((l r) 54.6 90))
(see 16 ((f r t) 72.2 -44) ((f r b) 54.1 20) ((f g r b) 52.5 -10) ((g r) 54.1 -17)
((l r) 51.4 -89))
Simulator sends such type of sensor informatio in each cycle(100-200 msec).
The main format of the information is:
(see Time ObjInfo ObjInfo . . . )
The ObjInfos are of the format below:
(ObjName Distance Direction
[DistChange DirChange [BodyFac- ingDir
HeadFacingDir]])
where the objects are like:
(b) Ball, (g r) Right goal, (f ...) represents various flags.
What I want is to parse this information and store/update in some database(record) to use for analysis.
The main difficulty I am facing is to Parse this information.
Please suggest me some way of doing this? (does Erlang contain any library for such work)
Yecc and Leex are your friends: http://erlang.org/doc/apps/parsetools/index.html
Leex is a lexical analyzer generator for Erlang which will tokenize your data. Yecc is LALR-1 parser generator that can parse your tokens into meaningful structures.
There's a good blog post by Relops, Leex And Yecc, detailing some of the basics.
If you load LFE (Lisp Flavoured Erlang) it contains a lisp scanner and parser. The modules you need are lfe_scan, lfe_parse and lfe_io which wraps the other two. The scanner is written using leex (source is lfe_scan.xrl) while the parser is hand written as there are some features of how yecc works which didn't quite fit.
The correct approach would be to just write a small LISP reader.
The quick and (very) dirty way (for initial testing ONLY): Substitute whitespace with a comma, "(" with "{" and ")" with "}". Then you have an erlang literal.
Have a look at erl_scan and erl_parse.
Related
I wanted to write a parser based on John Hughes' paper Generalizing Monads to Arrows. When reading through and trying to reimplement his code I realized there were some things that didn't quite make sense. In one section he lays out a parser implementation based on Swierstra and Duponchel's paper Deterministic, error-correcting combinator parsers using Arrows. The parser type he describes looks like this:
data StaticParser ch = SP Bool [ch]
data DynamicParser ch a b = DP (a, [ch]) -> (b, [ch])
data Parser ch a b = P (StaticParser ch) (DynamicParser ch a b)
with the composition operator looking something like this:
(.) :: Parser ch b c -> Parser ch a b -> Parser ch a c
P (SP e2 st2) (DP f2) . P (SP e1 st1) (DP f1) =
P (SP (e1 && e2) (st1 `union` if e1 then st2 else []))
(DP $ f2 . f1)
The issue is that the composition of parsers q . p 'forgets' q's starting symbols. One possible interpretation I thought of is that Hughes' expects all our DynamicParsers to be total such that a symbol parser's type signature would be symbol :: ch -> Parser ch a (Maybe ch) instead of symbol :: ch -> Parser ch a ch. This still seems awkward though since we have to duplicate information putting starting symbol information in both the StaticParser and DynamicParser. Another issue is that almost all parsers will have the potential to throw which means we will have to spend a lot of time inside Maybe or Either creating what is essentially the "monads do not compose problem." This could be remedied by rewriting DynamicParser itself to handle failure or as an Arrow transformer, but this is straying quite a bit from the paper. None of these issues are addressed in the paper, and the Parser is presented as if it obviously works, so I feel like I must me missing something basic. If someone can catch what I missed that would be super helpful.
I think the deterministic parsers described by Swierstra and Duponcheel are a bit different from traditional parsers: they do not handle failure at all, only choice.
See also the invokeDet function in the S&D paper:
invokeDet :: Symbol s => DetPar s a -> Input s -> a
invokeDet (_, p) inp = case p inp [] of (a, _) -> a
This function clearly assumes it will always be able to find a valid parse.
With the arrow version of the parsers described by Hughes you can write a examples like this:
main = do
let p = symbol 'a' >>> (symbol 'b' <+> symbol 'c')
print $ invokeDet p "ab"
print $ invokeDet p "ac"
Which will print the expected:
'b'
'c'
However, if you write a "failing" parse:
main = do
let p = symbol 'a' >>> (symbol 'b' <+> symbol 'c')
print $ invokeDet p "ad"
It will still print:
'c'
To make this behavior a bit more sensible, Swierstra and Duponcheel also introduce error-correction. The output 'c' is expected if we assume the erroneous character d has been corrected to be a c in the input. This requires an extra mechanism which presumably was too complicated to include in Hughes' paper.
I have uploaded the implementation I used to get these results here: https://gist.github.com/noughtmare/eced4441332784cc8212e9c0adb68b35
For more information about a more practical parser in the same style (but no longer deterministic and no longer limited to LL(1)) I really like the "Combinator Parsing: A Short Tutorial" by Swierstra. An interesting excerpt from section 9.3:
A subtle point here is the question how to deal with monadic parsers. As we described in [13] the static analysis does not go well with monadic computations, since in that case we dynamically build new parses based on the input produced thus far: the whole idea of a static analysis is that it is static. This observation has lead John Hughes to propose arrows for dealing with such situations [7]. It is only recently that we realised that, although our arguments still hold in general, they do not apply to the case of the LL(1) analysis. If we want to compute the symbols which can be recognised as the first symbol by a parser of the form p >>= q then we are only interested in the starting symbols of the right hand side if the left hand side can recognise the empty string; the good news is that in that case we statically know what value will be returned as a witness, and can pass this value on to q, and analyse the result of this call statically too. Unfortunately we will have to take special precautions in case the left hand side operator contains a call to pErrors in one of the empty derivations, since then it is no longer true that the witness of this alternative can be determined statically.
The full parser implementation by Swierstra can be found in the uu-parsinglib package, although I do not know how many of the extensions are implemented there.
We try to change the way maxima translates multiplication when converting to tex.
By default maxima gives a space: \,
We changed this to our own latex macro that looks like a space, but in that way we conserve the sementical meaning which makes it easier to convert the latex back to maxima.
:lisp (setf (get 'mtimes 'texsym) '("\\invisibletimes "));
However, we have one problem, and that is when we put simplification on. We use this for generating steps in the explanation of a solution. For example:
tex1(block([simp: false], 2*3));
Of course when multiplying numbers we can want an explicit multiplication (\cdot).
So we would like it that if both arguments of the multiplication are numbers, that we then have a \cdot when translating to tex.
Is that possible?
Yes, if there is a function named by the TEX property, that function is called to process an expression. The function named by TEX takes 3 arguments, namely an expression with the same operator to which the TEX property is attached, stuff to the left, and stuff to the right, and the TEX function returns a list of strings which are the bits of TeX which should be output.
You can say :lisp (trace tex-mtimes) to see how that works. You can see the functions attached to MTIMES or other operators by saying :lisp (symbol-plist 'mtimes) or in general :lisp (symbol-plist 'mfoo) for another MFOO operator.
So if you replace TEX-MTIMES (by :lisp (setf (get 'mtimes 'tex) 'my-tex-mtimes)) by some other function, then you can control the output to a greater extent. Here is an outline of a suitable function for your purpose:
(defun my-tex-mtimes (e l r)
(if $simp
(tex-nary e l r) ;; punt to default handler
(tex-mtimes-special-case e l r)))
You can make TEX-MTIMES-SPECIAL-CASE as complicated as you want. I assume that you can carry out the Lisp programming for that. The simplest thing to try, perhaps a point of departure for further efforts, is to just temporarily replace TEXSYM with \cdot. Something like:
(defun tex-mtimes-special-case (e l r)
(let ((prev-texsym (get 'mtimes 'texsym)))
(prog2 (setf (get 'mtimes 'texsym) (list "\\cdot "))
(tex-nary e l r)
(setf (get 'mtimes 'texsym) prev-texsym))))
Is it possible to use one of the parsing libraries (e.g. Parsec) for parsing something different than a String? And how would I do this?
For the sake of simplicity, let's assume the input is a list of ints [Int]. The task could be
drop leading zeros
parse the rest into the pattern (S+L+)*, where S is a number less than 10, and L is a number larger or equal to ten.
return a list of tuples (Int,Int), where fst is the product of the S and snd is the product of the L integers
It would be great if someone could show how to write such a parser (or something similar).
Yes, as user5402 points out, Parsec can parse any instance of Stream, including arbitrary lists. As there are no predefined token parsers (as there are for text) you have to roll your own, (myToken below) using e.g. tokenPrim
The only thing I find a bit awkward is the handling of "source positions". SourcePos is an abstract type (rather than a type class) and forces me to use its "filename/line/column" format, which feels a bit unnatural here.
Anyway, here is the code (without the skipping of leading zeroes, for brevity)
import Text.Parsec
myToken :: (Show a) => (a -> Bool) -> Parsec [a] () a
myToken test = tokenPrim show incPos $ justIf test where
incPos pos _ _ = incSourceColumn pos 1
justIf test x = if (test x) then Just x else Nothing
small = myToken (< 10)
large = myToken (>= 10)
smallLargePattern = do
smallints <- many1 small
largeints <- many1 large
let prod = foldl1 (*)
return (prod smallints, prod largeints)
myIntListParser :: Parsec [Int] () [(Int,Int)]
myIntListParser = many smallLargePattern
testMe :: [Int] -> [(Int, Int)]
testMe xs = case parse myIntListParser "your list" xs of
Left err -> error $ show err
Right result -> result
Trying it all out:
*Main> testMe [1,2,55,33,3,5,99]
[(2,1815),(15,99)]
*Main> testMe [1,2,55,33,3,5,99,1]
*** Exception: "your list" (line 1, column 9):
unexpected end of input
Note the awkward line/column format in the error message
Of course one could write a function sanitiseSourcePos :: SourcePos -> MyListPosition
There is very likely a way to get Parsec to use [a] as the stream type, but the idea behind parser combinators is actually very simple, and it's not very difficult to roll your own library.
A very accessible resource I would recommend is Monadic Parsing in Haskell by Graham Hutton and Erik Meijer.
Indeed, right now Erik Meijer is teaching an intro Haskell/functional programming course on edx.org (link) and Lecture 7 is all about functional parsers. As he states in the intro to the lecture:
"... No one can follow the path towards mastering functional programming without writing their own parser combinator library. We start by explaining what parsers are and how they can naturally be viewed as side-effecting functions. Next we define a number of basic parsers and higher-order functions for combining parsers. ..."
Edit: I discovered a partial answer to my own question in the process of writing this, but I think it can easily be improved upon so I will post it anyway. Maybe there's a better solution out there?
I am looking for an easy way to define recursive functions in a let form without resorting to letfn. This is probably an unreasonable request, but the reason I am looking for this technique is because I have a mix of data and recursive functions that depend on each other in a way requires a lot of nested let and letfn statements.
I wanted to write the recursive functions that generate lazy sequences like this (using the Fibonacci sequence as an example):
(let [fibs (lazy-cat [0 1] (map + fibs (rest fibs)))]
(take 10 fibs))
But it seems in clojure that fibs cannot use it's own symbol during binding. The obvious way around it is using letfn
(letfn [(fibo [] (lazy-cat [0 1] (map + (fibo) (rest (fibo)))))]
(take 10 (fibo)))
But as I said earlier this leads to a lot of cumbersome nesting and alternating let and letfn.
To do this without letfn and using just let, I started by writing something that uses what I think is the U-combinator (just heard of the concept today):
(let [fibs (fn [fi] (lazy-cat [0 1] (map + (fi fi) (rest (fi fi)))))]
(take 10 (fibs fibs)))
But how to get rid of the redundance of (fi fi)?
It was at this point when I discovered the answer to my own question after an hour of struggling and incrementally adding bits to the combinator Q.
(let [Q (fn [r] ((fn [f] (f f)) (fn [y] (r (fn [] (y y))))))
fibs (Q (fn [fi] (lazy-cat [0 1] (map + (fi) (rest (fi))))))]
(take 10 fibs))
What is this Q combinator called that I am using to define a recursive sequence? It looks like the Y combinator with no arguments x. Is it the same?
(defn Y [r]
((fn [f] (f f))
(fn [y] (r (fn [x] ((y y) x))))))
Is there another function in clojure.core or clojure.contrib that provides the functionality of Y or Q? I can't imagine what I just did was idiomatic...
letrec
I have written a letrec macro for Clojure recently, here's a Gist of it. It acts like Scheme's letrec (if you happen to know that), meaning that it's a cross between let and letfn: you can bind a set of names to mutually recursive values, without the need for those values to be functions (lazy sequences are ok too), as long as it is possible to evaluate the head of each item without referring to the others (that's Haskell -- or perhaps type-theoretic -- parlance; "head" here might stand e.g. for the lazy sequence object itself, with -- crucially! -- no forcing involved).
You can use it to write things like
(letrec [fibs (lazy-cat [0 1] (map + fibs (rest fibs)))]
fibs)
which is normally only possible at top level. See the Gist for more examples.
As pointed out in the question text, the above could be replaced with
(letfn [(fibs [] (lazy-cat [0 1] (map + (fibs) (rest (fibs)))))]
(fibs))
for the same result in exponential time; the letrec version has linear complexity (as does a top-level (def fibs (lazy-cat [0 1] (map + fibs (rest fibs)))) form).
iterate
Self-recursive seqs can often be constructed with iterate -- namely when a fixed range of look-behind suffices to compute any given element. See clojure.contrib.lazy-seqs for an example of how to compute fibs with iterate.
clojure.contrib.seq
c.c.seq provides an interesting function called rec-seq, enabling things like
(take 10 (cseq/rec-seq fibs (map + fibs (rest fibs))))
It has the limitation of only allowing one to construct a single self-recursive sequence, but it might be possible to lift from it's source some implementation ideas enabling more diverse scenarios. If a single self-recursive sequence not defined at top level is what you're after, this has to be the idiomatic solution.
combinators
As for combinators such as those displayed in the question text, it is important to note that they are hampered by the lack of TCO (tail call optimisation) on the JVM (and thus in Clojure, which elects to use the JVM's calling conventions directly for top performance).
top level
There's also the option of putting the mutually recursive "things" at top level, possibly in their own namespace. This doesn't work so great if those "things" need to be parameterised somehow, but namespaces can be created dynamically if need be (see clojure.contrib.with-ns for implementation ideas).
final comments
I'll readily admit that the letrec thing is far from idiomatic Clojure and I'd avoid using it in production code if anything else would do (and since there's always the top level option...). However, it is (IMO!) nice to play with and it appears to work well enough. I'm personally interested in finding out how much can be accomplished without letrec and to what degree a letrec macro makes things easier / cleaner... I haven't formed an opinion on that yet. So, here it is. Once again, for the single self-recursive seq case, iterate or contrib might be the best way to go.
fn takes an optional name argument with that name bound to the function in its body. Using this feature, you could write fibs as:
(def fibs ((fn generator [a b] (lazy-seq (cons a (generator b (+ a b))))) 0 1))
What is "monadic reflection"?
How can I use it in F#-program?
Is the meaning of term "reflection" there same as .NET-reflection?
Monadic reflection is essentially a grammar for describing layered monads or monad layering. In Haskell describing also means constructing monads. This is a higher level system so the code looks like functional but the result is monad composition - meaning that without actual monads (which are non-functional) there's nothing real / runnable at the end of the day. Filinski did it originally to try to bring a kind of monad emulation to Scheme but much more to explore theoretical aspects of monads.
Correction from the comment - F# has a Monad equivalent named "Computation Expressions"
Filinski's paper at POPL 2010 - no code but a lot of theory, and of course his original paper from 1994 - Representing Monads. Plus one that has some code: Monad Transformers and Modular Interpreters (1995)
Oh and for people who like code - Filinski's code is on-line. I'll list just one - go one step up and see another 7 and readme. Also just a bit of F# code which claims to be inspired by Filinski
I read through the first Google hit, some slides:
http://www.cs.ioc.ee/mpc-amast06/msfp/filinski-slides.pdf
From this, it looks like
This is not the same as .NET reflection. The name seems to refer to turning data into code (and vice-versa, with reification).
The code uses standard pure-functional operations, so implementation should be easy in F#. (once you understand it)
I have no idea if this would be useful for implementing an immutable cache for a recursive function. It look like you can define mutable operations and convert them to equivalent immutable operations automatically? I don't really understand the slides.
Oleg Kiselyov also has an article, but I didn't even try to read it. There's also a paper from Jonathan Sobel (et al). Hit number 5 is this question, so I stopped looking after that.
As previous answers links describes, Monadic reflection is a concept to bridge call/cc style and Church style programming. To describe these two concepts some more:
F# Computation expressions (=monads) are created with custom Builder type.
Don Syme has a good blog post about this. If I write code to use a builder and use syntax like:
attempt { let! n1 = f inp1
let! n2 = failIfBig inp2
let sum = n1 + n2
return sum }
the syntax is translated to call/cc "call-with-current-continuation" style program:
attempt.Delay(fun () ->
attempt.Bind(f inp1,(fun n1 ->
attempt.Bind(f inp2,(fun n2 ->
attempt.Let(n1 + n2,(fun sum ->
attempt.Return(sum))))))))
The last parameter is the next-command-to-be-executed until the end.
(Scheme-style programming.)
F# is based on OCaml.
F# has partial function application, but it also is strongly typed and has value restriction.
But OCaml don't have value restriction.
OCaml can be used in Church kind of programming, where combinator-functions are used to construct any other functions (or programs):
// S K I combinators:
let I x = x
let K x y = x
let S x y z = x z (y z)
//examples:
let seven = S (K) (K) 7
let doubleI = I I //Won't work in F#
// y-combinator to make recursion
let Y = S (K (S I I)) (S (S (K S) K) (K (S I I)))
Church numerals is a way to represent numbers with pure functions.
let zero f x = x
//same as: let zero = fun f -> fun x -> x
let succ n f x = f (n f x)
let one = succ zero
let two = succ (succ zero)
let add n1 n2 f x = n1 f (n2 f x)
let multiply n1 n2 f = n2(n1(f))
let exp n1 n2 = n2(n1)
Here, zero is a function that takes two functions as parameters: f is applied zero times so this represent the number zero, and x is used to function combination in other calculations (like add). succ function is like plusOne so one = zero |> plusOne.
To execute the functions, the last function will call the other functions with last parameter (x) as null.
(Haskell-style programming.)
In F# value restriction makes this hard. Church numerals can be made with C# 4.0 dynamic keyword (which uses .NET reflection inside). I think there are workarounds to do that also in F#.