I've found several questions about how to read from STDIN in Erlang, but each of them seems focused on reading one piece of data, or one line at at time.
If I'm piping data to my Erlang program, how can I read all of the data that's been piped to STDIN and then do something with it once the input buffer is flushed? My use case is parsing JSON.
This snippet seems to do the trick:
read_all_stdin() ->
read_all_stdin([]).
read_all_stdin(Acc) ->
case io:get_line("") of
eof -> lists:reverse(Acc);
Line -> read_all_stdin([Line|Acc])
end.
Related
I am interested in learning how to send data efficiently between Haskell programs using standard input and output. Suppose I want to pipe two programs together: "P1" outputs the number 5 to stdout, and "P2" takes an integer from stdin, adds 1, and outputs it to stdout again. Right now, the best way I know to do this involves outputting the data as text from P1, parsing that text back to an integer in P2, and proceeding from there. For example:
P1.hs:
module Main where
main = do
print 5
P2.hs:
module Main where
main = fmap manipulateData getLine >>= print
where
manipulateData = (+ 1) . (read :: String -> Int)
Output:
$ (stack exec p1) | (stack exec p2)
6
I'd like to use standard i/o to send an integer without treating it as text, if possible. I'm assuming this still requires some sort of parsing to work, but I'm hoping it's possible to parse the data as binary and get a faster program.
Does Haskell have any way to make this straightforward? Since I am going from one fundamental Haskell datatype (Int) to the same type again with a pass through standard i/o in the middle, I'm wondering if there is an easy solution that doesn't require writing a custom binary parser (which I don't know how to do). Can anyone provide such a method?
Here is the code that I ended up with:
module Main where
import qualified Data.ByteString.Lazy as BS
import qualified Data.Binary as B
main :: IO ()
main = do
dat <- BS.getContents
print $ (B.decode dat :: Int) + 1
The other program uses similar imports and outputs 5 with the following line:
BS.putStr $ B.encode (5 :: Int)
The resulting programs can be piped together, and the resulting program behaves as required.
Is there a way to make erlang print the full string even if one has used ~P in a io:format function?
Im having some troubles with EDoc and it keeps wrapping the error messages to ....
Is there any flags or other way to force erlang to print the entire string?
The only method I have found is to use io_lib:print(Term, Column, LineLength, Depth). That function allows you to specify the starting column, the line length to control wrapping, etc. It returns a string which you can then print using io:format with ~s format.
This question already has answers here:
A Haskell function of type: IO String-> String
(4 answers)
Closed 7 years ago.
Rather simple problem, but one that I am having trouble to overcome. None of the examples I find online works, not sure if its because they are outdated and something in IO were changed in last 2-3 years, or I a missing something obvious.
I know that reading file with readFile returns IO String and there is no easy way to get rid of it, but supposedly easy way to read file into normal String is s<- readFile "filename" which works in command line, but I just cant make it work in a function.
getString ::IO String
getString = readFile "Input.txt"
Is rather simple, but returns IO String instead of String, and I am having trouble making s<- readFile "filename" work.
All I really want is simple function that returns file in a String and then I can deal with data in that string.
Ps I would love to see simple way to read file line by line function as well, as all examples seems to be incredibly complicated for what they are supposed to do and how easy is to perform those actions in any imperative programming language.
There isn't a function that turns an IO String into a String. You can use do notation like so*:
main = do
s <- readFile "somefile.txt"
doSomethingWith s
doSomethingWith :: String -> IO ()
doSomethingWith str = putStrLn str
but you will only have access to s as a String inside the do block, or in functions that are applied to s, like doSomethingWith.
* You can also use the equivalent desugared version, main = readFile "someFile.txt" >>= \s -> doSomethingWith s.
when using io:getline("prompt") in the erlang shell , the function returns immediately with a return value of "\n"
io:get_line("prompt").
prompt
"\n"
but as suggested in another thread doing the following reads from standard_io correctly.
spawn(fun() -> timer:sleep(100),io:get_line("prompt") end).
waits for user input and reads from standard io (shell). It was mentioned that it was a race condition . can anyone tell me why is it so and how is it possible to read a value from the erlang shell ?
io:get_line/1 and io:get_line/2 returns data with \n every time.
get_line(Prompt) -> Data | server_no_data()
Where:
Data
The characters in the line terminated by a LF (or end of file). If the
IO device supports Unicode, the data may represent codepoints larger
than 255 (the latin1 range). If the I/O server is set to deliver
binaries, they will be encoded in UTF-8 (regardless of if the IO
device actually supports Unicode or not).
In first case you got \n, and try to get result of io:get_line in second case:
spawn(fun() ->
timer:sleep(100),
Result = io:get_line("prompt"),
io:format("Result: ~p~n", [Result])
end).
Let's break it down...
Why io:get_line/1 returns a \n?
io:get_line/1 returns a "line" and a \n ("end of a line" or "new line") constitutes a line together with the string you entered.
> io:get_line("Prompt:").
Prompt:TheStringIEntered
"TheStringIEntered\n"
How to read a value from the Erlang shell?
> Data = string:strip(io:get_line("Prompt:"), right, $\n).
Prompt:TheStringIEntered
"TheStringIEntered"
> Data.
"TheStringIEntered"
Note that the value (Data) here is always of string type. You can convert it into other types, but you always start with a string.
Why does spawn(fun() -> timer:sleep(100),io:get_line("prompt") end). behave differently?
Because spawn spawns a new process that temporarily takes over the shell. Once this process gets TheStringIEntered, it also reaches the end of its life. So it dies without having its return value (TheStringIEntered) printed to the shell.
I'm trying to run an external program in SBCL and capture its output.
The output is binary data (a png image), while SBCL insists on interpreting it as strings.
I tried a number of ways, like
(trivial-shell:shell-command "/path/to/png-generator" :input "some input")
(with-input-from-string (input "some input")
(with-output-to-string (output)
(run-program "/path/to/png-generator" () :input input :output output))
(with-input-from-string (input "some input")
(flexi-streams:with-output-to-sequence (output)
(run-program "/path/to/png-generator" () :input input :output output))
But I get errors like
Illegal :UTF-8 character starting at byte position 0.
It seems to me that SBCL is trying to interpret the binary data as a text and decode it. How do I change this behaviour ? I'm interested only in obtaining a vector of octets.
Edit: Since it is not clear from the text above, I'd like to add that at least in the case of flexi-stream, the element-type of the stream is a flexi-streams:octect (which is a (unsigned-byte 8)).
I would expect at least in this case run-program to read the raw bytes without many issues. Instead I get a message like Don't know how to copy to stream of element-type (UNSIGNED-BYTE 8)
Edit: I got angry at not being able to do this very simple task and solved the problem.
Functionally, the ability to send a stream of type UNSIGNED-BYTE into run-program and have it work correctly is severely limited, for reasons I don't understand. I tried gray streams, flexi-streams, fd streams, and a few other mechanisms, like you.
However, perusing run-program's source (for the fifth or sixth time), I noticed that there's an option :STREAM you can pass to output. Given that, I wondered if read-byte would work... and it did. For more performant work, one could determine how to get the length of a non-file stream and run READ-SEQUENCE on it.
(let*
;; Get random bytes
((proc-var (sb-ext:run-program "head" '("-c" "10" "/dev/urandom")
:search t
;; let SBCL figure out the storage type. This is what solved the problem.
:output :stream))
;; Obtain the streams from the process object.
(output (process-output proc-var))
(err (process-error proc-var)))
(values
;;return both stdout and stderr, just for polish.
;; do a byte read and turn it into a vector.
(concatenate 'vector
;; A byte with value 0 is *not* value nil. Yay for Lisp!
(loop for byte = (read-byte output nil)
while byte
collect byte))
;; repeat for stderr
(concatenate 'vector
(loop for byte = (read-byte err nil)
while byte
collect byte))))
If you're willing to use some external libraries, this can be done with babel-streams. This is a function I use to safely get content from a program. I use :latin-1 because it maps the first 256 bytes just to the characters. You could remove the octets-to-string and have the vector.
If you wanted stderr as well, you could use nested 'with-output-to-sequence' to get both.
(defun safe-shell (command &rest args)
(octets-to-string
(with-output-to-sequence (stream :external-format :latin-1)
(let ((proc (sb-ext:run-program command args :search t :wait t :output stream)))
(case (sb-ext:process-status proc)
(:exited (unless (zerop (sb-ext:process-exit-code proc))
(error "Error in command")))
(t (error "Unable to terminate process")))))
:encoding :latin-1))
Paul Nathan already gave a pretty complete answer as to how to read I/O from a program as binary, so I'll just add why your code didn't work: because you explicitely asked SBCL to interpret the I/O as a string of UTF-8 characters, using with-{in,out}put-to-string.
Also, I'd like to point that you don't need to go as far as run-program's source code to get to the solution. It's clearly documented in SBCL's manual.