I'm totally new to Scheme, functional programming, and specifically streams. What are the first five integers in the following stream?
(define (mystery x y z)
(cons x
(lambda ()
(mystery y z (+ x y z)))))
(mystery 1 2 3)
How does this work, and how can I use it in Racket?
We can examine the contents of an infinite stream by implementing a procedure that consumes a given number of elements and returns them in a list, for example:
(define (print strm n)
(if (zero? n)
'()
(cons (car strm)
(print ((cdr strm)) (sub1 n)))))
If we apply it to your stream, here's what we obtain:
(print (mystery 1 2 3) 5)
=> '(1 2 3 6 11)
Although in this case, it'll be more useful if we try to understand what's happening under the hood. In the end, this is just a series of procedure invocations, if we take note of the parameters that get passed at each call it's easy to find the answer. Take a look at the first column in the following table, and remember that the stream is being built by consing all the x's:
x y z
--------
1 2 3
2 3 6
3 6 11
6 11 20
11 20 37
This returns a list consisting of a number and a function. The function is - after the first call - the same as you'd get by calling (mystery 2 3 6)
How does it work? cons just makes a list of its two arguments, which in this case are a value and the result of evaluating a lambda function, which is itself a function
Related
(define ones (cons-stream 1 ones))
(stream-cdr ones)
returns an infinite sequence of evaluated 1s - i.e. I get
;Value: #0={1 1 1 1 1 1 1 and so on... - not the symbolic {1 1 ...} I would expect ...
On the other end if I define ints and cdr into it,
(define ints (cons-stream 1 (stream-map + ones ints)))
(stream-cdr ints)
I obtain the expected {1 2 ...}
Can anyone explain me why? I expect the stream-map definition to be not too far from
(define (mystream-map proc . argstreams)
(if (stream-null? (car argstreams))
the-empty-stream
(cons-stream
(apply proc (map stream-car argstreams))
(apply mystream-map (cons proc (map stream-cdr argstreams))))))
which I defined in ex 3.50 and returns the same result ... but this seems to (via (map stream-cdr argstreams)) stream-cdr into ones, which I would expect to result in the infinite sequence I get above!
Even though, if I understand correctly, due to cons-stream being a macro the second part inside the stream-map will only be lazily evaluated, at some point while I cdr into the ints stream I would expect to have to cdr into ones as well.. which instead doesn't seem to be happening! :/
Any help in understanding this would be very much appreciated!
(I also didn't override any scheme symbol and I'm running everything in MIT Scheme - Release 11.2 on OS X.)
(define ones (cons-stream 1 ones))
(stream-cdr ones)
returns an infinite sequence of evaluated ones.
No it does not. (stream-cdr ones) = (stream-cdr (cons-stream 1 ones)) = ones. It is just one ones, not the sequence of ones.
And what is ones?
Its stream-car is 1, and its stream-cdr is ones.
Whose stream-car is 1, and stream-cdr is ones again.
And so if you (stream-take n ones) (with an obvious implementation of stream-take) you will receive an n-long list of 1s, whatever the (non-negative) n is.
In other words, repeated cdring into ones produces an unbounded sequence of 1s. Or symbolically, {1 1 1 ...} indeed.
After your edit it becomes clear the question is about REPL behavior. What you're seeing most probably has to do with the difference between sharing and recalculation, just like in letrec with the true reuse (achieved through self-referring binding) vs. the Y combinator with the recalculation ((Y g) == g (Y g)).
You probably can see the behavior you're expecting, with the re-calculating definition (define (onesF) (cons-stream 1 (onesF))).
I only have Racket, where
(define ones (cons-stream 1 ones))
(define (onesF) (cons-stream 1 (onesF)))
(define ints (cons-stream 1 (stream-map + ones ints)))
(stream-cdr ones)
(stream-cdr (onesF))
(stream-cdr ints)
prints
(mcons 1 #<promise>)
(mcons 1 #<promise>)
(mcons 2 #<promise>)
Furthermore, having defined
(define (take n s)
(if (<= n 1)
(if (= n 1) ; prevent an excessive `force`
(list (stream-car s))
'())
(cons (stream-car s)
(take (- n 1)
(stream-cdr s)))))
we get
> (display (take 5 ones))
(1 1 1 1 1)
> (display (take 5 (onesF)))
(1 1 1 1 1)
> (display (take 5 ints))
(1 2 3 4 5)
This question already has answers here:
reverse list - scheme
(10 answers)
Closed 2 years ago.
I am trying to reverse a linked list in Scheme; and I have no idea why this isn't working. I am using Chez Scheme by the way. It is giving me this error: Exception: incorrect number of arguments to #procedure reverse-ll at reverse-linked-list.scm:59. Does anyone have an idea as to what is happening? Thanks!
(define pi '(3 1 4 1 5 9 2 6 5 3 5))
(define reverse-ll
(lambda (orig rev)
(if (null? (cdr orig)) rev
(reverse-ll (cons (car orig) rev)))))
(define reverse-pi (reverse-ll pi '()))
(display reverse-pi)
The error says that a call to reverse-ll does not have enough arguments, and there are only two such calls. The call (reverse-ll pi '()) is fine, since reverse-ll takes two list arguments. The other call is the problem:
(reverse-ll (cons (car orig) rev))
Here reverse-ll is given only one argument: (cons (car orig) rev). This expression is adding the first element of the input list to the accumulator. But the code is missing the reduced input list for the recursive step. Instead:
(reverse-ll (cdr orig) (cons (car orig) rev))
There is another problem. As written, the code does not include the last element of the input list in the reversed output. This is because of the base case:
(if (null? (cdr orig)) rev ;; )
In this code the base case is reached when (cdr orig) is '(), and that happens when the input list has been reduced to a length of one element. The last element is then not processed by the reverse-ll procedure. Instead you want an empty list in the input to trigger the return of the accumulator:
(define reverse-ll
(lambda (orig rev)
(if (null? orig) rev
(reverse-ll (cdr orig) (cons (car orig) rev)))))
Here is the result, showing both input and output:
> (load "scratch.ss")
(3 1 4 1 5 9 2 6 5 3 5)
(5 3 5 6 2 9 5 1 4 1 3)
I am trying to learn the use of delay and force in Scheme. I created a function which outputs the even numbers in the list. Now I am trying to use delay and force with function. But I am getting error : unbound symbol : "delay" [].
Can someone explain what I am doing wrong?
(define (even-filter ls)
(if (null? ls) '()
(filter even? ls)))
(even-filter '(1 2 3 4 5 88))
(let ((delayed (delay (even-filter '(1 2 3 4 5 88)))))
(force delayed))
Make sure that you're using the right language, because both force and delay are primitive forms, part of the standard language. If using Racket, in the bottom-left corner select "Determine language from source", and type the following:
#lang racket
(define (even-filter ls)
(filter even? ls))
(let ((delayed (delay (even-filter '(1 2 3 4 5 88)))))
(force delayed))
It works as expected:
'(2 4 88)
I'm working with a data set that is too big to fit into memory and therefore would like to use a stream to process the data. However, I'm finding that the stream isn't limiting the amount of memory used as I expected. The sample code illustrates the problem and will run out of memory if the memory limit is set to 128mb. I know I could up the memory limit, but with the sort of big data set that I want to use this won't be an option. How should I keep the memory use down?
#lang racket
(struct prospects (src pos num max-num)
#:methods gen:stream
[(define (stream-empty? stream)
(equal? (prospects-num stream) (prospects-max-num stream)))
;
(define (stream-first stream)
(list-ref (prospects-src stream) (prospects-pos stream)))
;
(define (stream-rest stream)
(let ([next-pos (add1 (prospects-pos stream))]
[src (prospects-src stream)]
[next-num (add1 (prospects-num stream))]
[max-num (prospects-max-num stream)])
(if (< next-pos (length src))
(prospects src next-pos next-num max-num)
(prospects src 0 next-num max-num))))])
(define (make-prospects src num)
(prospects src 0 0 num))
(define (calc-stats prospects)
(let ([just-a (stream-filter
(λ (p) (equal? "a" (vector-ref p 0)))
prospects)]
[just-b (stream-filter
(λ (p) (equal? "b" (vector-ref p 0)))
prospects)])
;
(let ([num-a (stream-length just-a)]
[num-b (stream-length just-b)]
[sum-ref1-a (for/sum ([p (in-stream just-a)])
(vector-ref p 1))]
[sum-ref1-b (for/sum ([p (in-stream just-b)])
(vector-ref p 1))])
;
#|
; Have also tried with stream-fold instead of for/sum as below:
[sum-ref1-a (stream-fold
(λ (acc p) (+ acc (vector-ref p 1)))
0 just-a)]
[sum-ref1-b (stream-fold
(λ (acc p) (+ acc (vector-ref p 1)))
0 just-b)])
|#
;
(list num-a num-b sum-ref1-a sum-ref1-b))))
;================================
; Main
;================================
(define num-prospects 800000)
(define raw-prospects (list #("a" 2 2 5 4 5 6 2 4 2 45 6 2 4 5 6 3 4 5 2)
#("b" 1 3 5 2 4 3 2 4 5 34 3 4 5 3 2 4 5 6 3)))
(calc-stats (make-prospects raw-prospects num-prospects))
Note: This program was created just to demonstrate the problem; the real stream would access a database to bring in the data.
The main problem in your code was that you were trying to make multiple passes through the stream. (Each call to stream-length is one pass, and each of your calls to for/sum (or stream-fold, for that matter) is another pass.) This means that you had to materialise the whole stream without allowing the earlier stream elements to be garbage-collected.
Here's a modification of your code to make only one pass. Note that I made num-prospects to be 8,000,000 in my version, since even the multi-pass version didn't run out of memory on my system with only 800,000:
#lang racket
(require srfi/41)
(define (make-prospects src num)
(stream-take num (apply stream-constant src)))
(define (calc-stats prospects)
(define default (const '(0 . 0)))
(define ht (for/fold ([ht #hash()])
([p (in-stream prospects)])
(hash-update ht (vector-ref p 0)
(λ (v)
(cons (add1 (car v))
(+ (cdr v) (vector-ref p 1))))
default)))
(define stats-a (hash-ref ht "a" default))
(define stats-b (hash-ref ht "b" default))
(list (car stats-a) (car stats-b) (cdr stats-a) (cdr stats-b)))
;================================
; Main
;================================
(define num-prospects 8000000)
(define raw-prospects '(#("a" 2 2 5 4 5 6 2 4 2 45 6 2 4 5 6 3 4 5 2)
#("b" 1 3 5 2 4 3 2 4 5 34 3 4 5 3 2 4 5 6 3)))
(calc-stats (make-prospects raw-prospects num-prospects))
I should clarify that the use of srfi/41 is simply to enable writing a more-efficient version of make-prospects (though, the reference implementation of stream-constant isn't very efficient, but still more efficient than what your prospects stream generator did); calc-stats doesn't use it.
I upvoted Chris' excellent answer and suggest you pick it to mark as accepted.
However, what would use the least memory for a data set that doesn't fit into RAM? Probably something like the following pseudo code:
(require db)
(define dbc <open a db connection>)
(define just-a (query-value dbc "SELECT Count(*) FROM t WHERE n = $1" "a"))
(define just-b (query-value dbc "SELECT Count(*) FROM t WHERE n = $1" "b"))
Why this is a somewhat smartypants answer:
Ostensibly you asked about using Racket streams to handle things that don't fit in memory.
If you need more complex aggregations than count (or sum/min/max), you'll need to write more complex SQL queries, and probably want to make them stored procedures on the server.
Why it isn't necessarily smartypants:
You did mention your real use case involved a database. ;)
A speciality of a DB server and SQL is large data sets that don't fit in memory. Taking advantage of the server often will beat something DB-ish re-implemented in a general-purpose language (as well as probably being friendlier to other uses/users of the same DB server).
You are running out of memory because you are hanging onto the head of the stream while traversing it. The GC can't collect anything because since you have a pointer to the head, every element of the stream is still reachable.
To demonstrate, with this stream:
(define strm (make-prospects raw-prospects num-prospects))
this blows up:
(define just-a (stream-filter (λ (p) (equal? "a" (vector-ref p 0))) strm))
(stream-length just-a)
while this is fine:
(stream-length (stream-filter (λ (p) (equal? "a" (vector-ref p 0))) strm))
How does the following process work:
(define integers
(cons-stream 1
(stream-map (lambda (x) (+ x 1))
integers))
The important thing to realize here that only those expressions are evaluated which are neccessary to calculate the element of the list you're accessing.
So when you access the first element, it evaluates the first argument to cons-stream which is 1.
When you access the second element, it evaluates the first element of stream-map (lambda (x) (+ x 1)) integers. For that it needs to get the first element of integers which is 1 and then adds 1 to that and you get 2.
When you access the third element, it evaluates the second element of stream-map (lambda (x) (+ x 1)) integers. So it takes the second element of integers (2) and adds 1 to that to get 3. And so on.