PostScript String Token - token

Question

I have a loop that takes out each word in a string using token, then I want to be able to do computations on it like this:
(1 2 add 3 4 add)
But any way you code it i keep getting
7
add
2
1
I want it to be
7
3
This is what I am working with
{ %loop
pstack
(repl> )print flush
(%lineedit)(r)file
dup bytesavailable string readstring pop
{
token
{}{exit}ifelse
exch
dup () eq {pop exec exit}if
exec
}loop
}loop

Answer 1

I would advise you to write stack comments at the end of each line. It really helps.
{ %loop
pstack
(repl> )print flush
(%lineedit)(r)file % f
dup bytesavailable string readstring pop % s
{
token % s t b
{}{exit}ifelse % s t
exch % t s
dup () eq {pop exec exit}if % t s
exec % t
}loop
}loop
So you're executing the remaining substring instead of the token. You need another exch in there before the exec in the inner loop. That would execute the token instead of the substring. But a problem with this is that the string is sitting there on the stack. So add won't work because it will find a string on top of the stack instead of the numbers below.
So it may be better to save the substring by name before exec-ing, and then put it back before the next iteration.
{ % s
token % s t b
{}{exit}ifelse % s t
exch % t s
dup () eq {pop exec exit}if % t s
/rem exch def % t
exec
rem % s
}loop
This portion may be more confusing than helpful to those very new to postscript. Read on at your own peril. If you get lost in the middle, be sure to see the very end where there is a final super-simple technique.
The next question a postscript hacker should ask is: "How can I do this without polluting the name space with this rem name?"
The insane trick I would use for this is to exploit the loop operator to make a procedure body with extra storage.
{ procedure body } exec
{ procedure body exit extra storage } loop
Both constructs above will execute procedure body and then return control. But using loop with an explicit exit lets us pack extra things into the array.
So, we take the inner loop from above.
token{}{exit}ifelse exch dup()eq{pop exec exit}if/rem exch def exec rem
Wrap it in an "exit-loop".
{
token{}{exit}ifelse exch dup()eq{pop exec exit}if/rem exch def exec rem
exit } loop
And we're going to store the string remainder just after exit.
{
token{}{exit}ifelse exch dup()eq{pop exec exit}if/rem exch def exec rem
exit STR } loop
Replace the /name exch def with code that stores into an array. This array will be a subarray of the loop body which just holds the [ STR ] extra storage.
/rem exch def --> ARR exch 0 exch put
rem --> ARR 0 get
{
token{}{exit}ifelse exch dup()eq{pop exec exit}if ARR exch 0 exch put exec ARR 0 get
exit STR } loop
This loop is of course a straight-shot: it doesn't actually loop. So to replace the inner loop from above, we wrap it in another loop.
{ {
token{}{exit}ifelse exch dup()eq{pop exec exit}if ARR exch 0 exch put exec ARR 0 get
exit STR } loop } loop
Then we need to insert the subarray where ARR is in the code. This is the subarray of the (inner) inner loop that contains the (dummy) STR token.
% 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20
{ { token{}{exit}ifelse exch dup()eq{pop exec exit}if ARR exch 0 exch put exec ARR 0 get exit STR } loop } loop
So we need a subarray [20, 1] to be inserted at array[10] and array[16]. And we can do this before calling loop at the outer scope.
{ {
token{}{exit}ifelse exch dup()eq{pop exec exit}if ARR exch 0 exch put exec ARR 0 get
exit STR } loop }
dup 0 get % loop-body inner-loop get a copy of the exit-loop
dup 20 1 getinterval % loop-body inner-loop [STR] take a subarray of the exit-loop
2 copy 10 exch put % loop-body inner-loop [STR] insert in position 10
16 exch put % loop-body' insert in position 16
loop % call the loop operator
There, a loop with no name. :)
Notice, we still have the dummy name STR in the code, and that's ok. It will parse as a name and allocate an extra slot in the array. And it doesn't need to be defined anywhere because it never gets executed.
An improvement over the above. We really do not need the second ARR in the template code. We can store the string directly into the procedure array at the position where it is needed. Then we don't even need the "exit-loop". So the template becomes:
% 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16
{ token{}{exit}ifelse exch dup()eq{pop exec exit}if ARR exch 0 exch put exec STR } loop
And the fully articulated loop
{ token{}{exit}ifelse exch dup()eq{pop exec exit}if ARR exch 0 exch put exec STR }
dup dup 10 exch % loop loop 10 loop prepare stack
16 1 getinterval % loop loop 10 [STR] take a subarray
put % loop insert in position 10
loop % call loop operator
An improvement of the improvement. We don't actually need a subarray either. We can store the entire loop array in the ARR position and use the index 16 instead of 0 in the storing code.
{ token not{exit}if exch dup()eq{pop exec exit}if ARR exch 16 exch put exec STR }
dup 10 1 index % loop loop 10 loop prepare stack
put % loop insert in position 10
loop % call loop operator
--
Much later...
This is way more complicated than it needs to be. We can simply make a little array to sequence the execution of these two things.
{exec rem}
Thus:
{ % s
token % s t b
not{exit}if % s t
exch % t s
dup () eq {pop exec exit}if % t s
/exec cvx exch 2 array astore cvx % t {exec s}
exec
}loop

Answer 2

Token simply returns the 'tokenised' object on the stack, it doesn't do anything further with it. If you want to perform the operation you will have to check the type of the returned object and 'exec' the executable ones.