headers
Slava Pestov | 22 Apr 21:18

Proposal: some changes to streams

Hi all,

Right now we have an 'stdio' variable and various words which operate  
on it,

: read stdio get stream-read ;
: readln stdio get stream-readln ;
: write stdio get stream-write ;
: print stdio get stream-print ;

with-stream rebinds it and closes it after, with-stream* rebinds it  
without closing.

This makes for very nice and concise code, but there's one problem  
with it and that it cannot express the case where you want to read  
from one file and write to another at the same time. Instead, you need  
to do something like this,

"in.txt" utf8 <file-reader> [
     "out.txt" latin1 <file-writer> [
         <duplex-stream> [
             ...
         ] with-stream*
     ] with-disposal
] with-disposal

Or if you want to read from a file and print to the console in the  
same code,

stdio get
(Continue reading)

Permalink | Reply |
headers
Eduardo Cavazos | 22 Apr 21:38

Re: Proposal: some changes to streams

Slava Pestov wrote:

> "in.txt" utf8 <file-reader> [
>      "out.txt" latin1 <file-writer> [
>          <duplex-stream> [
>              ...
>          ] with-stream*
>      ] with-disposal
> ] with-disposal
>
> Or if you want to read from a file and print to the console in the
> same code,
>
> stdio get
> "in.txt" utf8 <file-reader> [
>      swap <duplex-stream> [
>          ...
>      ] with-stream*
> ] with-disposal

Slava,

What would those examples look like using the proposed changes?

Ed

-------------------------------------------------------------------------
This SF.net email is sponsored by the 2008 JavaOne(SM) Conference 
Don't miss this year's exciting event. There's still time to save $100. 
Use priority code J8TL2D2.
(Continue reading)

Permalink | Reply |
headers
Slava Pestov | 22 Apr 21:41

Re: Proposal: some changes to streams


On Apr 22, 2008, at 2:38 PM, Eduardo Cavazos wrote:

> What would those examples look like using the proposed changes?

"in.txt" utf8 [
     "out.txt" latin1 [
         ...
     ] with-file-writer
] with-file-reader

"in.txt" utf8 [
     ...
] with-file-reader

-------------------------------------------------------------------------
This SF.net email is sponsored by the 2008 JavaOne(SM) Conference 
Don't miss this year's exciting event. There's still time to save $100. 
Use priority code J8TL2D2. 
http://ad.doubleclick.net/clk;198757673;13503038;p?http://java.sun.com/javaone
Permalink | Reply |
headers
Eduardo Cavazos | 22 Apr 22:19

Factoring out the reader/writer pattern

Slava Pestov wrote:

> "in.txt" utf8 [
>      "out.txt" latin1 [
>          ...
>      ] with-file-writer
> ] with-file-reader

I'd like to point out that even with the change you proposed, given that the 
pattern looks like the above, I'd still be inclined it out in to a 
with-reader-writer ( path path quot -- ).

Ed

-------------------------------------------------------------------------
This SF.net email is sponsored by the 2008 JavaOne(SM) Conference 
Don't miss this year's exciting event. There's still time to save $100. 
Use priority code J8TL2D2. 
http://ad.doubleclick.net/clk;198757673;13503038;p?http://java.sun.com/javaone
Permalink | Reply |
headers
Slava Pestov | 22 Apr 22:22

Re: Factoring out the reader/writer pattern


On Apr 22, 2008, at 3:19 PM, Eduardo Cavazos wrote:

> I'd like to point out that even with the change you proposed, given  
> that the
> pattern looks like the above, I'd still be inclined it out in to a
> with-reader-writer ( path path quot -- ).

Sure, that would be useful, and the implementation of with-reader- 
writer would be simpler with my changes too.

Slava

-------------------------------------------------------------------------
This SF.net email is sponsored by the 2008 JavaOne(SM) Conference 
Don't miss this year's exciting event. There's still time to save $100. 
Use priority code J8TL2D2. 
http://ad.doubleclick.net/clk;198757673;13503038;p?http://java.sun.com/javaone
Permalink | Reply |
headers
kochismo | 23 Apr 01:24

Re: Factoring out the reader/writer pattern

Slava Pestov <slava@...> writes:

> 
> 
> On Apr 22, 2008, at 3:19 PM, Eduardo Cavazos wrote:
> 
> > I'd like to point out that even with the change you proposed, given  
> > that the
> > pattern looks like the above, I'd still be inclined it out in to a
> > with-reader-writer ( path path quot -- ).
> 
> Sure, that would be useful, and the implementation of with-reader- 
> writer would be simpler with my changes too.
> 
> Slava
> 
> -------------------------------------------------------------------------
> This SF.net email is sponsored by the 2008 JavaOne(SM) Conference 
> Don't miss this year's exciting event. There's still time to save $100. 
> Use priority code J8TL2D2. 
> http://ad.doubleclick.net/clk;198757673;13503038;p?http://java.sun.com/javaone
> 

What if you don't have exactly one reader and exactly one writer?  Maybe we can
have with-reader-reader-writer(path path path quot -- )?  And
with-reader-writer-writer(path path path quot -- )?  And so on for every likely
number of simultaneous readers/writers.

Perhaps something like the following would be preferable?
(Continue reading)

Permalink | Reply |
headers
Slava Pestov | 23 Apr 01:34

Re: Factoring out the reader/writer pattern

The input-stream/output-stream variables are intended (as the stdio  
variable does) to optimize for the common use case.

You can already do

"foo.txt" <file-reader> [ | in1 | .... in1 stream-read ... ] with- 
disposal

Using locals. So there's no problem there.

Slava

On Apr 22, 2008, at 6:24 PM, kochismo wrote:

> What if you don't have exactly one reader and exactly one writer?

-------------------------------------------------------------------------
This SF.net email is sponsored by the 2008 JavaOne(SM) Conference 
Don't miss this year's exciting event. There's still time to save $100. 
Use priority code J8TL2D2. 
http://ad.doubleclick.net/clk;198757673;13503038;p?http://java.sun.com/javaone
Permalink | Reply |
headers
Eduardo Cavazos | 22 Apr 21:40

First example

Slava Pestov wrote:

> "in.txt" utf8 <file-reader> [
>      "out.txt" latin1 <file-writer> [
>          <duplex-stream> [
>              ...
>          ] with-stream*
>      ] with-disposal
> ] with-disposal

I think you've answered this question before but...

Why couldn't the above example be written like this:

"in.txt"  utf8   <file-reader>
"out.txt" latin1 <file-writer>
<duplex-stream>
  [ ... ]
with-stream

Ed

-------------------------------------------------------------------------
This SF.net email is sponsored by the 2008 JavaOne(SM) Conference 
Don't miss this year's exciting event. There's still time to save $100. 
Use priority code J8TL2D2. 
http://ad.doubleclick.net/clk;198757673;13503038;p?http://java.sun.com/javaone
Permalink | Reply |
headers
Slava Pestov | 22 Apr 21:42

Re: First example

Because if creating the writer fails, you don't dispose the reader.

Also because I'm kind of hoping to remove duplex-stream too :-)

On Apr 22, 2008, at 2:40 PM, Eduardo Cavazos wrote:

> Why couldn't the above example be written like this:
>
> "in.txt"  utf8   <file-reader>
> "out.txt" latin1 <file-writer>
> <duplex-stream>
>  [ ... ]
> with-stream

-------------------------------------------------------------------------
This SF.net email is sponsored by the 2008 JavaOne(SM) Conference 
Don't miss this year's exciting event. There's still time to save $100. 
Use priority code J8TL2D2. 
http://ad.doubleclick.net/clk;198757673;13503038;p?http://java.sun.com/javaone
Permalink | Reply |
headers
Eduardo Cavazos | 22 Apr 22:02

Re: Proposal: some changes to streams

Slava Pestov wrote:

> "in.txt" utf8 <file-reader> [
>      "out.txt" latin1 <file-writer> [
>          <duplex-stream> [
>              ...
>          ] with-stream*
>      ] with-disposal
> ] with-disposal

> This is too awkward for a language named "Factor".

Why wouldn't you just "Factor" out that pattern? Something like:

	: with-reader-writer ( path path quot -- ) ... ;

So the above code is just:

	"in.txt" "out.txt"
	  [ ... ]
	with-reader-writer

Well, assuming the proposed "path objects with encoding slots", it would look 
like this:

	"in.txt" utf8 "out.txt" latin1
	  [ ... ]
	with-reader-writer

Ed
(Continue reading)

Permalink | Reply |
headers
Slava Pestov | 22 Apr 22:05

Re: Proposal: some changes to streams

Because its not general enough. Only works for input from a file and  
output from a file.

You might want input from a file and output to a process stream, input  
from a socket and output to the console, etc.

On Apr 22, 2008, at 3:02 PM, Eduardo Cavazos wrote:

> Why wouldn't you just "Factor" out that pattern? Something like:
>
> 	: with-reader-writer ( path path quot -- ) ... ;
>
> So the above code is just:
>
> 	"in.txt" "out.txt"
> 	  [ ... ]
> 	with-reader-writer
>
> Well, assuming the proposed "path objects with encoding slots", it  
> would look
> like this:
>
> 	"in.txt" utf8 "out.txt" latin1
> 	  [ ... ]
> 	with-reader-writer
>
> Ed
>
> -------------------------------------------------------------------------
> This SF.net email is sponsored by the 2008 JavaOne(SM) Conference
(Continue reading)

Permalink | Reply |
headers
Eduardo Cavazos | 22 Apr 22:17

with-input-output

Slava Pestov wrote:

> Because its not general enough. Only works for input from a file and
> output from a file.

I agree. When I was thinking about with-reader-writer, I also thought about 
having with-input-output.

	: with-input-output ( stream stream quot -- ) ... ;

If you pass 'f' as a stream, it uses stdio for that stream.

> You might want input from a file and output to a process stream, input
> from a socket and output to the console, etc.

Those are covered by with-input-output (again, with the utf8 defaults) :

"in.txt" { "wc" } <process-stream> [ ... ] with-input-output

"www.apple.com" "http" <inet> <client> f [ ... ] with-input-output

Ed

-------------------------------------------------------------------------
This SF.net email is sponsored by the 2008 JavaOne(SM) Conference 
Don't miss this year's exciting event. There's still time to save $100. 
Use priority code J8TL2D2. 
http://ad.doubleclick.net/clk;198757673;13503038;p?http://java.sun.com/javaone
Permalink | Reply |
headers
Slava Pestov | 22 Apr 22:21

Re: with-input-output


On Apr 22, 2008, at 3:17 PM, Eduardo Cavazos wrote:

> Slava Pestov wrote:
>
>> Because its not general enough. Only works for input from a file and
>> output from a file.
>
> I agree. When I was thinking about with-reader-writer, I also  
> thought about
> having with-input-output.
>
> 	: with-input-output ( stream stream quot -- ) ... ;

So then you'd have to do

"in.txt" utf8 <file-reader> [
     "out.txt" utf8 <file-writer> [
         [
             ...
         ] with-input-output
     ] with-disposal
] with-disposal

Slava

-------------------------------------------------------------------------
This SF.net email is sponsored by the 2008 JavaOne(SM) Conference 
Don't miss this year's exciting event. There's still time to save $100. 
Use priority code J8TL2D2.
(Continue reading)

Permalink | Reply |
headers
Eduardo Cavazos | 22 Apr 22:09

Removing duplex streams

Slava Pestov wrote:

> A more radical idea is to remove duplex streams altogether. Instead,
> words like <process-stream> and <client> would return a pair of
> streams, a reader and a writer. A with-streams combinator with stack
> effect ( in out quot -- ) could be used here.

What's the main motivation for removing duplex streams?

I'd like to point out that it sort of moves in away from the direction I've 
argued for in general; that of "combining" objects on the stack instead of 
splitting them up. Areas where I've emphasized this principle are the 
encodings api and cursors.

Let me just say that exceptions to principles should be made when they're 
justified. You might not subscribe to this principle, but from my 
perspective, I'm very interested in the reasons behind the change.

Ed

-------------------------------------------------------------------------
This SF.net email is sponsored by the 2008 JavaOne(SM) Conference 
Don't miss this year's exciting event. There's still time to save $100. 
Use priority code J8TL2D2. 
http://ad.doubleclick.net/clk;198757673;13503038;p?http://java.sun.com/javaone
Permalink | Reply |
headers
Slava Pestov | 22 Apr 22:15

Re: Removing duplex streams

Three reasons:

- I believe they wouldn't be needed anymore with the new variables/ 
combinators.
- Because I/O through a duplex stream entails an extra level of  
indirection.
- Implementations of things like setting the timeout, changing the  
encoding is ugly with duplex streams.
- The implementation of duplex streams themselves contains a fair bit  
of boilerplate. I could move Dan's delegate library to the core but  
I'd rather not do that if duplex streams are the only use-case in the  
core.

Basically I think they would become redundant and since they're kind  
of ugly its best not to have them.

Slava

On Apr 22, 2008, at 3:09 PM, Eduardo Cavazos wrote:

> What's the main motivation for removing duplex streams?
>
> I'd like to point out that it sort of moves in away from the  
> direction I've
> argued for in general; that of "combining" objects on the stack  
> instead of
> splitting them up. Areas where I've emphasized this principle are the
> encodings api and cursors.
>
> Let me just say that exceptions to principles should be made when
(Continue reading)

Permalink | Reply |
headers
Eduardo Cavazos | 22 Apr 22:47

Duplex streams and delegation

Slava Pestov wrote:

> - The implementation of duplex streams themselves contains a fair bit
> of boilerplate. I could move Dan's delegate library to the core but
> I'd rather not do that if duplex streams are the only use-case in the
> core.

Where is the duplex streams code which uses delegation? I took a look 
at 'io.streams.duplex'.

Ed

-------------------------------------------------------------------------
This SF.net email is sponsored by the 2008 JavaOne(SM) Conference 
Don't miss this year's exciting event. There's still time to save $100. 
Use priority code J8TL2D2. 
http://ad.doubleclick.net/clk;198757673;13503038;p?http://java.sun.com/javaone
Permalink | Reply |
headers
Slava Pestov | 22 Apr 22:57

Re: Duplex streams and delegation


On Apr 22, 2008, at 3:47 PM, Eduardo Cavazos wrote:

> Slava Pestov wrote:
>
>> - The implementation of duplex streams themselves contains a fair bit
>> of boilerplate. I could move Dan's delegate library to the core but
>> I'd rather not do that if duplex streams are the only use-case in the
>> core.
>
> Where is the duplex streams code which uses delegation? I took a look
> at 'io.streams.duplex'.

It hasn't been written. But you can imagine the manual delegation  
being replaced with calls to Dan's CONSULT: word.

Slava

-------------------------------------------------------------------------
This SF.net email is sponsored by the 2008 JavaOne(SM) Conference 
Don't miss this year's exciting event. There's still time to save $100. 
Use priority code J8TL2D2. 
http://ad.doubleclick.net/clk;198757673;13503038;p?http://java.sun.com/javaone
Permalink | Reply |
headers
Eduardo Cavazos | 23 Apr 06:22

Re: Removing duplex streams

Slava Pestov wrote:

> - Implementations of things like setting the timeout, changing the
> encoding is ugly with duplex streams.

I'd like to see this code. I see that 'set-timeout' is a generic. Are you 
referring to the method on 'duplex-stream'?

Ed

-------------------------------------------------------------------------
This SF.net email is sponsored by the 2008 JavaOne(SM) Conference 
Don't miss this year's exciting event. There's still time to save $100. 
Use priority code J8TL2D2. 
http://ad.doubleclick.net/clk;198757673;13503038;p?http://java.sun.com/javaone
Permalink | Reply |
headers
Slava Pestov | 23 Apr 07:02

Re: Removing duplex streams


On Apr 22, 2008, at 11:22 PM, Eduardo Cavazos wrote:

> I'd like to see this code. I see that 'set-timeout' is a generic.  
> Are you
> referring to the method on 'duplex-stream'?

Yes.

I want to clarify something. By 'remove duplex streams', all I really  
mean is moving them out of the core. If people find the concept useful  
(I don't), I can leave it in extra/.

Slava

-------------------------------------------------------------------------
This SF.net email is sponsored by the 2008 JavaOne(SM) Conference 
Don't miss this year's exciting event. There's still time to save $100. 
Use priority code J8TL2D2. 
http://ad.doubleclick.net/clk;198757673;13503038;p?http://java.sun.com/javaone
Permalink | Reply |
headers
Eduardo Cavazos | 23 Apr 07:11

Re: Removing duplex streams

Slava Pestov wrote:

> I want to clarify something. By 'remove duplex streams', all I really
> mean is moving them out of the core.

Don't you also mean that you'd like to change '<client>', '<process-stream>', 
and 'accept'? They all use duplex-streams. And of course, this isn't just an 
implementation detail change; the stack effects will change.

Ed

-------------------------------------------------------------------------
This SF.net email is sponsored by the 2008 JavaOne(SM) Conference 
Don't miss this year's exciting event. There's still time to save $100. 
Use priority code J8TL2D2. 
http://ad.doubleclick.net/clk;198757673;13503038;p?http://java.sun.com/javaone
Permalink | Reply |
headers
Eduardo Cavazos | 23 Apr 07:01

Timeouts and encodings on duplex-streams vs input/output


> A more radical idea is to remove duplex streams altogether. Instead,
> words like <process-stream> and <client> would return a pair of
> streams, a reader and a writer. A with-streams combinator with stack
> effect ( in out quot -- ) could be used here.

OK. But then let's look at one of the reasons you want to do away with 
duplex-stream:

> - Implementations of things like setting the timeout, changing the  
> encoding is ugly with duplex streams.

If <process-stream> retuns two streams, you will have to set the timeout 
twice; you will have to change the encoding twice. The "ugly" is being 
shifted somewhere else.

Moreover, it's being shifted from a place where it was an abstrated pattern, 
into every place where a timeout/encoding is changed. This seems 
like "unfactoring".

Alternatively, let's say you wait until you're inside the "with-streams" 
quotation to modify the timeout or encoding. If you want to change the 
encoding, you'll have to do so on 'input' and 'output'.

Ed

-------------------------------------------------------------------------
This SF.net email is sponsored by the 2008 JavaOne(SM) Conference 
Don't miss this year's exciting event. There's still time to save $100. 
Use priority code J8TL2D2.
(Continue reading)

Permalink | Reply |

Gmane