headers
Andrew Lentvorski | 2 Aug 2006 18:00

Re: BytePickle examples?

Philipp Haller wrote:

>> The question I have is: How do I get an integer to always pickle as a 
>> 4 byte, network order thing?
> 
> I attached some code with a pickler that always pickles integers as 4 
> bytes. I don't know which order you need, though. If you think a pickler 
> for integers as 4 bytes in "network order" is generally useful, we might 
> add this to our pickler library.

Well, at the very least, the example of how to do this needs to be 
somewhere.  The library is probably as good a place as any.

Having a way to serialize an int into 4 byte network order is probably a 
useful construct.  I'm not sure how much further I would go, though. 
Probably I'd add 2 byte network order as well and stop there.  After 
4-byte and 2-byte (nat seems to work for 1 byte) ints the usefulness 
drops off really quickly.

Two things still aren't very clear to me yet, though.

First, I see some "polymorphic" behavior in the Actor library in 
TcpSerializerComb.scala, but I don't quite grok it.  What I am trying to 
do is read a network packet and unpickle.  The first four bytes are an 
integer, and are the packet type.  I would like to read the first 4 
bytes, figure out what the packet is, and then unpickle the rest of the 
packet to a case class which is the actual packet.  Part of the 
complication is my weak understanding of pickling and part of the 
complication is my weak understanding of matching in Scala.
(Continue reading)

Permalink | Reply |
headers
philipp.haller | 2 Aug 2006 23:07
Picon
Picon
Favicon

Re: BytePickle examples?

I didn't have time to come up with concrete code, yet. Anyway, here is some
advice before you go wild: ;-)

> First, I see some "polymorphic" behavior in the Actor library in
> TcpSerializerComb.scala, but I don't quite grok it.  What I am trying to
> do is read a network packet and unpickle.  The first four bytes are an
> integer, and are the packet type.  I would like to read the first 4
> bytes, figure out what the packet is, and then unpickle the rest of the
> packet to a case class which is the actual packet.  Part of the
> complication is my weak understanding of pickling and part of the
> complication is my weak understanding of matching in Scala.

OK, so what you want is a datatype pickler. In the library there is a `data'
pickler which uses integers to tag the different alternatives. For your pickler,
you could define a pickler similar to `data' which uses the `int4' pickler, like
this:
  def myData[a](tag: a => int, ps: List[()=>SPU[a]]): SPU[a] =
    sequ(tag, int4, (x: int) => ps.apply(x)());

  Usage: (example from my thesis  <at>  http://lamp.epfl.ch/~phaller/actors.html)

  def termPickler: SPU[Term] =
    myData((t: Term) => t match {
      case Var(_) => 0
      case Lam(_,_) => 1
      case App(_,_) => 2
    },
    List(() => varPickler, () => lamPickler, () => appPickler))

  where varPickler, lamPickler and appPickler are pickler for Var, Lam and App
(Continue reading)

Permalink | Reply |
headers
Andrew Lentvorski | 3 Aug 2006 09:09

Re: BytePickle examples?

philipp.haller <at> epfl.ch wrote:

>   def myData[a](tag: a => int, ps: List[()=>SPU[a]]): SPU[a] =
>     sequ(tag, int4, (x: int) => ps.apply(x)());

Does data actually work?  I can't seem to find a use of it in the
entire scala codebase.

I can't seem to get this to work with types that don't inherit
from a common ancestor.  Am I doing something wrong?

I had to rewrite data into foodata and rearrange the genericity terms
using a lot of Any and AnyRef.  It works, but it sure feels wrong.

I would welcome advice about how to get this right.

Thanks,
-a

package testpickle;

import scala.collection.mutable.ArrayBuffer

import scala.io._
import scala.io.BytePickle._

object TypePickleTest {
   def dumpBytes(bs: Array[byte]) = {
     val buf = new StringBuffer(bs.length)
     for (val i <- List.range(0, bs.length)) {
(Continue reading)

Permalink | Reply |
headers
Philipp Haller | 3 Aug 2006 13:10
Picon
Picon
Favicon

Re: BytePickle examples?

>>   def myData[a](tag: a => int, ps: List[()=>SPU[a]]): SPU[a] =
>>     sequ(tag, int4, (x: int) => ps.apply(x)());
> 
> 
> Does data actually work?  I can't seem to find a use of it in the
> entire scala codebase.

Yes, `data' works fine. I attached a self-contained example using `data' 
or `myData' (just replace the use of `data' by `myData').

> I can't seem to get this to work with types that don't inherit
> from a common ancestor.  Am I doing something wrong?

A common ancestor is necessary. Otherwise the result of data could not 
be typed.

> I had to rewrite data into foodata and rearrange the genericity terms
> using a lot of Any and AnyRef.  It works, but it sure feels wrong.

If you want to switch between Strings and ints surely you have no 
choice. Compare this to the standard Java de-serialization: The result 
has to have type Object because before you read the bytes you have no 
clue. You can only assign a more precise static type if you know before 
hand that the type is contained in some closed set. Thus, yes, you have 
to provide a common supertype.

-- Philipp

package examples.pickle
(Continue reading)

Permalink | Reply |
headers
Andrew Lentvorski | 3 Aug 2006 19:32

Re: BytePickle examples?

Philipp Haller wrote:
>> I had to rewrite data into foodata and rearrange the genericity terms
>> using a lot of Any and AnyRef.  It works, but it sure feels wrong.
> 
> If you want to switch between Strings and ints surely you have no 
> choice. Compare this to the standard Java de-serialization: The result 
> has to have type Object because before you read the bytes you have no 
> clue. You can only assign a more precise static type if you know before 
> hand that the type is contained in some closed set. Thus, yes, you have 
> to provide a common supertype.

Hmmm, okay, that explains why you wrote the code in 
TcpSerializerComb.scala the way you did.  Once things don't have a 
common ancestor, there is very little reason to bend things to match the 
data combinator.

It's not that I necessarily want to switch between Strings and ints, but 
I was kinda expecting that the data combinator was a bit more "magical" 
than it seems to be.

The fault is clearly in my understanding.  I'm a little slow at 
unwinding all of the typing and anonymous function implications.

Thanks for the explanation,
-a
Permalink | Reply |

Gmane