Ben Gamari | 30 May 06:27 2012
Picon

Unboxed Vectors of newtype'd values

One of the reasons I find Haskell so well suited to my work is its
ability to easily work with large quantities of data. In particular, I
find myself using Data.Vector.Unbox quite frequently.

Another of my reasons for using Haskell is the type safety it
provides. I find myself using newtypes very frequently to force myself
to think about invariants that a more weakly typed language would allow
me to simply ignore.

Sadly, these two features don't interact particularly well. While the
Data.Vector.Unbox documentation claims that "Implementing unboxed
vectors for new data types can be very easy", it then goes on to list an
abridged version of the Complex instance---dozens of lines of
code. While this code certainly isn't difficult to write, it is time
consuming, error-prone, and, above else, utterly mind deadeningly dull
(making it quite uncharacteristic for Haskell). So dull that I generally
avoid newtypes at all cost in code that might need to use unboxed
vectors. This boilerplate is largely due to Vector's use of type
families as this precludes the use of (the otherwise quite cunning)
GeneralizedNewtypeDeriving to automatically derive the necessary
instances.

What can be done to fix this unfortunate state of affairs? The obvious
solution here seems to be Template Haskell, but this seems a bit of an
unfortunate hack around what might be a deficiency in the type families
mechanism (or at least this application of it). The newtype package
provides a nice mechanism to pack and unpack newtypes, but providing
blanket Unbox instances for Newtype instances seems like an awful idea
(and, frankly, I'm not sure how this would work with type
families). Other than these two possibilities I am at a loss. Thoughts?
(Continue reading)

Jake McArthur | 30 May 14:19 2012
Picon

Re: Unboxed Vectors of newtype'd values

I don't have a proposal, but I'd like to echo my support for thinking about this problem. It is very annoying to use my own types with unboxed vectors.

On May 30, 2012 12:28 AM, "Ben Gamari" <bgamari.foss <at> gmail.com> wrote:
One of the reasons I find Haskell so well suited to my work is its
ability to easily work with large quantities of data. In particular, I
find myself using Data.Vector.Unbox quite frequently.

Another of my reasons for using Haskell is the type safety it
provides. I find myself using newtypes very frequently to force myself
to think about invariants that a more weakly typed language would allow
me to simply ignore.

Sadly, these two features don't interact particularly well. While the
Data.Vector.Unbox documentation claims that "Implementing unboxed
vectors for new data types can be very easy", it then goes on to list an
abridged version of the Complex instance---dozens of lines of
code. While this code certainly isn't difficult to write, it is time
consuming, error-prone, and, above else, utterly mind deadeningly dull
(making it quite uncharacteristic for Haskell). So dull that I generally
avoid newtypes at all cost in code that might need to use unboxed
vectors. This boilerplate is largely due to Vector's use of type
families as this precludes the use of (the otherwise quite cunning)
GeneralizedNewtypeDeriving to automatically derive the necessary
instances.

What can be done to fix this unfortunate state of affairs? The obvious
solution here seems to be Template Haskell, but this seems a bit of an
unfortunate hack around what might be a deficiency in the type families
mechanism (or at least this application of it). The newtype package
provides a nice mechanism to pack and unpack newtypes, but providing
blanket Unbox instances for Newtype instances seems like an awful idea
(and, frankly, I'm not sure how this would work with type
families). Other than these two possibilities I am at a loss. Thoughts?
I'd appreciate any ideas folks could offer.

Cheers,

- Ben


_______________________________________________
Libraries mailing list
Libraries <at> haskell.org
http://www.haskell.org/mailman/listinfo/libraries
_______________________________________________
Libraries mailing list
Libraries <at> haskell.org
http://www.haskell.org/mailman/listinfo/libraries
Bryan O'Sullivan | 30 May 18:50 2012

Re: Unboxed Vectors of newtype'd values

On Wed, May 30, 2012 at 5:19 AM, Jake McArthur <jake.mcarthur <at> gmail.com> wrote:

I don't have a proposal, but I'd like to echo my support for thinking about this problem. It is very annoying to use my own types with unboxed vectors.

Likewise, if anyone is listening :-) 
_______________________________________________
Libraries mailing list
Libraries <at> haskell.org
http://www.haskell.org/mailman/listinfo/libraries
Johan Tibell | 30 May 18:57 2012
Picon

Re: Unboxed Vectors of newtype'd values

On Wed, May 30, 2012 at 9:50 AM, Bryan O'Sullivan <bos <at> serpentine.com> wrote:
> Likewise, if anyone is listening :-)

Here too.

I'd like some compiler support for abstracting over unboxed values in
general and e.g. generate Unbox instances automatically. Then we could
have other unboxed structures than Vector.
Simon Peyton-Jones | 4 Jun 12:43 2012
Picon

RE: Unboxed Vectors of newtype'd values

| I'd like some compiler support for abstracting over unboxed values in
| general and e.g. generate Unbox instances automatically. Then we could
| have other unboxed structures than Vector.

Could someone post an example or two of the problem being described here?

Simon

| -----Original Message-----
| From: libraries-bounces <at> haskell.org [mailto:libraries-
| bounces <at> haskell.org] On Behalf Of Johan Tibell
| Sent: 30 May 2012 17:57
| To: Bryan O'Sullivan
| Cc: Jake McArthur; libraries <at> haskell.org
| Subject: Re: Unboxed Vectors of newtype'd values
| 
| On Wed, May 30, 2012 at 9:50 AM, Bryan O'Sullivan <bos <at> serpentine.com>
| wrote:
| > Likewise, if anyone is listening :-)
| 
| Here too.
| 
| I'd like some compiler support for abstracting over unboxed values in
| general and e.g. generate Unbox instances automatically. Then we could
| have other unboxed structures than Vector.
| 
| _______________________________________________
| Libraries mailing list
| Libraries <at> haskell.org
| http://www.haskell.org/mailman/listinfo/libraries
Bas van Dijk | 4 Jun 21:40 2012
Picon

Re: Unboxed Vectors of newtype'd values

On 4 June 2012 12:43, Simon Peyton-Jones <simonpj <at> microsoft.com> wrote:
> | I'd like some compiler support for abstracting over unboxed values in
> | general and e.g. generate Unbox instances automatically. Then we could
> | have other unboxed structures than Vector.
>
> Could someone post an example or two of the problem being described here?

The following is some boring code I wrote a while back at work for
storing UUIDs[1] in unboxed vectors:

{-# LANGUAGE MultiParamTypeClasses, TypeFamilies #-}

import Control.Monad (liftM)
import Data.Word (Word32)

import Data.UUID (UUID)
import qualified Data.UUID as UUID

import qualified Data.Vector.Unboxed         as VU
import qualified Data.Vector.Generic         as VG
import qualified Data.Vector.Generic.Mutable as VGM

newtype instance VU.MVector s UUID =
	MV_UUID (VU.MVector s (Word32, Word32, Word32, Word32))

newtype instance  VU.Vector UUID =
	V_UUID (VU.Vector (Word32, Word32, Word32, Word32))

instance VU.Unbox UUID

instance VGM.MVector VU.MVector UUID where
  {-# INLINE basicLength #-}
  {-# INLINE basicUnsafeSlice #-}
  {-# INLINE basicOverlaps #-}
  {-# INLINE basicUnsafeNew #-}
  {-# INLINE basicUnsafeReplicate #-}
  {-# INLINE basicUnsafeRead #-}
  {-# INLINE basicUnsafeWrite #-}
  {-# INLINE basicClear #-}
  {-# INLINE basicSet #-}
  {-# INLINE basicUnsafeCopy #-}
  {-# INLINE basicUnsafeGrow #-}
  basicLength (MV_UUID v) =
    VGM.basicLength v

  basicUnsafeSlice i n (MV_UUID v) =
    MV_UUID $ VGM.basicUnsafeSlice i n v

  basicOverlaps (MV_UUID v1) (MV_UUID v2) =
    VGM.basicOverlaps v1 v2

  basicUnsafeNew n =
    MV_UUID `liftM` VGM.basicUnsafeNew n

  basicUnsafeReplicate n uuid =
    MV_UUID `liftM` VGM.basicUnsafeReplicate n (UUID.toWords uuid)

  basicUnsafeRead (MV_UUID v) i =
    fromQuadruple `liftM` VGM.basicUnsafeRead v i

  basicUnsafeWrite (MV_UUID v) i uuid =
    VGM.basicUnsafeWrite v i (UUID.toWords uuid)

  basicClear (MV_UUID v) =
    VGM.basicClear v

  basicSet (MV_UUID v) uuid =
    VGM.basicSet v (UUID.toWords uuid)

  basicUnsafeCopy (MV_UUID v1) (MV_UUID v2) =
    VGM.basicUnsafeCopy v1 v2

  basicUnsafeMove (MV_UUID v1) (MV_UUID v2) =
    VGM.basicUnsafeMove v1 v2

  basicUnsafeGrow (MV_UUID v) n =
    MV_UUID `liftM` VGM.basicUnsafeGrow v n

instance VG.Vector VU.Vector UUID where
  {-# INLINE basicUnsafeFreeze #-}
  {-# INLINE basicUnsafeThaw #-}
  {-# INLINE basicLength #-}
  {-# INLINE basicUnsafeSlice #-}
  {-# INLINE basicUnsafeIndexM #-}
  {-# INLINE elemseq #-}
  basicUnsafeFreeze (MV_UUID v) =
    V_UUID `liftM` VG.basicUnsafeFreeze v

  basicUnsafeThaw (V_UUID v) =
    MV_UUID `liftM` VG.basicUnsafeThaw v

  basicLength (V_UUID v) =
    VG.basicLength v

  basicUnsafeSlice i n (V_UUID v) =
    V_UUID $ VG.basicUnsafeSlice i n v

  basicUnsafeIndexM (V_UUID v) i =
    fromQuadruple `liftM` VG.basicUnsafeIndexM v i

  basicUnsafeCopy (MV_UUID mv) (V_UUID v) =
    VG.basicUnsafeCopy mv v

  elemseq _ uuid z = VG.elemseq (undefined :: VU.Vector a) a
                   $ VG.elemseq (undefined :: VU.Vector a) b
                   $ VG.elemseq (undefined :: VU.Vector a) c
                   $ VG.elemseq (undefined :: VU.Vector a) d z
      where
        (a,b,c,d) = UUID.toWords uuid

fromQuadruple :: (Word32, Word32, Word32, Word32) -> UUID
fromQuadruple (a,b,c,d) = UUID.fromWords a b c d

Regards,

Bas

[1] http://hackage.haskell.org/packages/archive/uuid/1.2.5/doc/html/Data-UUID.html
Johan Tibell | 4 Jun 21:57 2012
Picon

Re: Unboxed Vectors of newtype'd values

My desire probably doesn't overlap much with what others have
described in this thread, apologies. I *think* they want generalized
newtype deriving to work for Unbox instances.

I want to be able to write definition like this (pseudo code):

  data UnpackedList a = forall a. Unbox a => Cons {-# UNPACK #-} !a
(UnpackedList a) | Nil

and have GHC generate appropriate data type definitions at call sites.
In other words, I want polymorphic unpacking to work as long as the
unpacked field is unboxable (i.e. is member of some Unobx type class.)

I believe I've described this desire to you before. Unfortunately it
seems like a really hard problem.

-- Johan
Simon Peyton-Jones | 5 Jun 09:06 2012
Picon

RE: Unboxed Vectors of newtype'd values

I'm sorry I'm still struggling.  In your example below you bind 'a' twice, once with the "data UnpackedList
a" and once with the "forall a".  Did you intend an existential?

Also I don't know what you mean by "generate appropriate type definitions at call sites".

I do remember a dinner conversation in Tokyo where I couldn't figure out how to implement what you wanted,
but I've forgotten the details.  Maybe it's worth a wiki page to explain -- or maybe not if its un-implementable.

Anyway thanks for distinguishing the two threads.  If you can clarify the generalised newtype deriving
problem too that would be great.

Simon

| -----Original Message-----
| From: Johan Tibell [mailto:johan.tibell <at> gmail.com]
| Sent: 04 June 2012 20:58
| To: Bas van Dijk
| Cc: Simon Peyton-Jones; Bryan O'Sullivan; Jake McArthur;
| libraries <at> haskell.org
| Subject: Re: Unboxed Vectors of newtype'd values
| 
| My desire probably doesn't overlap much with what others have described
| in this thread, apologies. I *think* they want generalized newtype
| deriving to work for Unbox instances.
| 
| I want to be able to write definition like this (pseudo code):
| 
|   data UnpackedList a = forall a. Unbox a => Cons {-# UNPACK #-} !a
| (UnpackedList a) | Nil
| 
| and have GHC generate appropriate data type definitions at call sites.
| In other words, I want polymorphic unpacking to work as long as the
| unpacked field is unboxable (i.e. is member of some Unobx type class.)
| 
| I believe I've described this desire to you before. Unfortunately it
| seems like a really hard problem.
| 
| -- Johan
Johan Tibell | 5 Jun 18:32 2012
Picon

Re: Unboxed Vectors of newtype'd values

On Tue, Jun 5, 2012 at 12:06 AM, Simon Peyton-Jones
<simonpj <at> microsoft.com> wrote:
> I'm sorry I'm still struggling.  In your example below you bind 'a' twice, once with the "data
UnpackedList a" and once with the "forall a".  Did you intend an existential?

I didn't intend the existential. I haven't made any real progress on
my unpacking problem since we last talked so lets let it rest for now.
I'll try to find some time to write down a wiki page so we at least
know where we got stuck last time. Perhaps someone reading it will
have an aha moment that will let us move forward.

> Also I don't know what you mean by "generate appropriate type definitions at call sites".

If I write:

    module A

    data UnpackedList a = Unbox a => Cons {-# UNPACK #-} !a
(UnpackedList a) | Nil
    { -# INLINABLE_DATA UnpackedList #-}  -- Like INLINABLE, but for data types

    module B

    f :: UnpackedList Int -> ...

I'd like GHC to generate

    data UnpackedListInt = ConsInt {-# UNPACK #-} !Int UnpackedListInt | NilInt

and a specialized version of f, using that type, in B. It's a bit like
the call-site specialization afforded by the INLINABLE pragma, but
applied also to data type definitions.

You might protest and say that I could just just use type families,
but they don't work well if my type has more than one type parameter
(e.g. Map), as I need to write O(n^2) instances:

instance Map Int Int where
    type Map = MapIntInt ...{-# UNPACK #-} !Int {-# UNPACK #-} !Int...

instance Map Int Char where
    type Map = MapIntChar ...

instance Map Int (Int, Int) where
   type Map = MapIntIntInt ...

That's 50,625 instances to just cover the basic types (e.g. Int, Word,
etc) and pairs. The reason you'd want call-site generation of data
types (ala C++ templates) is that a real program would only use no
more than 100 (for a big program) of the potentially infinite number
of combinations.

> Anyway thanks for distinguishing the two threads.  If you can clarify the generalised newtype deriving
problem too that would be great.

I'll leave it to others. I'm not quite sure what the issue is.

-- Johan
Simon Peyton-Jones | 5 Jun 09:02 2012
Picon

RE: Unboxed Vectors of newtype'd values

I see the boring code, but I don't see what you *want*!  I'm guessing you want "generalised newtype deriving"
but why does that not work?

S

| -----Original Message-----
| From: Bas van Dijk [mailto:v.dijk.bas <at> gmail.com]
| Sent: 04 June 2012 20:41
| To: Simon Peyton-Jones
| Cc: Johan Tibell; Bryan O'Sullivan; Jake McArthur; libraries <at> haskell.org
| Subject: Re: Unboxed Vectors of newtype'd values
| 
| On 4 June 2012 12:43, Simon Peyton-Jones <simonpj <at> microsoft.com> wrote:
| > | I'd like some compiler support for abstracting over unboxed values
| > | in general and e.g. generate Unbox instances automatically. Then we
| > | could have other unboxed structures than Vector.
| >
| > Could someone post an example or two of the problem being described
| here?
| 
| The following is some boring code I wrote a while back at work for
| storing UUIDs[1] in unboxed vectors:
| 
| {-# LANGUAGE MultiParamTypeClasses, TypeFamilies #-}
| 
| import Control.Monad (liftM)
| import Data.Word (Word32)
| 
| import Data.UUID (UUID)
| import qualified Data.UUID as UUID
| 
| import qualified Data.Vector.Unboxed         as VU
| import qualified Data.Vector.Generic         as VG
| import qualified Data.Vector.Generic.Mutable as VGM
| 
| newtype instance VU.MVector s UUID =
| 	MV_UUID (VU.MVector s (Word32, Word32, Word32, Word32))
| 
| newtype instance  VU.Vector UUID =
| 	V_UUID (VU.Vector (Word32, Word32, Word32, Word32))
| 
| instance VU.Unbox UUID
| 
| instance VGM.MVector VU.MVector UUID where
|   {-# INLINE basicLength #-}
|   {-# INLINE basicUnsafeSlice #-}
|   {-# INLINE basicOverlaps #-}
|   {-# INLINE basicUnsafeNew #-}
|   {-# INLINE basicUnsafeReplicate #-}
|   {-# INLINE basicUnsafeRead #-}
|   {-# INLINE basicUnsafeWrite #-}
|   {-# INLINE basicClear #-}
|   {-# INLINE basicSet #-}
|   {-# INLINE basicUnsafeCopy #-}
|   {-# INLINE basicUnsafeGrow #-}
|   basicLength (MV_UUID v) =
|     VGM.basicLength v
| 
|   basicUnsafeSlice i n (MV_UUID v) =
|     MV_UUID $ VGM.basicUnsafeSlice i n v
| 
|   basicOverlaps (MV_UUID v1) (MV_UUID v2) =
|     VGM.basicOverlaps v1 v2
| 
|   basicUnsafeNew n =
|     MV_UUID `liftM` VGM.basicUnsafeNew n
| 
|   basicUnsafeReplicate n uuid =
|     MV_UUID `liftM` VGM.basicUnsafeReplicate n (UUID.toWords uuid)
| 
|   basicUnsafeRead (MV_UUID v) i =
|     fromQuadruple `liftM` VGM.basicUnsafeRead v i
| 
|   basicUnsafeWrite (MV_UUID v) i uuid =
|     VGM.basicUnsafeWrite v i (UUID.toWords uuid)
| 
|   basicClear (MV_UUID v) =
|     VGM.basicClear v
| 
|   basicSet (MV_UUID v) uuid =
|     VGM.basicSet v (UUID.toWords uuid)
| 
|   basicUnsafeCopy (MV_UUID v1) (MV_UUID v2) =
|     VGM.basicUnsafeCopy v1 v2
| 
|   basicUnsafeMove (MV_UUID v1) (MV_UUID v2) =
|     VGM.basicUnsafeMove v1 v2
| 
|   basicUnsafeGrow (MV_UUID v) n =
|     MV_UUID `liftM` VGM.basicUnsafeGrow v n
| 
| instance VG.Vector VU.Vector UUID where
|   {-# INLINE basicUnsafeFreeze #-}
|   {-# INLINE basicUnsafeThaw #-}
|   {-# INLINE basicLength #-}
|   {-# INLINE basicUnsafeSlice #-}
|   {-# INLINE basicUnsafeIndexM #-}
|   {-# INLINE elemseq #-}
|   basicUnsafeFreeze (MV_UUID v) =
|     V_UUID `liftM` VG.basicUnsafeFreeze v
| 
|   basicUnsafeThaw (V_UUID v) =
|     MV_UUID `liftM` VG.basicUnsafeThaw v
| 
|   basicLength (V_UUID v) =
|     VG.basicLength v
| 
|   basicUnsafeSlice i n (V_UUID v) =
|     V_UUID $ VG.basicUnsafeSlice i n v
| 
|   basicUnsafeIndexM (V_UUID v) i =
|     fromQuadruple `liftM` VG.basicUnsafeIndexM v i
| 
|   basicUnsafeCopy (MV_UUID mv) (V_UUID v) =
|     VG.basicUnsafeCopy mv v
| 
|   elemseq _ uuid z = VG.elemseq (undefined :: VU.Vector a) a
|                    $ VG.elemseq (undefined :: VU.Vector a) b
|                    $ VG.elemseq (undefined :: VU.Vector a) c
|                    $ VG.elemseq (undefined :: VU.Vector a) d z
|       where
|         (a,b,c,d) = UUID.toWords uuid
| 
| fromQuadruple :: (Word32, Word32, Word32, Word32) -> UUID fromQuadruple
| (a,b,c,d) = UUID.fromWords a b c d
| 
| Regards,
| 
| Bas
| 
| [1]
| http://hackage.haskell.org/packages/archive/uuid/1.2.5/doc/html/Data-
| UUID.html
Bryan O'Sullivan | 5 Jun 19:56 2012

Re: Unboxed Vectors of newtype'd values

On Tue, Jun 5, 2012 at 12:02 AM, Simon Peyton-Jones <simonpj <at> microsoft.com> wrote:
I see the boring code, but I don't see what you *want*!  I'm guessing you want "generalised newtype deriving" but why does that not work?

Yes, I (and I assume Bas) want generalised newtype deriving to work, but it doesn't.

I want to write something very simple:

{-# LANGUAGE GeneralizedNewtypeDeriving #-}
newtype Foo = Foo Int
    deriving (Eq, Show, Unbox)

But with the above, GHC says:

    No instances for (M.MVector MVector Foo, G.Vector Vector Foo)
      arising from the 'deriving' clause of a data type declaration

And then we begin the journey that eventually gets us to Bas's rather wordy code.
_______________________________________________
Libraries mailing list
Libraries <at> haskell.org
http://www.haskell.org/mailman/listinfo/libraries
Andres Löh | 5 Jun 21:54 2012

Re: Unboxed Vectors of newtype'd values

Hi Bryan.

> Yes, I (and I assume Bas) want generalised newtype deriving to work, but it
> doesn't.
>
> I want to write something very simple:
>
> {-# LANGUAGE GeneralizedNewtypeDeriving #-}
> newtype Foo = Foo Int
>     deriving (Eq, Show, Unbox)
>
> But with the above, GHC says:
>
>     No instances for (M.MVector MVector Foo, G.Vector Vector Foo)
>       arising from the 'deriving' clause of a data type declaration

Yes, because these are superclasses of Unbox. So can't you simply say this:

> newtype Foo = Foo Int
>   deriving (Eq, Show, Unbox, M.MVector MVector, G.Vector Vector)

?

Cheers,
  Andres

--

-- 
Andres Löh, Haskell Consultant
Well-Typed LLP, http://www.well-typed.com
Daniel Peebles | 5 Jun 21:56 2012
Picon

Re: Unboxed Vectors of newtype'd values

Yep, that works! Sweet!

On Tue, Jun 5, 2012 at 3:54 PM, Andres Löh <andres <at> well-typed.com> wrote:
Hi Bryan.

> Yes, I (and I assume Bas) want generalised newtype deriving to work, but it
> doesn't.
>
> I want to write something very simple:
>
> {-# LANGUAGE GeneralizedNewtypeDeriving #-}
> newtype Foo = Foo Int
>     deriving (Eq, Show, Unbox)
>
> But with the above, GHC says:
>
>     No instances for (M.MVector MVector Foo, G.Vector Vector Foo)
>       arising from the 'deriving' clause of a data type declaration

Yes, because these are superclasses of Unbox. So can't you simply say this:

> newtype Foo = Foo Int
>   deriving (Eq, Show, Unbox, M.MVector MVector, G.Vector Vector)

?

Cheers,
 Andres

--
Andres Löh, Haskell Consultant
Well-Typed LLP, http://www.well-typed.com

_______________________________________________
Libraries mailing list
Libraries <at> haskell.org
http://www.haskell.org/mailman/listinfo/libraries

_______________________________________________
Libraries mailing list
Libraries <at> haskell.org
http://www.haskell.org/mailman/listinfo/libraries
Ben Gamari | 6 Jun 12:05 2012
Picon

Re: Unboxed Vectors of newtype'd values

Andres Löh <andres <at> well-typed.com> writes:

> Hi Bryan.
>
> Yes, because these are superclasses of Unbox. So can't you simply say this:
>
>> newtype Foo = Foo Int
>>   deriving (Eq, Show, Unbox, M.MVector MVector, G.Vector Vector)
>
Wonderful! I suppose that should have been clear from the error. Thanks!

Cheers,

- Ben

_______________________________________________
Libraries mailing list
Libraries <at> haskell.org
http://www.haskell.org/mailman/listinfo/libraries
Roman Leshchinskiy | 6 Jun 12:44 2012
Picon
Picon

Re: Unboxed Vectors of newtype'd values

Andres Löh wrote:
>
> Yes, because these are superclasses of Unbox. So can't you simply say
> this:
>
>> newtype Foo = Foo Int
>>   deriving (Eq, Show, Unbox, M.MVector MVector, G.Vector Vector)

I don't think this should work. This is the basic set up:

class G.Vector v a where
  basicLength :: v a -> Int
  ...

data family Vector a

newtype instance Vector Int = V_Int ...

instance G.Vector Vector Int where
  basicLength (V_Int ...) = ...
  ...

I don't understand what GeneralizedNewtypeDeriving does with
deriving(G.Vector Vector) in this situation but it can't possibly do
anything sensible without a data instance Vector Foo. The fact that it
doesn't fail is just a bug, IMO.

Roman
Andres Löh | 6 Jun 12:58 2012

Re: Unboxed Vectors of newtype'd values

>> Yes, because these are superclasses of Unbox. So can't you simply say
>> this:
>>
>>> newtype Foo = Foo Int
>>>   deriving (Eq, Show, Unbox, M.MVector MVector, G.Vector Vector)
>
> I don't think this should work. This is the basic set up:

[...]

> I don't understand what GeneralizedNewtypeDeriving does with
> deriving(G.Vector Vector) in this situation but it can't possibly do
> anything sensible without a data instance Vector Foo. The fact that it
> doesn't fail is just a bug, IMO.

I've actually started wondering the same after I posted this. I tried
to use -ddump-deriv to get at the code, but GHC seems to be cheating
and doesn't show what it generates. I guess I understand
newtype-deriving less than I thought ...

Cheers,
  Andres

--

-- 
Andres Löh, Haskell Consultant
Well-Typed LLP, http://www.well-typed.com
Roman Leshchinskiy | 6 Jun 13:13 2012
Picon
Picon

Re: Unboxed Vectors of newtype'd values

Andres Löh wrote:
>>> Yes, because these are superclasses of Unbox. So can't you simply say
>>> this:
>>>
>>>> newtype Foo = Foo Int
>>>>   deriving (Eq, Show, Unbox, M.MVector MVector, G.Vector Vector)
>>
>> I don't think this should work. This is the basic set up:
>
> [...]
>
>> I don't understand what GeneralizedNewtypeDeriving does with
>> deriving(G.Vector Vector) in this situation but it can't possibly do
>> anything sensible without a data instance Vector Foo. The fact that it
>> doesn't fail is just a bug, IMO.
>
> I've actually started wondering the same after I posted this. I tried
> to use -ddump-deriv to get at the code, but GHC seems to be cheating
> and doesn't show what it generates. I guess I understand
> newtype-deriving less than I thought ...

I don't really understand it, either, and try to stay well away from it
:-) It simply doesn't seem to work with GADTs, type families etc. I've
created a ticket: http://hackage.haskell.org/trac/ghc/ticket/6147.

Roman
Simon Peyton-Jones | 5 Jun 22:45 2012
Picon

RE: Unboxed Vectors of newtype'd values

So Andres has explained how to do what Johan asks.  Does that mean that Bas’s problem is solved?

 

S

 

From: Bryan O'Sullivan [mailto:bos <at> serpentine.com]
Sent: 05 June 2012 18:57
To: Simon Peyton-Jones
Cc: Bas van Dijk; Johan Tibell; Jake McArthur; libraries <at> haskell.org
Subject: Re: Unboxed Vectors of newtype'd values

 

On Tue, Jun 5, 2012 at 12:02 AM, Simon Peyton-Jones <simonpj <at> microsoft.com> wrote:

I see the boring code, but I don't see what you *want*!  I'm guessing you want "generalised newtype deriving" but why does that not work?

 

Yes, I (and I assume Bas) want generalised newtype deriving to work, but it doesn't.

 

I want to write something very simple:

 

{-# LANGUAGE GeneralizedNewtypeDeriving #-}

newtype Foo = Foo Int

    deriving (Eq, Show, Unbox)

 

But with the above, GHC says:

 

    No instances for (M.MVector MVector Foo, G.Vector Vector Foo)

      arising from the 'deriving' clause of a data type declaration

 

And then we begin the journey that eventually gets us to Bas's rather wordy code.

_______________________________________________
Libraries mailing list
Libraries <at> haskell.org
http://www.haskell.org/mailman/listinfo/libraries
Bryan O'Sullivan | 5 Jun 22:46 2012

Re: Unboxed Vectors of newtype'd values

On Tue, Jun 5, 2012 at 1:45 PM, Simon Peyton-Jones <simonpj <at> microsoft.com> wrote:

So Andres has explained how to do what Johan asks.  Does that mean that Bas’s problem is solved?


I think so - thanks, Andres! I believe that Johan's problem is another beast entirely. 
_______________________________________________
Libraries mailing list
Libraries <at> haskell.org
http://www.haskell.org/mailman/listinfo/libraries
Johan Tibell | 5 Jun 22:49 2012
Picon

Re: Unboxed Vectors of newtype'd values

On Tue, Jun 5, 2012 at 1:46 PM, Bryan O'Sullivan <bos <at> serpentine.com> wrote:
> I think so - thanks, Andres! I believe that Johan's problem is another beast
> entirely.

Just to make it clear why I butted into this thread to begin with
(apologies for that.)

I also want a deriving Unbox mechanism, but with a more magic Unbox
class (e.g. one that knew enough about the representation of a data
type to unpack it.) That was the connection.

-- Johan
Bas van Dijk | 6 Jun 00:46 2012
Picon

Re: Unboxed Vectors of newtype'd values

On 5 June 2012 22:45, Simon Peyton-Jones <simonpj <at> microsoft.com> wrote:
> So Andres has explained how to do what Johan asks.  Does that mean that
> Bas’s problem is solved?

My problem can't be solved with GeneralizedNewtypeDeriving since a
UUID is not a newtype but an abstract data type defined in another
package[1].

However, the template-haskell from the vector-th-unbox package which
Reiner mentioned does the job perfectly:

derivingUnbox "UUID"
    [d| instance Unbox' UUID (Word32, Word32, Word32, Word32) |]
    [| \ uuid -> UUID.toWords uuid|]
    [| \ (a,b,c,d) -> UUID.fromWords a b c d |]

Thanks Reiner for pointing me to this package!

Cheers,

Bas

[1] http://hackage.haskell.org/packages/archive/uuid/latest/doc/html/src/Data-UUID-Internal.html#UUID

_______________________________________________
Libraries mailing list
Libraries <at> haskell.org
http://www.haskell.org/mailman/listinfo/libraries
Simon Peyton-Jones | 6 Jun 09:59 2012
Picon

RE: Unboxed Vectors of newtype'd values

| > So Andres has explained how to do what Johan asks.  Does that mean
| > that Bas’s problem is solved?
| 
| My problem can't be solved with GeneralizedNewtypeDeriving since a UUID
| is not a newtype but an abstract data type defined in another
| package[1].
| 
| However, the template-haskell from the vector-th-unbox package which
| Reiner mentioned does the job perfectly:
| 
| derivingUnbox "UUID"
|     [d| instance Unbox' UUID (Word32, Word32, Word32, Word32) |]
|     [| \ uuid -> UUID.toWords uuid|]
|     [| \ (a,b,c,d) -> UUID.fromWords a b c d |]

Ah, well, that is another matter.

GeneralisedNewtypeDeriving respects the abstraction boundaries imposed by the package author.  I think
Template Haskell does not.  It probably should; see [1].

So it seems that 
 * generalised newtype deriving will do your job, if you
   can persuade the package author to expose the representation
 * Template Haskell can do the job, but only because of a bug (arguably)

So this is really a library design question, not a language design one.  If a package author makes a type
abstract, he's saying that he does not want you to look at the representation; and yet you must if you want to
unbox it. So you must negotiate with the package author.  I don't see how the language can (a) respect
abstractions and (b) let you retrospectively unbox abstract types.


[1] http://stackoverflow.com/questions/10857030/whats-so-bad-about-template-haskell

_______________________________________________
Libraries mailing list
Libraries <at> haskell.org
http://www.haskell.org/mailman/listinfo/libraries
Bas van Dijk | 6 Jun 10:33 2012
Picon

Re: Unboxed Vectors of newtype'd values

On 6 June 2012 09:59, Simon Peyton-Jones <simonpj <at> microsoft.com> wrote:
>  * generalised newtype deriving will do your job, if you
>   can persuade the package author to expose the representation

Just exposing the representation is not enough to get
GeneralizedNewtypeDeriving to work since UUIDs are not newtypes but
datatypes. The only way to get it to work is to persuade the author to
expose the representation _and_ turn it into a newtype. But a newtype
of what? newtype UUID = UUID (Word32,Word32,Word32,Word32) is not
better than the current fully unboxed representation.

>  * Template Haskell can do the job, but only because of a bug (arguably)

I don't think vector-th-unbox is exploiting a bug. It doesn't need to
inspect the private representation of a UUID. The only thing that it
needs is a type that is isomorphic to a UUID and conversion functions
between them.

> So this is really a library design question, not a language design one.  If a package author makes a type
abstract, he's saying that he does not want you to look at the representation; and yet you must if you want to
unbox it. So you must negotiate with the package author.  I don't see how the language can (a) respect
abstractions and (b) let you retrospectively unbox abstract types.

Agreed.

Regards,

Bas

_______________________________________________
Libraries mailing list
Libraries <at> haskell.org
http://www.haskell.org/mailman/listinfo/libraries
Roman Leshchinskiy | 30 May 21:41 2012
Picon
Picon

Re: Unboxed Vectors of newtype'd values

On 30/05/2012, at 05:27, Ben Gamari wrote:

> Sadly, these two features don't interact particularly well. While the
> Data.Vector.Unbox documentation claims that "Implementing unboxed
> vectors for new data types can be very easy", it then goes on to list an
> abridged version of the Complex instance---dozens of lines of
> code. While this code certainly isn't difficult to write, it is time
> consuming, error-prone, and, above else, utterly mind deadeningly dull
> (making it quite uncharacteristic for Haskell). So dull that I generally
> avoid newtypes at all cost in code that might need to use unboxed
> vectors. This boilerplate is largely due to Vector's use of type
> families as this precludes the use of (the otherwise quite cunning)
> GeneralizedNewtypeDeriving to automatically derive the necessary
> instances.

I completely agree that the current situation is less than ideal but don't really have a good solution.
Frankly, the easiest way of defining Unbox instances for newtypes is probably a preprocessor macro. I use
a lot of those in vector to reduce boilerplate and they work well. I'll provide one for newtypes in the next
release. 

As you say, TH is another possibility but one I'm rather wary of.

Roman
Daniel Peebles | 31 May 07:03 2012
Picon

Re: Unboxed Vectors of newtype'd values

It seems like the most elegant long-term solution here is to make GeneralizedNewtypeDeriving more cunning. Is there some fundamental reason it can't work with the type families involved?


But in the short term, I agree and TH is probably best.

-Dan

On Wed, May 30, 2012 at 3:41 PM, Roman Leshchinskiy <rl <at> cse.unsw.edu.au> wrote:
On 30/05/2012, at 05:27, Ben Gamari wrote:

> Sadly, these two features don't interact particularly well. While the
> Data.Vector.Unbox documentation claims that "Implementing unboxed
> vectors for new data types can be very easy", it then goes on to list an
> abridged version of the Complex instance---dozens of lines of
> code. While this code certainly isn't difficult to write, it is time
> consuming, error-prone, and, above else, utterly mind deadeningly dull
> (making it quite uncharacteristic for Haskell). So dull that I generally
> avoid newtypes at all cost in code that might need to use unboxed
> vectors. This boilerplate is largely due to Vector's use of type
> families as this precludes the use of (the otherwise quite cunning)
> GeneralizedNewtypeDeriving to automatically derive the necessary
> instances.


I completely agree that the current situation is less than ideal but don't really have a good solution. Frankly, the easiest way of defining Unbox instances for newtypes is probably a preprocessor macro. I use a lot of those in vector to reduce boilerplate and they work well. I'll provide one for newtypes in the next release.

As you say, TH is another possibility but one I'm rather wary of.

Roman



_______________________________________________
Libraries mailing list
Libraries <at> haskell.org
http://www.haskell.org/mailman/listinfo/libraries

_______________________________________________
Libraries mailing list
Libraries <at> haskell.org
http://www.haskell.org/mailman/listinfo/libraries
Reiner Pope | 31 May 18:17 2012
Picon

Re: Unboxed Vectors of newtype'd values

In case anyone missed it, Liyang HU wrote a TH deriver for Unbox instances[1]. 

Also, I have tried using GHC's new generic deriving[2] to derive instances - this is almost possible, but is blocked by a GHC issue[3]. IMHO generic deriving would be the best solution (once it works), because it works for product types as well as newtypes, and requires no extra GHC assistance or TH.


Cheers,
Reiner

On 30 May 2012 22:03, Daniel Peebles <pumpkingod <at> gmail.com> wrote:
It seems like the most elegant long-term solution here is to make GeneralizedNewtypeDeriving more cunning. Is there some fundamental reason it can't work with the type families involved?

But in the short term, I agree and TH is probably best.

-Dan


On Wed, May 30, 2012 at 3:41 PM, Roman Leshchinskiy <rl <at> cse.unsw.edu.au> wrote:
On 30/05/2012, at 05:27, Ben Gamari wrote:

> Sadly, these two features don't interact particularly well. While the
> Data.Vector.Unbox documentation claims that "Implementing unboxed
> vectors for new data types can be very easy", it then goes on to list an
> abridged version of the Complex instance---dozens of lines of
> code. While this code certainly isn't difficult to write, it is time
> consuming, error-prone, and, above else, utterly mind deadeningly dull
> (making it quite uncharacteristic for Haskell). So dull that I generally
> avoid newtypes at all cost in code that might need to use unboxed
> vectors. This boilerplate is largely due to Vector's use of type
> families as this precludes the use of (the otherwise quite cunning)
> GeneralizedNewtypeDeriving to automatically derive the necessary
> instances.


I completely agree that the current situation is less than ideal but don't really have a good solution. Frankly, the easiest way of defining Unbox instances for newtypes is probably a preprocessor macro. I use a lot of those in vector to reduce boilerplate and they work well. I'll provide one for newtypes in the next release.

As you say, TH is another possibility but one I'm rather wary of.

Roman



_______________________________________________
Libraries mailing list
Libraries <at> haskell.org
http://www.haskell.org/mailman/listinfo/libraries


_______________________________________________
Libraries mailing list
Libraries <at> haskell.org
http://www.haskell.org/mailman/listinfo/libraries


_______________________________________________
Libraries mailing list
Libraries <at> haskell.org
http://www.haskell.org/mailman/listinfo/libraries
Ben Gamari | 31 May 19:16 2012
Picon

Re: Unboxed Vectors of newtype'd values

Reiner Pope <reiner.pope <at> gmail.com> writes:

> In case anyone missed it, Liyang HU wrote a TH deriver for Unbox
> instances[1].
>
Thanks! I don't know how I missed this. This is quite sufficient for
most of my purposes.

> Also, I have tried using GHC's new generic deriving[2] to derive instances
> - this is almost possible, but is blocked by a GHC issue[3]. IMHO generic
> deriving would be the best solution (once it works), because it works for
> product types as well as newtypes, and requires no extra GHC assistance or
> TH.
>
This will be quite nice and it sounds like it should cover nearly all of
the cases I'm interested in. I'm not terribly familiar with generics
support so I'm not sure whether this is possible or not, but it would be
great if there were a way to specify whether an SoA or AoS
representation is derived. This way one could write nearly cache-optimal
code while keeping the mechanics of data layout nicely contained.

Cheers,

- Ben

Gmane