Oliver Charles | 1 Feb 11:06 2013
Picon

Optimizing performance problems with Aeson rendering large Text arrays

Hello,

In summary, i'm working on an application that responds to a users query, a
sequence index, with the union of a list of UUIDs that have "changed" 
since that
same sequence index, split into 6 sections. I wish to respond to these 
queries
via JSON to provide an easy to use web service, and for the most part, 
what I
have works.

The problem I am having is that profiling seems to show that the 
majority of the
time spent in my application is encoding this to JSON, and also that the
application is only 60% productive with 40% allocations happening in
Data.Aeson.encode (and friends).

Here's an overview of what I'm doing, the full code can be found at the 
end of
this email.

I am storing my data in memory as an IntMap, from sequence index to a 
changeset:

     IntMap ChangeSet

Where a ChangeSet is essentially a tuple of HashSet's of UUIDs:

     data ChangeSet = ChangeSet { artistChanges :: !(HashSet MBID)
                                , labelChanges :: !(HashSet MBID)
(Continue reading)

Oliver Charles | 1 Feb 13:13 2013
Picon

Re: Optimizing performance problems with Aeson rendering large Text arrays

Urgh, the formatting got totally destroyed in sending, I think. If so, 
here's a paste of my email as I intended it to be sent:

http://hpaste.org/81648

Sorry about that!
- Ocharles
Bas van Dijk | 1 Feb 14:44 2013
Picon

Re: Optimizing performance problems with Aeson rendering large Text arrays


On Feb 1, 2013 1:15 PM, "Oliver Charles" <ollie <at> ocharles.org.uk> wrote:
>
> Urgh, the formatting got totally destroyed in sending, I think. If so, here's a paste of my email as I intended it to be sent:
>
> http://hpaste.org/81648
>
> Sorry about that!
> - Ocharles
>
>
> _______________________________________________
> Haskell-Cafe mailing list
> Haskell-Cafe <at> haskell.org
> http://www.haskell.org/mailman/listinfo/haskell-cafe

If I make a special case for text based UUIDs in aeson:

data Value = ... | UUID Text | ...

Data.Aeson.Encode.fromValue (UUID s) = singleton '"' <> fromText s <> singleton '"'

Then encoding time improves by 20%.

So a big part of the time is spent encoding the UUID strings.

Bas

_______________________________________________
Haskell-Cafe mailing list
Haskell-Cafe <at> haskell.org
http://www.haskell.org/mailman/listinfo/haskell-cafe
Oliver Charles | 1 Feb 21:04 2013
Picon

Re: Optimizing performance problems with Aeson rendering large Text arrays

On 02/01/2013 01:44 PM, Bas van Dijk wrote:
>
> If I make a special case for text based UUIDs in aeson:
>
> data Value = ... | UUID Text | ...
>
> Data.Aeson.Encode.fromValue (UUID s) = singleton '"' <> fromText s <> 
> singleton '"'
>
> Then encoding time improves by 20%.
>
> So a big part of the time is spent encoding the UUID strings.
>
> Bas
>
This might work, but it doesn't seem to be something I could actually 
distribute, unless I shipped modified Aeson source code... the String 
fromValue pattern seems to do a lot of escaping which won't apply to my 
UUIDs, but I'm not sure what the right solution to that is.

- Ocharles

Gmane