headers
Adam Lally | 17 May 20:58
Picon

Re: UIMA internals memory footprint

Kirk,

In this test are you running a CPE or just an AnalysisEngine?  If it
is a CPE do you know what your CAS Pool size is?

When a CAS is created it does allocate a large heap which is then
filled as you create annotations.  By default I believe this is
500,000 cells (2MB) per CAS, but this can be overridden (see
UIMAFramework.getDefaultPerformanceTuningPropeties()).  So this can
defintely be one source of memory overhead.  As you saw it does not
grow with larger documents, it will only grow if you create enough
annotations to fill up the allocated space.

-Adam

On 5/17/07, Kirk True <kirk@...> wrote:
> Hi all,
>
> I have begun getting seeing heavy memory use when processing largish
> documents through a UIMA pipeline. I wanted to make sure what I'm
> seeing with regard to UIMA's internal memory use is on par with
> expectations.
>
> It looks like either for a 1,500,000 byte or a 15,000,000 byte document
> with the same annotations (100,000 10-character annotations), we incur
> a ~13 MB "overhead" for internal UIMA data structures. Is this in line
> with expectations?
>
> Details:
>
(Continue reading)

Permalink | Reply |
headers
Kirk True | 18 May 01:28
Gravatar

Re: UIMA internals memory footprint

Hi Adam,

> Kirk,
> 
> In this test are you running a CPE or just an AnalysisEngine?  If it
> is a CPE do you know what your CAS Pool size is?

It's an AnalysisEngine.

> When a CAS is created it does allocate a large heap which is then
> filled as you create annotations.  By default I believe this is
> 500,000 cells (2MB) per CAS, but this can be overridden (see
> UIMAFramework.getDefaultPerformanceTuningPropeties()).  So this can
> defintely be one source of memory overhead.  As you saw it does not
> grow with larger documents, it will only grow if you create enough
> annotations to fill up the allocated space.

I noticed that this is tweak-able and set it to something insanely
small (like 100). But, as you said, it grows as the number of
annotations grow. Since the parameter is under the umbrella of
performance, I'd assume that it would actually be better to
pre-allocate close to what we're going to use.

Thanks!
Kirk

> On 5/17/07, Kirk True <kirk@...> wrote:
> > Hi all,
> >
> > I have begun getting seeing heavy memory use when processing
(Continue reading)

Permalink | Reply |
headers
Thilo Goetz | 18 May 09:55
Picon
Picon

Re: UIMA internals memory footprint

Kirk True wrote:
> Hi Adam,
> 
>> Kirk,
>>
>> In this test are you running a CPE or just an AnalysisEngine?  If it
>> is a CPE do you know what your CAS Pool size is?
> 
> It's an AnalysisEngine.
> 
>> When a CAS is created it does allocate a large heap which is then
>> filled as you create annotations.  By default I believe this is
>> 500,000 cells (2MB) per CAS, but this can be overridden (see
>> UIMAFramework.getDefaultPerformanceTuningPropeties()).  So this can
>> defintely be one source of memory overhead.  As you saw it does not
>> grow with larger documents, it will only grow if you create enough
>> annotations to fill up the allocated space.
> 
> I noticed that this is tweak-able and set it to something insanely
> small (like 100). But, as you said, it grows as the number of
> annotations grow. Since the parameter is under the umbrella of
> performance, I'd assume that it would actually be better to
> pre-allocate close to what we're going to use.
[...]

Yes.

You can estimate data use on the heap as follows.  Each FS uses at least one
int for the type information, plus whatever features it has.  So a vanilla
annotation is 3 ints, one for the type, and one for the start and end features,
(Continue reading)

Permalink | Reply |
headers
Adam Lally | 21 May 15:38
Picon

Re: UIMA internals memory footprint

On 5/18/07, Thilo Goetz <twgoetz@...> wrote:
> You can estimate data use on the heap as follows.  Each FS uses at least one
> int for the type information, plus whatever features it has.  So a vanilla
> annotation is 3 ints, one for the type, and one for the start and end features,
> respectively.  If you have two additional features, that's 5 ints, so 20 bytes.
> If you use the JCas, you incur an additional overhead of a Java object for
> each annotation.  It's small, but I can't say off the top of my head how small
> exactly.  Plus, the JCas objects are held in a HashMap (or some such, Marshall
> correct me if I'm wrong), which incurs additional memory overhead.
>
> In my experience, the CAS can easily reach 10 to 20 times the size of the input
> document.  If you have information reach token annotations, that's not really
> surprising.  (And this is without using JCas).  Imagine you were to manually
> create Java objects that carry the same information, you would see roughly
> the same kind of overhead.
>

Using these numbers can we account for the 9,300,000 bytes of integer arrays?

100,000 annotations of size 5 cells = 500,000 ints, which is exactly
the default heap size.  But with the Sofa FS this will exceed the
default heap size.  It will grow by another 500,000 (I think).

So that accounts for 1,000,000 ints = 4,000,000 bytes.

Where are the other 5,300,000?

Likewise, what about the 1,600,000 bytes of Integers.  The JCAS hash
map only accounts for one per annotation, which in this case should
only be 400,000 bytes.
(Continue reading)

Permalink | Reply |
headers
Marshall Schor | 22 May 15:46

Re: UIMA internals memory footprint

The indexes use int[] arrays. 

Kirk - what indexes do you have defined (if any)?  Do you 
"addToIndexes..." any of
the annotations you create?

-Marshall

Adam Lally wrote:
> On 5/18/07, Thilo Goetz <twgoetz@...> wrote:
>> You can estimate data use on the heap as follows.  Each FS uses at 
>> least one
>> int for the type information, plus whatever features it has.  So a 
>> vanilla
>> annotation is 3 ints, one for the type, and one for the start and end 
>> features,
>> respectively.  If you have two additional features, that's 5 ints, so 
>> 20 bytes.
>> If you use the JCas, you incur an additional overhead of a Java 
>> object for
>> each annotation.  It's small, but I can't say off the top of my head 
>> how small
>> exactly.  Plus, the JCas objects are held in a HashMap (or some such, 
>> Marshall
>> correct me if I'm wrong), which incurs additional memory overhead.
>>
>> In my experience, the CAS can easily reach 10 to 20 times the size of 
>> the input
>> document.  If you have information reach token annotations, that's 
>> not really
(Continue reading)

Permalink | Reply |
headers
Kirk True | 22 May 18:42
Gravatar

Re: UIMA internals memory footprint

Hi Marshall,

> The indexes use int[] arrays. 
> 
> Kirk - what indexes do you have defined (if any)?  Do you 
> "addToIndexes..." any of
> the annotations you create?

Yes - I'm adding all annotations to the indexes.

If it helps, here's the source code for the annotator and the shim
application from which it is run:

    http://www.mustardgrain.com/files/testcaseannotator.zip

Thanks for all the feedback!

Kirk

> -Marshall
> 
> Adam Lally wrote:
> > On 5/18/07, Thilo Goetz <twgoetz@...> wrote:
> >> You can estimate data use on the heap as follows.  Each FS uses at
> 
> >> least one
> >> int for the type information, plus whatever features it has.  So a
> 
> >> vanilla
> >> annotation is 3 ints, one for the type, and one for the start and
(Continue reading)

Permalink | Reply |
headers
Adam Lally | 22 May 21:23
Picon

Re: UIMA internals memory footprint

On 5/22/07, Kirk True <kirk@...> wrote:
> If it helps, here's the source code for the annotator and the shim
> application from which it is run:
>
>     http://www.mustardgrain.com/files/testcaseannotator.zip
>

Kirk,

Just so we have all the legal bases covered, could you attach this to
the JIRA issue UIMA-412
(https://issues.apache.org/jira/browse/UIMA-412) and check the box
that says you grant license to the ASF?

Thanks,
  -Adam
Permalink | Reply |
headers
Marshall Schor | 20 May 02:58

Re: UIMA internals memory footprint

Thilo Goetz wrote:
> Kirk True wrote:
>> Hi Adam,
>>
>>> Kirk,
>>>
>>> In this test are you running a CPE or just an AnalysisEngine?  If it
>>> is a CPE do you know what your CAS Pool size is?
>>
>> It's an AnalysisEngine.
>>
>>> When a CAS is created it does allocate a large heap which is then
>>> filled as you create annotations.  By default I believe this is
>>> 500,000 cells (2MB) per CAS, but this can be overridden (see
>>> UIMAFramework.getDefaultPerformanceTuningPropeties()).  So this can
>>> defintely be one source of memory overhead.  As you saw it does not
>>> grow with larger documents, it will only grow if you create enough
>>> annotations to fill up the allocated space.
>>
>> I noticed that this is tweak-able and set it to something insanely
>> small (like 100). But, as you said, it grows as the number of
>> annotations grow. Since the parameter is under the umbrella of
>> performance, I'd assume that it would actually be better to
>> pre-allocate close to what we're going to use.
> [...]
>
> Yes.
>
> You can estimate data use on the heap as follows.  Each FS uses at 
> least one
(Continue reading)

Permalink | Reply |
headers
Thilo Goetz | 20 May 08:25
Picon
Picon

Re: UIMA internals memory footprint

Marshall Schor wrote:
> Thilo Goetz wrote:
[...]
>> If you use the JCas, 
> or you create FeatureStructure Java objects (which are Java Objects),

True.  My point was (which maybe I should have mentioned ;-) that JCas
objects stick around, while plain old FeatureStructures get can get
garbage collected after each annotator has run.  So JCas objects behave
like the rest of the CAS in that respect, and unlike FeatureStructure
objects.  Not beating on the JCas, just trying to explain sources of
memory consumption in the final analysis, after processing, so to speak.

--Thilo
Permalink | Reply |

Gmane