headers
Robert Muir (JIRA | 6 Aug 2012 04:24
Picon
Favicon

[jira] [Created] (LUCENE-4289) highlighter idf calculation problems

Robert Muir created LUCENE-4289:
-----------------------------------

             Summary: highlighter idf calculation problems
                 Key: LUCENE-4289
                 URL: https://issues.apache.org/jira/browse/LUCENE-4289
             Project: Lucene - Core
          Issue Type: Bug
            Reporter: Robert Muir
         Attachments: LUCENE-4289.patch

* highlighter uses numDocs instead of maxDoc
* fastvectorhighlighter uses numDocs - numDeletedDocs instead of maxDoc (will go negative if more than
half of docs are marked deleted)
* fastvectorhighlighter calls docFreq and computes IDF per-position when it won't change (inefficient)

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
Permalink | Reply |
headers
Robert Muir (JIRA | 6 Aug 2012 04:24
Picon
Favicon

[jira] [Updated] (LUCENE-4289) highlighter idf calculation problems


     [
https://issues.apache.org/jira/browse/LUCENE-4289?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Robert Muir updated LUCENE-4289:
--------------------------------

    Attachment: LUCENE-4289.patch

> highlighter idf calculation problems
> ------------------------------------
>
>                 Key: LUCENE-4289
>                 URL: https://issues.apache.org/jira/browse/LUCENE-4289
>             Project: Lucene - Core
>          Issue Type: Bug
>            Reporter: Robert Muir
>         Attachments: LUCENE-4289.patch
>
>
> * highlighter uses numDocs instead of maxDoc
> * fastvectorhighlighter uses numDocs - numDeletedDocs instead of maxDoc (will go negative if more than
half of docs are marked deleted)
> * fastvectorhighlighter calls docFreq and computes IDF per-position when it won't change (inefficient)

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
(Continue reading)

Permalink | Reply |
headers
Koji Sekiguchi (JIRA | 6 Aug 2012 05:02
Picon
Favicon

[jira] [Commented] (LUCENE-4289) highlighter idf calculation problems


    [
https://issues.apache.org/jira/browse/LUCENE-4289?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13428962#comment-13428962
] 

Koji Sekiguchi commented on LUCENE-4289:
----------------------------------------

+1. Thanks, Robert.

> highlighter idf calculation problems
> ------------------------------------
>
>                 Key: LUCENE-4289
>                 URL: https://issues.apache.org/jira/browse/LUCENE-4289
>             Project: Lucene - Core
>          Issue Type: Bug
>            Reporter: Robert Muir
>         Attachments: LUCENE-4289.patch
>
>
> * highlighter uses numDocs instead of maxDoc
> * fastvectorhighlighter uses numDocs - numDeletedDocs instead of maxDoc (will go negative if more than
half of docs are marked deleted)
> * fastvectorhighlighter calls docFreq and computes IDF per-position when it won't change (inefficient)

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
(Continue reading)

Permalink | Reply |
headers
Uwe Schindler (JIRA | 6 Aug 2012 09:08
Picon
Favicon

[jira] [Commented] (LUCENE-4289) highlighter idf calculation problems


    [
https://issues.apache.org/jira/browse/LUCENE-4289?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13429001#comment-13429001
] 

Uwe Schindler commented on LUCENE-4289:
---------------------------------------

Too funny, +1 to fix. Backport?

> highlighter idf calculation problems
> ------------------------------------
>
>                 Key: LUCENE-4289
>                 URL: https://issues.apache.org/jira/browse/LUCENE-4289
>             Project: Lucene - Core
>          Issue Type: Bug
>            Reporter: Robert Muir
>         Attachments: LUCENE-4289.patch
>
>
> * highlighter uses numDocs instead of maxDoc
> * fastvectorhighlighter uses numDocs - numDeletedDocs instead of maxDoc (will go negative if more than
half of docs are marked deleted)
> * fastvectorhighlighter calls docFreq and computes IDF per-position when it won't change (inefficient)

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
(Continue reading)

Permalink | Reply |
headers
Gili Nachum (JIRA | 6 Aug 2012 09:27
Picon
Favicon

[jira] [Commented] (LUCENE-2501) ArrayIndexOutOfBoundsException in ByteBlockPool.allocSlice


    [
https://issues.apache.org/jira/browse/LUCENE-2501?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13429006#comment-13429006
] 

Gili Nachum commented on LUCENE-2501:
-------------------------------------

Issue resolved successfully. Even when increasing the degree of concurrency, I can no longer reproduce
with 16 threads over 4 core machine. 
Thank you Michael!

> ArrayIndexOutOfBoundsException in ByteBlockPool.allocSlice
> ----------------------------------------------------------
>
>                 Key: LUCENE-2501
>                 URL: https://issues.apache.org/jira/browse/LUCENE-2501
>             Project: Lucene - Core
>          Issue Type: Bug
>          Components: core/index
>    Affects Versions: 3.0.1
>            Reporter: Tim Smith
>         Attachments: LUCENE-2501.patch
>
>
> I'm seeing the following exception during indexing:
> {code}
> Caused by: java.lang.ArrayIndexOutOfBoundsException: 14
> at org.apache.lucene.index.ByteBlockPool.allocSlice(ByteBlockPool.java:118)
> at org.apache.lucene.index.TermsHashPerField.writeByte(TermsHashPerField.java:490)
(Continue reading)

Permalink | Reply |
headers
Robert Muir (JIRA | 6 Aug 2012 17:36
Picon
Favicon

[jira] [Resolved] (LUCENE-4289) highlighter idf calculation problems


     [
https://issues.apache.org/jira/browse/LUCENE-4289?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Robert Muir resolved LUCENE-4289.
---------------------------------

       Resolution: Fixed
    Fix Version/s: 3.6.2
                   5.0
                   4.0

I backported too. Note in 3.6 fast-vector-highlighter is unaffected, 
it doesn't compute IDF.

> highlighter idf calculation problems
> ------------------------------------
>
>                 Key: LUCENE-4289
>                 URL: https://issues.apache.org/jira/browse/LUCENE-4289
>             Project: Lucene - Core
>          Issue Type: Bug
>            Reporter: Robert Muir
>             Fix For: 4.0, 5.0, 3.6.2
>
>         Attachments: LUCENE-4289.patch
>
>
> * highlighter uses numDocs instead of maxDoc
> * fastvectorhighlighter uses numDocs - numDeletedDocs instead of maxDoc (will go negative if more than
(Continue reading)

Permalink | Reply |

Gmane