FFmpeg | 27 Feb 20:35 2013

#2310(undetermined:new): x264: x264_cqm_init access beyond array causes undefined behavior in gcc48

#2310: x264: x264_cqm_init access beyond array causes undefined behavior in gcc48
-------------------------------------+-------------------------------------
             Reporter:  Safari       |                     Type:  defect
               Status:  new          |                 Priority:  normal
            Component:               |                  Version:  git-
  undetermined                       |  master
             Keywords:               |               Blocked By:
             Blocking:               |  Reproduced by developer:  0
Analyzed by developer:  0            |
-------------------------------------+-------------------------------------
 Summary of the bug:
 can't x264 bugs be submitted with trac..? anyways...

 i can get to 7, whereas quant4_mf quant4_bias0 quant4_mf dequant4_mf and
 unquant4_mf are defined as array [4], therefore undefined behavior happens
 with gcc-4.8-0.14.fc19 (infinite loop).

 How to reproduce:

 run x264 and it segfaults in x264_cqm_init.

 small test case which causes infinite loop

 #include <stdio.h>
 #include <string.h>
 #include <inttypes.h>
 #include <stdlib.h>

 uint32_t (*quant4_mf[4])[16];

(Continue reading)

FFmpeg | 27 Feb 22:41 2013

Re: #2310(undetermined:closed): x264: x264_cqm_init access beyond array causes undefined behavior in gcc48

#2310: x264: x264_cqm_init access beyond array causes undefined behavior in gcc48
-------------------------------------+-------------------------------------
             Reporter:  Safari       |                    Owner:
                 Type:  defect       |                   Status:  closed
             Priority:  normal       |                Component:
              Version:  git-master   |  undetermined
             Keywords:  libx264      |               Resolution:  invalid
             Blocking:               |               Blocked By:
Analyzed by developer:  0            |  Reproduced by developer:  0
-------------------------------------+-------------------------------------
Changes (by cehoyos):

 * keywords:   => libx264
 * status:  new => closed
 * resolution:   => invalid

Comment:

 Please consider posting on http://mailman.videolan.org/listinfo/x264-devel

--

-- 
Ticket URL: <https://ffmpeg.org/trac/ffmpeg/ticket/2310#comment:1>
FFmpeg <http://ffmpeg.org>
FFmpeg issue tracker
FFmpeg | 11 Apr 00:47 2013

Re: #2310(undetermined:closed): x264: x264_cqm_init access beyond array causes undefined behavior in gcc48

#2310: x264: x264_cqm_init access beyond array causes undefined behavior in gcc48
-------------------------------------+-------------------------------------
             Reporter:  Safari       |                    Owner:
                 Type:  defect       |                   Status:  closed
             Priority:  normal       |                Component:
              Version:  git-master   |  undetermined
             Keywords:  libx264      |               Resolution:  invalid
             Blocking:               |               Blocked By:
Analyzed by developer:  0            |  Reproduced by developer:  0
-------------------------------------+-------------------------------------

Comment (by rogerdpack):

 I think I may have just run into this on windows (cross compiling) with
 4.8
 Was this reported to x264-devel does anybody know?

 Program received signal SIGSEGV, Segmentation fault.
 0x759c79c8 in msvcrt!memcmp () from C:\Windows\syswow64\msvcrt.dll
 (gdb) bt
 #0  0x759c79c8 in msvcrt!memcmp () from C:\Windows\syswow64\msvcrt.dll
 #1  0x004a303c in x264_cqm_init ()

 Also related: http://forum.doom9.org/showthread.php?p=1623567

--

-- 
Ticket URL: <https://ffmpeg.org/trac/ffmpeg/ticket/2310#comment:2>
FFmpeg <http://ffmpeg.org>
FFmpeg issue tracker
(Continue reading)


Gmane