Oracle OIT IX SDK libvs_pdf Tj Operator Denial of Service Vulnerability(CVE-2016-3576)

### DESCRIPTION When parsing a specialy crafted PDF document, a NULL pointer dereference leading to a process termination. A pointer value from a memory structure initialized to zero is reference without check. ### TESTED VERSIONS Oracle Outside In IX SDK 8.5.1 ### PRODUCT URLs http://www.oracle.com/technetwork/middleware/content-management/oit-all-085236.html ### DETAILS While executing a `Tj` operator on a piece of text contained in a stream, a memory structure probably containing charset mappings is referenced. No NULL pointer check is made and since the sturcture is zero initialized this can result in a crash. The supplied testcase succesfully crashes the sample `ixsample` application supplied with the SDK. In the supplied testcase, after the parser successfully decodes the /FlateDecode encoded stream data, it proceeds to execute the operators contained whitin. In this case the decoded stream data is : ``` 'BT\r/F2 1 Tf\r12 0 0 12 90.001 708.017 Tm\r0 g\r/GS1 gs\r0 Tc\r0 Tw\r(Results)Tj\r/F3 1 Tf\r0 -1.04 TD\r0.0009 Tc\r0.0087 Tw\r(The tasters s' ``` The problematic code is triggered while `Tj` operator is being executed with it's argument being string "Results". Function `OIT_cmdTj` in libvs_pdf.so implements this operator. Eventually the function `sub_B74E190C` is reached (libvs_pdf.so base address being 0xB74BF000) and the crash is triggered by the following basic block specifically: ``` .text:B74E1E20 mov edx, [esp+0BCh+arg_4] .text:B74E1E27 movzx eax, byte ptr [edx+1F9Ch] .text:B74E1E2E mov edx, ebp [1] .text:B74E1E30 movzx ecx, dl [2] .text:B74E1E33 shl eax, 5 .text:B74E1E36 mov edi, [esp+0BCh+arg_4] .text:B74E1E3D lea edx, [eax+edi] .text:B74E1E40 mov eax, [edx+1F18h] [3] .text:B74E1E46 movzx edi, byte ptr [eax+ecx] [4] .text:B74E1E4A mov eax, edi .text:B74E1E4C test al, al .text:B74E1E4E jz loc_B7 ``` At the time of the crash, initial value of `ebp` at [1] contains the first character of the `Tj` operator argument, in this case "R", which ends up in `ecx` and is subsequently used as an offset into the memory structure at [4]. At [2], value of `dl` is zero extended into ecx limiting our control over it. At [3], final value of `eax` is set from offset 0x1f18 into `edx`. Value of `eax` can be NULL but isn't checked resulting in a near NULL pointer dereference. It is worth nothing that when the same memory address is accessed in other parts of the code, the pointer is properly checked beforehand. ### TIMELINE * 2016-04-12 - Discovery * 2016-07-19 – Public Disclosure