Oracle OIT IX SDK TIFF file parsing heap buffer overflow(CVE-2016-3582)

### Description While parsing a specially crafted TIFF file, a parser confusion can lead to a heap buffer overflow resulting in out of bounds memory overwrite and possibly leading to arbitrary code execution. ### Tested Versions Outside In IX sdk 8.5.1. ### Product URLs http://www.oracle.com/technetwork/middleware/content-management/oit-all-085236.html ### Details While parsing a specially crafted TIFF file with PhotometricInterpretation set to 5, indicating a CMYK image, the same amount of memory is allocated as when processing the original file (with PhotometricInterpretation set to 3 in this case), but the parser takes a different path, ending in a loop that uses the size value as a counter. The overflow happens because in each iteration of the loop, 3 bytes are written to the destination thus leading to heap buffer overflow and overwrite of adjacent heap structures. Technical information below: Size of the allocation is based on ImageWidth value (at offset 0x6c in the supplied testcase) and is calculated in the following basic block in `libvs_tiff.so` (image base is 0xB74E1000): ``` .text:B74E8AEF mov ecx, [esp+8Ch+arg_4] .text:B74E8AF6 movzx eax, word ptr [ecx+60h] .text:B74E8AFA imul eax, [ecx+30h] .text:B74E8AFE movzx edx, word ptr [ecx+0D8h] .text:B74E8B05 imul eax, edx .text:B74E8B08 add eax, 7 .text:B74E8B0B shr eax, 3 .text:B74E8B0E mov [ecx+3CCh], eax [1] .text:B74E8B14 cmp word ptr [ecx+0F8h], 0 .text:B74E8B1C jnz loc_B74E8C01 ``` At [1], the final value of the allocation is saved in `ecx+0x3cc`. The arithmetic can be translated into the following formula: ``` size = ((4*ImageWidth)+7)>>3 ` The allocation in question happens at the following basic block: ` .text:B74E6866 mov [esp+2Ch+var_18], eax .text:B74E686A mov esi, edx .text:B74E686C mov eax, [edx+3CCh] [1] .text:B74E6872 mov [esp+2Ch+s], eax .text:B74E6875 call _SYSNativeAlloc .text:B74E687A mov [esi+3D8h], eax .text:B74E6880 test eax, eax .text:B74E6882 jz loc_B74E6A ``` At [1], previously computed value is set as a parameter to `SYSNativeAlloc` which is a wrapper to `malloc`. After the successful allocation, the parser eventually ends up in the following basic block where the overflow happens: ``` .text:B74EC2F3 mov [edi], al [1] .text:B74EC2F5 mov eax, esi .text:B74EC2F7 mov [edi+1], al [2] .text:B74EC2FA mov [edi+2], dl [3] .text:B74EC2FD mov edx, [esp+26Ch+var_220] .text:B74EC301 movzx eax, word ptr [edx+0D8h] .text:B74EC308 add ebp, eax [4] .text:B74EC30A add edi, 3 [5] .text:B74EC30D cmp [edx+3CCh], ebp [6] .text:B74EC313 ja short loc_B74 ``` At [1],[2] and [3] in the above basic block, 3 bytes are written to the destination buffer in `edi` which gets incremented by 3 at [5] in each iteration of the loop. At [4] `eax` is always 1, increasing the counter in `ebp` only by 1, resulting in an overflow as the counter is compared against the previously calculated allocation size at [6]. The 1 value in `eax` at [4] comes from SamplesPerPixel (at offset 0xD8 in the supplied testcase). Regular TIFF file with CMYK color map should have SamplesPerPixel value of 4. To summarize, by controlling the size of the allocation, we can control the size of the overflow, as the number of bytes written in the loop that causes the overflow is `SamplesPerPixel*SizeOfAllocation`. Control over the data that gets written past the end of the buffer is subject to arithmetic transformations in the previous basic block. In the supplied testcase, the values have been malformed in such a way as to result in a function pointer overwrite. The application crashes when the overwritten function pointer is dereferenced during a call. This issue can be triggered by running the supplied testcase in the `ixsample` application supplied with the SDK. ### Timeline * 2016-04-12 - Vendor Notification * 2016-07-19 – Public Disclosure