Question

Describe, in your own words, how branch functions make dissassembly more difficult. Your answer should briefly describe how linear sweep and recursive traversal disassembly work, and how branch functions affect both of them.

Answer 1

Linear Sweep : It is a basic algorithm taking all the section marked as code (collection of bytes) and disassembling (at the beginning, decoding 1st byte and so on) it by reading the instructions one after another until the end of the segment.

Recursive traversal disassembly: This is based on the concept of control flow to disassembly a program. Whenever this algorithm detects an instruction that can take more than one path. The key element to this approach is analysis of each instruction to determine if it is referenced from any other location. There are several classifications.

a)Sequential flow Instructions

b)Conditional Branching Instructions

c)Unconditional Branching Instructions

d)Function Call Instructions

e)Return Instructions

Generally dissemblers follow one of the two approaches. The drawback of linear sweep is that they are error prone. When junk bytes are inserted at unreachable locations then it has huge impact on linear sweep. But insertion of junk bytes has no impact on recursive dissembler. The problem with recursive approach is that the control flow cannot always reconstructed precisely. When a jump or a call cannot be determined (for an indirect jump), the recursive dissemblers cannot analyze the parts of the program's code. Recursive traversal relies on branch function. Thus creates possibilities to insert junk bytes and mislead both types of dissemblers. A normal call to a subroutine is replaced with a call to the branch function which uses indirect jump to transfer control to the original subroutine. Also, an offset value is added to the return address of the subroutine. When subroutine is done, the control is transferred to the address after the call instruction.

When we are using branch function then we can jump to any instruction and can decode it anytime but when we are performing linear sweep and recursive traversal , we must follow a specific order. Thus using branch function affects both of them.