Lehrstuhl fr Systemsicherheit Virtual Machine-based Fingerprints - - PowerPoint PPT Presentation

Lehrstuhl für Systemsicherheit

Virtual Machine-based Fingerprints SPRING 9 Bochum, 31.07 - 01.08.2014

Implementing a Virtual Machine-based Fingerprinting Scheme HORST GÖRTZ INSTITUT FÜR IT-SICHERHEIT | SPRING 9

• 1. Background
• 1. Fingerprinting
• 2. Virtual Machines
• 2. Implemented Schemes
• 1. Permutation-based Fingerprints
• 2. Dynamic branch-based Fingerprints
• 3. Fingerprints based on Encoding Choice
• 3. Conclusion

Table of Contents

Implementing a Virtual Machine-based Fingerprinting Scheme HORST GÖRTZ INSTITUT FÜR IT-SICHERHEIT | SPRING 9

Background

Implementing a Virtual Machine-based Fingerprinting Scheme HORST GÖRTZ INSTITUT FÜR IT-SICHERHEIT | SPRING 9

• Two phases:
• 1. Embed an unique identifier (“mark”) into object
• 2. Identify the object by extracting the fingerprint

mark

• Fingerprint mark identifies party that uses the object
• In contrast to watermarking (claim ownership)
• Software use case: given a copy of the software, find
• ut who it has been sold to

Fingerprinting I

Implementing a Virtual Machine-based Fingerprinting Scheme HORST GÖRTZ INSTITUT FÜR IT-SICHERHEIT | SPRING 9

• Three types of fingerprints, determined by extraction

phase:

• 1. Static
• 2. Dynamic
• 3. Abstract
• Balance properties:
• 1. Stealth
• 2. Data Rate
• 3. Resilience

Fingerprinting II

Implementing a Virtual Machine-based Fingerprinting Scheme HORST GÖRTZ INSTITUT FÜR IT-SICHERHEIT | SPRING 9

• Structure commonly used in software protection systems
• Basic idea: Translate (parts of) native code into a custom

architecture and embed interpreter (VM)

• breaks existing tools
• non-trivial to attack generically
• hides original semantic and tamper-proofs
• Set of handlers describe semantics

Virtual Machines I

Implementing a Virtual Machine-based Fingerprinting Scheme HORST GÖRTZ INSTITUT FÜR IT-SICHERHEIT | SPRING 9

Virtual Machines II

fetch operands vm_mov_reg_imm handler 00 ... 5A ... 7F 80 ... FF 32

• pcode

parameters

0xdeadbeef

0x0f00 0xbeef

FE

0x0f00

0xcafebabe

0x0f00 0xbeef

07

0xdead

0x1badf00d

0xb00b

32

bytecode 0xdead 0x0f00 0xdead

FE

0x0f00

5A 5A 5A FE [pointer] handler tbl

VM context

[pointer] vIP 0xdeadbeef native eax 0x1badc0de native ecx ... ...

entry value

vm_and_reg_reg ... vm_mov_reg_imm ... vm_add_reg_reg vm_xor_reg_reg ... vm_mov_reg_reg

handler table

update ctx calculate dispatch next

handler code

Implementing a Virtual Machine-based Fingerprinting Scheme HORST GÖRTZ INSTITUT FÜR IT-SICHERHEIT | SPRING 9

Implemented Schemes

Implementing a Virtual Machine-based Fingerprinting Scheme HORST GÖRTZ INSTITUT FÜR IT-SICHERHEIT | SPRING 9

• Based on patent by Davidson and Myhrvold (1996)
• Embeds the mark in order of basic blocks of a

function

• Mark extracted by comparing order in binary to

canonical ordering

• But: Prone to subsequent application!
• Approach here: Embed mark in permutation of handler

table

• Subsequent application results in non-functional

program!

Permutation-based Fingerprints

Implementing a Virtual Machine-based Fingerprinting Scheme HORST GÖRTZ INSTITUT FÜR IT-SICHERHEIT | SPRING 9

Permutation-based Fingerprints

0040AFC4 00407513 0040645A 0040699E 004070A1 0040640A ... 00407F72 FE 39 01 12 2A 00 ... 42

canonical

0040640A 0040645A 004064AB 004064FF 0040654F 004065A0 ... 0040AF72 00 01 02 03 04 05 ... FF

Canonical Form Extracted

Handler Table

Perm. lookup handler index

Fingerprinted Binary Code

Implementing a Virtual Machine-based Fingerprinting Scheme HORST GÖRTZ INSTITUT FÜR IT-SICHERHEIT | SPRING 9

• Based on method by Linn et al., extension by Collberg et

al.

• Mark encoded in (unstealthy!) series of unconditional

branches

• Branch direction encodes one bit
• Extraction using Execution Trace
• Approach here: Transferred verbatim, but extraction phase

problematic due to VM layer

• Circumvent VM layer without lowering its security?
• VM Trapdooring: constant (secret) seed when generating

components

Branch-based Fingerprints

Implementing a Virtual Machine-based Fingerprinting Scheme HORST GÖRTZ INSTITUT FÜR IT-SICHERHEIT | SPRING 9

Branch-based Fingerprints

vm_mov_reg_imm vm_mov_reg_reg 0040645A 0040699E vm_mov_reg_imm 0040640A ... vm_mov_reg_reg jmp 07 jmp 08 ... jmp target jmp 00 ... jmp 24 01 02 ... 07 08 ... ... ... jmp 02 ... 23 24 ... ... 00 35 12 jmp 35 jmp 01

1 1 1 1

virtualized code encoding fingerprint 0b1010101 ... VM code handler table verify vIP update

track target immediate

verify VM sequence

track dst register

intercept handler execution vm_mov_reg_imm observer vm_mov_reg_reg observer

jmp target (IA-32) mov_reg_imm tmp, target mov_reg_reg vIP, tmp

Implementing a Virtual Machine-based Fingerprinting Scheme HORST GÖRTZ INSTITUT FÜR IT-SICHERHEIT | SPRING 9

• Handler Duplication: duplicate handler code
• Multiple handlers encode same semantics
• Multiple opcodes per virtual instruction
• We have a choice when encoding bytecode
• Approach here: Group equivalent handlers and assign

values to each member in a group (cf. Monden et al.)

• Every encoded virtual instruction embeds a few bits

based on the handler it chooses

• Embed mark in all emitted instructions

FPs based on Encoding Choice

Implementing a Virtual Machine-based Fingerprinting Scheme HORST GÖRTZ INSTITUT FÜR IT-SICHERHEIT | SPRING 9

FPs based on Encoding Choice

??

• pcode

parameters

0xdeadbeef

0x0f00 0xbeef

??

0x0f00

0xcafebabe

0x0f00 0xbeef

??

0xdead

0x1badf00d

0xb00b

??

bytecode handler table 0xdead

??

0x0f00 0xdead

??

0x0f00

?? ?? ?? vm_mov_reg_reg vm_add_reg_imm vm_mov_reg_imm vm_add_reg_imm vm_and_reg_reg vm_mov_reg_imm 00 01 02 03 04 vm_mov_reg_imm 05 ... vm_mov_reg_imm ... FF 06

• pcode

semantics

enc.

bits

... ... ... ... ... 00 01 10 11

Implementing a Virtual Machine-based Fingerprinting Scheme HORST GÖRTZ INSTITUT FÜR IT-SICHERHEIT | SPRING 9

Conclusion

Implementing a Virtual Machine-based Fingerprinting Scheme HORST GÖRTZ INSTITUT FÜR IT-SICHERHEIT | SPRING 9

• Schemes draw from resilience provided by VM
• Exploit specific VM traits, tied to VM layer
• Comes at the cost of increased time/space complexity
• Refrain from protecting performance-critical sections

Conclusion

Implementing a Virtual Machine-based Fingerprinting Scheme HORST GÖRTZ INSTITUT FÜR IT-SICHERHEIT | SPRING 9

• Robert I. Davidson and Nathan Myhrvold. Method and system for generating and

auditing a signature for a computer program, September 24 1996. US Patent 5,559,884.

• Cullen Linn, Saumya Debray, and John Kececioglu. Enhancing Software Tamper-

Resistance via Stealthy Address Computations. In Proceedings of the 19th Annual Computer Security Applications Conference (ACSAC 2003). Citeseer, 2003.

• Akito Monden, Hajimu Iida, K-i Matsumoto, Katsuro Inoue, and Koji Torii. A Practical

Method for Watermarking Java Programs. In Computer Software and Applications Conference, 2000. COMPSAC 2000. The 24th Annual International, pages 191-197. IEEE, 2000.

• Christian Collberg and Jasvir Nagra. Surreptitious Software. Upper Saddle River, NJ:

Addision-Wesley Professional, 2010.

• Patrick Cousot and Radhia Cousot. An Abstract Interpretation-Based Framework for

Software Watermarking. In ACM SIGPLAN Notices, volume 39, pages 173-185. ACM, 2004.

Bibliography

Implementing a Virtual Machine-based Fingerprinting Scheme HORST GÖRTZ INSTITUT FÜR IT-SICHERHEIT | SPRING 9

Thank you for your attention!

Any questions? @dwuid