DHTs and Sharding Aurojit Panda Announcements Announcements Fill - PowerPoint PPT Presentation

Consistent Hashing • Depends on what hashing algorithm you use. • But one pretty common strategy (with some additional details missing) 16 buckets h: h[k] = v

Consistent Hashing • Depends on what hashing algorithm you use. • But one pretty common strategy (with some additional details missing) 16 buckets h: h[k] = v h[hash(k) % len(h)] = v

Consistent Hashing • Depends on what hashing algorithm you use. • But one pretty common strategy (with some additional details missing) 16 buckets h: h[k] = v h[hash(k) % len(h)] = v h[hash(k) % 16] = v

Consistent Hashing • Depends on what hashing algorithm you use. • But one pretty common strategy (with some additional details missing) 16 buckets h: v h[k] = v h[hash(k) % len(h)] = v h[hash(k) % 16] = v

Consistent Hashing 16 buckets h: v h[k] = v h[hash(k) % len(h)] = v h[hash(k) % 16] = v x = h[k]

Consistent Hashing 16 buckets h: v h[k] = v h[hash(k) % len(h)] = v h[hash(k) % 16] = v x = h[k] x = h[hash(k) % len(h)]

Consistent Hashing 16 buckets h: v h[k] = v h[hash(k) % len(h)] = v h[hash(k) % 16] = v x = h[k] x = h[hash(k) % len(h)] x = h[hash(k) % 16]

Consistent Hashing 20 buckets h: v x = h[k] x = h[hash(k) % len(h)] x = h[hash(k) % 16] Added 4 buckets x = h[k] x = h[hash(k) % len(h)] x = h[hash(k) % 20]

Consistent Hashing 20 buckets h: v x = h[k] x = h[hash(k) % len(h)] How many items need to move? x = h[hash(k) % 16] Added 4 buckets x = h[k] x = h[hash(k) % len(h)] x = h[hash(k) % 20]

Consistent Hashing • David Karger, et al. 1997 at MIT: Mechanism to minimize reassignments.

Consistent Hashing • David Karger, et al. 1997 at MIT: Mechanism to minimize reassignments. • Monotonicity : When buckets are added items move from old to new .

Consistent Hashing • David Karger, et al. 1997 at MIT: Mechanism to minimize reassignments. • Monotonicity : When buckets are added items move from old to new . • No items move between old buckets.

Consistent Hashing • David Karger, et al. 1997 at MIT: Mechanism to minimize reassignments. • Monotonicity : When buckets are added items move from old to new . • No items move between old buckets. • Also require other properties (e.g., balance) which are more traditional.

Consistent Hashing

Consistent Hashing • Map each bucket to a point in the unit interval .

Consistent Hashing • Map each bucket to a point in the unit interval . • How? Use a hash function h b .

Consistent Hashing • Map each bucket to a point in the unit interval . • How? Use a hash function h b . • Map each key to a point in the unit interval .

Consistent Hashing • Map each bucket to a point in the unit interval . • How? Use a hash function h b . • Map each key to a point in the unit interval . • How? Use another hash function h k .

Consistent Hashing • Map each bucket to a point in the unit interval . • How? Use a hash function h b . • Map each key to a point in the unit interval . • How? Use another hash function h k . • Map key k to bucket b such that: |h k (k) - h b (b)| is minimal.

Consistent Hashing • Map each bucket to a point in the unit interval . • How? Use a hash function h b . • Map each key to a point in the unit interval . • How? Use another hash function h k . • Map key k to bucket b such that: |h k (k) - h b (b)| is minimal. • Is this sufficient for monotonicity?

Chord • Use consistent hashing to build a distributed hash table. • Keys must be allocated across different nodes. • Balanced, i.e., no more than (1 + ε )K/N keys go to a single node. • Minimize amount of data that needs to be synchronized. • Especially important when nodes leave or join.

Chord Slow Lookup 2 1 35 Successor pointer

Chord Slow Lookup lookup(k) 2 hash(k) = 7 1 35 Successor pointer

Chord Slow Lookup lookup(k) 2 hash(k) = 7 1 35 Successor pointer

Chord Join 2: ... 2 37: ... 1 42: ... 3: ... 35 0: ... 4: ... 27: ... Successor pointer

Chord Join 2: ... 2 8 37: ... 1 42: ... 3: ... 35 0: ... 4: ... 27: ... Successor pointer

Chord Join 2: ... 2 8 37: ... 1 42: ... 3: ... 35 0: ... 4: ... 27: ... Predecessor pointer Successor pointer

Chord Join 2: ... 2 8 37: ... 1 42: ... 3: ... 35 0: ... 4: ... 27: ... Predecessor pointer Successor pointer

Chord Join 2: ... 2 8 3: ... 4: ... 37: ... 1 42: ... 27: ... 35 0: ... Predecessor pointer Successor pointer

Chord Leave 2: ... 2 8 3: ... 4: ... 37: ... 1 42: ... 27: ... 35 0: ... Predecessor pointer Successor pointer

Chord Leave 2: ... 2 37: ... 1 42: ... 27: ... 35 0: ... Predecessor pointer Successor pointer

Chord Leave 2: ... 2 Oops: Need successor list to fix this problem 37: ... 1 42: ... 27: ... 35 0: ... Predecessor pointer Successor pointer

Chord Leave 2: ... 2 37: ... 1 42: ... 27: ... 35 0: ... Predecessor pointer Successor pointer

Fingers 0 2 3.lookup(k) 38 hash(k) = 42 3 34 6 32 30 8 m=6 (2 m = 64) 28 10 26 12 24 14 22 16 20 18

Fingers 0 2 3.lookup(k) 38 hash(k) = 42 3 34 6 32 30 8 m=6 (2 m = 64) 28 10 26 12 24 14 22 16 20 18

<latexit sha1_base64="o0lJPfhPEwESip2b8DuHRmlkxGM=">ACHnicbVDLSgMxFM34tr6qLt0Ei6CIZUYU3QiG5cKVoV2LJn0Tg1NMkNyRyzDfIkbf8WNC0UEV/o3po+FrwMXTs65l9x7olQKi7/6Y2Mjo1PTE5Nl2Zm5+YXyotLFzbJDIcaT2RiriJmQoNRQo4So1wFQk4TLqHPf8y1swViT6HLsphIq1tYgFZ+ikZnm3gXCHeSx0G0xR74T0gA4km3FerGu6Sbev85WUGzQhkpa7qU2muWKX/X7oH9JMCQVMsRps/zeaCU8U6CRS2ZtPfBTDHNmUHAJRamRWUgZ7A21B3VTIEN8/5BV1zSovGiXGlkfbV7xM5U9Z2VeQ6FcMb+9vrif959Qzj/TAXOs0QNB98FGeSYkJ7WdGWMBRdh1h3Ai3K+U3zDCOLtGSCyH4fJfcrFdDfxqcLZTOTwaxjFVsgqWScB2SOH5ISckhrh5J48kmfy4j14T96r9zZoHfGM8vkB7yPLztQoTU=</latexit> <latexit sha1_base64="o0lJPfhPEwESip2b8DuHRmlkxGM=">ACHnicbVDLSgMxFM34tr6qLt0Ei6CIZUYU3QiG5cKVoV2LJn0Tg1NMkNyRyzDfIkbf8WNC0UEV/o3po+FrwMXTs65l9x7olQKi7/6Y2Mjo1PTE5Nl2Zm5+YXyotLFzbJDIcaT2RiriJmQoNRQo4So1wFQk4TLqHPf8y1swViT6HLsphIq1tYgFZ+ikZnm3gXCHeSx0G0xR74T0gA4km3FerGu6Sbev85WUGzQhkpa7qU2muWKX/X7oH9JMCQVMsRps/zeaCU8U6CRS2ZtPfBTDHNmUHAJRamRWUgZ7A21B3VTIEN8/5BV1zSovGiXGlkfbV7xM5U9Z2VeQ6FcMb+9vrif959Qzj/TAXOs0QNB98FGeSYkJ7WdGWMBRdh1h3Ai3K+U3zDCOLtGSCyH4fJfcrFdDfxqcLZTOTwaxjFVsgqWScB2SOH5ISckhrh5J48kmfy4j14T96r9zZoHfGM8vkB7yPLztQoTU=</latexit> <latexit sha1_base64="o0lJPfhPEwESip2b8DuHRmlkxGM=">ACHnicbVDLSgMxFM34tr6qLt0Ei6CIZUYU3QiG5cKVoV2LJn0Tg1NMkNyRyzDfIkbf8WNC0UEV/o3po+FrwMXTs65l9x7olQKi7/6Y2Mjo1PTE5Nl2Zm5+YXyotLFzbJDIcaT2RiriJmQoNRQo4So1wFQk4TLqHPf8y1swViT6HLsphIq1tYgFZ+ikZnm3gXCHeSx0G0xR74T0gA4km3FerGu6Sbev85WUGzQhkpa7qU2muWKX/X7oH9JMCQVMsRps/zeaCU8U6CRS2ZtPfBTDHNmUHAJRamRWUgZ7A21B3VTIEN8/5BV1zSovGiXGlkfbV7xM5U9Z2VeQ6FcMb+9vrif959Qzj/TAXOs0QNB98FGeSYkJ7WdGWMBRdh1h3Ai3K+U3zDCOLtGSCyH4fJfcrFdDfxqcLZTOTwaxjFVsgqWScB2SOH5ISckhrh5J48kmfy4j14T96r9zZoHfGM8vkB7yPLztQoTU=</latexit> <latexit sha1_base64="o0lJPfhPEwESip2b8DuHRmlkxGM=">ACHnicbVDLSgMxFM34tr6qLt0Ei6CIZUYU3QiG5cKVoV2LJn0Tg1NMkNyRyzDfIkbf8WNC0UEV/o3po+FrwMXTs65l9x7olQKi7/6Y2Mjo1PTE5Nl2Zm5+YXyotLFzbJDIcaT2RiriJmQoNRQo4So1wFQk4TLqHPf8y1swViT6HLsphIq1tYgFZ+ikZnm3gXCHeSx0G0xR74T0gA4km3FerGu6Sbev85WUGzQhkpa7qU2muWKX/X7oH9JMCQVMsRps/zeaCU8U6CRS2ZtPfBTDHNmUHAJRamRWUgZ7A21B3VTIEN8/5BV1zSovGiXGlkfbV7xM5U9Z2VeQ6FcMb+9vrif959Qzj/TAXOs0QNB98FGeSYkJ7WdGWMBRdh1h3Ai3K+U3zDCOLtGSCyH4fJfcrFdDfxqcLZTOTwaxjFVsgqWScB2SOH5ISckhrh5J48kmfy4j14T96r9zZoHfGM8vkB7yPLztQoTU=</latexit> Fingers 0 2 3.lookup(k) 38 hash(k) = 42 3 34 6 32 30 8 m=6 (2 m = 64) 28 10 26 12 24 14 22 16 20 18 finger[ k ] = succ( n + 2 k − 1 ) mod 2 m )

<latexit sha1_base64="o0lJPfhPEwESip2b8DuHRmlkxGM=">ACHnicbVDLSgMxFM34tr6qLt0Ei6CIZUYU3QiG5cKVoV2LJn0Tg1NMkNyRyzDfIkbf8WNC0UEV/o3po+FrwMXTs65l9x7olQKi7/6Y2Mjo1PTE5Nl2Zm5+YXyotLFzbJDIcaT2RiriJmQoNRQo4So1wFQk4TLqHPf8y1swViT6HLsphIq1tYgFZ+ikZnm3gXCHeSx0G0xR74T0gA4km3FerGu6Sbev85WUGzQhkpa7qU2muWKX/X7oH9JMCQVMsRps/zeaCU8U6CRS2ZtPfBTDHNmUHAJRamRWUgZ7A21B3VTIEN8/5BV1zSovGiXGlkfbV7xM5U9Z2VeQ6FcMb+9vrif959Qzj/TAXOs0QNB98FGeSYkJ7WdGWMBRdh1h3Ai3K+U3zDCOLtGSCyH4fJfcrFdDfxqcLZTOTwaxjFVsgqWScB2SOH5ISckhrh5J48kmfy4j14T96r9zZoHfGM8vkB7yPLztQoTU=</latexit> <latexit sha1_base64="o0lJPfhPEwESip2b8DuHRmlkxGM=">ACHnicbVDLSgMxFM34tr6qLt0Ei6CIZUYU3QiG5cKVoV2LJn0Tg1NMkNyRyzDfIkbf8WNC0UEV/o3po+FrwMXTs65l9x7olQKi7/6Y2Mjo1PTE5Nl2Zm5+YXyotLFzbJDIcaT2RiriJmQoNRQo4So1wFQk4TLqHPf8y1swViT6HLsphIq1tYgFZ+ikZnm3gXCHeSx0G0xR74T0gA4km3FerGu6Sbev85WUGzQhkpa7qU2muWKX/X7oH9JMCQVMsRps/zeaCU8U6CRS2ZtPfBTDHNmUHAJRamRWUgZ7A21B3VTIEN8/5BV1zSovGiXGlkfbV7xM5U9Z2VeQ6FcMb+9vrif959Qzj/TAXOs0QNB98FGeSYkJ7WdGWMBRdh1h3Ai3K+U3zDCOLtGSCyH4fJfcrFdDfxqcLZTOTwaxjFVsgqWScB2SOH5ISckhrh5J48kmfy4j14T96r9zZoHfGM8vkB7yPLztQoTU=</latexit> <latexit sha1_base64="o0lJPfhPEwESip2b8DuHRmlkxGM=">ACHnicbVDLSgMxFM34tr6qLt0Ei6CIZUYU3QiG5cKVoV2LJn0Tg1NMkNyRyzDfIkbf8WNC0UEV/o3po+FrwMXTs65l9x7olQKi7/6Y2Mjo1PTE5Nl2Zm5+YXyotLFzbJDIcaT2RiriJmQoNRQo4So1wFQk4TLqHPf8y1swViT6HLsphIq1tYgFZ+ikZnm3gXCHeSx0G0xR74T0gA4km3FerGu6Sbev85WUGzQhkpa7qU2muWKX/X7oH9JMCQVMsRps/zeaCU8U6CRS2ZtPfBTDHNmUHAJRamRWUgZ7A21B3VTIEN8/5BV1zSovGiXGlkfbV7xM5U9Z2VeQ6FcMb+9vrif959Qzj/TAXOs0QNB98FGeSYkJ7WdGWMBRdh1h3Ai3K+U3zDCOLtGSCyH4fJfcrFdDfxqcLZTOTwaxjFVsgqWScB2SOH5ISckhrh5J48kmfy4j14T96r9zZoHfGM8vkB7yPLztQoTU=</latexit> <latexit sha1_base64="o0lJPfhPEwESip2b8DuHRmlkxGM=">ACHnicbVDLSgMxFM34tr6qLt0Ei6CIZUYU3QiG5cKVoV2LJn0Tg1NMkNyRyzDfIkbf8WNC0UEV/o3po+FrwMXTs65l9x7olQKi7/6Y2Mjo1PTE5Nl2Zm5+YXyotLFzbJDIcaT2RiriJmQoNRQo4So1wFQk4TLqHPf8y1swViT6HLsphIq1tYgFZ+ikZnm3gXCHeSx0G0xR74T0gA4km3FerGu6Sbev85WUGzQhkpa7qU2muWKX/X7oH9JMCQVMsRps/zeaCU8U6CRS2ZtPfBTDHNmUHAJRamRWUgZ7A21B3VTIEN8/5BV1zSovGiXGlkfbV7xM5U9Z2VeQ6FcMb+9vrif959Qzj/TAXOs0QNB98FGeSYkJ7WdGWMBRdh1h3Ai3K+U3zDCOLtGSCyH4fJfcrFdDfxqcLZTOTwaxjFVsgqWScB2SOH5ISckhrh5J48kmfy4j14T96r9zZoHfGM8vkB7yPLztQoTU=</latexit> Fingers 0 2 3.lookup(k) 38 hash(k) = 42 3 34 6 32 30 8 m=6 (2 m = 64) 28 10 26 12 24 14 22 16 20 18 finger[ k ] = succ( n + 2 k − 1 ) mod 2 m ) Finger

<latexit sha1_base64="o0lJPfhPEwESip2b8DuHRmlkxGM=">ACHnicbVDLSgMxFM34tr6qLt0Ei6CIZUYU3QiG5cKVoV2LJn0Tg1NMkNyRyzDfIkbf8WNC0UEV/o3po+FrwMXTs65l9x7olQKi7/6Y2Mjo1PTE5Nl2Zm5+YXyotLFzbJDIcaT2RiriJmQoNRQo4So1wFQk4TLqHPf8y1swViT6HLsphIq1tYgFZ+ikZnm3gXCHeSx0G0xR74T0gA4km3FerGu6Sbev85WUGzQhkpa7qU2muWKX/X7oH9JMCQVMsRps/zeaCU8U6CRS2ZtPfBTDHNmUHAJRamRWUgZ7A21B3VTIEN8/5BV1zSovGiXGlkfbV7xM5U9Z2VeQ6FcMb+9vrif959Qzj/TAXOs0QNB98FGeSYkJ7WdGWMBRdh1h3Ai3K+U3zDCOLtGSCyH4fJfcrFdDfxqcLZTOTwaxjFVsgqWScB2SOH5ISckhrh5J48kmfy4j14T96r9zZoHfGM8vkB7yPLztQoTU=</latexit> <latexit sha1_base64="o0lJPfhPEwESip2b8DuHRmlkxGM=">ACHnicbVDLSgMxFM34tr6qLt0Ei6CIZUYU3QiG5cKVoV2LJn0Tg1NMkNyRyzDfIkbf8WNC0UEV/o3po+FrwMXTs65l9x7olQKi7/6Y2Mjo1PTE5Nl2Zm5+YXyotLFzbJDIcaT2RiriJmQoNRQo4So1wFQk4TLqHPf8y1swViT6HLsphIq1tYgFZ+ikZnm3gXCHeSx0G0xR74T0gA4km3FerGu6Sbev85WUGzQhkpa7qU2muWKX/X7oH9JMCQVMsRps/zeaCU8U6CRS2ZtPfBTDHNmUHAJRamRWUgZ7A21B3VTIEN8/5BV1zSovGiXGlkfbV7xM5U9Z2VeQ6FcMb+9vrif959Qzj/TAXOs0QNB98FGeSYkJ7WdGWMBRdh1h3Ai3K+U3zDCOLtGSCyH4fJfcrFdDfxqcLZTOTwaxjFVsgqWScB2SOH5ISckhrh5J48kmfy4j14T96r9zZoHfGM8vkB7yPLztQoTU=</latexit> <latexit sha1_base64="o0lJPfhPEwESip2b8DuHRmlkxGM=">ACHnicbVDLSgMxFM34tr6qLt0Ei6CIZUYU3QiG5cKVoV2LJn0Tg1NMkNyRyzDfIkbf8WNC0UEV/o3po+FrwMXTs65l9x7olQKi7/6Y2Mjo1PTE5Nl2Zm5+YXyotLFzbJDIcaT2RiriJmQoNRQo4So1wFQk4TLqHPf8y1swViT6HLsphIq1tYgFZ+ikZnm3gXCHeSx0G0xR74T0gA4km3FerGu6Sbev85WUGzQhkpa7qU2muWKX/X7oH9JMCQVMsRps/zeaCU8U6CRS2ZtPfBTDHNmUHAJRamRWUgZ7A21B3VTIEN8/5BV1zSovGiXGlkfbV7xM5U9Z2VeQ6FcMb+9vrif959Qzj/TAXOs0QNB98FGeSYkJ7WdGWMBRdh1h3Ai3K+U3zDCOLtGSCyH4fJfcrFdDfxqcLZTOTwaxjFVsgqWScB2SOH5ISckhrh5J48kmfy4j14T96r9zZoHfGM8vkB7yPLztQoTU=</latexit> <latexit sha1_base64="o0lJPfhPEwESip2b8DuHRmlkxGM=">ACHnicbVDLSgMxFM34tr6qLt0Ei6CIZUYU3QiG5cKVoV2LJn0Tg1NMkNyRyzDfIkbf8WNC0UEV/o3po+FrwMXTs65l9x7olQKi7/6Y2Mjo1PTE5Nl2Zm5+YXyotLFzbJDIcaT2RiriJmQoNRQo4So1wFQk4TLqHPf8y1swViT6HLsphIq1tYgFZ+ikZnm3gXCHeSx0G0xR74T0gA4km3FerGu6Sbev85WUGzQhkpa7qU2muWKX/X7oH9JMCQVMsRps/zeaCU8U6CRS2ZtPfBTDHNmUHAJRamRWUgZ7A21B3VTIEN8/5BV1zSovGiXGlkfbV7xM5U9Z2VeQ6FcMb+9vrif959Qzj/TAXOs0QNB98FGeSYkJ7WdGWMBRdh1h3Ai3K+U3zDCOLtGSCyH4fJfcrFdDfxqcLZTOTwaxjFVsgqWScB2SOH5ISckhrh5J48kmfy4j14T96r9zZoHfGM8vkB7yPLztQoTU=</latexit> Fingers 0 2 3.lookup(k) 38 hash(k) = 42 3 34 1 6 32 30 8 m=6 (2 m = 64) 28 10 26 12 24 14 22 16 20 18 finger[ k ] = succ( n + 2 k − 1 ) mod 2 m ) Finger

<latexit sha1_base64="o0lJPfhPEwESip2b8DuHRmlkxGM=">ACHnicbVDLSgMxFM34tr6qLt0Ei6CIZUYU3QiG5cKVoV2LJn0Tg1NMkNyRyzDfIkbf8WNC0UEV/o3po+FrwMXTs65l9x7olQKi7/6Y2Mjo1PTE5Nl2Zm5+YXyotLFzbJDIcaT2RiriJmQoNRQo4So1wFQk4TLqHPf8y1swViT6HLsphIq1tYgFZ+ikZnm3gXCHeSx0G0xR74T0gA4km3FerGu6Sbev85WUGzQhkpa7qU2muWKX/X7oH9JMCQVMsRps/zeaCU8U6CRS2ZtPfBTDHNmUHAJRamRWUgZ7A21B3VTIEN8/5BV1zSovGiXGlkfbV7xM5U9Z2VeQ6FcMb+9vrif959Qzj/TAXOs0QNB98FGeSYkJ7WdGWMBRdh1h3Ai3K+U3zDCOLtGSCyH4fJfcrFdDfxqcLZTOTwaxjFVsgqWScB2SOH5ISckhrh5J48kmfy4j14T96r9zZoHfGM8vkB7yPLztQoTU=</latexit> <latexit sha1_base64="o0lJPfhPEwESip2b8DuHRmlkxGM=">ACHnicbVDLSgMxFM34tr6qLt0Ei6CIZUYU3QiG5cKVoV2LJn0Tg1NMkNyRyzDfIkbf8WNC0UEV/o3po+FrwMXTs65l9x7olQKi7/6Y2Mjo1PTE5Nl2Zm5+YXyotLFzbJDIcaT2RiriJmQoNRQo4So1wFQk4TLqHPf8y1swViT6HLsphIq1tYgFZ+ikZnm3gXCHeSx0G0xR74T0gA4km3FerGu6Sbev85WUGzQhkpa7qU2muWKX/X7oH9JMCQVMsRps/zeaCU8U6CRS2ZtPfBTDHNmUHAJRamRWUgZ7A21B3VTIEN8/5BV1zSovGiXGlkfbV7xM5U9Z2VeQ6FcMb+9vrif959Qzj/TAXOs0QNB98FGeSYkJ7WdGWMBRdh1h3Ai3K+U3zDCOLtGSCyH4fJfcrFdDfxqcLZTOTwaxjFVsgqWScB2SOH5ISckhrh5J48kmfy4j14T96r9zZoHfGM8vkB7yPLztQoTU=</latexit> <latexit sha1_base64="o0lJPfhPEwESip2b8DuHRmlkxGM=">ACHnicbVDLSgMxFM34tr6qLt0Ei6CIZUYU3QiG5cKVoV2LJn0Tg1NMkNyRyzDfIkbf8WNC0UEV/o3po+FrwMXTs65l9x7olQKi7/6Y2Mjo1PTE5Nl2Zm5+YXyotLFzbJDIcaT2RiriJmQoNRQo4So1wFQk4TLqHPf8y1swViT6HLsphIq1tYgFZ+ikZnm3gXCHeSx0G0xR74T0gA4km3FerGu6Sbev85WUGzQhkpa7qU2muWKX/X7oH9JMCQVMsRps/zeaCU8U6CRS2ZtPfBTDHNmUHAJRamRWUgZ7A21B3VTIEN8/5BV1zSovGiXGlkfbV7xM5U9Z2VeQ6FcMb+9vrif959Qzj/TAXOs0QNB98FGeSYkJ7WdGWMBRdh1h3Ai3K+U3zDCOLtGSCyH4fJfcrFdDfxqcLZTOTwaxjFVsgqWScB2SOH5ISckhrh5J48kmfy4j14T96r9zZoHfGM8vkB7yPLztQoTU=</latexit> <latexit sha1_base64="o0lJPfhPEwESip2b8DuHRmlkxGM=">ACHnicbVDLSgMxFM34tr6qLt0Ei6CIZUYU3QiG5cKVoV2LJn0Tg1NMkNyRyzDfIkbf8WNC0UEV/o3po+FrwMXTs65l9x7olQKi7/6Y2Mjo1PTE5Nl2Zm5+YXyotLFzbJDIcaT2RiriJmQoNRQo4So1wFQk4TLqHPf8y1swViT6HLsphIq1tYgFZ+ikZnm3gXCHeSx0G0xR74T0gA4km3FerGu6Sbev85WUGzQhkpa7qU2muWKX/X7oH9JMCQVMsRps/zeaCU8U6CRS2ZtPfBTDHNmUHAJRamRWUgZ7A21B3VTIEN8/5BV1zSovGiXGlkfbV7xM5U9Z2VeQ6FcMb+9vrif959Qzj/TAXOs0QNB98FGeSYkJ7WdGWMBRdh1h3Ai3K+U3zDCOLtGSCyH4fJfcrFdDfxqcLZTOTwaxjFVsgqWScB2SOH5ISckhrh5J48kmfy4j14T96r9zZoHfGM8vkB7yPLztQoTU=</latexit> Fingers 0 2 3.lookup(k) 38 hash(k) = 42 3 34 1 2 6 32 30 8 m=6 (2 m = 64) 28 10 26 12 24 14 22 16 20 18 finger[ k ] = succ( n + 2 k − 1 ) mod 2 m ) Finger

<latexit sha1_base64="o0lJPfhPEwESip2b8DuHRmlkxGM=">ACHnicbVDLSgMxFM34tr6qLt0Ei6CIZUYU3QiG5cKVoV2LJn0Tg1NMkNyRyzDfIkbf8WNC0UEV/o3po+FrwMXTs65l9x7olQKi7/6Y2Mjo1PTE5Nl2Zm5+YXyotLFzbJDIcaT2RiriJmQoNRQo4So1wFQk4TLqHPf8y1swViT6HLsphIq1tYgFZ+ikZnm3gXCHeSx0G0xR74T0gA4km3FerGu6Sbev85WUGzQhkpa7qU2muWKX/X7oH9JMCQVMsRps/zeaCU8U6CRS2ZtPfBTDHNmUHAJRamRWUgZ7A21B3VTIEN8/5BV1zSovGiXGlkfbV7xM5U9Z2VeQ6FcMb+9vrif959Qzj/TAXOs0QNB98FGeSYkJ7WdGWMBRdh1h3Ai3K+U3zDCOLtGSCyH4fJfcrFdDfxqcLZTOTwaxjFVsgqWScB2SOH5ISckhrh5J48kmfy4j14T96r9zZoHfGM8vkB7yPLztQoTU=</latexit> <latexit sha1_base64="o0lJPfhPEwESip2b8DuHRmlkxGM=">ACHnicbVDLSgMxFM34tr6qLt0Ei6CIZUYU3QiG5cKVoV2LJn0Tg1NMkNyRyzDfIkbf8WNC0UEV/o3po+FrwMXTs65l9x7olQKi7/6Y2Mjo1PTE5Nl2Zm5+YXyotLFzbJDIcaT2RiriJmQoNRQo4So1wFQk4TLqHPf8y1swViT6HLsphIq1tYgFZ+ikZnm3gXCHeSx0G0xR74T0gA4km3FerGu6Sbev85WUGzQhkpa7qU2muWKX/X7oH9JMCQVMsRps/zeaCU8U6CRS2ZtPfBTDHNmUHAJRamRWUgZ7A21B3VTIEN8/5BV1zSovGiXGlkfbV7xM5U9Z2VeQ6FcMb+9vrif959Qzj/TAXOs0QNB98FGeSYkJ7WdGWMBRdh1h3Ai3K+U3zDCOLtGSCyH4fJfcrFdDfxqcLZTOTwaxjFVsgqWScB2SOH5ISckhrh5J48kmfy4j14T96r9zZoHfGM8vkB7yPLztQoTU=</latexit> <latexit sha1_base64="o0lJPfhPEwESip2b8DuHRmlkxGM=">ACHnicbVDLSgMxFM34tr6qLt0Ei6CIZUYU3QiG5cKVoV2LJn0Tg1NMkNyRyzDfIkbf8WNC0UEV/o3po+FrwMXTs65l9x7olQKi7/6Y2Mjo1PTE5Nl2Zm5+YXyotLFzbJDIcaT2RiriJmQoNRQo4So1wFQk4TLqHPf8y1swViT6HLsphIq1tYgFZ+ikZnm3gXCHeSx0G0xR74T0gA4km3FerGu6Sbev85WUGzQhkpa7qU2muWKX/X7oH9JMCQVMsRps/zeaCU8U6CRS2ZtPfBTDHNmUHAJRamRWUgZ7A21B3VTIEN8/5BV1zSovGiXGlkfbV7xM5U9Z2VeQ6FcMb+9vrif959Qzj/TAXOs0QNB98FGeSYkJ7WdGWMBRdh1h3Ai3K+U3zDCOLtGSCyH4fJfcrFdDfxqcLZTOTwaxjFVsgqWScB2SOH5ISckhrh5J48kmfy4j14T96r9zZoHfGM8vkB7yPLztQoTU=</latexit> <latexit sha1_base64="o0lJPfhPEwESip2b8DuHRmlkxGM=">ACHnicbVDLSgMxFM34tr6qLt0Ei6CIZUYU3QiG5cKVoV2LJn0Tg1NMkNyRyzDfIkbf8WNC0UEV/o3po+FrwMXTs65l9x7olQKi7/6Y2Mjo1PTE5Nl2Zm5+YXyotLFzbJDIcaT2RiriJmQoNRQo4So1wFQk4TLqHPf8y1swViT6HLsphIq1tYgFZ+ikZnm3gXCHeSx0G0xR74T0gA4km3FerGu6Sbev85WUGzQhkpa7qU2muWKX/X7oH9JMCQVMsRps/zeaCU8U6CRS2ZtPfBTDHNmUHAJRamRWUgZ7A21B3VTIEN8/5BV1zSovGiXGlkfbV7xM5U9Z2VeQ6FcMb+9vrif959Qzj/TAXOs0QNB98FGeSYkJ7WdGWMBRdh1h3Ai3K+U3zDCOLtGSCyH4fJfcrFdDfxqcLZTOTwaxjFVsgqWScB2SOH5ISckhrh5J48kmfy4j14T96r9zZoHfGM8vkB7yPLztQoTU=</latexit> Fingers 0 2 3.lookup(k) 38 hash(k) = 42 3 34 1 2 6 32 3 30 8 m=6 (2 m = 64) 28 10 26 12 24 14 22 16 20 18 finger[ k ] = succ( n + 2 k − 1 ) mod 2 m ) Finger

<latexit sha1_base64="o0lJPfhPEwESip2b8DuHRmlkxGM=">ACHnicbVDLSgMxFM34tr6qLt0Ei6CIZUYU3QiG5cKVoV2LJn0Tg1NMkNyRyzDfIkbf8WNC0UEV/o3po+FrwMXTs65l9x7olQKi7/6Y2Mjo1PTE5Nl2Zm5+YXyotLFzbJDIcaT2RiriJmQoNRQo4So1wFQk4TLqHPf8y1swViT6HLsphIq1tYgFZ+ikZnm3gXCHeSx0G0xR74T0gA4km3FerGu6Sbev85WUGzQhkpa7qU2muWKX/X7oH9JMCQVMsRps/zeaCU8U6CRS2ZtPfBTDHNmUHAJRamRWUgZ7A21B3VTIEN8/5BV1zSovGiXGlkfbV7xM5U9Z2VeQ6FcMb+9vrif959Qzj/TAXOs0QNB98FGeSYkJ7WdGWMBRdh1h3Ai3K+U3zDCOLtGSCyH4fJfcrFdDfxqcLZTOTwaxjFVsgqWScB2SOH5ISckhrh5J48kmfy4j14T96r9zZoHfGM8vkB7yPLztQoTU=</latexit> <latexit sha1_base64="o0lJPfhPEwESip2b8DuHRmlkxGM=">ACHnicbVDLSgMxFM34tr6qLt0Ei6CIZUYU3QiG5cKVoV2LJn0Tg1NMkNyRyzDfIkbf8WNC0UEV/o3po+FrwMXTs65l9x7olQKi7/6Y2Mjo1PTE5Nl2Zm5+YXyotLFzbJDIcaT2RiriJmQoNRQo4So1wFQk4TLqHPf8y1swViT6HLsphIq1tYgFZ+ikZnm3gXCHeSx0G0xR74T0gA4km3FerGu6Sbev85WUGzQhkpa7qU2muWKX/X7oH9JMCQVMsRps/zeaCU8U6CRS2ZtPfBTDHNmUHAJRamRWUgZ7A21B3VTIEN8/5BV1zSovGiXGlkfbV7xM5U9Z2VeQ6FcMb+9vrif959Qzj/TAXOs0QNB98FGeSYkJ7WdGWMBRdh1h3Ai3K+U3zDCOLtGSCyH4fJfcrFdDfxqcLZTOTwaxjFVsgqWScB2SOH5ISckhrh5J48kmfy4j14T96r9zZoHfGM8vkB7yPLztQoTU=</latexit> <latexit sha1_base64="o0lJPfhPEwESip2b8DuHRmlkxGM=">ACHnicbVDLSgMxFM34tr6qLt0Ei6CIZUYU3QiG5cKVoV2LJn0Tg1NMkNyRyzDfIkbf8WNC0UEV/o3po+FrwMXTs65l9x7olQKi7/6Y2Mjo1PTE5Nl2Zm5+YXyotLFzbJDIcaT2RiriJmQoNRQo4So1wFQk4TLqHPf8y1swViT6HLsphIq1tYgFZ+ikZnm3gXCHeSx0G0xR74T0gA4km3FerGu6Sbev85WUGzQhkpa7qU2muWKX/X7oH9JMCQVMsRps/zeaCU8U6CRS2ZtPfBTDHNmUHAJRamRWUgZ7A21B3VTIEN8/5BV1zSovGiXGlkfbV7xM5U9Z2VeQ6FcMb+9vrif959Qzj/TAXOs0QNB98FGeSYkJ7WdGWMBRdh1h3Ai3K+U3zDCOLtGSCyH4fJfcrFdDfxqcLZTOTwaxjFVsgqWScB2SOH5ISckhrh5J48kmfy4j14T96r9zZoHfGM8vkB7yPLztQoTU=</latexit> <latexit sha1_base64="o0lJPfhPEwESip2b8DuHRmlkxGM=">ACHnicbVDLSgMxFM34tr6qLt0Ei6CIZUYU3QiG5cKVoV2LJn0Tg1NMkNyRyzDfIkbf8WNC0UEV/o3po+FrwMXTs65l9x7olQKi7/6Y2Mjo1PTE5Nl2Zm5+YXyotLFzbJDIcaT2RiriJmQoNRQo4So1wFQk4TLqHPf8y1swViT6HLsphIq1tYgFZ+ikZnm3gXCHeSx0G0xR74T0gA4km3FerGu6Sbev85WUGzQhkpa7qU2muWKX/X7oH9JMCQVMsRps/zeaCU8U6CRS2ZtPfBTDHNmUHAJRamRWUgZ7A21B3VTIEN8/5BV1zSovGiXGlkfbV7xM5U9Z2VeQ6FcMb+9vrif959Qzj/TAXOs0QNB98FGeSYkJ7WdGWMBRdh1h3Ai3K+U3zDCOLtGSCyH4fJfcrFdDfxqcLZTOTwaxjFVsgqWScB2SOH5ISckhrh5J48kmfy4j14T96r9zZoHfGM8vkB7yPLztQoTU=</latexit> Fingers 0 2 3.lookup(k) 38 hash(k) = 42 3 34 1 2 6 32 3 30 8 m=6 (2 m = 64) 28 10 26 4 12 24 14 22 16 20 18 finger[ k ] = succ( n + 2 k − 1 ) mod 2 m ) Finger

<latexit sha1_base64="o0lJPfhPEwESip2b8DuHRmlkxGM=">ACHnicbVDLSgMxFM34tr6qLt0Ei6CIZUYU3QiG5cKVoV2LJn0Tg1NMkNyRyzDfIkbf8WNC0UEV/o3po+FrwMXTs65l9x7olQKi7/6Y2Mjo1PTE5Nl2Zm5+YXyotLFzbJDIcaT2RiriJmQoNRQo4So1wFQk4TLqHPf8y1swViT6HLsphIq1tYgFZ+ikZnm3gXCHeSx0G0xR74T0gA4km3FerGu6Sbev85WUGzQhkpa7qU2muWKX/X7oH9JMCQVMsRps/zeaCU8U6CRS2ZtPfBTDHNmUHAJRamRWUgZ7A21B3VTIEN8/5BV1zSovGiXGlkfbV7xM5U9Z2VeQ6FcMb+9vrif959Qzj/TAXOs0QNB98FGeSYkJ7WdGWMBRdh1h3Ai3K+U3zDCOLtGSCyH4fJfcrFdDfxqcLZTOTwaxjFVsgqWScB2SOH5ISckhrh5J48kmfy4j14T96r9zZoHfGM8vkB7yPLztQoTU=</latexit> <latexit sha1_base64="o0lJPfhPEwESip2b8DuHRmlkxGM=">ACHnicbVDLSgMxFM34tr6qLt0Ei6CIZUYU3QiG5cKVoV2LJn0Tg1NMkNyRyzDfIkbf8WNC0UEV/o3po+FrwMXTs65l9x7olQKi7/6Y2Mjo1PTE5Nl2Zm5+YXyotLFzbJDIcaT2RiriJmQoNRQo4So1wFQk4TLqHPf8y1swViT6HLsphIq1tYgFZ+ikZnm3gXCHeSx0G0xR74T0gA4km3FerGu6Sbev85WUGzQhkpa7qU2muWKX/X7oH9JMCQVMsRps/zeaCU8U6CRS2ZtPfBTDHNmUHAJRamRWUgZ7A21B3VTIEN8/5BV1zSovGiXGlkfbV7xM5U9Z2VeQ6FcMb+9vrif959Qzj/TAXOs0QNB98FGeSYkJ7WdGWMBRdh1h3Ai3K+U3zDCOLtGSCyH4fJfcrFdDfxqcLZTOTwaxjFVsgqWScB2SOH5ISckhrh5J48kmfy4j14T96r9zZoHfGM8vkB7yPLztQoTU=</latexit> <latexit sha1_base64="o0lJPfhPEwESip2b8DuHRmlkxGM=">ACHnicbVDLSgMxFM34tr6qLt0Ei6CIZUYU3QiG5cKVoV2LJn0Tg1NMkNyRyzDfIkbf8WNC0UEV/o3po+FrwMXTs65l9x7olQKi7/6Y2Mjo1PTE5Nl2Zm5+YXyotLFzbJDIcaT2RiriJmQoNRQo4So1wFQk4TLqHPf8y1swViT6HLsphIq1tYgFZ+ikZnm3gXCHeSx0G0xR74T0gA4km3FerGu6Sbev85WUGzQhkpa7qU2muWKX/X7oH9JMCQVMsRps/zeaCU8U6CRS2ZtPfBTDHNmUHAJRamRWUgZ7A21B3VTIEN8/5BV1zSovGiXGlkfbV7xM5U9Z2VeQ6FcMb+9vrif959Qzj/TAXOs0QNB98FGeSYkJ7WdGWMBRdh1h3Ai3K+U3zDCOLtGSCyH4fJfcrFdDfxqcLZTOTwaxjFVsgqWScB2SOH5ISckhrh5J48kmfy4j14T96r9zZoHfGM8vkB7yPLztQoTU=</latexit> <latexit sha1_base64="o0lJPfhPEwESip2b8DuHRmlkxGM=">ACHnicbVDLSgMxFM34tr6qLt0Ei6CIZUYU3QiG5cKVoV2LJn0Tg1NMkNyRyzDfIkbf8WNC0UEV/o3po+FrwMXTs65l9x7olQKi7/6Y2Mjo1PTE5Nl2Zm5+YXyotLFzbJDIcaT2RiriJmQoNRQo4So1wFQk4TLqHPf8y1swViT6HLsphIq1tYgFZ+ikZnm3gXCHeSx0G0xR74T0gA4km3FerGu6Sbev85WUGzQhkpa7qU2muWKX/X7oH9JMCQVMsRps/zeaCU8U6CRS2ZtPfBTDHNmUHAJRamRWUgZ7A21B3VTIEN8/5BV1zSovGiXGlkfbV7xM5U9Z2VeQ6FcMb+9vrif959Qzj/TAXOs0QNB98FGeSYkJ7WdGWMBRdh1h3Ai3K+U3zDCOLtGSCyH4fJfcrFdDfxqcLZTOTwaxjFVsgqWScB2SOH5ISckhrh5J48kmfy4j14T96r9zZoHfGM8vkB7yPLztQoTU=</latexit> Fingers 0 2 3.lookup(k) 38 hash(k) = 42 3 34 1 2 6 32 3 30 8 m=6 (2 m = 64) 28 10 26 4 12 24 14 5 22 16 20 18 finger[ k ] = succ( n + 2 k − 1 ) mod 2 m ) Finger

<latexit sha1_base64="o0lJPfhPEwESip2b8DuHRmlkxGM=">ACHnicbVDLSgMxFM34tr6qLt0Ei6CIZUYU3QiG5cKVoV2LJn0Tg1NMkNyRyzDfIkbf8WNC0UEV/o3po+FrwMXTs65l9x7olQKi7/6Y2Mjo1PTE5Nl2Zm5+YXyotLFzbJDIcaT2RiriJmQoNRQo4So1wFQk4TLqHPf8y1swViT6HLsphIq1tYgFZ+ikZnm3gXCHeSx0G0xR74T0gA4km3FerGu6Sbev85WUGzQhkpa7qU2muWKX/X7oH9JMCQVMsRps/zeaCU8U6CRS2ZtPfBTDHNmUHAJRamRWUgZ7A21B3VTIEN8/5BV1zSovGiXGlkfbV7xM5U9Z2VeQ6FcMb+9vrif959Qzj/TAXOs0QNB98FGeSYkJ7WdGWMBRdh1h3Ai3K+U3zDCOLtGSCyH4fJfcrFdDfxqcLZTOTwaxjFVsgqWScB2SOH5ISckhrh5J48kmfy4j14T96r9zZoHfGM8vkB7yPLztQoTU=</latexit> <latexit sha1_base64="o0lJPfhPEwESip2b8DuHRmlkxGM=">ACHnicbVDLSgMxFM34tr6qLt0Ei6CIZUYU3QiG5cKVoV2LJn0Tg1NMkNyRyzDfIkbf8WNC0UEV/o3po+FrwMXTs65l9x7olQKi7/6Y2Mjo1PTE5Nl2Zm5+YXyotLFzbJDIcaT2RiriJmQoNRQo4So1wFQk4TLqHPf8y1swViT6HLsphIq1tYgFZ+ikZnm3gXCHeSx0G0xR74T0gA4km3FerGu6Sbev85WUGzQhkpa7qU2muWKX/X7oH9JMCQVMsRps/zeaCU8U6CRS2ZtPfBTDHNmUHAJRamRWUgZ7A21B3VTIEN8/5BV1zSovGiXGlkfbV7xM5U9Z2VeQ6FcMb+9vrif959Qzj/TAXOs0QNB98FGeSYkJ7WdGWMBRdh1h3Ai3K+U3zDCOLtGSCyH4fJfcrFdDfxqcLZTOTwaxjFVsgqWScB2SOH5ISckhrh5J48kmfy4j14T96r9zZoHfGM8vkB7yPLztQoTU=</latexit> <latexit sha1_base64="o0lJPfhPEwESip2b8DuHRmlkxGM=">ACHnicbVDLSgMxFM34tr6qLt0Ei6CIZUYU3QiG5cKVoV2LJn0Tg1NMkNyRyzDfIkbf8WNC0UEV/o3po+FrwMXTs65l9x7olQKi7/6Y2Mjo1PTE5Nl2Zm5+YXyotLFzbJDIcaT2RiriJmQoNRQo4So1wFQk4TLqHPf8y1swViT6HLsphIq1tYgFZ+ikZnm3gXCHeSx0G0xR74T0gA4km3FerGu6Sbev85WUGzQhkpa7qU2muWKX/X7oH9JMCQVMsRps/zeaCU8U6CRS2ZtPfBTDHNmUHAJRamRWUgZ7A21B3VTIEN8/5BV1zSovGiXGlkfbV7xM5U9Z2VeQ6FcMb+9vrif959Qzj/TAXOs0QNB98FGeSYkJ7WdGWMBRdh1h3Ai3K+U3zDCOLtGSCyH4fJfcrFdDfxqcLZTOTwaxjFVsgqWScB2SOH5ISckhrh5J48kmfy4j14T96r9zZoHfGM8vkB7yPLztQoTU=</latexit> <latexit sha1_base64="o0lJPfhPEwESip2b8DuHRmlkxGM=">ACHnicbVDLSgMxFM34tr6qLt0Ei6CIZUYU3QiG5cKVoV2LJn0Tg1NMkNyRyzDfIkbf8WNC0UEV/o3po+FrwMXTs65l9x7olQKi7/6Y2Mjo1PTE5Nl2Zm5+YXyotLFzbJDIcaT2RiriJmQoNRQo4So1wFQk4TLqHPf8y1swViT6HLsphIq1tYgFZ+ikZnm3gXCHeSx0G0xR74T0gA4km3FerGu6Sbev85WUGzQhkpa7qU2muWKX/X7oH9JMCQVMsRps/zeaCU8U6CRS2ZtPfBTDHNmUHAJRamRWUgZ7A21B3VTIEN8/5BV1zSovGiXGlkfbV7xM5U9Z2VeQ6FcMb+9vrif959Qzj/TAXOs0QNB98FGeSYkJ7WdGWMBRdh1h3Ai3K+U3zDCOLtGSCyH4fJfcrFdDfxqcLZTOTwaxjFVsgqWScB2SOH5ISckhrh5J48kmfy4j14T96r9zZoHfGM8vkB7yPLztQoTU=</latexit> Fingers 0 2 3.lookup(k) 6 38 hash(k) = 42 3 34 1 2 6 32 3 30 8 m=6 (2 m = 64) 28 10 26 4 12 24 14 5 22 16 20 18 finger[ k ] = succ( n + 2 k − 1 ) mod 2 m ) Finger

Some Concerns • What happens with simultaneous joins and leaves? • Can you ever loose track of a node? • Can you ever get to a state where a node joins and then disappears?

An aside on hashing • Why SHA-1/SHA-256/...?

An aside on hashing • Why SHA-1/SHA-256/...? • Do we need cryptographic hash functions for this purpose?

An aside on hashing • Why SHA-1/SHA-256/...? • Do we need cryptographic hash functions for this purpose? • There are other hash functions that are much faster to compute.

An aside on hashing • Why SHA-1/SHA-256/...? • Do we need cryptographic hash functions for this purpose? • There are other hash functions that are much faster to compute. • Why are cryptographic hash functions well suited to this purpose?

Two Problems - One Solution? • Distributed data storage and management. • Load distribution.

Sharding and Load Balancing