SLIDE 1
Hardware Accelerated Similarity Search George Williams Who Am I? - - PowerPoint PPT Presentation
Hardware Accelerated Similarity Search George Williams Who Am I? - - PowerPoint PPT Presentation
Hardware Accelerated Similarity Search George Williams Who Am I? Director, GSI Technology Previously, Chief Data Scientist Senior Data Scientist AI Research Scientist Software Engineer Recent Headlines Convergence and Integration That was
SLIDE 2
SLIDE 3
Recent Headlines
SLIDE 4
Convergence and Integration
That was then...
SLIDE 5
This is Now: Technology Disintegration
SLIDE 6
More Innovation Around The Corner
?
SLIDE 7
GSI’s Similarity Search Accelerator
SLIDE 8
Agenda
- Chip Explosion
- GSI Technology
- What is Vector Similarity Search?
- GSI’s Similarity Search Accelerator
- Integration Case Studies: Bio, Database
- Early Adopters Program
SLIDE 9
Who Is GSI Technology?
SLIDE 10
Aerospace, Government, R&D High Performance SRAM and DRAM GSI Vector Similarity Search Accelerator Chip
What We Do
SLIDE 11
What Is Vector Similarity Search?
SLIDE 12
What is Vector Similarity Search?
Numeric Representation Bit-vector 0110000100 Coordinates (2.3, 5.6)
SLIDE 13
Numeric Representation Simple “Distance” Function d = Func (a, b)
What is Vector Similarity Search?
SLIDE 14
K = 5 K = 3
Numeric Representation Simple “Distance” Function K Nearest Neighbor (Top-K)
What is Vector Similarity Search?
SLIDE 15
K = 5 K = 3
Numeric Representation Simple “Distance” Functions K Nearest Neighbor (Top-K) Search is Computational E-Commerce, Bioinformatics
What is Vector Similarity Search?
SLIDE 16
E-Commerce: Visual Search
SLIDE 17
Visual Search
Binary Codes, Continuous Embeddings Euclidean, L1, Hamming, Cosine >1 Billion Images
SLIDE 18
Visual Search: Embedding Space
SLIDE 19
Bioinformatics: Molecule Similarity
Fingerprints Tanimoto Many Large DBs 100s GB
SLIDE 20
Molecule Similarity: Tanimoto
Jaccard Intersection / Union
SLIDE 21
Bioinformatics: Molecule Similarity
Drug Discovery of Novel Molecules Virtual Screening Activity (Toxicity) Prediction
SLIDE 22
Programming Interfaces
Idiomatic SQL Integrate Into Data Pipelines Leverage Skills of Data Eng & Scientists
SLIDE 23
Many Domains and Applications
E-Commerce / Recommendations Bioinformatics / Genomics Healthcare / Medical Records Cybersecurity / Malware Detection Computer Vision / Video Surveillance
SLIDE 24
GSI’s Similarity Search Accelerator
SLIDE 25
Computational Memory
SLIDE 26
“In-Place” Associative Processing
Bit Logic
- Programmable
- 2 million
SLIDE 27
Consumer Board Solution
16GB Memory
PCIe Card
128Mb 128Mb
2 Chips Per Board On Board DDR4 Main Memory SRAM Cache Per Chip
SLIDE 28
1 Chassis (4U) Solution
16GB Memory
PCIe Card
128Mb 128Mb
16GB Memory
PCIe Card
128Mb 128Mb
16GB Memory
PCIe Card
128Mb 128Mb
16GB Memory
PCIe Card
128Mb 128Mb
Chassis
4 Boards Per Chassis
SLIDE 29
Multiple Chassis Solution
One Chassis Is The Master
SLIDE 30
Network Attached Storage
RDMA support (NAS As Data Source)
Network
..
16GB Memory
PCIe Card
128Mb 128Mb
16GB Memory
PCIe Card
128Mb 128Mb
16GB Memory
PCIe Card
128Mb 128Mb
16GB Memory
PCIe Card
128Mb 128Mb
Chassis
...
SLIDE 31
Segmentation by Clustering
Offline Clustering / K-Means Avoids Full DB “Scan” Faster Performance
SLIDE 32
Availability
Q2, 2019 Q1, 2019
demo boards mass production chip
Q4, 2018
16GB Memory
PCIe Card
128Mb 128Mb
SLIDE 33
Weizmann Institute Case Study
“GSI’s [Accelerator] can dramatically reduce the time required to search our small molecules database...”
- Dr. Efrat Ben-Zeev, Computational Chemist
SLIDE 34
Weizmann Institute Case Study
Molecule Similarity Search Biovia Pipeline Pilot Application Query of 34M Molecule DB Takes 10 Minutes ! Using GSI Accelerator (estimated) Query Latency Reduced To 300ms 400 Queries In 1 Second
SLIDE 35
Biovia Application Integration
Application Python Library C Library DRIVERS GSI Accelerator
SLIDE 36
In-Memory Database Integration
Database C Library DRIVERS GSI Accelerator
SLIDE 37
In-Memory: Expected Performance
Memory Speed Vector Size Throughput (imgs/ sec) MemSQL ~50GB / sec 4 KB
12.5 million images/sec
GSI ~100GB / sec 4 KB
25 million images/sec
SLIDE 38
Early Adopters Program
Consult With Our Hardware and AI Experts Co-Development and App Integration Access to simulator and test hardware Co-Marketing Opportunity
SLIDE 39
GSI Upcoming Events
Nov, Open Data Science Panel, Visual Search Nov, PyData ( Washington DC ) Dec, GSI Similarity Search Accelerator Workshop Coming Soon, GSI’s Tech Meetup 2019, First Chips and Boards Available
SLIDE 40
Contact Us
Twitter: @gsitechnology @cgeorgewilliams Blogs: gsitechnology.com medium.com/gsitechnology Email: associativecomputing@gsitechnology.com
SLIDE 41
The End. Thanks !
SLIDE 42
SLIDE 43
Query
Option 1: INFERENCE IN PRODUCTION
- Image2Vec (VGG, Resnet)
- NLP
- LSTN
Option 2: Query Vector done by External Application
- Fingerprint Query
- In Memory Vector (Cosine)
Option 3: 3rd Party Inference Vector
SLIDE 44
Single Board
Small Database Fit All Data Into Cache For Lowest Latency If Larger, Paging Occurs To Memory Cluster Techniques
16GB Memory
PCIe Card
128Mb 128Mb
SLIDE 45
Large Database
Large Database (<1TB, Flat) Pharma, Drug Search, Weizmann Molecule Search, In-Memory
16GB Memory
PCIe Card
128Mb 128Mb
16GB Memory
PCIe Card
128Mb 128Mb
16GB Memory
PCIe Card
128Mb 128Mb
16GB Memory
PCIe Card
128Mb 128Mb
Chassis
...
SLIDE 46
Multi Board Solutions
For Huge Databases ( ~ 1TB )
16GB Memory
PCIe Card
128Mb 128Mb
16GB Memory
PCIe Card
128Mb 128Mb
16GB Memory
PCIe Card
128Mb 128Mb
16GB Memory
PCIe Card
128Mb 128Mb
Chassis
...
For Throughput: Batch Queries Split Across Boards Chassis Host Master Merges The Results
SLIDE 47
Offline Data Preparation
Training Inference Optimize For Cache and Memory
SLIDE 48
Clustering For Large Databases
Offline Clustering Centroids List <16GB Reduces Storage Only Centroids Are Kept Local For Real-Time Performance
SLIDE 49
Huge Database
Exact Nearest Neighbors For Small Vectors
16GB Memory
PCIe Card
128Mb 128Mb
16GB Memory
PCIe Card
128Mb 128Mb
16GB Memory
PCIe Card
128Mb 128Mb
16GB Memory
PCIe Card
128Mb 128Mb
Chassis
...
Approx For Large Vectors (Quantization) Large Scale Sim Search, FAISS
SLIDE 50
Biovia Application Integration
Biovia Application Used By Thousands of Bio-Tech Companies
SLIDE 51
Weizmann: Load A Database
SLIDE 52
Weizmann: 3rd Party Search
SLIDE 53
Weizmann: Select Search Method
SLIDE 54
Weizmann: Search
SLIDE 55
Weizmann: Define Parameters
SLIDE 56