Convolutional Poisson Gamma Belief Network Chaojie Wang Bo Chen - - PowerPoint PPT Presentation
Convolutional Poisson Gamma Belief Network Chaojie Wang Bo Chen Sucheng Xiao Mingyuan Zhou National Laboratory of Radar Signal Processing, Xidian University, Xi'an, Shaanxi, China McCombs School of Business, The
♖ National Laboratory of Radar Signal Processing, Xidian University, Xi'an, Shaanxi, China ♘ McCombs School of Business, The University of Texas at Austin, Austin, TX, USA
Chaojie Wang♖ Bo Chen♖ Sucheng Xiao♖ Mingyuan Zhou♘
1
2019-6-11
Xidian University & UT-Austin
ü Preserve all textual information ※ Extremely large and sparse matrices ※ Burdens of calculation and storage ※ Difficult to model directly ü Term-document frequency count matrix ※ Lose word order ü Project words to low-dimensional vectors ※ Require additional large corpora
q Simplified Lossy Representation
Simplified
“I love it” Document One-hot Sequence don't hate I it love I love it
1 1 1
Most basic representation Ø A sequence of one-hot vectors Ø Bag-of-words Ø Word embeddings
q Basic Lossless Representation
Challenge 2019-6-11
2
Xidian University & UT-Austin
q Generative model of CPFA
ü Preserve word order information ü Directly model sparse matrices ü Take advantages of the sparsity ü Support parallel computation ü Capture pharse-level topics
Probabilistic Convolutional Layer
2019-6-11
3
Xidian University & UT-Austin
q Generative model of CPGBN q Probabilistic Pooling Layer
ü Transfer the messages from deeper layers ü Jointly Train all the other layers ü Deep extention can boost performance ü Hierachical pharse-level topic Equivalent
2019-6-11
4
Xidian University & UT-Austin
q Convolutional inference network
ü Fast in out-of-sample prediction ü Parallel scalable inference ü Easy extension (e.g., modeling document labels)
q Weibull Reparameterization
Approximate
2019-6-11
5
Xidian University & UT-Austin
2019-6-11
6
Xidian University & UT-Austin
♖ National Laboratory of Radar Signal Processing, Xidian University, Xi'an, Shaanxi, China ♘ McCombs School of Business, The University of Texas at Austin, Austin, TX, USA
Chaojie Wang♖ Bo Chen♖ Sucheng Xiao♖ Mingyuan Zhou♘
2019-6-11
7
Xidian University & UT-Austin