F i n e - g r a i n e d F l o w e r a n d F u - - PowerPoint PPT Presentation

▶
f i n e g r a i n e d f l o w e r a n d f u n g i c l a s
SMART_READER_LITE
LIVE PREVIEW

F i n e - g r a i n e d F l o w e r a n d F u - - PowerPoint PPT Presentation

Aug 15, 2023 231 likes •396 views

F i n e - g r a i n e d F l o w e r a n d F u n g i C l a s s i fj c a t i o n a t C MP Mi l a n u l c , L u k P i c e k , J i Ma t a s 1 2 1 1 V i s u a l

slide-1
SLIDE 1

Mi l a n Š u l c F i n e

  • g

r a i n e d F l

  • w

e r a n d F u n g i C l a s s i fj c a t i

  • n

a t C MP

F i n e

  • g

r a i n e d F l

  • w

e r a n d F u n g i C l a s s i fj c a t i

  • n

a t C MP

Mi l a n Š u l c

1

, L u k á š P i c e k

2

, J i ř í Ma t a s

1

1 V i s u a l R e c

  • g

n i t i

  • n

G r

  • u

p , C e n t e r f

  • r

Ma c h i n e P e r c e p t i

  • n

, D e p t .

  • f

C y b e r n e t i c s , F E E C z e c h T e c h n i c a l U n i v e r s i t y i n P r a g u e 2 D e p t .

  • f

C y b e r n e t i c s , F A S U n i v e r s i t y

  • f

We s t B

  • h

e m i a

slide-2
SLIDE 2

Mi l a n Š u l c F i n e

  • g

r a i n e d F l

  • w

e r a n d F u n g i C l a s s i fj c a t i

  • n

a t C MP

Label Distributions

2 / 1 5

FGVCx Flower and Fungi Classification datasets available for training follow a “long-tail distribution” of classes, which may not correspond with the test-time distribution. FGVCx Flowers FGVCx Fungi

200 400 600 800 Class (sorted) 0.0 0.1 0.2 0.3 0.4 0.5 0.6 % samples T raining set 200 400 600 800 1000 1200 Class (sorted) 0.0 0.1 0.2 0.3 0.4 0.5 % samples T raining set Validation set

slide-3
SLIDE 3

Mi l a n Š u l c F i n e

  • g

r a i n e d F l

  • w

e r a n d F u n g i C l a s s i fj c a t i

  • n

a t C MP

10 20 Class (sorted by Ntrain) 500 1000 1500 2000 2500 # images T est set T raining set

Label Distributions

3 / 1 5

2000 4000 6000 8000 10000 Class (sorted by Ntrain) 500 1000 1500 2000 2500 # images T est set T raining set

We recently observed a similar problem in the LifeCLEF plant identification challenge: majority of training data comes from the web, while test images come from a different source. Can we compensate for this imbalance?

Figure: PlantCLEF 2017 label distribution in the “trusted” training set.

[1] Improving CNN classifiers by estimating test-time priors. Milan Šulc and Jiří Matas. arXiv:1805.08235 [cs.CV], 2018.

slide-4
SLIDE 4

Mi l a n Š u l c F i n e

  • g

r a i n e d F l

  • w

e r a n d F u n g i C l a s s i fj c a t i

  • n

a t C MP

Training neural networks (f with parameters ) by cross-entropy loss minimization means training it to estimate the posterior probabilities: where Then:

CNN Outputs as Posterior Estimates

4 / 1 5

slide-5
SLIDE 5

Mi l a n Š u l c F i n e

  • g

r a i n e d F l

  • w

e r a n d F u n g i C l a s s i fj c a t i

  • n

a t C MP

Experiment on selected subsets of CIFAR-100 with different class priors: How well do the posterior estimates marginalize over dataset samples?

CNN Outputs as Posterior Estimates

5 / 1 5 [1] Improving CNN classifiers by estimating test-time priors. Milan Šulc and Jiří Matas. arXiv:1805.08235 [cs.CV], 2018.

slide-6
SLIDE 6

Mi l a n Š u l c F i n e

  • g

r a i n e d F l

  • w

e r a n d F u n g i C l a s s i fj c a t i

  • n

a t C MP

Assuming that the probability density function remains unchanged: The mutual relation of the posteriors is:

Adjusting Estimates to New Priors

6 / 1 5

slide-7
SLIDE 7

Mi l a n Š u l c F i n e

  • g

r a i n e d F l

  • w

e r a n d F u n g i C l a s s i fj c a t i

  • n

a t C MP

When Test Set Priors Are Unknown

7 / 1 5

How to estimate the test-set priors? Saerens et al. [1] proposed a simple EM procedure to maximize the likelihood L(x0,x1,x2,...): This procedure is equivalent [2] to fixed-point-iteration minimization of the KL divergence between and .

[1] Adjusting the outputs of a classifier to new a priori probabilities: a simple procedure. Marco Saerens, Patrice Latinne, and Christine Decaestecker. Neural computation 14.1 (2002): 21-41. [2] Semi-supervised learning of class balance under class-prior change by distribution matching. Marthinus Christoffel Du Plessis and Masashi Sugiyama. Neural Networks, 50:110–119, 2014.

slide-8
SLIDE 8

Mi l a n Š u l c F i n e

  • g

r a i n e d F l

  • w

e r a n d F u n g i C l a s s i fj c a t i

  • n

a t C MP

Test Set Prior Estimation in LifeCLEF

8 / 1 5

Preliminary experiments (using the 2017 test set for validation):

  • When the whole test set is available:

Inception-ResNet-v2: 82.9% → 85.8% Inception-v4: 82.8% → 86.3%

  • On-line [1] after each new test image:

[1] Improving CNN classifiers by estimating test-time priors. Milan Šulc and Jiří Matas. arXiv:1805.08235 [cs.CV], 2018.

slide-9
SLIDE 9

Mi l a n Š u l c F i n e

  • g

r a i n e d F l

  • w

e r a n d F u n g i C l a s s i fj c a t i

  • n

a t C MP

When New Priors Are Known

9 / 1 5

200 400 600 800 1000 1200 Class (sorted) 0.0 0.1 0.2 0.3 0.4 0.5 % samples T raining set Validation set

FGVCx Fungi 2018

slide-10
SLIDE 10

Mi l a n Š u l c F i n e

  • g

r a i n e d F l

  • w

e r a n d F u n g i C l a s s i fj c a t i

  • n

a t C MP

When New Priors Are Known

1 / 1 5

Note: in the iNaturalist 2017 challenge, the winning GMV submission [1] approached the change in priors as follows: “To compensate for the imbalanced training data, the models were further fine-tuned on the 90% subset of the validation data that has a more balanced distribution.” We, instead, only use the validation set statistics – i.e. uniform class distribution in this case.

[1] The iNaturalist Species Classification and Detection Dataset-Supplementary Material. Grant Van Horn, Oisin Mac Aodha, Yang Song, Yin Cui, Chen Sun, Alex Shepard, Hartwig Adam, Pietro Perona, and Serge Belongie. Reptilia 32, no. 400: 5426.

slide-11
SLIDE 11

Mi l a n Š u l c F i n e

  • g

r a i n e d F l

  • w

e r a n d F u n g i C l a s s i fj c a t i

  • n

a t C MP

When New Priors Are Known

1 1 / 1 5

200 400 600 800 1000 1200 Class (sorted) 0.0 0.1 0.2 0.3 0.4 0.5 % samples T raining set Validation set 50000 100000 150000 200000 250000 300000 350000 400000 T raining steps 35.0 37.5 40.0 42.5 45.0 47.5 50.0 52.5 Accuracy [%] CNN output accuracy Known (fmat) test distr.

[1] Improving CNN classifiers by estimating test-time priors. Milan Šulc and Jiří Matas. arXiv:1805.08235 [cs.CV], 2018.

FGVCx Fungi 2018

Inception v4

slide-12
SLIDE 12

Mi l a n Š u l c F i n e

  • g

r a i n e d F l

  • w

e r a n d F u n g i C l a s s i fj c a t i

  • n

a t C MP

Tricks used in both challenges

1 2 / 1 5

Predictions re-weighted simply assuming uniform class priors. Moving average of trained variables (exponential decay). Training time augmentation:

  • Random crops
  • Color distortions

Test-time data augmentation: 14 per image : 7 crops 2 (mirror) ⨯ ⨯

slide-13
SLIDE 13

Mi l a n Š u l c F i n e

  • g

r a i n e d F l

  • w

e r a n d F u n g i C l a s s i fj c a t i

  • n

a t C MP

Final Ensembles

1 3 / 1 5

FGVCx Fungi: 6 nets (averaged) 2x Inception-v4 299x299 initialized from ImageNet and LifeCLEF ckpts 2x Inception-v4 598x598 initialized from ImageNet and LifeCLEF ckpts 2x Inception-ResNet-v2 299x299 from ImageNet and LifeCLEF ckpts FGVCx Flowers: 5 nets (modus) 3x Inception-v4 299x299 initialized from ImageNet, LifeCLEF, iNaturalist ckpts 1x Inception-v4 598x598 initialized from LifeCLEF ckpt 1x Inception-ResNet-v2 299x299 initialized from LifeCLEF ckpt

slide-14
SLIDE 14

Mi l a n Š u l c F i n e

  • g

r a i n e d F l

  • w

e r a n d F u n g i C l a s s i fj c a t i

  • n

a t C MP

Leaderboard

1 4 / 1 5

FGVCx Fungi FGVCx Flowers

slide-15
SLIDE 15

Mi l a n Š u l c F i n e

  • g

r a i n e d F l

  • w

e r a n d F u n g i C l a s s i fj c a t i

  • n

a t C MP

  • Standard CNN classifiers (and their ensembles) achieve

best results in plant and fungi recognition.

  • Future work: Learning from Ensembles?
  • Important to take into account change in class prior distribution [1]
  • New priors can be estimated on-line, as new test-samples appear.
  • Q & A sulcmila@cmp.felk.cvut.cz

1 5 / 1 5

Discussion

[1] Improving CNN classifiers by estimating test-time priors. Milan Šulc and Jiří Matas. arXiv:1805.08235 [cs.CV], 2018.