MACHINE VISION Euripides G.M. Petrakis http://www.ced.tuc/~petrakis - - PowerPoint PPT Presentation

E.G.M. Petrakis Machine Vision (Introduction) 1

TECHNICAL UNIVERSITY OF CRETE

DEPARTMENT OF ELECTRONIC AND COMPUTER ENGINEERING

MACHINE VISION

Euripides G.M. Petrakis http://www.ced.tuc/~petrakis

Chania 2001

E.G.M. Petrakis Machine Vision (Introduction) 2

Machine Vision

• The goal of Machine Vision is to create a

model of the real world from images

– A machine vision system recovers useful information about a scene from its two dimensional projections – The world is three dimensional – Two dimensional digitized images

E.G.M. Petrakis Machine Vision (Introduction) 3

Machine Vision (2)

• Knowledge about the objects (regions) in a

scene and projection geometry is required.

• The information which is recovered differs

depending on the application

– Satellite, medical images etc.

• Processing takes place in stages:

– Enhancement, segmentation, image analysis and matching (pattern recognition).

Illumination Scene 2D Digital Image Image Description Image Acquisition Machine Vision System Feedback The goal of a machine vision system is to compute a meaningful description of the scene (e.g., object)

E.G.M. Petrakis Machine Vision (Introduction) 5

Machine Vision Stages

• Analog to digital

conversion

• Remove noise/patterns,

improve contrast

• Find regions (objects) in

the image

• Take measurements of
• bjects/relationships
• Match the above

description with similar description of known

• bjects (models)

Image Acquisition (by cameras, scanners etc) Image Processing Image Enhancement Image Restoration Image Segmentation Image Analysis (Binary Image Processing) Model Matching Pattern Recognition

E.G.M. Petrakis Machine Vision (Introduction) 6

Image Processing

Image Processing

• Image transformation

– image enhancement (filtering, edge detection, surface detection, computation of depth). – Image restoration (remove point/pattern degradation: there exist a mathematical expression of the type of degradation like e.g. Added multiplicative noise, sin/cos pattern degradation etc). Input Image Output Image

E.G.M. Petrakis Machine Vision (Introduction) 7

Image Segmentation

Image Segmentation

• Classify pixels into groups (regions/objects of interest)

sharing common characteristics.

– Intensity/Color, texture, motion etc.

• Two types of techniques:

– Region segmentation: find the pixels of a region. – Edge segmentation: find the pixels of its outline contour. Input Image Regions/Objects

E.G.M. Petrakis Machine Vision (Introduction) 8

Image Analysis

Image Analysis

• Take useful measurements from pixels, regions, spatial

relationships, motion etc.

– Grey scale / color intensity values; – Size, distance; – Velocity; Input Image Segmented Image (regions, objects) Measurements

E.G.M. Petrakis Machine Vision (Introduction) 9

Pattern Recognition

Model Matching Pattern Recognition

• Classify an image (region) into one of a number of known

classes

– Statistical pattern recognition (the measurements form vectors which are classified into classes); – Structural pattern recognition (decompose the image into primitive structures). Image/regions

• Measurements, or
• Structural description

Class identifier

E.G.M. Petrakis Machine Vision (Introduction) 10

Digital Image Representation

• Image: 2D array of gray level or color values

– Pixel: array element; – Pixel value: arithmetic value of gray level or color intensity.

• Gray level image: f = f(x,y)
• 3D image f=f(x,y,z)
• Color image (multi-spectral)

f = {Rred(x,y), Ggreen(x,y), Bblue(x,y)}

E.G.M. Petrakis Machine Vision (Introduction) 11

What a computer “sees” is very different from what a human sees. A computer sees pixels (arithmetic values) while a human sees shapes, structures etc.

E.G.M. Petrakis Machine Vision (Introduction) 12

Relationships to other fields

• Image Processing (IP)
• Pattern Recognition (PR)
• Computer Graphics (CG)
• Artificial Intelligence (AI)
• Neural Networks (NN)
• Psychophysics

E.G.M. Petrakis Machine Vision (Introduction) 13

Image Processing (IP)

• IP transforms images to images

– Image filtering, compression, restoration

• IP is applied at the early stages of machine

vision.

– IP is usually used to enhance particular information and to suppress noise.

E.G.M. Petrakis Machine Vision (Introduction) 14

Pattern Recognition (PR)

• PR classifies numerical and symbolic data.

– Statistical: classify feature vectors. – Structural: represent the composition of an

• bject in terms of primitives and parse this

description.

• PR is usually used to classify objects but
• bject recognition in machine vision usually

requires many other techniques.

E.G.M. Petrakis Machine Vision (Introduction) 15

Statistical Pattern Recognition

• Pattern: the description of an an object

– Feature vector – (size, roundness, color, texture)

• Pattern class: set of patterns with similar

characteristics.

• Take measurements from a population of

patterns.

• Classification: Map each pattern to a class.

E.G.M. Petrakis Machine Vision (Introduction) 16

Structure of PR Systems

input

Sensor Processing Measurements Classification class

E.G.M. Petrakis Machine Vision (Introduction) 17

Example of Statistical PR

• Two classes:

I. W1 Basketball players II. W2 jockeys

• Description: X = (X1, X2) = (height, weight)

X1 . . . . . .. . . … .. … … .. .. .. …… W2 W1 D(X) = AX1 + BX2 + C = 0 Decision function

• +

X2

E.G.M. Petrakis Machine Vision (Introduction) 18

Syntactic Pattern Recognition

• The structure is important
• Identify primitives

– E.g., Shape primitives

• Break down an image (shape) into a sequence of

such primitives.

• The way the primitives are related to each other to

form a shape is unique.

– Use a grammar/algorithm – Parse the shape

E.G.M. Petrakis Machine Vision (Introduction) 19

• Primitives
• G1,L(G1) : submedian Grammar
• G2,L(G2) : telocentric Grammar

E.G.M. Petrakis Machine Vision (Introduction) 20

• Each digit is represented by a waveform representing

black/white, white/black transitions (scan the image from Left to right.

E.G.M. Petrakis Machine Vision (Introduction) 21

Computer Graphics (CG)

• Machine vision is the analysis of images

while CG is the decomposition of images:

– CG generates images from geometric primitives (lines, circles, surfaces). – Machine vision is the inverse: estimate the geometric primitives from an image.

• Visualization and virtual reality bring these

two fields closer.

E.G.M. Petrakis Machine Vision (Introduction) 22

Artificial Intelligence (AI)

• Machine vision is considered to be sub-field of AI.
• AI studies the computational aspects of

intelligence.

• CV is used to analyze scenes and compute

symbolic representations from them.

• AI: perception, cognition, action

– Perception translates signals to symbols; – Cognition manipulates symbols; – Action translates symbols to signals that effect the world.

E.G.M. Petrakis Machine Vision (Introduction) 23

Psychophysics

• Psychophysics and cognitive science have

studied human vision for a long time.

• Many techniques in machine vision are

related to what is known about human vision.

E.G.M. Petrakis Machine Vision (Introduction) 24

Neural Networks (NN)

• NNs are being increasingly applied to solve

many machine vision problems.

• NN techniques are usually applied to solve

PR tasks.

– Image recognition/classification.

• They have also applied to segmentation and
• ther machine vision tasks.

E.G.M. Petrakis Machine Vision (Introduction) 25

Machine Vision Applications

• Robotics
• Medicine
• Remote Sensing
• Cartography
• Meteorology
• Quality inspection
• Reconnaissance

E.G.M. Petrakis Machine Vision (Introduction) 26

Robot Vision

• Machine vision can make a robot manipulator

much more versatile.

– Allow it to deal with variations in parts position and

• rientation.

E.G.M. Petrakis Machine Vision (Introduction) 27

Remote Sensing

• Take images from

high altitudes (from aircrafts, satellites).

• Find ships in the aerial

image of the dock.

– Find if new ships have arrived. – What kind of ships?

E.G.M. Petrakis Machine Vision (Introduction) 28

Remote Sensing (2)

• Analyze the image

– Generate a description – Match this descriptions with the descriptions of empty docs

• There are four ships

– Marked by “+”

E.G.M. Petrakis Machine Vision (Introduction) 29

Medical Applications

• Assist a physician to

reach a diagnosis.

• Construct 2D, 3D

anatomy models of the human body.

– CG geometric models.

• Analyze the image to

extract useful features.

E.G.M. Petrakis Machine Vision (Introduction) 30

Machine Vision Systems

• There is no universal machine vision system

– One system for each application

• Assumptions:

– Good lighting; – Low noise; – 2D images

• Passive - Active environment

– Changes in the environment call for different actions (e.g., turn left, push the break etc).

E.G.M. Petrakis Machine Vision (Introduction) 31

Vision by Man and Machine

• What is the mechanism of human vision?

– Can a machine do the same thing? – There are many studies; – Most are empirical.

• Humans and machines have different

– Software – Hardware

E.G.M. Petrakis Machine Vision (Introduction) 32

Human “Hardware”

• Photoreceptors take measurements of light signals.

– About 106 Photoreceptors.

• Retinal ganglion cells transmit electric and

chemical signals to the brain

– Complex 3D interconnections; – What the neurons do? In what sequence? – Algorithms?

• Heavy Parallelism.

E.G.M. Petrakis Machine Vision (Introduction) 33

Machine Vision Hardware

• PCs, workstations etc.
• Signals: 2D image arrays gray level/color values.
• Modules: low level processing, shape from

texture, motion, contours etc.

• Simple interconnections.
• No parallelism.

E.G.M. Petrakis Machine Vision (Introduction) 34

Course Outline

• Introduction to machine vision, applications,

Image formation, color, reflectance, depth, stereopsis.

• Basic image processing techniques (filtering,

digitization, restoration), Fourier transform.

• Binary image processing and analysis, Distance

transform, morphological operators.

E.G.M. Petrakis Machine Vision (Introduction) 35

Course Outline (2)

• Image segmentation (region segmentation, edge

segmentation).

• Edge detection, edge enhancement and
• linking. Thresholding, region growing, region

merging/splitting.

• Relaxation labeling, Hough transform.
• Image analysis, shape analysis. Polygonal

approximation, splines, skeletons. Shape features, multi-resolution representations.

E.G.M. Petrakis Machine Vision (Introduction) 36

Course Outline (3)

• Image representation, image - shape recognition

and classification. Attributed relational graphs, semantic nets.

• Image - shape matching (Fourier descriptors,

moments, matching in scale space).

• Texture representation and recognition, statistical

and structural methods.

• Motion, motion detection, optical flow.

E.G.M. Petrakis Machine Vision (Introduction) 37

Bibliography

• “Machine Vision”, Ramesh Jain, Rangachar

Kasturi, Brian G. Schunck, Mc Graw-Hill, 1995 (highly recommended!).

• "Image Processing, Analysis and Machine

Vision", Milan Sonka, Vaclav Hlavac, Roger Boyle, PWS Publishing, Second Edition.

• "Machine Vision, Theory, Algorithms,

Practicalities'', E. R. Davies, Academic Press, 1997.

E.G.M. Petrakis Machine Vision (Introduction) 38

• "Practical Computer Vision Using C'', J.
• R. Parker, John Wiley & Sons Inc., 1994.
• Selected articles from the literature.
• Lecture notes

(http://www.ced.tuc/~petrakis).

E.G.M. Petrakis Machine Vision (Introduction) 39

Grading Scheme

• Final Exam (F): 40%, min 5
• Assignments (Α): 40%
• Two assignments

– obligatory