Self-Organizing MapsSince the second edition of this book came out in early 1997, the number of scientific papers published on the Self-Organizing Map (SOM) has increased from about 1500 to some 4000. Also, two special workshops dedicated to the SOM have been organized, not to mention numerous SOM sessions in neural network conferences. In view of this growing interest it was felt desirable to make extensive revisions to this book. They are of the following nature. Statistical pattern analysis has now been approached more carefully than earlier. A more detailed discussion of the eigenvectors and eigenvalues of symmetric matrices, which are the type usually encountered in statistics, has been included in Sect. 1.1.3: also, new probabilistic concepts, such as factor analysis, have been discussed in Sect. 1.3.1. A survey of projection methods (Sect. 1.3.2) has been added, in order to relate the SOM to classical paradigms. Vector Quantization is now discussed in one main section, and derivation of the point density of the codebook vectors using the calculus of variations has been added, in order to familiarize the reader with this otherwise com plicated statistical analysis. It was also felt that the discussion of the neural-modeling philosophy should include a broader perspective of the main issues. A historical review in Sect. 2.2, and the general philosophy in Sects. 2.3, 2.5 and 2.14 are now expected to especially help newcomers to orient themselves better amongst the profusion of contemporary neural models. |
Contents
1 Mathematical Preliminaries | 1 |
11 Mathematical Concepts and Notations | 2 |
112 Matrix Notations | 8 |
113 Eigenvectors and Eigenvalues of Matrices | 11 |
114 Further Properties of Matrices | 13 |
115 On Matrix Differential Calculus | 15 |
12 Distance Measures for Patterns | 17 |
122 Measures of Similarity and Distance Between Symbol Strings | 21 |
572 The Batch Map for Strings | 206 |
The SOM of Phonemic Transcriptions | 207 |
59 EvolutionaryLearning SOM | 211 |
592 SelfOrganization According to a Fitness Function | 212 |
510 Supervised SOM | 215 |
511 The AdaptiveSubspace SOM ASSOM | 216 |
5112 Relation Between Invariant Features and Linear Subspaces | 218 |
5113 The ASSOM Algorithm | 222 |
123 Averages Over Nonvectorial Variables | 28 |
13 Statistical Pattern Analysis | 29 |
132 Projection Methods | 34 |
133 Supervised Classification | 39 |
134 Unsupervised Classification | 44 |
14 The Subspace Methods of Classification | 46 |
142 Adaptation of a Model Subspace to Input Subspace | 49 |
143 The Learning Subspace Method LSM | 53 |
| 59 | |
152 Derivation of the VQ Algorithm | 60 |
153 Point Density in VQ | 62 |
16 Dynamically Expanding Context | 64 |
161 Setting Up the Problem | 65 |
162 Automatic Determination of ContextIndependent Productions | 66 |
163 Conflict Bit | 67 |
164 Construction of Memory for the ContextDependent Productions | 68 |
166 Estimation Procedure for Unsuccessful Searches | 69 |
2 Neural Modeling | 71 |
22 A History of Some Main Ideas in Neural Modeling | 72 |
23 Issues on Artificial Intelligence | 75 |
24 On the Complexity of Biological Nervous Systems | 76 |
25 What the Brain Circuits Are Not | 78 |
26 Relation Between Biological and Artificial Neural Networks | 79 |
27 What Functions of the Brain Are Usually Modeled? | 81 |
29 Transformation Relaxation and Decoder | 82 |
210 Categories of ANNs | 85 |
211 A Simple Nonlinear Dynamic Model of the Neuron | 87 |
212 Three Phases of Development of Neural Models | 89 |
213 Learning Laws | 91 |
2132 The RiccatiType Learning Law | 92 |
2133 The PCAType Learning Law | 95 |
214 Some Really Hard Problems | 96 |
215 Brain Maps | 99 |
3 The Basic SOM | 105 |
31 A Qualitative Introduction to the SOM | 106 |
32 The Original Incremental SOM Algorithm | 109 |
33 The DotProduct SOM | 115 |
34 Other Preliminary Demonstrations of TopologyPreserving Mappings | 116 |
342 Demonstrations of Ordering of Responses in the Output Space | 120 |
35 Basic Mathematical Approaches to SelfOrganization | 127 |
351 OneDimensional Case | 128 |
352 Constructive Proof of Ordering of Another OneDimensional SOM | 132 |
36 The Batch Map | 138 |
37 Initialization of the SOM Algorithms | 142 |
38 On the Optimal LearningRate Factor | 143 |
39 Effect of the Form of the Neighborhood Function | 145 |
310 Does the SOM Algorithm Ensue from a Distortion Measure? | 146 |
311 An Attempt to Optimize the SOM | 148 |
312 Point Density of the Model Vectors | 152 |
3122 Numerical Check of Point Densities in a Finite OneDimensional SOM | 153 |
313 Practical Advice for the Construction of Good Maps | 159 |
314 Examples of Data Analyses Implemented by the SOM | 161 |
Poverty Map | 165 |
316 Interpretation of the SOM Mapping | 166 |
3162 Contribution of a Variable to Cluster Structures | 169 |
317 Speedup of SOM Computation | 170 |
3172 Increasing the Number of Units in the SOM | 172 |
3173 Smoothing | 175 |
3174 Combination of Smoothing Lattice Growing and SOM Algorithm | 176 |
4 Physiological Interpretation of SOM | 177 |
42 Two Different Lateral Control Mechanisms | 178 |
421 The WTA Function Based on Lateral Activity Control | 179 |
422 Lateral Control of Plasticity | 184 |
43 Learning Equation | 185 |
45 Recapitulation of the Features of the Physiological SOM Model | 188 |
461 Magnification | 189 |
5 Variants of SOM | 191 |
52 Adaptive Tensorial Weights | 194 |
53 TreeStructured SOM in Searching | 197 |
54 Different Definitions of the Neighborhood | 198 |
55 Neighborhoods in the Signal Space | 200 |
56 Dynamical Elements Added to the SOM | 204 |
57 The SOM for Symbol Strings | 205 |
5114 Derivation of the ASSOM Algorithm by Stochastic Approximation | 226 |
5115 ASSOM Experiments | 228 |
512 FeedbackControlled AdaptiveSubspace SOM FASSOM | 242 |
6 Learning Vector Quantization | 245 |
62 The LVQ1 | 246 |
63 The OptimizedLearningRate LVQ1 OLVQ1 | 250 |
64 The BatchLVQ1 | 251 |
65 The BatchLVQ1 for Symbol Strings | 252 |
67 The LVQ3 | 253 |
68 Differences Between LVQ1 LVQ2 and LVQ3 | 254 |
610 The HypermapType LVQ | 256 |
611 The LVQSOM | 261 |
7 Applications | 263 |
71 Preprocessing of Optic Patterns | 264 |
711 Blurring | 265 |
712 Expansion in Terms of Global Features | 266 |
714 Expansion in Terms of Local Features Wavelets | 267 |
72 Acoustic Preprocessing | 268 |
73 Process and Machine Monitoring | 269 |
732 Analysis of Large Systems | 270 |
74 Diagnosis of Speech Voicing | 274 |
76 Texture Analysis | 280 |
77 Contextual Maps | 281 |
771 Artifically Generated Clauses | 283 |
772 Natural Text | 285 |
78 Organization of Large Document Files | 286 |
782 Construction of Very Large WEBSOM Maps by the Projection Method | 292 |
783 The WEBSOM of All Electronic Patent Abstracts | 296 |
79 RobotArm Control | 299 |
792 Another Simple RobotArm Control | 303 |
710 Telecommunications | 304 |
7102 Channel Equalization in the Adaptive QAM | 305 |
7103 ErrorTolerant Transmission of Images by a Pair of SOMs | 306 |
711 The SOM as an Estimator | 308 |
7112 Asymmetric Heteroassociative Mapping | 309 |
8 Software Tools for SOM | 311 |
82 Desirable Auxiliary Features | 313 |
83 SOM Program Packages | 315 |
832 SOM Toolbox | 317 |
833 Nenet Neural Networks Tool | 318 |
84 Examples of the Use of SOIV_PAK | 319 |
842 Description of the Programs in SOM_PAK | 322 |
843 A Typical Training Sequence | 326 |
85 NeuralNetworks Software with the SOM Option | 327 |
9 Hardware for SOM | 329 |
92 Fast Digital Classifier Circuits | 332 |
93 SIMD Implementation of SOM | 337 |
94 Transputer Implementation of SOM | 339 |
95 SystolicArray Implementation of SOM | 341 |
96 The COKOS Chip | 342 |
98 NBISOM_25 Chip | 344 |
10 An Overview of SOM Literature | 347 |
102 Early Works on Competitive Learning | 348 |
103 Status of the Mathematical Analyses | 349 |
1032 Alternative Topological Mappings | 350 |
1034 Functional Variants | 351 |
1035 Theory of the Basic SOM | 352 |
104 The Learning Vector Quantization | 358 |
1052 Optical Character and Script Reading | 360 |
1054 Acoustic and Musical Studies | 361 |
1055 Signal Processing and Radar Measurements | 362 |
1058 Process Control | 363 |
1059 Robotics | 364 |
10512 Chemistry | 365 |
10514 Neurophysiological Research | 366 |
10516 Linguistic and AI Problems | 367 |
10517 Mathematical and Other Theoretical Problems | 368 |
106 Applications of LVQ | 369 |
107 Survey of SOM and LVQ Implementations | 370 |
11 Glossary of Neural Terms | 373 |
References | 403 |
| 487 | |
Other editions - View all
Common terms and phrases
accuracy adaptive algorithm analysis applications approximation array Artificial Neural Networks ASSOM basic basis vectors brain cells classification clustering codebook vectors components computing convergence corresponding cortex defined denoted density function described dimensionality distance dot product dynamic elements equation Euclidean feature filters IEEE Service Center input data input samples input vector labels lattice learning learning-rate factor Levenshtein distance linear linear subspace map units mathematical matrix method neighborhood function Networks IEEE Service Neural Networks IEEE neuron nodes nonlinear operation optimal orthogonal output parameters pattern recognition phoneme Piscataway point density probability density function problem Proc projection quantization error random reference vectors respectively scalar Self-Organizing Map sequence similar simulation speech recognition statistical step stochastic stochastic approximation strings subset subspace supervised learning symbol synaptic tion topology transformation two-dimensional values variables vector quantization Voronoi tessellation wavelets whereby winner zero