Generation of Pattern Classifiers Based on Linear Nongroup CA

Abstract

Nongroup Cellular Automata(CA) having two trees in the state transition diagram of a CA is suitable for pattern classifier which divides pattern set into two classes. Maji et al. [1] classified patterns by using multiple attractor cellular automata as a pattern classifier with dependency vector. In this paper we propose a method of generation of a pattern classifier using feature vector which is the extension of dependency vector. In addition, we propose methods for finding nonreachable states in the 0-tree of the state transition diagram of TPMACA corresponding to the given feature vector for the analysis of the state transition behavior of the generated pattern classifier.

1. INTRODUCTION

Pattern recognition has applications in computer vision, radar processing, and text classification. Pattern recognition is to classify input data by classes to distinguish with extracting important characteristics and attributes from data. It is necessary to classify patterns into classes by automated devices for a given pattern set. The ultimate goal of pattern recognition system is to classify the recognized patterns into a member of corresponding pattern classes by detecting and extracting the common features from the given pattern set[2]. The important prerequisites for designing the pattern classifier are high throughput, small storage state space and low cost hardware implementation.

A Cellular Automata(CA) is a discrete dynamical system, which evolves in discrete space and time[3-9]. CA is divided into linear CA and nonlinear CA according to the type of rules applying to the state transition. If the rule of a CA cell employs only XOR logic, then the CA is called a linear CA, otherwise it is called a nonlinear CA[3]. In addition, CA is divided into group CA and nongroup CA. Linear CA are characterized utilizing matrix algebra. In this paper, we concentrate on linear CA with two-state 3-neighborhood dependency. Also we limit on discussion for null boundary condition only. A CA generating both cyclic and non-cyclic states is a nongroup CA. For a nongroup CA, there are some states that can not be reachable from any other state. These states are referred to as nonreachable states. The nongroup CA for which the state transition diagram consists of a set of disjoint components forming tree-like structures rooted at attractors are referred to as multiple attractor CA(MACA)[10]. Two predecessor MACA(TPMACA) having two trees is suitable for natural pattern classifier which divides pattern set into two classes.

Maji et al.[1] has designed a pattern classifier that can effectively classify 2-class patterns in a manner that minimizes the amount of memory by synthesizing MACA. They proposed a method for identifying and classifying patterns using a dependency vector. Cho et al. proposed method for synthesizing 90/150 maximum length CA, 90/150 TPMACA with two attractor trees and a two predecessor SACA(TPSACA)[8,11]. And Cho et al.[12] proposed a theoretical method of synthesizing an n-cell TPMACA derived from an (n-1)-cell TPMACA (n ≥ 4). Moreover, until now there does not exist any method for synthesizing MACA for the feature vectors of the forms (0 ∗⋯∗ 0) or (∗⋯∗ 100 ⋯ 01 ∗⋯∗). To overcome this problem, we propose a pattern classifier based on a linear nongroup CA that can classify patterns according to a given feature vector.

In this paper we propose two algorithms for generations of TPMACA and MACA corresponding to the given feature vectors. In addition, we propose methods for finding nonreachable states in the 0-tree of the state transition diagram of TPMACA corresponding to the given feature vector for the analysis of the state transition behavior of the generated pattern classifier.

 

2. PRELIMINARIES AND RELATED WORKS

CA has a simple, regular, modular and cascadable structure with logical neighborhood interconnection. The simple structure of CA with logical interconnections are ideally suited for hardware implementation[9]. The next state of the cell at time (t + 1) is affected by its state and the states of its neighbors (left and right neighbors) at time t. The next state of the ith CA cell is specified by fi as

where represents the state of the ith cell at the tth instant of time and fi is the next state function and referred to as the rule of CA. If the rule of a CA cell involves only XOR logic, then it is called a linear rule. A CA with all the cells having linear rules is called a linear CA. Table 1 shows linear rules which are used in this paper. The state transition matrix T of an n-cell linear CA can be represented as the following tridiagonal matrix which is called the state transition matrix of the CA[8]. For the state transition matrix T the next state Y of X is Y=TX, where X is the present state of the CA.

Table 1.Linear rules and state transition functions

An MACA with m attractors can be viewed as a pattern classifier. It classifies a given set of patterns into m distinct classes[13,14]. The following lemma shows properties of MACA.

Lemma 2.1.[10] For an n-cell d-depth MACA with characteristic polynomial xd(x+1)n-d and minimal polynomial xd(x+1),

Remark A. Let ℂ be a TPMACA whose characteristic polynomial and minimal polynomial are the same as xn-1(x+1). Then from Lemma 2.1 the number of attractors of the state transition diagram of ℂ is 2 and the number of states in each attractor tree is the same and equal to 2n-1. Also the depth of each tree is n-1.

We want to use feature vectors in order to classify n-bit patterns. Let P be a test pattern set and let P be divided into two classes S1 and S2 where S1 ∩ S2 = ∅ . Let V be a vector satisfying V⋅x = 0 and V⋅y≠ 0 for all x ∈ S1 and y ∈S2. This V is called a feature vector. Fig. 1 shows that the set of all 4-bit patterns is divided into two classes by V, where V=(1001).

Fig. 1.Pattern classification for V = (1001).

Lemma 2.2. [12,13] Let TM be the state transition matrix of an n-cell TPMACA whose characteristic polynomial and minimal polynomial are the same as xn-1(x+1) corresponding to a feature vector V of length n. Then the following hold.

Remark B. Let ℂ be an n-cell CA and let TM be the state transition matrix of ℂ satisfying the conditions (i) and (ii) in Lemma 2.2. Then ℂ becomes becomes a TPMACA with two trees. In fact (i) and (ii) in Lemma 2.2 are necessary and sufficient condition for ℂ to be TPMACA with two trees.

The following special forms of vectors VS1 and VS2 can not synthesize TPMACA[12,13].

 

3. ALGORITHMS FOR GENERATION OF A LINEAR NONGROUP CA CORRESPONDING TO FEATURE VECTORS

3.1 Generation of TPMACA corresponding to feature vectors

In this subsection we propose a method of synthesizing an n-cell TPMACA whose characteristic polynomial and minimal polynomial are the same as xn-1(x+1) by decomposing the given feature vector into the basic forms in Table 2. In order to synthesize the TPMACA corresponding to the given feature vector V, let (10⋯0 ), (0⋯01), (1⋯ 1) and (101) be the basic forms of V. For example if V=(00111101), then V consists of the basic forms (001), (1111) and (101). Table 2 shows rules of TPMACA corresponding to basic forms of V. The (i+1,i) entry of the state transition matrix TM=(tij) of an n-cell TPMACA corresponding to the basic forms of type (i), type (iii) or type (v) is ti+1,i = 1. Also the (i,i+1) entry of TM corresponding to the basic forms of type (ii), type (iv) or type (vi) is ti,i+1=1. Therefore, the rules used to synthesize TPMACA will be synthesized by using rules of type (i), type (iii) or type (v), or synthesized by using rules of type (ii), type (iv) or type (vi).

Table 2.Rules of TPMACA corresponding to feature vector V

All the feature vectors can be decomposed into basic forms in Table 2. For example, if V=(00111101011), then V is decomposed into v1=(001), v2=(1111), v3=(101), v4=(101) and v5=(11). Hereafter we represent V as V=[v1 v2 v3 v4 v5].

Suppose that V is an n-bit vector and V can be decomposed into k number of vi(i=1,2,⋯k). And if the length of each vi is ni, then the following holds.

If ℂ is an n-cell TPMACA corresponding to V=(v1v2 ⋯ vn)(vi ∈ {0,1}, then we can synthesize the pattern classifier using the following algorithm. The following algorithm shows a synthesis algorithm of a TPMACA C corresponding to V=(v1v2 ⋯ vn).

Algorithm 1 SynthesisOfTPMACA

Step 1. If V = (1v2v3 ⋯ vn-10), then we synthesize by using type (iii) and type (v), and finally by using type (i).

Step 2. If V = (0v2v3 ⋯ vn-11), then we synthesize by using type (ii) firstly, and then synthesize by using type (iv) and type (vi).

Step 3. If V = (1v2v3 ⋯ vn-11), then we synthesize by using type (iii) and type (v), or by using type (iv) and type (vi).

In case of V = (0v2v3 ⋯ vn-10), there does not exist an n-cell TPMACA corresponding to V. But, in the subsection 3.2, we can show that there exists an n-cell MACA corresponding to V by concatenating TPMACA which are obtained by Algorithm 1.

The following shows important conditions that must be taken into account when synthesizing TPMACA using the basic forms.

Theorem 3.1. Let ℂ be an n-cell TPMACA corresponding to the feature vector V = [v1 v2 ⋯ vk] (k≤n). Then the following hold.

(1) If ℂ is synthesized by using type (i), type (iii) and type (v), then the last rule of the vi is changed by the first rule of vi+1.

(2) If ℂ is synthesized by using type (ii), type (iv) and type (vi), then the first rule of the vi+1 is changed by the last rule of vi.

Proof. (1) [vi vi+1]can be decomposed into type (v, iii), type (v, i), type (v, v), type (iii, i) and type (iii, v), where type (i), type (iii) and type (v) are in Table 2. There are five possible cases to consider. Firstly we show (1) for the case of type (v, iii). In this case vi=(101) and vi+1=(11 ⋯ 1). The rule vector of 3-cell CA corresponding to vi is <150, 60, 150>. Also the rule vector of ni+1-cell CA corresponding to vi+1 is <240, …, 240, 60>. If <150, 60, 240, …, 240, 60> is the rule vector of (2 + ni+1)-cell CA ℂ, then the state transition matrix The number of 1’s of the diagonal elements in T is 3 and rank(T) = n-1, and rank(T ⊕ I)=n-1. Thus by Remark B, ℂ is a (2+ni+1)-cell TPMACA. Let Then

By (3.2)

and so

Therefore by Lemma 2.2(ii), ℂ is the TPMACA corresponding to V=(101 ⋯ 1). The proof of (1) for the cases of type (v, i), type (v, v), type (iii, i) and type (iii, v) is similar to the proof for type (v, iii). (2) It can be proved by the similar method as (1).

Example 3.2. Let V = (10110111)=[ v1 v2 v3 v4 ], where v1 = v3 = (101), v2 = (11) and v4 = (111). Then the synthesized CA is an 8-cell TPMACA with the rule vector <150, 60, 240, 150, 60, 240, 240, 60>. Fig. 2 shows the synthesis of TPMACA with the rule vector <150, 60, 240, 150, 60, 240, 240, 60>.

Fig. 2.The synthesis of TPMACA.

Table 3 shows the synthesis of TPMACA corresponding to 10-bit V. The fifth V in Table 3 can also be synthesized with type (vi), type (vi), type (vi) and type (iv) in Table 2. In this case the rule vector of the TPMACA is <150, 102, 150, 102, 150, 102, 150, 170, 170, 170>.

Table 3.Synthesis of TPMACA corresponding to 10-bit V

The following theorem shows a method for finding a nonreachable state in the 0-tree of TPMACA synthesized by Algorithm 1 and Theorem 3.1.

Theorem 3.3. Let V1 = (1v2v3 ⋯ v n-10)(resp. V2 = (0v2v3 ⋯ v n-11) and V3 = (1v2v3 ⋯ v n-11) be the n-bit feature vector. Then the following state NS is a nonreachable state of the 0-tree in the state transition diagram of the n-cell TPMACA ℂ obtained by Algorithm 1 and Theorem 3.1.

where tij is the (i,j) entry of the state transition matrix TM = (tij) of ℂ.

Proof. Let V1 = (1v2v3 ⋯ v n-10) and v2 = 1. Then NS = (110 ⋯ 0) and V = (11v3 ⋯ vn-10). Since V ⋅ NS =0, NS is a state of the 0-tree in the state transition diagram of ℂ. Let TM be the state transition matrix of ℂ. Then because the depth of the 0-tree is n-1. To show that NS is a nonreachable state we must show that Since V = (11v3 ⋯ vn-10), V is one of the forms (111 ∗⋯∗ 0), (1101 ∗⋯∗ 0) or (1100 ⋯ 0). Thus by Theorem 3.1 and Table 2 the first column of TM and second column of TM is (010 ⋯ 0)t and (0 ∗ 10 ⋯ 0)t respectively. If ci is the ith column of On the other hand, ℂ is synthesized by Algorithm 1 with type (i), type (iii) and type (v) in Table 2. Thus we can get the (j,k) entry as the following :

Therefore c1 = (0 ∗ ⋯ 10)t and c2 = (0 ∗ ⋯ ∗ 1)t. So c1 + c2 ≠ 0. Hence and thus NS is a nonreachable state of the 0-tree of ℂ. We can show that remaining states NS are all nonreachable states by the similar method.

Corollary 3.4. Let V be the n-bit feature vector V = (10 ⋯ 0) (resp. V = (0 ⋯ 01)). Then NS =(0 ⋯ 01) (resp. NS = (010 ⋯ 0)) is a nonreachable state of the 0-tree in the state transition diagram of the n-cell TPMACA ℂ obtained by Algorithm 1.

3.2 Generation of MACA corresponding to feature vectors

Now consider the case there is no TPMACA for a given feature vector VS of the form in (2.3) and (2.4). So in this subsection we propose a method for synthesizing an n-cell MACA for such feature vectors. Let VS = (V1||V2|| ⋯ |VK) be the concatenation of Vi (i = 1,2, ⋯ k). Here Vi is the feature vector of a TPMACA ℂi (i = 1,2, ⋯ k) generated by Algorithm 1. And let TMi be the state transition matrix of ℂi whose characteristic polynomial and minimal polynomial are the same as xdi(x+1), where the length of Vi is ni = di + 1. Let

and let ℂ be the n-cell MACA whose state transition matrix is T, where n = n1 + n2 +⋯ nk. Then this ℂ is the MACA corresponding to VS. The characteristic polynomial and the minimal polynomial of T is Xn-k(x+1)k and Xd(x+1) respectively, where d = max (d1,d2, ⋯ dk). Also the number of trees in the state transition diagram of ℂ is 2k.

To increase the efficiency of pattern classifier we need to minimize d and k. Especially to minimize d it is necessary for Vi to have similar lengths.

The following algorithm is a synthesis algorithm of linear MACA which is the pattern classifier corresponding to VS = (V1||V2|| ⋯ ||VK).

Algorithm 2 Synthesis Of Linear MACA Pattern Classifier

Step 1.Decompose the given feature vector VS into the minimum number of Vi's with similar sizes.

Step 2.Synthesize TPMACA ℂi corresponding to Vi by Algorithm 1.

Step 3.Concatenate ℂi's (i =1,2,⋯,k).

Step 4.Change the jth rule and the (j + 1)th rule by using Table 4.

Table 4.Change of the state transition rules occurring in the synthesis process of linear MACA

Table 4 shows change of the state transition rules occurring in the synthesis process of linear MACA.

Example 3.5. Let VS = (0111010). Then VS = (W1||W2) = (01||11010) or VS = (V1||V2) =(011||101). The minimal polynomial of the pattern classifier synthesized using VS = (W1||W2) = (01||11010) is x4(x+1). But the minimal polynomial of the pattern classifier synthesized using VS = (V1||V2) =(011||101) is x3(x+1). Therefore, it is more effective to decompose VS into VS = (V1||V2) raher than VS = (W1||W2). Thus by Algorithm 1, ℂ1 and ℂ2 are <170,102,170> and <150,60,60,240> respectively. Therefore, the linear MACA ℂ corresponding to VS is = ˂170,102,0,102,60,60,240˃ by Algorithm 2 and Table 4.

Fig. 3 shows the pattern classifier corresponding to feature vector VS=(0111010). In Fig. 3, S={0,1,2, ⋯ ,126,127} and VS=(0111010). By using Algorithm 1 and Algorithm 2 we obtain T. Then we can classify S into P0 and P1 with T. Here P0={0,1,4,5,10, ⋯ 117,122,127} and P1=S＼P0={2,3,6,7, ⋯ 125,126}.

Fig. 3.Pattern classifier corresponding to the feature vector VS = (0111010).

In Example 3.5 NS1 = (011)(resp. NS2 = (1010)) is the nonreachable state of ℂ1 (resp. ℂ2) by Theorem 3.3. Thus the nonreachable states of ℂ can be obtained by linear combination of (NS1ǁ0) or (0ǁNS2).

 

4. CONCLUSION

In this paper we proposed a pattern classifier based on a linear nongroup CA that can classify patterns according to a given feature vector and two algorithms for generations of TPMACA and MACA corresponding to the given feature vectors. In addition, we found methods for finding nonreachable states in the 0-tree of the state transition diagram of TPMACA corresponding to the given feature vector for the analysis of the state transition behavior of the generated pattern classifier.