A.V. Litavrin. Endomorphisms of finite commutative groupoids related with multilayer feedforward neural networks ... P. 130-145

In this paper, we introduce commutative, but generally not associative, groupoids $\mathrm{AGS}(\mathcal{N})$ consisting of idempotents. The groupoid $ (\mathrm{AGS}(\mathcal{N}),+)$ is closely related to the multilayer feedforward neural networks $\mathcal{N}$ (hereinafter just a neural network). It turned out that in such neural networks, specifying a subnet of a fixed neural network is tantamount to specifying some special tuple composed of finite sets of neurons in the original network. All special tuples defining some subnet of the neural network $\mathcal{N}$ are contained in the set $\mathrm{AGS}(\mathcal{N})$. The rest of the tuples from $\mathrm{AGS}(\mathcal{N})$ also have a neural network interpretation. Thus, $\mathrm{AGS}(\mathcal{N})=F_1\cup F_2$, where $F_1$ is the set of tuples that induce subnets and $F_2$ is the set of other tuples. If two subnets of a neural network are specified, then two cases arise. In the first case, a new subnet can be obtained from these subnets by merging the sets of all neurons of these subnets. In the second case, such a merger is impossible due to neural network reasons. The operation $(+)$ for any tuples from $\mathrm{AGS}(\mathcal{N})$ returns a tuple that induces a subnet or returns a neutral element that does not induce subnets. In particular, if for two elements from $F_1$ the operation $(+)$ returns a neutral element, then the subnets induced by these elements cannot be combined into one subnet. For any two elements from $\mathrm{AGS}(\mathcal{N})$, the operation has a neural network interpretation. In this paper, we study the algebraic properties of the groupoids $\mathrm{AGS}(\mathcal{N})$ and construct some classes of endomorphisms of such groupoids. It is shown that every subnet $\mathcal{N}'$ of the net $\mathcal{N}$ defines a subgroupoid $T$ in the groupoid $\mathrm{AGS}(\mathcal{N})$ isomorphic to $\mathrm{AGS}(\mathcal{N}')$. It is proved that for every finite monoid $G$ there is a neural network $\mathcal{N}$ such that $G$ is isomorphically embeddable into the monoid of all endomorphisms $\mathrm {End}(\mathrm{AGS}(\mathcal{N}))$. This statement is the main result of the work.

Keywords: groupoid endomorphism, multilayer feedforward neural networks, multilayer neural network subnet

Received January 11, 2021

Revised February 14, 2021

Accepted February 24, 2021

Funding Agency:  This work is supported by the Krasnoyarsk Mathematical Center and financed by the Ministry of Science and Higher Education of the Russian Federation in the framework of the establishment and development of regional Centers for Mathematics Research and Education (Agreement No. 075-02-2020-1534/1).

Andrey Viktorovich Litavrin, Cand. Sci. (Phys.-Math.), Siberian Federal University, Krasnoyarsk, 660041 Russia, e-mail: anm11@rambler.ru

REFERENCES

1.   Golovko V.A., Krasnoproshin V.V. Neirosetevye tekhnologii obrabotki dannykh; ucheb. posobie [Neural network data processing technologies; textbook]. Minsk: Belorus. Gos. Univ. Publ, 2017, 263 p. ISBN: 978-985-566-467-4 .

2.   Gorban’ A.N. Generalized approximation theorem and computational capabilities of neural networks. Sib. Zh. Vychisl. Mat., 1998, vol. 1, no. 1, pp. 11–24 (in Russian).

3.   McCulloch W., Pitts W. A logical calculus of the ideas immanent in nervous activity. Bulletin Math. Biophys., 1943, vol. 5, no. 4, pp. 115–133. doi: 10.1007/BF02478259 

4.   Rosenblatt F. Principles of neurodynamics: Perceptrons and the theory of brain mechanisms. Washington: Spartan Books, 1962, 616 p. Translated to Russian under the title Printsipy neirodinamiki: perseptron i teoriya mekhanizmov mozga. Moscow: Mir, 1965. 478 p.

5.   Litavrin A.V. Endomorphisms of some groupoids of order $k + k^2$. Bulletin Irkutsk State Univ. Ser. Mathematics, 2020, vol. 32, pp. 64–78. doi: 10.26516/1997-7670.2020.32.64 

6.   Tsarkov O.I. Endomorphisms of the semigroup $G_2(r)$ over partially ordered commutative rings without zero divisors and with 1/2. J. Math. Sci., 2014, vol. 201, no. 4, pp. 534–551. doi: 10.1007/s10958-014-2010-0 

7.   Zhuchok Yu.V. Endomorphism semigroups of some free products. J. Math. Sci., 2012, vol. 187, no. 2, pp. 146–152. doi: 10.1007/s10958-012-1057-z 

8.   Slepovichev I.I. Algebraic properties of abstract neural network. Izv. Saratov Univ. (N.S.), Ser. Math. Mech. Inform., 2016, vol. 16, no. 1, pp. 96–103 (in Russian). doi: 10.18500/1816-9791-2016-16-1-96-103 

9.   Greenglaz L.Ja. , Gvaramija A.A., Plotkin B.I. Algebraic structures in automata and database theory. Singapore: World Scientific, 1992, 296 p. ISBN: 9789814505666 . Translated to Russian under the title Elementy algebraicheskoi teorii avtomatov. Moscow: Vysshaya Shkola Publ., 1994, 192 p. ISBN: 5-06-002738-4 .

10.   Molchanov V.A., Khvorostukhina E.V. On problem of abstract definability of universal hypergraphic automata by input symbol semigroup. Chebyshevskii Sb., 2019, vol. 20, no. 2, pp. 259–272 (in Russian). doi: 10.22405/2226-8383-2018-20-2-259-272 

11.   Litinskii L.B. On the problem of neural network decomposition into some subnetworks. Matem. Mod., 1996, vol. 8, no. 11, pp. 119–127 (in Russian).

12.   Timofeev A.V., Derin O.A. Principles of hierarchical neural networks for analysis of multi-images. Tr. SPIIRAN, 2009, vol. 10, pp. 160–166 (in Russian).

13.   Kurosh A.G. Lektsii po obshchei algebre (Lectures in general algebra). St. Petersburg: Lan’ Publ., 2007, 560 p. ISBN: 978-5-8114-0617-3 .

Cite this article as: A.V. Litavrin. Endomorphisms of finite commutative groupoids related with multilayer feedforward neural networks, Trudy Instituta Matematiki i Mekhaniki UrO RAN, 2021, vol. 27, no. 1, pp. 130–145.