ALaRI Hang Glider

Search form

Education and Innovation in Embedded Systems Design

USI Università della Svizzera italiana, USI Faculty of Informatics, Advanced Learning and Research Institute USI Università della Svizzera italiana USI Faculty of Informatics USI Advanced Learning and Research Institute
TitleInvestigating echo state networks dynamics by means of recurrence analysis
Publication TypeJournal Article
Year of Publication2018
AuthorsBianchi, F. Maria, L. Livi, and C. Alippi
JournalIEEE Transactions on Neural Networks and Learning Systems
Volume29
Pagination 427 - 439
Date Published02/2018
Abstract

In this paper, we elaborate over the well-known
interpretability issue in echo state networks. The idea is to investigate
the dynamics of reservoir neurons with time-series analysis
techniques taken from research on complex systems. Notably, we
analyze time-series of neuron activations with Recurrence Plots
(RPs) and Recurrence Quantification Analysis (RQA), which
permit to visualize and characterize high-dimensional dynamical
systems. We show that this approach is useful in a number of
ways. First, the two-dimensional representation offered by RPs
provides a way for visualizing the high-dimensional dynamics of
a reservoir. Our results suggest that, if the network is stable,
reservoir and input denote similar line patterns in the respective
RPs. Conversely, the more unstable the ESN, the more the RP
of the reservoir presents instability patterns. As a second result,
we show that the Lmax measure is highly correlated with the
well-established maximal local Lyapunov exponent. This suggests
that complexity measures based on RP diagonal lines distribution
provide a valuable tool to quantify the degree of network stability.
Finally, our analysis shows that all RQA measures fluctuate on
the proximity of the so-called edge of stability, where an ESN
typically achieves maximum computational capability. We verify
that the determination of the edge of stability provided by such
RQA measures is more accurate than two well-known criteria
based on the Jacobian matrix of the reservoir. Therefore, we
claim that RPs and RQA-based analyses can be used as valuable
tools to design an effective network given a specific problem.

DOI10.1109/TNNLS.2016.2630802