Mutual Information and Redundancy Analysis

Fraser and Swinney introduced an analysis in which information-theoretic properties estimated from time series data is used to deduce properties of the dynamics on the attractor. In their first paper, they proposed that the best value of the time delay was the value of the smallest time lag for which the mutual information is a minimum, where the mutual information in bits is defined as

(1)

MI, so defined, measures the general dependence of on p(t) while a measure like autocorrelation measures only linear dependencies. In a second paper, they generalized the procedure to reconstructions of arbitrary embedding dimensions, by introducing the total redundancy of a multi-dimensional distribution as

(2)

and its marginal redundancy as

(3)

They suggested that the time delay for the d dimensional delay embedding should correspond to the minimum of , and that d itself should be such that it is the smallest value for which the marginal redundancy lies close to the asymptotic accumulation line. Figures 3a and 3b show and for the present data for embedding dimensions from 2 to 5. It was calculated using the 65536 points over 910 years after leaving out the first 100 years, using their code. The number of data points are most likely insufficient for the higher dimensional embeddings. The autocorrelation at different time lags for the same data is shown in Fig 3c. The first minimum of occurs at about 14 days for all the embeddings considered, while the first zero-crossing of the autocorrelation function is at 78 days. Again, keeping in mind that the statistic for the higher dimensional embeddings might not have reliably converged, the stastic indicates that an embedding dimension of 4 may be sufficient.

  

Figure 2: Total and Marginal Redundancy and Autocorrelation