The self-organization of recurrent feature-discovery networks is studied from the perspective of dynamical systems. Bifurcation theory reveals pa(cid:173) rameter regimes in which multiple equilibria or limit cycles coexist with the equilibrium at which the networks perform principal component analysis.