Signal and Noise

I’ll be writing about machine learning, quant stuff, and probability theory. There may be some random musings too.

The Streaming PCA Problem and Oja’s Algorithm

In the first post of this series, I introduce Oja’s algorithm and the streaming PCA problem. We start from the Riemannian gradient flow, derive an exact solution, and then show how the stochastic gradient version inherits the same time-scale structure. I then sketch out a roadmap for proving that the expected squared sine distance between the last iterate of the algorithm and the principal component of the data is small under an iid generative assumption.

Nov 29, 2025