Abstract:  In this talk, I will discuss how the game-theoretic concept of regret minimization plays a pivotal role towards generalization of modern optimization methods in machine learning.  I will illustrate this through some curious theoretical results and phenomena concerning Stochastic Gradient Descent (SGD) that challenge the current conventional wisdom in machine learning.  The talk will not assume any prior knowledge in machine learning and/or optimization.