We prove that in every normal form n-player game with m actions for each player, there exists an approximate Nash equilibrium in which each player randomizes uniformly among a set of O(log m + log n) pure actions. This result induces an O(Nlog log N)-time algorithm for computing an approximate Nash equilibrium in games where the number of actions is polynomial in the number of players (m=poly(n)); here N=nmn is the size of the game (the input size). Furthermore, when the number of actions is a fixed constant (m=O(1)) the same algorithm runs in O(Nlog log log N) time. In addition, we establish an inverse connection between the entropy of Nash equilibria in the game, and the time it takes to find such an approximate Nash equilibrium using the random sampling method. We also consider other relevant notions of equilibria. Specifically, we prove the existence of approximate correlated equilibrium of support size polylogarithmic in the number of players, n, and the number of actions per player, m. In particular, using the probabilistic method, we show that there exists a multiset of action profiles of polylogarithmic size such that the uniform distribution over this multiset forms an approximate correlated equilibrium. Along similar lines, we establish the existence of approximate coarse correlated equilibrium with logarithmic support. We complement these results by considering the computational complexity of determining small-support approximate equilibria. We show that random sampling can be used to efficiently determine an approximate coarse correlated equilibrium with logarithmic support. But, such a tight result does not hold for correlated equilibrium, i.e., sampling might generate an approximate correlated equilibrium of support size Ω(m) where m is the number of actions per player. Finally, we show that finding an exact correlated equilibrium with smallest possible support is NP-hard under Cook reductions, even in the case of two-player zero-sum games.