The notion of Abelian complexity of infinite words was recently used by the three last authors to investigate various Abelian properties of words. In particular, using van der Waerden's theorem, they proved that if a word avoids Abelian k-powers for some integer k, then its Abelian complexity is unbounded. This suggests the following question: How frequently do Abelian k-powers occur in a word having bounded Abelian complexity? In particular, does every uniformly recurrent word having bounded Abelian complexity begin in an Abelian k-power? While this is true for various classes of uniformly recurrent words, including for example the class of all Sturmian words, in this paper we show the existence of uniformly recurrent binary words, having bounded Abelian complexity, which admit an infinite number of suffixes which do not begin in an Abelian square. We also show that the shift orbit closure of any infinite binary overlap-free word contains a word which avoids Abelian cubes in the beginning. We also consider the effect of morphisms on Abelian complexity and show that the morphic image of a word having bounded Abelian complexity has bounded Abelian complexity. Finally, we give an open problem on avoidability of Abelian squares in infinite binary words and show that it is equivalent to a well-known open problem of Pirillo-Varricchio and Halbeisen-Hungerbühler.