We study the convergence of the slow (or "essential") components of singularly perturbed stochastic differential systems to solutions of lower dimensional stochastic systems (the "effective", or "coarse" dynamics). We prove strong, mean-square convergence in systems where both fast and slow components are driven by noise, with full coupling between fast and slow components. We analyze a class of "projective integration" methods, which consist of a hybridization between a standard solver for the slow components, and short runs for the fast dynamics, which are used to estimate the effect that the fast components have on the slow ones. We obtain explicit bounds for the discrepancy between the results of the projective integration method and the slow components of the original system.