We bound the condition number of the Jacobian in pseudo-arclength continuation problems, and we quantify the effect of this condition number on the linear system solution in a Newton-GMRES solve. Pseudo-arclength continuation solves parameter dependent nonlinear equations G(u, lambda) = 0 by introducing a new parameter s, which approximates arclength, and viewing the vector x = (u, lambda) as a function of s. In this way simple fold singularities can be computed directly by solving a larger system F( x, s) = 0 by simple continuation in the new parameter s. It is known that the Jacobian F-x of F with respect to x = (u, lambda) is nonsingular if the path contains only regular points and simple fold singularities. We introduce a new characterization of simple folds in terms of the singular value decomposition, and we use it to derive a new bound for the norm of F-x(-1). We also show that the convergence rate of GMRES in a Newton step for F(x, s) = 0 is essentially the same as that of the original problem G(u, lambda) = 0. In particular, we prove that the bounds on the degrees of the minimal polynomials of the Jacobians F-x and G(u) differ by at most 2. We illustrate the effectiveness of our bounds with an example from radiative transfer theory.