We study the effect of doping on the linear and nonlinear optical properties of Au19M clusters, M being the alkali metal atoms, Li, Na, K, Rb, and Cs, as well as the coinage metal atoms, Ag and Cu, by employing response theory within time-dependent density functional theory. We consider doping at both exohedral and endohedral locations and use several optimized geometries already reported in our earlier work on the ground-state structures and electronic properties of these clusters. Using these structures, we calculate the dipole polarizability and first-order hyperpolarizability characterizing linear and nonlinear optical properties, respectively, of these doped gold clusters. We find that the nonlinear optical response property depends crucially on the nature and the location of the dopant atom. The alkali atom doped gold clusters with the dopant atom sitting at the vertex of the tetrahedral structure are found to yield the highest value of the first-order hyperpolarizability. On the other hand, the endohedrally doped clusters are found to be significantly less hyperpolarizable. We rationalize the nonlinear optical properties by studying the low-energy UV_vis optical absorption band obtained by employing time-dependent density functional theory.