We have shown here for the first time that the completeness relation provides a simple unified theoretical framework for deriving different kinds of new recurrence formulae for Riemann Zeta Functions, Dirichlet series and Other Allied Series by selecting only different forms of complete set of ortho normalfunction(CSOF) in contrast to the expansion method(EM) where one needs to select not only different kinds of CSOF but also suitable arbitrary function. Anewproofis alsogivenby selecting only orthogonal Bessel functions for the well known identity corresponding to the sum of squares of the reciprocals of zeros for the m-th order Bessel function. In addition, we have shown here that,in comparison to the EM and other methods, ourpresent methodhas far-reaching implications, viz. (i)All proofs are based on completeness relation and proper selection of orthogonal functions without selecting any arbitrary functions. (ii) Simpler proofs are possible for new identities corresponding to infinite number of infinite series,sum of each having a fixed value pi(Π). (iii) New proofs emerge not only for the identities corresponding to the Associated Clausen functions but also the sum of new classes of infinite series which resemble the associated Clausen functions.