In this work, the electronic structures of quantum dots (QDs) of nine direct band gap semiconductor materials belonging to the group II-VI and III-V families are investigated, within the empirical tight-binding framework, in the effective bond orbital model. This methodology is shown to accurately describe these systems, yielding, at the same time, qualitative insights into their electronic properties. Various features of the bulk band structure such as band-gaps, band curvature, and band widths around symmetry points affect the quantum confinement of electrons and holes. These effects are identified and quantified. A comparison with experimental data yields good agreement with the calculations. These theoretical results would help quantify the optical response of QDs of these materials and provide useful input for applications.