"How come there is such a massive gap between black holes and super massive black holes?" do you mean gap in the mass? There isn't - there are "small" black holes that form at the end of life of massive stars when they explode as supernova. They are from a few to a few ten solar masses a bit hard to find, but we know a bunch of them.
Supermassive black holes in centers of galaxies are much more massive - millions to billions solar masses. We are not yet 100% sure how they formed --