King was born Benjamin Earl Nelson in Henderson, NC, in 1938, and sang with his church choir before the family moved to Harlem in 1947. In junior high, he began performing with a street corner doo wop group called the Four B's, which won second place in an Apollo Theater talent contest. While still in high school, he was offered a chance to join the Moonglows, but was simply too young and inexperienced to stick. He subsequently worked at his father's restaurant as a singing waiter, which led to an invitation to become the baritone singer in a doo wop outfit called the Five Crowns in 1958. The Five Crowns performed several gigs at the Apollo Theater along with the Drifters, whose career had begun to flounder in the years since original lead singer Clyde McPhatter departed. Drifters manager George Treadwell, dissatisfied with the group members' unreliability and lack of success, fired them all in the summer of 1958 and hired the Five Crowns to assume the name of the Drifters (which he owned).
The new Drifters toured for about a year, playing to often hostile audiences who knew they were a completely different group. In early 1959, they went into the studio with producers Jerry Leiber and Mike Stoller to cut their first records. A song Nelson (still performing under his given name) co-wrote called "There Goes My Baby" became his first lead vocal and the lush backing arrangement made highly unorthodox (in fact, virtually unheard-of) use of a string section. "There Goes My Baby" became a massive hit, laying the groundwork for virtually every smooth/uptown soul production that followed. Over the next two years, Nelson sang lead on several other Drifters classics, including "Dance With Me," "This Magic Moment," "Save the Last Dance for Me," and "I Count the Tears."