A 7-foot center, Spivey starred at the University of Kentucky for coach Adolph Rupp. He was a two-time All-American who averaged 19.3 ppg. His 1950-51 team went 32-2 and won the NCAA tournament. He had 22 points and 21 rebounds in the championship game. The Helms Foundation named him player of the year of the Best Team.
When a huge point shaving scandal broke in 1951 Spivey was indicted for perjury because of his refusal to testify against teammates Alex Groza, Ralph Beard and Dale Barnstable (who confessed to the point shaving charges). Though never convicted, Spivey was suspended from the Kentucky team and barred from the NBA for life.
Spivey, thought by many to be one of the best big men in the nation, spent the remainder of his playing career in the back waters of pro ball. He played for the barnstorming Detroit Vagabonds in 1952-52. He played for the Boston Whirlwinds, the House of David, the Washington Generals and other teams that were foils for the Harlem Globetrotters. He spent several years touring with teams that played the Harlem Magicians.
In 1958 Spivey signed with the Wilkes-Barre Barons of the Eastern League winning the MVP in 1959 after leading them to the championship. He played with several teams in the Eastern League until February 4, 1968.
When Abe Saperstein, George Steinbrenner and others formed the American Basketball League in 1961, Spivey became the center on Saperstein's Los Angeles-Hawaii-Long Beach squad. He averaged 29.9 ppg for the life of the ABL. After that league collapsed, he returned to the Eastern League. His 5,510 career points put him at #15 on the all-time Continental Basketball Association (the successor to the Eastern League) scoring list.
Those who knew said that he never recovered from the events of his youth. Spivey might have been the most dominating center in the NBA, had he been permitted to play. George Mikan was in the twilight of his career and Wilt Chamberlain had not yet come up. But Spivey never got the chance.