Ballard attended the University of Oregon where he played at the collegiate level at the forward position. He was drafted by the Washington Bullets in the 1st round (4th pick) of the 1977 NBA Draft and won an NBA Championship with the Bullets in 1978. He played for the Golden State Warriors for the 1986 and 1987 seasons. He finished his career with the Seattle SuperSonics in 1989 with career stats of 12.1 points, and 6.2 rebounds, 2.2 assists per game.
Ballard turned to coaching after his playing career wound up, and in 1989 he moved to Italy to serve as an assistant coach for Il Messaggero Roma. He is currently an assistant coach for the Atlanta Hawks.