The Rice Lake, Wisconsin native was the head coach of the Ohio State University from 1922 to 1946. In 1946 he became the first head coach of the Basketball Association of America (the forerunner of the NBA) Chicago Stags, where he coached almost three seasons before being replace by Philip Brownstein. Olsen also coached a single season at Northwestern University (1950-51).
While playing at University of Wisconsin (1913-17), Olsen was named two-time All-Big Ten. After graduating from Wisconsin, he began his coaching career at Ripon College. In 1922 Olsen followed George Trautman as head coach of the Ohio State University. In 24 years he guided the Buckeyes to a 265-81 record, as well as five Big Ten championships (1925, 1933, 1939, 1944, 1946). In 1939, Olsen spearheaded efforts to create the NCAA postseason national playoffs, now known as the NCAA Division I Men's Tournament. Olsen also helped initiate the 10-second rule.
In 1959 he was inducted to the Naismith Memorial Basketball Hall of Fame as a contributor as part of the inaugural class of inductees.