Karl Olsen was born on a farm on the Norwegian island of Bergsoya on 27 July 1921 to Anders "Andrew" Olsen of Aspoya and Kristine Jordal Olsen of Bergsoya. Karl's father immigrated to the United States to live with four of his siblings whom had already settled in the McKinley Hill neighborhood (also called Sneuse Hill) of Tacoma, Washington. Andrew's brother married a Norwegian girl named Lena Jordal in 1910 and in 1912 Andrew married Lena's sister Kristine. Andrew became a sawyer at the Saint Paul Mill. In 1919, the Olsen family returned to Norway. Karl Olsen was born a few years later. Andrew eventually returned to Tacoma and his wife and children followed in 1929. Olsen began attending Pacific Lutheran University but signed up for the army when World War II broke out shortly thereafter. He was stationed in the South Pacific. He returned and on 17 August 1946 Karl married his longtime girlfriend, Lois, and they had three children. When he graduated PLU, Karl worked as a teacher and school principal in Tacoma for thrity years. He and family were very active in the Luther League at Christ Lutheran Church in Tacoma, and they traveled extensively all over the world.