His family was poor, and he was largely self-taught. He showed early art talent, and as a child copied pictures from books. At age 15, he apprenticed to a house painter, and at age 20, moved to London and assisted portrait and marine artist George Johansen.

In 1901, he arrived in San Francisco from Sweden, having jumped ship as a seaman on the "S.S. Arizonan. " He walked the rail track to Los Angeles, and learned painting techniques from William Wendt, well-known landscape artist.

Sponsored by Phoebe Hearst, mother of newspaper tycoon William Randolph Hearst, Borg studied art in Paris and Rome, and with Hearst's encouragement, also painted Indian portraits. He then taught at the California Art Institute in Los Angeles, spent six months in Honduras, and from 1918-24, was an instructor at the School of Arts in Santa Barbara.

From 1924-1935, he was in California and Arizona doing commissioned paintings of Southwest Indian tribal ceremonies for Hearst and also did Grand Canyon landscapes. He traveled in the country when war broke out and was forced to spend World War II in Sweden where his desert and Indian portraits became much sought after.

After the war, he returned to Santa Barbara and died there on May 8, 1947.

One of his paintings of the Grand Canyon is in the collection of Supreme Court Justice Sandra Day O'Connor, from Arizona.