While still young, he studied the piano at the Bush Conservatory of Music in Chicago.
He eventually moved to New York City to become a songwriter.
His first composition was "Do You, Don't You, Will You, Won't You," published in 1923.
In 1925 he joined ASCAP (American Society of Composers, Authors and Publishers), and remained a member for 63 years.
In 1929 he co-authored his most famous song, "When You're Smiling" (As with many other of his songs, this was a collaboration with Joe Goodwin and Mark Fisher; see Shay, Fisher, and Goodwin).
In the 1930s, Metro-Goldwyn-Mayer hired him to become their music director and Shay and his wife Doris moved from New York to Hollywood.
In that capacity, he hired Bing Crosby, who was paid $50 a day by MGM for his first picture.
He published over 300 songs in his life.
He died in Newport Beach, California.
He had a daughter, Dawn Russum, who is now president of the Channel Cities Jazz Society in Ventura, California.