Whilst the Girl Don't Leave Me side, to these ears, is the (slightly) better side vocally and lyrically I am plumping for the Girl (You're So Sweet, You're So Fine) side because of the instrumentation - wah wah guitar intro, sax and trombone breaks and better production.