I remember in 1979 as a 17 year old, I was already into Northern Soul (since 1976) and then got into Ska/Reggae via the 2-Tone movement. Oh, how the old '69 Skinheads rebuked us...."we were there first time around, the Specials are just copying Prince Buster etc, etc".
Because of 2-Tone. which was dissed by the old firm, I now have a nice collection of original authentic Jamaican Ska/Reggae 45's and L.P's alongside my Northern Soul collection.
I suppose the point is there's no set point where you have to begin liking a certain type of music....