Hi Chris,
Great photo and story, and close to home for me. Plate "2" was originally issued to E. S. Rymill of East Terrace, Adelaide. There was in fact a plate "1", issued to J. Bannigan of Malvern. The first registrations were in September 1906.
Your photo and your great grandfather's bike is early 1930s (I'm not expert on this era, so someone else will fill in the details). To get "2" he would have to either be E. S. Rymill, have bought one of Rymill's bikes carrying "2", or have had more sway than most with the Registrar of motor vehicles.
What was his name?
Leon
Edit: Here's the photo as jpg