A client of mine asked this question: "How can I allow registered users to log in and see the site even in offline mode, without giving them Super user privileges?"
Yes, that's the default Joomla behaviour. Fortunately the fix is quite simple.
To control what user groups have access to an offline site, go to Global Configuration (System -> Global Configuration) and select the Permissions tab. Select the user group you want to give access on the right hand side (for example "Editor"), and set Offline Access to "Allowed". (Default setting is "Inherited", thus Not Allowed.
Enjoy! Your registered users will now be able to log in even if your site is offline.