Usually when this happens, it's because the site is being viewed in multiple tabs and one of them is not using a secure connection. To avoid this, always start with one tab, and log into the admin. To see the public site from there, use the 'View Public Site' link in the upper left, which will make sure you stay in your secure session while you view the site.
