Authentication systems must be strong enough to ensure confidentiality, accountability, and integrity for a reasonable cost--and be user friendly. Good, publicly available guidance does exist; for instance, see [1]. Usability, on the other hand, is mostly treated informally. The authors report that only four of the 23 publications report the results of an empirical usability study, and none of the systems are analyzed using a standard usability metric.

This paper uses a formal approach, systems usability scale (SUS), to assess the usability of seven different web authentication systems. It provides reliable, replicable results.

The seven web authentication systems are split into three groups: federated single sign-on, email-based, and QR code-based. For each of the groups, the authors conduct a separate usability study, and the system with the highest SUS score in each study is selected as a winner. The three winners are then compared again. Of these seven systems, federated and smartphone-based single sign-on receive the best overall usability ratings.

The authors also collected feedback and proposals. These provide more insight into what makes an authentication system usable. Note that the study was executed with young people with a medium level of IT skills, somewhat more males than females, in a lab setting.

Single sign-on (SSO) is preferred, but there are security concerns related to the SSO provider. Combining it with low-entropy passwords per site is suggested. They also like transparency, but too much raises suspicion: is this really secure? Both of these findings indicate that we may underestimate the users’ willingness to assist and be involved with secure access.

New, more innovative authentication mechanisms were found attractive, and biometric mechanisms (not in the test) were suggested. Given the age of most participants in the test, this might not be generally so.

The mean time to authenticate did not seem to play a role, whereas this is a common measure currently. It may be because it was a lab test, not one with people dealing with numerous systems each day. When using a physical token (that is, a WebTicket or smartphone), participants want to have a fallback mechanism. This concern is well known.

I can only welcome more scientific approaches in security for a key pillar: the user. SUS seems to be fit for the purpose. The significant improvement of one of the systems based on the analysis should inspire authentication product creators. The classic idea of users just wanting to get the job done and not caring about the authentication seems to be a mistake: less hiding and more explaining may be a new path to consider.