BlackIoT: IoT Botnet of High Wattage Devices Can Disrupt the Power Grid

Link to the Paper: https://www.usenix.org/system/files/conference/usenixsecurity18/sec18-soltan.pdf

Summary:

This paper focused on the theoretical attack where an IOT bot net took over an entire system of high wattage smart devices. This seems very implausible due to the lack of a common framework for these types of devices, but the paper points out that by simply messing just a little bit with power demand, it can create major issues for the power grid as a whole.

What I liked:

1. The study focused not on ways the grid good be disrupted but the costs could be increased for an actor

2. These types of attacks are almost impossible to detect because of how distributed they are

3. Good cost analysis. IE simulations show 5% more energy costs 20% more

4. Good historical examples of times where this type of attack could have occurred ie historical black outs.

5. A realistic modeling of how many devices the attacker would need in order to carry out a successful attack

What I didn't like:

1. I think they need a specific term for this type of attack especially as we start fleshing into different types of IOT attacks

2. Specific instances of the grid attacks happen on days where we have a peak ie Poland in 2008. What other devices are not being used to increase the energy usage?

3. There is a lack of discussion on things like how solar energy or green energy may help mitigate these issues in the grid

4. Didn't talk about how new things such as Tesla's batter might be able to mitigate these types of attacks at least until we get more power generation in

5. I don't think its plausible to compromise this many devices especially when there might not be a common framework connecting these devices.

Points for Discussion:

1. Why are things like ovens have a wifi connection? Is there such a thing as over connection in IOT

2. Would it be better for an adversary to take out the grid or to just increase the cost if they were able to economically benefit?

3. What would the recovery time be for this type of attack?

4. What key services would be taken out by this attack that do not have backup systems? What would the transition time be?

5. Costs for retrofitting the grid to prevent against these attacks.

New Ideas:

1. What steps need to be taken for dynamic power demand, more batteries?

2. What area's have the IOT device density and wattage to take down the grid?

3. Are there protocols at the home hub level that could detect this type of attack and prevent them from happening?

4. What mitigation techniques can be used in order to keep the frequency up? Are there technologies that allow for a great width of frequencies?

5. It'd be really cool to make a mini grid and test this out, at least for a small scale model

IoT Goes Nuclear: Creating a Zigbee Chain Reaction

Link to the Paper: https://eprint.iacr.org/2016/1047.pdf

Summary:

First thoughts on this paper was that this is really really cool - the paper's abstract kind of paints a picture of some type of dystopia in the near future where worms are infecting IOT devices left and right. The paper goes onto transition that idea into real life saying that they made a worm infection like that that could take down all the lights of a typical city like Paris (105 square kilometers). But while there is a lot of novel approaches to compromise IOT devices in this paper, a lot of the broad industry level information has already been touched on in past papers and this paper mainly focused on a single product and how to exploit that one product. That said I really enjoyed this paper - but I don't think that much can be taken away from it now that the manufacturers have patched what made the exploits very effective.

What I liked:

1. The paper identifies the key design flaw in Phillips products - that an attacker will not get in close physical proximity to one of their products

2. As I was reading how the smart lamps can be reset - it was a natural progression in my mind that these types of attacks can be carried out on some type of moving platform. I really loved how the paper addressed this and did its own tests

3. The entire attack was done with off the shelf activity - there was some custom implementations that needed to be done but it isn't out of the extreme of a hobbyist.

4. Very in depth explanations that don't go so deep into the weeds Ex: how they loaded the OTA image in to the chip by only setting a flag and knowing the offsets.

5. Paints a picture of the future in an IOT world and shows how future threats of network jamming, data exfiltration and denial of service attacks might happen

What I didn't like:

1. For the calculations on the attack on Paris - the paper makes the assumption that the smart lamps are randomly distributed around the city which I think isn't really likely. My assumption would be newer districts would have the smart lamps concentrated in their area while older ones would not have them at all - this prevents the worm from spreading and keeps the worm localized.

2. The paper points to issues with Phillips implementation of security measures, but doesn’t really target what the industry could do better to prevent these types of attacks in the future

3. It seems like the major possibility of a worm was brought up in O'Flynn's research. I think that this is a specific implementation of that worm - but I don't see how this paper is extremely unique from past papers.

4. Since the exploits have been patched I'm not really sure how this paper benefits the community as a whole now

5. They didn't actually build the worm and see how it would have worked even in an isolated setting

Points for discussion:

1. What does it mean to really be "adjacent" in an interconnected world of IOT devices?

2. How will the industry standard of IOT devices communication cryptography change in light of these exploits?

3. How did O'Flynn propose that a worm attack IOT devices in his past papers - how does that differ from this paper?

4. How did Phillips change the infection range to less than a meter to prevent these attacs from happening in the future?

5. Is type of attack novel to devices all on one network - or is adaptable to many different networks?

New Ideas:

1. Research what other IOT devices are similar to the lamps and can fall victim to similar types of attacks

2. Is there a way to dynamically change the master key for the ZLL certified products so that if the key gets leaked - it doesn't open the products up to new exploits?

3. The paper mentions that the Harvard architecture of the IOT devices in this case made it extremely hard to pass a software exploit through traditional means - what percentage of IOT devices have this? Are their alternatives that are more cost effective?

4. It seems like the worm is only possible due to fake firmware updates - is there a way to create a secret key for each device that is with in the Zigbee protocol that can act as a backstop against leaked master keys?

5. Is a common update framework helpful or hurtful in the spread of worms?

SoK: Exploiting Network Printers

Link to the Paper: https://oaklandsok.github.io/papers/muller2017.pdf

Summary:

In a room full of people, which each person being representative of a cyber security threat - this paper is probably the quietest of them all but probably has the most potential in the near term to do a lot of damage to the world if not taken seriously. The paper brings attention to how printers are usually very unprotected both in terms of a network sense and in terms of a physical sense meaning that they are open to a large amount of vectors of attack. The team uses open source software on a variety of printers to steal or corrupt data in a way most people wouldn't imagine a printer could do.

What I liked:

1. The fact the study brings a lot of attention to a problem that most people don't think about - that printers are unprotected but carry a lot of confidential information

2. The paper went for common trends in attacks ie the way they described most attacks on printers were on the implemented interpreters - PostScript and PJL.

3. There's a lot of different attacks that are looked at in the paper - really broad offering

4. The creation of the Printer Exploitation Toolkit seems very novel way of creating an attack framework - I'm glad that they used something that was open source instead of propietary in this study

5. The paper definitely builds on a lot of prior work - as seen in the references there are like 65 other papers

What I didn't like:

1. Over use of arbitrary acronyms in this paper kind of made it hard to read when I had to constantly flip between pages to remember what an acronym stood for

2. Study was constrained by the donation of old printers. It begs the question of if these printers were representative of printers currently in use today?

3. The paper considers certain attacks out of scope such as of any active network attacker was controlling the communication between the end user and the printer

4. Again the paper kind of went out against specific brands in certain places and doesn't really touch on what the industry as a whole could do to improve

5. The paper I don't think focused enough on how malicious fake firmware updates could become an attack vector in the future - especially after reading the Zigbee paper

Points for Discussion:

1. What other unprotected high access devices are in company networks?

2. Would industry specific rewards incentivize more white hat hackers to traditionally neglected cyber security fields?

3. Why can factory resets be done over the network? Why not restrict it to physical proximity?

4. How and why are credentials stored on printers?

5. How big of an issue are printer attacks currently?

New Ideas:

1. Does the diversity in printer manufacturers and their implementations make it harder to find a silver bullet hack?

2. I think this study needs to be redone with modern printers that are coming out today and are kept supported through firmware updates

3. It might be interesting to classify attacks on printers by hardware and software attacks to see if there are any other common trends that could be extrapolated

4. Is there different levels or tiers of security across different price points - we could do a study between personal and enterprise printers

5. Could proprietary information be stolen from 3d printers in an industrial setting?

Cloudy with a Chance of Breach: Forecasting Cyber Security Incidents

Link to the paper: https://www.usenix.org/system/files/conference/usenixsecurity15/sec15-paper-liu.pdf

Summary:

This study continued on the trend of papers looking to branch out of the normal cyber research and focus on the prediction of cyber events. Specifically in this case the study looked at 258 externally measurable features that made up a security posturing profile. From there the study made a model that tried predicting future cyberattacks using this model. One of the major flaws of this study though was the data they were using - in the end they only had enough data to test and train one event type web applications. Furthermore the study had a higher false positive rate than other methods of prediction like RiskTeller which has been proposed in other papers.

What I liked:

1. The study encompasses 258 externally measurable features meaning that there are a large amount of observation data the model can use

2. I really like the anaology to the patient in prediction vs detection - but I think there is a distinction that needs to be made. Prediction is a lot more valuable than detection. If a patient is sick and a doctor detects the sickness he or she can give medicine to make the problem go away - while in the case of cyber by the time you detect a problem the damage could be already done. So being able to predict where cyber attacks occur and being able to shore up defenses in a cost effective matter is a lot more needed than detection

3. Large amount of hacks from different event databases creates a diversity for the model to learn from

4. The study does a good job of weeding out attacks from their data that had nothing to do with security posturing ie internal attacks

5. Testing training data was done chronologically meaning the testing seemed more real life

What I didn’t like:

1. The study reports a 90% true positive rate, and a 10% false positive rate which is less effective than a lot of the other papers - specifically the Risk Teller paper had a 95% true positive

2. I don’t think the study defines how they evaluate what counts as a malicious activity in their security posture data which kind of begs the question of what exactly they are predicting

3. I don’t like the fact the study uses a collection of datasets that are off by a couple of months it seems like the data is disjointed and might not paint the right picture

4. One of the major issues with this study is that they claim to offer a snapshot of security that doesn't change to much month to month as compared to day to day snapshots that are in other studies like RiskTeller. I think this snapshot is even flawed because the data doesn't overlap in time meaning that the snapshot might not even be clear.

5. The study says the only incident they had enough data to test and train is the system for web app incidents - which means they really didn't come up with much with their data

Points to talk about:

1. This study has a higher false positive rate than the RiskTeller study - what methods from that study made it more effective in reducing false positives

2. Do cyberattack types vary from country to country or are they globalized ie the same across countries?

3. Are hosting companies like Go Daddy following the best cyber practices - the study omits web hoster's name's from the study to prevent biasing their model. But it begs the question why their names show up so many times in the attack details.

4. Why are attacks from the WHID Database detected less often than other attacks in Figure 6?

5. Does size of the network increase or decrease the risk of an attack?

 New Ideas:

1. Look into creating a study that uses all 3 of their datasets: mismanagement symptoms, malicious activities, and incident reports at the same time instead of staggered like this study

2. How would this study change if we included things like internal attacks which were left out in this study. I'm sure there are posturing techniques that restrict the ability for an internal attacker to really hurt the company?

3. How would the model change if the study kept the hosting information?

4. Does relying on multiple databases make this system more reliable in prediction? Can we create novel testing data and compare outcomes with other prediction methods such as Riskteller that only use one source of data?

5. Reconduct this study with a bigger data set so they can have valid models for different event types

RiskTeller: Predicting the Risk of Cyber Incidents

Link to the paper: https://acmccs.github.io/papers/p1299-bilgeA.pdf

*Also just a heads up there is some crazy cool data being used in this data courtesy of Symantec Research Labs

Summary:

The real novelty of this paper is the fact they went down the road less traveled in cyber security - prediction, as opposed to the 3 main paths: analysis, detection, and prevention. This paper is also really timely given as cyber attacks create more and more harm economically for companies - companies are in the market to buy cybersecurity insurance which many times does not have accurate and up to date models. This paper therefore creates a novel analysis tool that is able to detect so called "infected computers" at a rate of 95% with a relatively low false positive rate. The problem though is their definition of "infected" might not actually be valid given they weight a high amount of their modeling on the amount of unique files a device has which just really isn't realistic when considering specific user types such as developers.

What I liked:

1. The study uses a lot of very comprehensive information ie 1 year of data spread over 18 enterprises and 600k machines with over 4.4 binary file events.

2. The paper is unique in the fact that cybersecurity research of the past has been on analysis, detection, and prevention while this paper is focused on prediction which has not seen much work

3. When it comes to the risk prediction they are using a model that prioritizes lower false positives because that is what enterprise industry has demanded in order for a solution to be deployed

4. Study goes into factors that create malware incidents ie they check if the use downloaded the stuff from home or out of office hours which is an interesting facet

5. The study uses and builds on NISTS vulnerability Database instead of creating its own scoring system which makes it simpler to evaluate

What I didn’t like:

1. Very early on the paper acknowledges that the ground truth is the most important thing of this study which is based on observing malicious files and infection records but they acknowledge its nearly impossible to obtain a perfect ground truth with the way they have conducted their study

2. In the study set up details they only go into depth about how the study was set up for windows and they never really clarify whether the study encompasses other operating systems or just windows.

3. Probably my biggest issue with this study is how they classify clean and not clean devices. Essentially their methodology is going through the files on a device and penalizing the device if they have unique files. But this really doesn't make sense - say if the device is a developer or something else - you are simply flagging it for normal use which kinda trashes the study.

4. Again as part of the user profiles the prevalence based features of their data penalizes users like developers who create their own types of files.

5. The study even though it had so much data really didn't produce that many visualizations;. Out of the 82 factors or profile details they only displayed like 9 graphs of these factors.

Points to talk about:

1. The paper references a technique to figure out who is vulnerable to phishing emails  and have like extra layers of protection around those certain users which is very similar to what one of my classmates is working on.

2. How does this specific method detect new forms of malware when it only knows what it has seen?

3. How does the emergence of new malware end up messing with this prediction model? Is it something that is statically priced in at the end or is their other information that can be drawn from in order to make this model more dynamic

4. Did the system weight any of the factors more than others in determining whether computer was infected or not?

5. Is simply updating the data set enough to prevent concept drift in anti virus machine learning?

New Ideas:

1. Maybe we could use the same dataset except reveal the industries these 18 enterprises are in and map how certain industries are susceptible to different types of malware

2. The study clearly states that they do not seek to figure out the exact causes of the infection - but with the binary event log files it would not bee too hard to extrapolate certain causes for future study

3. Given that there are different users ie developers can their be more user types in this study to create a more accurate model of what an infected computer looks like? Would this lower effectiveness and/or false positives?

4. If the system weighted some factors more than others in determining what devices were infected or not can this be translated to a priority list for cyber security practitioners.

5. Possibly look into exporting this form of analysis into login data to detect false users when signing into portals?

Atom: Horizontally Scaling Strong Anonymity

Link to the paper: https://www.henrycg.com/files/academic/papers/sosp17atom.pdf

Summary

In this paper Kwon et al studies an anonymous message system called Atom which protects against traffic-analysis attacks. The novel development of this form of messaging system is that its capacity scales near-linearly with the number of servers on the network compared to prior methods scaled at a much slower rate. Atom brings together a lot of theory that has been published in prior papers and then puts into practice this theory with a number of work arounds specifically work arounds for the multiparty computation protocols which have been inefficient to deploy on a large scale. The result of atom is that at the time of publication the system is 23x faster than prior systems with similar privacy guarantees.

What I liked:

1. I like the fact that they not only showed the set up behind how something works and not just the theory, but the authors of the study further extrapolated some applications of the technology in a real world setting

2. I like that they were able to figure out a way around multiparty computation protocols (MPCs) which are generally too inefficient to use.

3. They used 1024 amazon servers to run their program and actually test the performance of it prior to publishing this paper - though this probably wouldn't be a real life scenario it’s an extra plus

4. The systems fail safe kicks in with only 2 honest users in a pool full of adversaries which is very good because it prevents a majority of dishonest users from taking over if there are still honest users

5. Two forms of tamper resistance? Both the NIZK proofs and the novel trap message based encryption. The trap message means that if there is a malicious server that edits a message it is a 50% chance that it’s a trap message.

 What I didn’t like:

1. The very start of the atom protocol dictates that volunteer servers are organized into small groups. How are these small groups created? Can they be done in a decentralized manner?

2. There are very strict setting in which atom is effective for anonymity that are spelled out in the report -- they are essentially betting that there is an honest server in each of the server groups which might be reasonable when there are thousands of servers but what if there are only a handful early on.

3. The study used the exact same servers from the exact same place - which doesn't replicate how a network would work if they are all scattered across the world with different bandwidths different server types etc

4. The system is extremely vulnerable when there is a small amount of users - which kinda begs the question why be an early adopter of the technology if it puts you more at risk?

5. A small problem the paper notes is intersection attacks - but on a strong healthy network this should not be a problem - it does how ever kind of build the argument against being an early adopter of this protocol.

Points for further discussion:

1. Why is latency a major problem here? IE is the constraint the number of servers or the latency between the links? The system atom uses can transit 1 million tweets in 30 minutes

2. The study points to an internal example where they rented out amazon servers but how will the system deploy in the real world when each user has different motives in the network?

3. Why should someone be an early adopter of this technology?

4. If we were to stagger different servers with different capabilities would this cut the idle time the paper was referring to by a significant amount? Would we even be able to predict this because the server groups are constantly changing?

5. Is there a way to make an atom network private - so that it is only available to a certain subset of users who all want to stay anonymous? 

New Ideas:

1. In bitcoin early adopters are incentivized to join the network by having the opportunity to cheaply mine new currency when it is easy to - how can Atom convince early adopters to come on board?

2. Is there a way to have a bunch of cheap servers bombard the network in the hope that the servers are put in a group with less than 2 honest users?

3. Can this architecture be used in other decentralized networks to preserve anonymity

4. Explore configuring the trap message so if a message is altered it is a 90% chance or a 99% chance that it is a trap message? Would this lower the amount of messages that could be transmitted on the network?

5. Explore on making the process computationally expensive to scale in order to prevent one person from getting hundreds of cheap servers to attack the network.

SoK: Secure Messaging

Link to the paper: http://cacr.uwaterloo.ca/techreports/2015/cacr2015-02.pdf

Summary:

As kind of a precursor I was really interested in the topic so on top of reading the paper I actually looked up the presentation the team gave at the symposium: SoK: Secure Messaging.

This paper goes through a lot of the personal communication systems out in the world today and tries to dig in to whether or not they deliver on the privacy solutions that a lot of consumers have heard hyped up in a post Snowden world. The point of this paper was to go through and create a rubric of what a good secure private messaging system should have and really look at private messaging systems from a broad standpoint - a little too broad I think for one paper. They specifically look at issues such as security, usability, and ease of adoption through 3 different "orthogonal" lenses: trust establishment, conversation security, and transport privacy.

What I liked:

1. The really good part about this paper is that it doesn't talk about privacy in an abstract way like a lot of other academic papers do - but instead takes a hands on approach which is different from others in the field.

2. The paper hammers in the fact that everyone builds different privacy tools but no one really digs into whether or not these privacy tools really work

3. The data visualization of the different methods and tools in the report.

4. I also liked the fact that the Electronic Frontier Foundation played a role in this paper - I personally consider them to be the gold standard in privacy

5. The Paper considers a wide variety of threat models from 3 different types of adversaries ie local adversaries, global adversaries, and service providers

What I didn’t like:

1. The author of the paper in the video acknowledges how secure messaging features like deniability and transcript consistency still have a lot of work needed before they can be deployed in a group chat setting - Multiparty OTR still lacking - so the paper is trying to evaluate some technology that hasn't really been created yet

2. The paper never really considers how implementation is handled ie if you have a lot of great theoretical methods for keeping secure how do you evaluate the privacy system for failures in implementation

3. I'm not sure how this paper is much more difficult than the EFF secure messaging scorecard, the only major difference that I can see is the addition of end to end encryption in the evaluation system

4. The paper isn't really focused and extremely broad. I would have preferred they stuck to one of the 3 issues they pointed out in the beginning of their paper and really dug into how that can be improved and evaluated all the different schemes from that aspect. I feel like they cover a lot but don't go into enough depth on some of the really interest aspects.

5. I don't think that block chains should necessarily be in the purview of this paper. Public distributed ledgers are the opposite of a private messaging system.

 Points for further discussion:

1. Is it better to have a secure messaging system tailored to a specific use case or a 1 size fit all method standardization?

2. Can this paper be used as a framework for how to evaluate future privacy projects

3. Why hasn't there been a standardization of terms in the cybersecurity world when it comes to secure messaging - the paper keeps needing to define terms about the different features a protocol offers

4. Something that came up in the symposium QandA was the demand for people using End to End encryption. The study cites that a high amount of Americans would like more privacy but how many would trade convenience for that privacy?

5. Why has their not been a prior review of "out in the wild" approaches? 

New Ideas:

1. "ScatterShot" encryption for messaging applications? Essentially each packet of information is randomly sent to servers at a set rate - anyone can intercept these packets. But to read them you need to have a specific private key that has been predetermined by a different scheme. The rate continues until you run out of new messages to relay - at which point we just keep looping through past messages.

2. Study usability in privacy among different populations ie general public vs journalists vs dissidents vs whistleblowers - how complex is too complicated

3. Can hashes of the messages which are published out in the open be used to create transcript finality? IE everytime some message is sent and opened the encrypted message is hashed and published --> then upon decryption we know when it is sent

4. At the end of the symposium presentation the author makes the point that security proofs are not enough and the developers are the ones who make the privacy happen? How can academia which usually focuses on the proofs make more of a focus on developers?

5. Possibly add an early adoption metric - as in how painful or risky is it for the first person to join the network - because of what we learned in atom we saw that as the network gets bigger it usually becomes safer for all users.

The Best and Worst of YubiKey Usability

Link to the paper: https://isrl.byu.edu/pubs/sp2018.pdf

Summary:

This paper is based on the entire aspect of 2FA authentication specifically the Yubikey which is a usb like piece of hardware which givens an access code when it has its main button pressed. The novelty of this study was the fact that this was the first major study where 2FA was tested among users outside of the enterprise setting. As part of the study the authors had 2 different aspects - the first was the users experience in the initial set up of 2FA. The second was their experience of the Yubikey using it in their personal lives to login into windows 10, google, and facebook. The results of the study should not be taken as one for the entire public as most of the participants of the study were of the younger generation. But the study found the on-boarding process was a little bit too hard for the general user, and some users experienced issues with usability over the course of the study.  

Points to talk about:

1. The study recommends standardizing the on-boarding process for 2FA In order to make it simpler. Which organization, and parties need to create these standards

2. In the process of standardizing on-boarding for 2FA - would this open the consumer up to more attacks if a vulnerability is found in the standard on-boarding process. Would this make the on-boarding process a higher priority for attack by hackers?

3. Why is there a split in current 2FA methods between business and consumers. Businesses use hardware while consumers use SMS more?

4. How does the Yubikey differ from other forms of 2FA? With similar tests, have the results been the same among 2FA methods?

5. How might this study change when the test group is composed of users who already use 2FA? Will they like Yubikey more or is this effect isolated to first time 2FA users?

 What I liked:

1. The fact they had 2 studies one for the actual set up of 2FA and the other for the daily usability gives a more complete picture of the field

2. I liked that they compared the login in times of users with other secure logins ie compared the 2FA with single sign on codes

3. The study went into depth about the very specific issues users ran into when trying to setup their keys on windows and Facebook

4. Why does Google have such a high success rate for 2FA as compared to other platforms? Possibly because they have enterprise companies already using some of their products?

5. Interesting that the consumers found the Yubikey better to use and preferred it over SMS after the set up process

What I didn’t like:

1. There's no indication of what percentage of the user's visit sites with 2FA as an option. Given that we aren't dealing with enterprise level - I think that it might be worthwhile to examine which consumer facing sites offer 2FA

2. The fact the study narrows 2FA only to Facebook, Google and Windows - not as broad as a normal consumer

3. The users they used in this study didn't know anything about 2FA and were only given 5 mins in the study to google any questions they have - I don’t think this would be representative of real life.

4. The average age of the study skewed very young and very male - which isn't really representative of the demographics of most consumers

5. I'm not sure how the study measures SUS (System Usability Scale) accurately in the set up process if the majority of users could not make it through the first 3 steps as the study said. Did they narrow the data set to only the users who were able to set it up correctly?

 New Ideas:

1. The study neglects to answer the background issue of what percentage of the general population is actually interested in 2FA. This is the first study to go outside the enterprise environment, and in the consumer space consumer wants and usability has historically traded off with security.

2. There's no indication of what percentage of the user's visit sites with 2FA as an option. Given that we aren't dealing with enterprise level - I think that it might be worthwhile to examine which consumer facing sites offer 2FA

3. We could look into way's of how to replicate Google's success with setting up 2FA and how to migrate that to other platforms

4. Look into the psychology of security ie I don't see a reason why SMS is less secure and more user intensive that Yubikey but for some reason a good amount of users in the study said they preferred using the physical key.

5.  How might we go about account sharing - possibly 2 different keys that have the same privileges?

The Rewards and Costs of Stronger Passwords in a University

Link to the paper: https://www.usenix.org/system/files/conference/usenixsecurity18/sec18-becker.pdf

Summary

The study took 100,000 staff and students from a major university and studied the strength of user passwords under a new password scheme. The novelty in the university's scheme was that they would vary the lifetime of the password with its strength. The study through this scheme observed over 200k password resets over the course of 14 months and came up with some interesting insights which didn't exactly line up with what other studies out of CMU had observed. The study found that the stronger passwords were the more likely they were to be reset along with a deficit in password strength among users who forgot their passwords more than once a year.

What I liked:

1. I thought that the following was a very interesting insight "Users who reset their password more than once per year (27% of users) choose passwords with over 10 days fewer lifetime, and while they also respond to the policy, maintain this deficit"

2. The fact that they were using 100,000 enrolled. Users and 200k passwords over 14 months gave the study a really good data set to go off of - they noted they are probably the largest study of this type

3. The study did not alter any of the regular systems of the university. "We were not involved in the design of the policy or the choice of password strength estimator"

4. New users are constantly coming into the system, so it creates different test groups. Students who transitioned over from the old system, and new users - it would be interesting to see how their password strength is different

5. I really liked the fact they looked at different password tiers, ie certain passwords are a lot more valuable to hackers than other passwords.

What I didn’t like:

1. The study acknowledges off the bat that Shannon entropy isn't the best way to measure password strength but uses it as the base of their study

2. The study makes use of 93 anecdotal interviews - which I'm not sure how anecdotal information is

3. Why wasn't an industry standard password cracking estimation method like zxcvbn used as the basis of this study as opposed to Shannon which doesn't necessarily correlate to password cracking strength.

4. The data collected on the study wasn't the user's passwords but instead a single number for the user's password strength. That indicator of strength really doesn't get a lot of scrutiny in the study

5. The study has very different outcomes to a similar study done by CMU - this kind of underscores the fact they had really weak data given that they acknowledge that their method of what they were seeing from the passwords the system was giving them "only weakly correlates to password strength"

 Points to talk about:

1. "The new policy took over 100 days to gain traction, but after that, average entropy rose steadily" - why did it take so long to gain traction? What could be done to shorten that amount of time?

2. The study casts doubts on some debunked myths on when it is time to change passwords. It begs the questions what merits a mandated password change - maybe taking into account their could be unknown compromises on the system

3. For password strength estimation why dies zxcvbn error increase as the password bits get longer? Are there any other algorithm's that can determine password strength?

4. The study makes the point that users don't want to go through a wide ranging security check unless something of value ie money or data is stored in that account. But doesn't every account that you have contain data on the user logging in?

5. The average age of the staff member surveyed in this study is 34.6 which strikes me as pretty young. Did they get a good representation of the entire university faculty, or was it skewed toward younger members?

New Ideas:

1. How significant is the tradeoff between security and convenience. Do companies who employ stricter security measures actually end up with fewer customers?

2. The study makes the distinction in password strength between systems with hard and soft system security transitions, how does the transition type effect security outcomes?

3. The study makes distinctions between different types of users in the system. I think it would be interesting to track how strong the passwords are among the different user groups

4. I think the very basis of how we define password strength is flawed. With some using zxcvbn, this study using Shannon, and others using different methods? It begs the question what is the best way to measure password strength going forward. This study says there are more intensive methods but says they were infeasible to be deployed real time when the user is making his or her password.

5. How does the password length correlate with the different user tier who made the password? I think this would be an interesting follow up study.

The Quest to Replace Passwords

Summary

Off the bat the paper starts about how there is a lack of standardization about password protocols. Many password protocols are too narrow while other try to go for a generic one size fit all approach. The paper starts with the premise that every proponent of a specific method of authentication has a different grading criteria - shaped by specific environments. In order to standardize the grading criteria the paper suggests 25 different factors, but does not specify a weight to each of these factors. The conclusion toward the end of the paper after digging into some of the most popular security authentication is that there really is no scheme that is truly perfect.

What I liked:

1. Very specific 25 actionable grading points which sets a standard instead of just saying a uniform standard is needed in the industry

2. I like the fact they focused on the human computer interface for security as opposed to making this a very broad study on things like machine to machine authentication

3. The study does note that it would be a bad idea to simply have each point weighted the same - it correctly notes that some features are more to be desired than others

4. The paper goes really in depth into the different sign in methods that are currently offered and does a good job explaining the general benefits- but shies away from ranking

5. The chart on page 11 was a good visualization tool to express the different benefits of a specific technology

 What I didn’t like:

1. The paper starts by pointing out how past grading criteria is too narrowly tailored or too generic - how does the proposal of 25 very actionable grading points not fall into the same issue the authors had with past studies?

2. The way the paper goes about the study I feel is problematic because of the rating system. Quasi vs Full vs 0. I'm sure there is more in depth ways to evaluate each of the 25 points. Maybe a 10 point scale possibly?

3. The study does not take any stance on how to rate systems which I feel is like a cop out. "In our experience, “the journey (the rating exercise) is the reward”: the important technical insights we gained about schemes by discussing whether our ratings were fair and consistent were worth much more to us than the actual scores produced."

4. In the paper's analysis of different authentication systems they come across pretty big factors that are not covered in their 25 point grading scale. "We do note however that it requires identity providers yield some control over trust decisions and possibly weaken their own brand [28], a deployment drawback not currently captured in our criteria."

5. The paper starts going into depth about the drawbacks of each scheme only at the very end - I think that they should make this more of a focus in their paper because it really shapes the reasons why one scheme would be preferred over another in a specific use case

 Points to talk about:

1. What is the current industry consensus on the tug of war battle between security and usability?

2. The paper assumes that the implementers of the protocol use best practices such as salting and hashing "even though we know they often don’t." How much more effective are these 25 data points as compared to  just using salting and hashing correctly

3. Is it even worthwhile to create a generic uniform approach to security - or is the targeted narrow approach to security which is more customizable to be desired

4. The study points to traditional passwords as scoring high in maturity because they are so common? What has the adoption trend looked like for alternative adoption methods?

5. What is the biggest reason that traditional passwords have not been phased out yet?

 New Ideas:

1. Security vs Usability: How could information tied to a person's life be used in passwords? Would it be possible to have multiple passwords in the forms of questions that are distributed in various authentication servers with some serving as honeypots?

2. How does new forms of identification compare to traditional passwords - ie the way you type https://www.creditcards.com/credit-card-news/how-type-move-mouse-help-catch-fraudsters.php

3. Is there a way to merge the best of both worlds from multiple password schemes - kind of addressed toward the end of the paper?

4. Perhaps there should be a weighting rubric dependent on the desired out come and balance of security vs usability vs other factors a practitioner would like to fit in when it comes to security

5. Could there be a standard third party ie a government to login in users using single sign on?