Autonomous Machines and ethics – why we need a comfortable answer to who dies before the era of autonomous anything.

We are a hairs breadth away from the era of autonomous machines. The possibilities that they offer are endless, and the way they will influence everything from how work gets done, to city design and transport mean a future that is barely recognizable. It looks like the first autonomous machines that society at large sees will be cars, but before they can become a part of our lives, we need to solve the problem of how our software decides who dies when there are no other choices.

The Trolley problem is a thought experiment in ethics that deals with situations in which act or a failure to act lead directly to a death. The only choice available is to cause death by action or do nothing and let a death occur. While the problem has been used to study how we weight lives – dealing with relatives vs strangers, old vs. young and many deaths vs. a few, the underlying problem is one that will be a reality for autonomous machines. Which is a problem, because someone has to tell them how to make it.

As an example scenario – a self driving car senses a pedestrian on a road way in front of the vehicle, it calculates that evasive action is possible, but the available evasive action is likely to result in the death of the driver. What should the car do in this situation? More pertinently, how should it be programmed to respond to this situation?

Computers are deterministic systems. Their programming encodes rules that they follow exactly. There is no randomness to their behavior (unless caused by bugs or bad input). Mostly, this is a great thing. When making life and death decisions though, it means that someone needs to tell the computer how to make the decision, that it will follow, exactly. Someone needs to tell the computer who to kill.

This means that we need to make the decision ahead of time about who will live, and who will die. More importantly, we need a societally acceptable way to program something to make the decision – and it doesn’t exist.

The problem is larger than autonomous machines. That there is no societally acceptable way to make a decision about who should die. It’s why legislation doesn’t tell us how much a life is worth. The decision is left to judges and actuaries.

In the context of autonomous vehicles, this means that the decision is going to get left to actuaries to game out, and programmers to execute. This means that ultimately, we won’t decide how to handle it. We’ll let a company make it, we’ll let someone die – and then we’ll wait years for a judge to decide who was liable, and if they made the decision acceptably – at which point it will become an insurance issue. The alternate scenario is that it becomes a political issue – and ultimately, I think that’s worse for everyone.

We don’t have legislation that tells us how much a life is worth because no politician wants to touch the issue. It is a no-win scenario. There is no political move forward in putting a price on a human life, or in providing a way to decide who should die. Politicians are also action oriented people, when was the last time a politician appeared on the news saying that they weren’t going to “do something”. Logically, the only thing a politician can do is impose a ban – or conditions that might as well be a ban.

This is one reason I think it’s going to be quite a while until we see a totally autonomous private vehicle – at least on a road shared with human drivers. At some point, there will be a real life trolley problem, and someone will die. If we have not decided how to make the decision in a societally acceptable way, we leave to chance that it will be a political process that makes it for us, and we may give up the next major advancement in the quality of our lives.

The gap in the electronic signature market that blockchain and cryptographic technology might be able to fill

Last year DocuSign IPO’d for over six Billion. They make a great product that provides a high level of convenience and a degree of certainty for the people that use it. At a recent conference I caught a session about digital signatures by Mark Henderson from Adelaide Law firm Kelledy Jones. It highlighted where electronic and digital signatures work well, and places where there are problems that make them uncertain. There is also one area in which they are currently unusable, and which I think provides a legitimate opportunity for the most over-hyped technology in the market to do something useful.

The requirements for a valid contract to be formed are well understood, and the legislation and legal precedents for entering into contracts legally via electronic means exist. In essence, there is not generally anything stopping an electronic contract in most situations. The quotation from one relevant case by Justice Harrison reads “Mr Stuart typed his name on the foot of the email. He signed it by doing so. It would be an almost lethal assault on common sense to take any other view”. There are however a number of areas in which electronic means of signing cannot currently be used.

These areas generally require one or both of two things – a witness, or the physical fixing of some form of stamp or seal. I am not qualified to go into the nuances of which instruments these are, but what intrigues me is that these seem like logical areas for a technology like Blockchain.

Witnesses and seals raise the volume and quality of information required to reduce the chance that a contract can be repudiated. As an example, a contract with a single signature on it could be repudiated by a company on the basis that the signature was forged. A contract signed by the chairman and CEO of the company, with the corporate seal affixed and witnessed by a non-interested third party on the other hand is far harder to repudiate. Each step adds information that reduces the chance that an apparently valid contract can be repudiated – and this is why blockchain might provide a technical solution in the presence of enabling legislation.

The centre piece of blockchain is non-repudiation. This means that it has technology that proves a transaction has not been tampered with. It does this by using cryptographic measures that verify the position of an item in the chain based on the information from the items in the chain prior to it. This means that once an item is entered in the chain, it cannot be changed unless every other item after it is then altered as well. Obviously this could be accomplished, but a blockchain is also required to be distributed among many peers and kept synchronised to make tampering more difficult – so someone trying to change one transaction would also have to get every other peer to change their chain too.

This all means that a blockchain could provide a publicly available repository of contract data – kept in public and distributed so that it could not be tampered with, and cryptographically signed by both sides of the transaction. Witnessing of the transaction could take the form of a future transaction by a neutral third party that references the original transaction. This witnessing could include cryptographic information that replaces the current function of a corporate seal using Public Key cryptography.

The simplicity of this idea is that it could scale to encompass scenarios requiring multiple signatures and witnesses, and also the affixing of a corporate seal. It could also include amendments and variations to contracts through self-referencing. Given the flexibility of blockchain, it would also be trivially easy to use a digital identity provider to verify identity and sign the transaction – and include information about liveness tests.

Personally, I’ve used DocuSign and formal digital signatures for many transactions and entered into contracts by email and other digital methods. The legal framework makes doing this simple and the actual technology largely irrelevant. In the case of transactions requiring additional information to verify their validity though, we have both a legal and technical challenge, I think that blockchain provides a useful way to solve the technical side of the issue – which would enable a common law test and might lead us to a world that no longer requires the signing ceremony.

The four questions you need to ask to balance external collaboration and risk management

In Darkest Hour there’s a scene in which Winston Churchill is almost begging Franklin Roosevelt for the warplanes that they had ordered, and now desperately needed. It got me thinking about collaboration and about how important it can be for gaining capability; and how badly it can go if we don’t assess the new risks that collaborating entails. I’ll refer back to this scenario throughout the post but please note that for the history buffs, I know I’m not being historically accurate.

Collaboration is essential to doing business – I don’t think this is in dispute (if you’d like to dispute it, I’ll happily debate you). What I don’t think gets talked about often enough is how we should think about the decision to collaborate. All collaboration should be cost-benefit based, with the decision predicated on whether the benefits outweigh the risks.

I’ve found that the frame below a useful way to think about the decision to collaborate with another organisation.

There are four basic questions that need answering:

  1. What benefit is there to collaborating?
  2. What new risks does the collaboration expose us to?
  3. Can you effectively mitigate the risks?
  4. Who has the authority to decide whether the balance is right?

1. What benefit is there to collaborating?

The UK needed planes to fight a resurgent Germany – but should they build them, or collaborate to get what they needed sooner and maybe even cheaper?

Stage 1 must be about quantifying the gain from collaboration. Collaboration is always about achieving objectives cost-effectively. If your organisation can already achieve the objective cost effectively, you don’t need to collaborate. Your thesis should be that collaborating will let you will gain access to information, capability or knowledge at a cost that makes the risks of collaborating worth the organisational gain. The deliverable from this stage should be a cost-benefit analysis.

2. What new risks does the collaboration expose you to?

Getting the planes sooner and cheaper gives the UK more capability, and more reserves to buy more capability if they need it – but what risks will relying on the US introduce?

Collaboration will always expose your business to new risks – so stage 2 should be identifying and quantifying them. New risks will principally come from two places –

  1.  The fact that your counter-party will always have better information about what they offer.
  2. The information you will expose about and from your business while collaborating.

To get this assessment right, you should seek input from experts in all areas of your business. Once you have an accurate risk register, and assessment of the magnitude of harm, you can move on to mitigation strategies.

3. Can you effectively mitigate the risks?

The war winning or losing question – if the US doesn’t deliver the planes when we need them, what do we do?

Once you understand the risks, mitigation measures can be defined. Risk mitigation will look to balance the incentives of involved parties, minimise the amount of information leakage, make sure adequate information about service quality is available, and ensure that adequate backup plans are in place. Once you understand your ability to effectively mitigate each risk, you can then make a call about whether to insure against or accept any residual risks.

4. Who has the authority to decide whether the balance is right?

Get it right – and the UK gets the planes and has the capability it needs to defend itself. Get it wrong, and the UK faces a resurgent Germany without what it needs to defend itself. Who had the authority to risk the future of the United Kingdom on this collaboration?

Someone in the organisation will need to accept the risks of a new collaboration. How high in the organisation they are will depend on the impact of loss. If you know who this person is, you should consider their appetite for risk early on and include that in your early go/no-go decisions. It goes without saying that putting together a great case for someone whose’ risk aversion won’t let you move forward should be avoided.

Conclusion

We all collaborate many times a day without thinking about it. When we go across the boundary of our organisation, the dynamic changes. We should be doing it because there is a cost-benefit relationship that lets us access information, capability and knowledge at far below the cost of developing it. Benefits though are only one side, we need to also consider the risks and whether we are able to take those risks on behalf of our organisation. If you keep this frame in mind, you should get good outcomes.

Does this reflect your view on collaboration and how it should be approached? If not, I’d love to hear from you and understand why.

The only four reasons why you should collaborate with another organisation

Collaboration is “the act of working together to produce something”. Over the last ten years it has been billed as a magic bullet, and like so many magic bullets, it has become unfashionable to question why, how and if we achieve value from it. Which I think is a mistake that has eroded our understanding of it, and lead to the current magic bullet view.

In a previous post, I outlined what you can get access to when you collaborate. Just because you can though, it doesn’t mean that you should. There are four reasons that I think make collaboration worth consideration:

  1. You need something faster than your organisation can deliver it.
  2. Developing or acquiring what you need in-house is too expensive.
  3. Your organisation cannot create what you need because of regulations.
  4. You’re looking for the capability to become a customer.

Under any other circumstances, you should seriously question the need to collaborate. Collaborating is a build or buy decision that has to stand up to a cost-benefit analysis. If your organisation can deliver what you need fast enough and at a market competitive price, without violating any regulations, what justifiable reason do you have for collaboration?

The three things you can get by collaborating with another organization.

Collaboration is “the act of working together to produce something”.

It’s fundamentally about going outside your organization to get access to something without having to build it. There are also times when you might not be able to get what you need internally for regulatory reasons, or because the capability to become a customer is a self-defeating cycle if you’re just buying your own stuff.

When you collaborate with another organization, you should think about the gap you are trying to fill in terms of one of the following three things –

  1. Information – facts about something or someone.
  2. Knowledge – the theoretical or practical understanding of a subject.
  3. Capability – the power or ability to do something.

Once you define your need in one of these terms, you’re much more likely to make the right decision about whether you should collaborate, and then to find an efficient supplier of what you need.

Where is multi factor authentication in the 2017 Australian Government Information Security Manual?

This post is going to be a bit dry, it is written to provide an accurate overview of specifically where you can find multi-factor authentication controls in the 2017 Australian Government Information Security Manual (ISM). It is accurate as at the 3rd of March 2017. If you are in a security or IT decision-making role, and are considering whether multiple factors of authentication should be part of your security apparatus, the ISM provides both a minimum standard for accreditation, and guidance that can be used to inform your risk assessment. Each control is contextual, and doesn’t apply to every situation – you should seek a qualified opinion from a member of the IRAP program to ensure that you are assessing the right controls.

The minimum standard is imposed through controls that are listed as “must” for compliance purposes. In areas where some consideration of control vs. ease of access is appropriate, the controls are listed as “should”. What is clear from the ISM is that for system administrative activities, it is not considered acceptable to act without multiple factors of authentication. In some individual user access scenarios though, multiple-factors are listed as “should”. This lessening of controls for end users provides scope for each agency to consider the level of risk associated with access to the system, the level of burden that it is appropriate for users of that system to bear, and the level of operational complexity that the additional factors add.

As always, prior to looking at controls, the grade of information the service will carry needs to be decided. The cost of achieving each higher classification rises substantially, and each successive classification focuses more on access control than ease of access. From a risk perspective, more consideration of whether “should” should become “must” should also be considered. Appropriately qualified security and risk management personnel should be engaged to advise on these matters.

From a pure control standpoint, the controls focused on multi-factor authentication are listed below, each applies to all classifications –

  • 0974 – “Agencies should use multi-factor authentication for all users.”
  • 1039 – “Agencies should use multi-factor authentication for access to gateways.”
  • 1173 – “Agencies must use multi-factor authentication for” – system and database administrators, privileged users, positions of trust and remote access.
  • 1384 – “Agencies must ensure that all privileged actions must pass through at least one multi-factor authentication process.”
  • 1401 – “Agencies using passphrases as part of a multi-factor authentication must ensure a minimum length of six alphabetic characters with no complexity requirement.”

Some discussion of Multi-factor authentication can also be found in the “Access Control” section of the ISM – Principles manual.

All the documentation you need can be found at https://www.asd.gov.au/infosec/ism/index.htm

The 2017 ISM – Controls can be found at – https://www.asd.gov.au/publications/Information_Security_Manual_2017_Controls.pdf

The 2016 ISM (the latest) – Principles can be found at – https://www.asd.gov.au/publications/Information_Security_Manual_2016_Principles.pdf

What access security really is, and the myth of “totally secure”

A good definition of security will let you do two things –

  • Make more objective decisions about how much to spend on security.
  • Sort out who the people who don’t know what they’re talking about (and the liars) are.

So what is security as it relates to access?

Access security is the process of making access cheap for people who are authorised, and expensive for people who are not authorised.

That’s a simple but objectively useful definition. It’s useful because it can be applied simply to every form of access security – armed guards, door locks, IT security, theft laws – it’s universal. With the definition in place, you can move on to a conversation about how expensive access should be, then think about how you invest. Any investment in security should introduce significantly more cost for someone attempting unauthorised access.

In the light of that definition, it’s also easy to see why “totally secure” is a myth. Any form of access means something is now only secure against a certain amount of expenditure.

Next time you’re talking to someone about security and they’re asking you to make an investment, ask them about how much access expense the investment will add for an unauthorised person. Then you can compare it to the cost of what you’re securing. If you can’t make sense of that, you should find someone who can help you can. Any investment in security that doesn’t make unauthorised access many times more expensive than its cost is just the purchase of a warm and fuzzy feeling.