Autonomous Machines and ethics – why we need a comfortable answer to who dies before the era of autonomous anything.

We are a hairs breadth away from the era of autonomous machines. The possibilities that they offer are endless, and the way they will influence everything from how work gets done, to city design and transport mean a future that is barely recognizable. It looks like the first autonomous machines that society at large sees will be cars, but before they can become a part of our lives, we need to solve the problem of how our software decides who dies when there are no other choices.

The Trolley problem is a thought experiment in ethics that deals with situations in which act or a failure to act lead directly to a death. The only choice available is to cause death by action or do nothing and let a death occur. While the problem has been used to study how we weight lives – dealing with relatives vs strangers, old vs. young and many deaths vs. a few, the underlying problem is one that will be a reality for autonomous machines. Which is a problem, because someone has to tell them how to make it.

As an example scenario – a self driving car senses a pedestrian on a road way in front of the vehicle, it calculates that evasive action is possible, but the available evasive action is likely to result in the death of the driver. What should the car do in this situation? More pertinently, how should it be programmed to respond to this situation?

Computers are deterministic systems. Their programming encodes rules that they follow exactly. There is no randomness to their behavior (unless caused by bugs or bad input). Mostly, this is a great thing. When making life and death decisions though, it means that someone needs to tell the computer how to make the decision, that it will follow, exactly. Someone needs to tell the computer who to kill.

This means that we need to make the decision ahead of time about who will live, and who will die. More importantly, we need a societally acceptable way to program something to make the decision – and it doesn’t exist.

The problem is larger than autonomous machines. That there is no societally acceptable way to make a decision about who should die. It’s why legislation doesn’t tell us how much a life is worth. The decision is left to judges and actuaries.

In the context of autonomous vehicles, this means that the decision is going to get left to actuaries to game out, and programmers to execute. This means that ultimately, we won’t decide how to handle it. We’ll let a company make it, we’ll let someone die – and then we’ll wait years for a judge to decide who was liable, and if they made the decision acceptably – at which point it will become an insurance issue. The alternate scenario is that it becomes a political issue – and ultimately, I think that’s worse for everyone.

We don’t have legislation that tells us how much a life is worth because no politician wants to touch the issue. It is a no-win scenario. There is no political move forward in putting a price on a human life, or in providing a way to decide who should die. Politicians are also action oriented people, when was the last time a politician appeared on the news saying that they weren’t going to “do something”. Logically, the only thing a politician can do is impose a ban – or conditions that might as well be a ban.

This is one reason I think it’s going to be quite a while until we see a totally autonomous private vehicle – at least on a road shared with human drivers. At some point, there will be a real life trolley problem, and someone will die. If we have not decided how to make the decision in a societally acceptable way, we leave to chance that it will be a political process that makes it for us, and we may give up the next major advancement in the quality of our lives.

The gap in the electronic signature market that blockchain and cryptographic technology might be able to fill

Last year DocuSign IPO’d for over six Billion. They make a great product that provides a high level of convenience and a degree of certainty for the people that use it. At a recent conference I caught a session about digital signatures by Mark Henderson from Adelaide Law firm Kelledy Jones. It highlighted where electronic and digital signatures work well, and places where there are problems that make them uncertain. There is also one area in which they are currently unusable, and which I think provides a legitimate opportunity for the most over-hyped technology in the market to do something useful.

The requirements for a valid contract to be formed are well understood, and the legislation and legal precedents for entering into contracts legally via electronic means exist. In essence, there is not generally anything stopping an electronic contract in most situations. The quotation from one relevant case by Justice Harrison reads “Mr Stuart typed his name on the foot of the email. He signed it by doing so. It would be an almost lethal assault on common sense to take any other view”. There are however a number of areas in which electronic means of signing cannot currently be used.

These areas generally require one or both of two things – a witness, or the physical fixing of some form of stamp or seal. I am not qualified to go into the nuances of which instruments these are, but what intrigues me is that these seem like logical areas for a technology like Blockchain.

Witnesses and seals raise the volume and quality of information required to reduce the chance that a contract can be repudiated. As an example, a contract with a single signature on it could be repudiated by a company on the basis that the signature was forged. A contract signed by the chairman and CEO of the company, with the corporate seal affixed and witnessed by a non-interested third party on the other hand is far harder to repudiate. Each step adds information that reduces the chance that an apparently valid contract can be repudiated – and this is why blockchain might provide a technical solution in the presence of enabling legislation.

The centre piece of blockchain is non-repudiation. This means that it has technology that proves a transaction has not been tampered with. It does this by using cryptographic measures that verify the position of an item in the chain based on the information from the items in the chain prior to it. This means that once an item is entered in the chain, it cannot be changed unless every other item after it is then altered as well. Obviously this could be accomplished, but a blockchain is also required to be distributed among many peers and kept synchronised to make tampering more difficult – so someone trying to change one transaction would also have to get every other peer to change their chain too.

This all means that a blockchain could provide a publicly available repository of contract data – kept in public and distributed so that it could not be tampered with, and cryptographically signed by both sides of the transaction. Witnessing of the transaction could take the form of a future transaction by a neutral third party that references the original transaction. This witnessing could include cryptographic information that replaces the current function of a corporate seal using Public Key cryptography.

The simplicity of this idea is that it could scale to encompass scenarios requiring multiple signatures and witnesses, and also the affixing of a corporate seal. It could also include amendments and variations to contracts through self-referencing. Given the flexibility of blockchain, it would also be trivially easy to use a digital identity provider to verify identity and sign the transaction – and include information about liveness tests.

Personally, I’ve used DocuSign and formal digital signatures for many transactions and entered into contracts by email and other digital methods. The legal framework makes doing this simple and the actual technology largely irrelevant. In the case of transactions requiring additional information to verify their validity though, we have both a legal and technical challenge, I think that blockchain provides a useful way to solve the technical side of the issue – which would enable a common law test and might lead us to a world that no longer requires the signing ceremony.

The four questions you need to ask to balance external collaboration and risk management

In Darkest Hour there’s a scene in which Winston Churchill is almost begging Franklin Roosevelt for the warplanes that they had ordered, and now desperately needed. It got me thinking about collaboration and about how important it can be for gaining capability; and how badly it can go if we don’t assess the new risks that collaborating entails. I’ll refer back to this scenario throughout the post but please note that for the history buffs, I know I’m not being historically accurate.

Collaboration is essential to doing business – I don’t think this is in dispute (if you’d like to dispute it, I’ll happily debate you). What I don’t think gets talked about often enough is how we should think about the decision to collaborate. All collaboration should be cost-benefit based, with the decision predicated on whether the benefits outweigh the risks.

I’ve found that the frame below a useful way to think about the decision to collaborate with another organisation.

There are four basic questions that need answering:

  1. What benefit is there to collaborating?
  2. What new risks does the collaboration expose us to?
  3. Can you effectively mitigate the risks?
  4. Who has the authority to decide whether the balance is right?

1. What benefit is there to collaborating?

The UK needed planes to fight a resurgent Germany – but should they build them, or collaborate to get what they needed sooner and maybe even cheaper?

Stage 1 must be about quantifying the gain from collaboration. Collaboration is always about achieving objectives cost-effectively. If your organisation can already achieve the objective cost effectively, you don’t need to collaborate. Your thesis should be that collaborating will let you will gain access to information, capability or knowledge at a cost that makes the risks of collaborating worth the organisational gain. The deliverable from this stage should be a cost-benefit analysis.

2. What new risks does the collaboration expose you to?

Getting the planes sooner and cheaper gives the UK more capability, and more reserves to buy more capability if they need it – but what risks will relying on the US introduce?

Collaboration will always expose your business to new risks – so stage 2 should be identifying and quantifying them. New risks will principally come from two places –

  1.  The fact that your counter-party will always have better information about what they offer.
  2. The information you will expose about and from your business while collaborating.

To get this assessment right, you should seek input from experts in all areas of your business. Once you have an accurate risk register, and assessment of the magnitude of harm, you can move on to mitigation strategies.

3. Can you effectively mitigate the risks?

The war winning or losing question – if the US doesn’t deliver the planes when we need them, what do we do?

Once you understand the risks, mitigation measures can be defined. Risk mitigation will look to balance the incentives of involved parties, minimise the amount of information leakage, make sure adequate information about service quality is available, and ensure that adequate backup plans are in place. Once you understand your ability to effectively mitigate each risk, you can then make a call about whether to insure against or accept any residual risks.

4. Who has the authority to decide whether the balance is right?

Get it right – and the UK gets the planes and has the capability it needs to defend itself. Get it wrong, and the UK faces a resurgent Germany without what it needs to defend itself. Who had the authority to risk the future of the United Kingdom on this collaboration?

Someone in the organisation will need to accept the risks of a new collaboration. How high in the organisation they are will depend on the impact of loss. If you know who this person is, you should consider their appetite for risk early on and include that in your early go/no-go decisions. It goes without saying that putting together a great case for someone whose’ risk aversion won’t let you move forward should be avoided.

Conclusion

We all collaborate many times a day without thinking about it. When we go across the boundary of our organisation, the dynamic changes. We should be doing it because there is a cost-benefit relationship that lets us access information, capability and knowledge at far below the cost of developing it. Benefits though are only one side, we need to also consider the risks and whether we are able to take those risks on behalf of our organisation. If you keep this frame in mind, you should get good outcomes.

Does this reflect your view on collaboration and how it should be approached? If not, I’d love to hear from you and understand why.

The only four reasons why you should collaborate with another organisation

Collaboration is “the act of working together to produce something”. Over the last ten years it has been billed as a magic bullet, and like so many magic bullets, it has become unfashionable to question why, how and if we achieve value from it. Which I think is a mistake that has eroded our understanding of it, and lead to the current magic bullet view.

In a previous post, I outlined what you can get access to when you collaborate. Just because you can though, it doesn’t mean that you should. There are four reasons that I think make collaboration worth consideration:

  1. You need something faster than your organisation can deliver it.
  2. Developing or acquiring what you need in-house is too expensive.
  3. Your organisation cannot create what you need because of regulations.
  4. You’re looking for the capability to become a customer.

Under any other circumstances, you should seriously question the need to collaborate. Collaborating is a build or buy decision that has to stand up to a cost-benefit analysis. If your organisation can deliver what you need fast enough and at a market competitive price, without violating any regulations, what justifiable reason do you have for collaboration?

The three things you can get by collaborating with another organization.

Collaboration is “the act of working together to produce something”.

It’s fundamentally about going outside your organization to get access to something without having to build it. There are also times when you might not be able to get what you need internally for regulatory reasons, or because the capability to become a customer is a self-defeating cycle if you’re just buying your own stuff.

When you collaborate with another organization, you should think about the gap you are trying to fill in terms of one of the following three things –

  1. Information – facts about something or someone.
  2. Knowledge – the theoretical or practical understanding of a subject.
  3. Capability – the power or ability to do something.

Once you define your need in one of these terms, you’re much more likely to make the right decision about whether you should collaborate, and then to find an efficient supplier of what you need.