AI Versus Human - Who is Responsible?

Dr. Yaozhang Pan, Head of Data Science, Shopee

Dr. Yaozhang Pan, Head of Data Science, Shopee

Artificial Intelligence (AI) technology is changing our world - most of this change will be for the good of humanity, such as the transformation of the healthcare, farming and transportation industries, prediction of natural disasters, and more. However, there is a risk of these AI-enabled systems being misused for dangerous or harmful purposes, either intentionally or unintentionally, to the detriment of humanity. Billionaire entrepreneur Elon Musk has often times suggested that AI could potentially be very dangerous; in fact, he once said that “AI will be the best or worst thing ever for humanity.” As AI continues to advance rapidly, and show no signs of slowing down, it is vital that we kick start the discussion on the ethics of AI, and determine how we can develop AI positively while minimising its destructive potential.

Let me pose a hypothetical situation — someone you love is hit by a car, and after you dial for emergency services, an AI-enabled ambulance is signaled. This ambulance manages to reach your loved one in time because it was able to calculate an optimal route using real-time and historic traffic data. When your loved one, the victim of the car accident, survives, you would naturally thank your lucky stars for AI. However, if you eventually learn that the vehicle at fault for the car accident was an AI-enabled self-driving car, how would you feel? Would your loved one have been hit if it was a human behind the wheel? Would the ambulance have gotten there in time if not for AI?

Now imagine if you were the CEO of a self-driving car company, when an AI-enabled car kills somebody, who is responsible? How would you prepare for this moment? What capabilities would you need? What kind of transparency? It turns out that many CEOs don’t know where to begin when asked these questions.

The above scenarios demonstrate the potential for AI to transform customer experience, but they also show the risks AI poses to human rights.

By examining the risks AI could bring to humanity, we can see that there are several challenges we need to overcome for AI-enabled systems:

Transparency and Explainability

AI-enabled systems are known to be a “black box”, which means that their internal workings are not known to the people using them, as they are made up of proprietary pieces of software developed by someone else.

However, it is important to make the systems transparent and explainable to users. AI is being widely used in many areas today, such as to decide on loans, insurance claims and job applications, and even in the judicial system. If your mortgage, job interviews, or written judgement are decided by an algorithm, you would want to be sure its assessment is fair. This is impossible if the people using AI do not understand how it works.

At Shopee, we rely heavily on AI to identify fraudsters and spam listings. We put a lot of effort into educating every local operation team to ensure that they have a good understanding of how the systems work. We also build numerous explanation tools to help teams understand why the models make certain decisions.

Data Security and Privacy

AI-enabled systems rely heavily on data, so AI experts need to access vast amounts of data, including sensitive personal data, in order to build AI models. As a result, the risk of a data breach arises.

At Shopee, we have a dedicated Developmental Operations team that controls data warehouse access hierarchically. The team also develops up-to-date network security technologies to avoid any intentional or unintentional data leakage.

Human Safety

People in the AI industry expect that autonomous systems such as self-driving cars, Unmanned Aerial Vehicles or robotics will be seamlessly integrated into society in the future. However, if the AI-enabled systems are deployed across public or industrial areas, we will first have to ensure that human safety will not be compromised by these autonomous systems.

Fairness and Nondiscriminatory

A machine learning AI-enabled system is developed using training data. A machine learning model typically relies on the training data to recognise and identify patterns and make predictions for new data. Since we only work with a small sample of data, this causes skewed and incomplete training data sets. Also, real-world data usually have natural bias. Some examples include over-policing of minority groups and discrimination in access to loans. If we do not correct biases in the data, it is likely that AI-enabled systems will further amplify existing discrimination. At Shopee, we specifically design algorithms to counter the possible bias of training data by balancing factors such as gender and age in our user data.

Human Control

As we build trust in AI-enabled systems, we should continue to exercise our ability to judge the correctness of decisions made by these systems. We should also actively challenge decisions made by AI-enabled systems that we believe to be unfair. As such, no matter how advanced AI becomes, we should always ensure human control of it. At Shopee, we make sure all AI-enabled systems follow Standard Operating Procedures to bring in regular human intervention.

As a result of the above challenges, we continuously drive the notion of responsible AI at Shopee. We make sure that every employee working with AI is aware of the impact that AI-enabled systems can have when implemented in different situations. We believe in building responsible AI-enabled systems and we make sure to account for decisions made by these systems. AI-enabled systems need to be programmed to act responsibly and fair within their boundaries for sustainable and trustworthy outcomes. This is the only way towards successful and sustainable use of AI in our organisation.