Transparency in AI

Introduction

Nowadays, AI is extensively used in multiple domains and affects our lives significantly. Banks use credit score prediction models to decide a person's eligibility for getting a loan. Airports use facial characteristics for detecting potential suspects. In such scenarios, if AI flags a person, they get affected by that decision in terms of money, time, and resources.

Therefore, the ML model should be able to explain its decisions so that a potential candidate does not get denied a loan and an innocent person does not get flagged as suspicious. This aspect of AI comes under the purview of 'Transparency in AI'.

Principle of Transparency in AI

All individuals affected by AI have the right to know the grounds on which they are affected. For example, a person getting flagged as suspicious in an airport has the right to know the reasons behind the same. We must ensure that AI models do not have any unfair bias while making such decisions.

What is Transparency?

Transparency is a system property that enables us to extract information about the system's inner workings. Some other terms are used as synonyms of transparency, such as explainability(xAI), interpretability, and understandability.

Transparency in AI

The extent of transparency in AI depends on the ethical issues we are solving. In essence, transparency is relevant to the following matters.

• Justification of decisions: Decisions that can morally or legally affect any individual should be non-arbitrary and justified. The grounds on which the decision is taken should be publicly accessible.
• Right to know: Individuals have the right to know the basis on which a decision affects them. They have the right to know about the usage of their data.
• Moral obligation to understand the consequences of our actions: A community's ethical duty is to thoroughly assess the risks associated with a system before unleashing them to the public.

Transparency as a Property of the System

Transparency in AI addresses how a model works internally. It is further divided into

• Simulatability: Functioning of a model
• Decomposability: Components of a model
• Algorithmic Transparency: Algorithms behind a model

Further, a system becomes a black box due to reasons like

• Complexity: Modern AI systems have millions of parameters learned using complex algorithms. Even if the parameters are known, extracting information from them is highly difficult.
• Difficulty in Building Explainability Solutions: Despite advancements in explainable AI, creating a user experience for easily understandable explanations is challenging.
• Risk Concerns: If AI algorithms are made more transparent, then adversaries might give inputs that generate unintended outputs leading to system malfunctioning.

How to Make Models More Transparent

As shown below, there can be various ways to increase transparency in AI models.

• Simpler Models: This can, however, lead to a decrease in accuracy.
• Combine Simpler and Sophisticated Models: Simpler models provide more transparency, whereas sophisticated models can handle complexity.
• Modify Input and Identify Input-Output Relations: This distinguishes crucial input data points and features.
• Design Models for User: This involves visualization of the model states and the input features to highlight the most critical factors for decision-making.
• Follow Latest Research: This involves using the various xAI techniques developed nowadays to increase transparency in AI models.

Why do We Need Transparency in AI?

We need transparency in AI due to the following reasons.

• Unexplained Algorithm: Some ML algorithms are pretty deep, and hence it is challenging to extract the decision-making grounds of the ML model.
• Lack of Visibility into Training Datasets: We must ensure the training dataset is tidy and well-labeled for a well-performing model.
• Lack of Visibility into Data Selection Methods: Apart from training data, we need to know if any data augmentation techniques are used and what portion of the data is used.
• Limited Understanding of Bias in Training Datasets: AI models should not be unfairly biased towards a social segment. Therefore, we must ensure that the training dataset is not socially or culturally biased.

Challenges for Transparency in AI

There are some challenges to transparency in AI. If there is transparency in AI models, we can detect when a model performs poorly and prevent any subsequent faulty decisions. If a model performs poorly and provides wrong predictions, this might be due to the following reasons.

• The dataset might have some unclean or error-strewn data points, leading to the AI models learning incorrect decision boundaries and parameters. This would further lead to inaccurate predictions.
• Tuning the hyperparameters is crucial for making the models learn the correct decision boundaries and parameters. Therefore, misconfiguration of the hyperparameters can also lead to incorrect learning and predictions.
• Even if the data is error-free and the model is trained right, we can have wrong predictions if the data points in the dataset are biased. This bias can pertain to cultural/social/gender or any other bias that prefers one class. This can happen when we do not consider the authenticity or biasedness of the source while collecting the data.

Best Approaches for Transparency in AI

A few best approaches for increasing transparency in AI are listed as follows.

• Keep humans in the loop: Humans must analyze decisions to prevent errors in AI projects.
• Eliminate Biased Datasets: For constructing non-discriminatory and accurate AI models, we need to ensure that the datasets do not contain unfair bias.
• Ensure Explainable Decisions: Explainable AI helps illustrate the work behind an AI model making decisions.
• Reliable Reproduction of Findings: AI models should always be consistent in making predictions for the same input.

Essential Roles of Transparency in AI

There can be various essential roles of transparency in AI which can be listed as follows.

• Legal Needs: For legal and regulatory explanations, it is required to use highly understandable algorithms.
• Severity: For important life missions, it is required to have a reasoning mechanism to improve teamwork with human factors.
• Transparency: This is highly needed when AI models are used in life-saving missions and affect someone's existence.
• Exposure: This involves the access control of the AI model.
• Dataset: It is always desirable to have a balanced and diverse dataset as much as possible.

Who is Responsible?

Following stakeholders affect the functioning of an AI system and, therefore, can be held responsible for increasing transparency in AI and any incident caused by the AI model.

• Hardware builder: Builds the sensors to feed input data into the prediction models.
• Software Engineer: Builds the software and the models essential for the working of the entire AI system.
• End-user of the AI model: Personalizes the AI system according to personal preferences to generate desirable actions.
• The AI model: Learns the various parameters and decision boundaries using a learning algorithm and accordingly takes decisions.

Use cases

There are various use cases of AI transparency, some of which are listed below.

• Transparent AI in Healthcare: Healthcare requires one of the highest transparency in AI systems. It is highly undesirable to wrongly diagnose a patient leading to incorrect treatments. Therefore, it is crucial to validate how an AI system diagnoses or detects a disease.
• Industrial IoT: Modern industries heavily depend upon automation using sensors. These sensor data are then fed into an AI model, which takes crucial operational decisions. Any flaw in the sensor data or the model can lead to high risks in operations and subsequent loss. Therefore, it is desirable to validate the decisions taken by an AI system.
• Autonomous Vehicles: Autonomous vehicles involve very high stakes, including the lives of other humans using the road. Therefore, a single wrong decision has the potential to cause deadly accidents. Hence, we need transparency in AI to check on such systems and maintain all the safety standards.
• Smart Devices: Devices like smart watches, home device control systems, and many more are extensively connected as well as the Internet. As a result, a flaw in such intelligent devices might lead to wrong interpretations of the environment leading to incorrect actions. Hence, intelligent devices also need transparency in AI to accurately handle the network of devices they are connected to.