Working towards achieving trustworthy AI by solving ethical challenges in AI applications

As a kid, I was fascinated by the stars in the universe. I used to read books related to space, for example, about dark matter and the big bang theory. Besides that, math was also one of my favorite subjects. The ability of numbers to express a variety of patterns that we see in nature, from those that appear on flower petals to the ones on the stars is something I find divine.

I used to love solving math problems from a very young age. In school, we used to have math speed games where the teacher would give us all a set of problems and whoever first solved all the problems correctly got a prize. I remember solving 30 to 40 problems involving binary to decimal conversions in less than 8 minutes. I also got the highest score in the National Talent Search Examination (*1), solving 100 analytical problems in about 45 minutes.

I obtained my master’s degree in Communications and Signal Processing. After I completed the master course, I chose to pursue a PhD degree in order to conduct specialized research in Computer Vision (*2). Back then when making my course selection, I was purely motivated by the potential of Computer Vision itself — the fact that it can address questions pertaining to diverse fields including astronomy, for example, to prove the existence of a black hole by computational photography (*3) methods.

This major also required students to have a strong math foundation, because developing algorithms necessitates quantification of abstract concepts into a format that machines can process. In short, I found computer vision quite fascinating, because it blended well with two of my favorite subjects, namely math and astronomy. But to be honest, understanding the societal impacts of machine learning, which is an extensive area of study for me these days, was not something in the forefront of my mind back then during my university years.

After graduation, I was primarily looking for a research position that would enable me to focus on developing future technologies. Fujitsu’s concept of Human Centric – aspiring for the realization of a digital society centered on people and focused towards creating a better society – resonated with my personal values. I find that working in Fujitsu has not only paved the way for me to work on cutting-edge technologies’ R&D, but also helped me to get a real sense of what customers actually require.

When I joined the company, I was working on machine learning research for healthcare and taking part in a project called ONE. It was an AI-based healthcare application project, concerning developing personalized recommendations designed to help people achieve a specific health goal, related to nutrition, exercise, and sleep etc. For this application, I used machine learning algorithms to understand and to model the preferences of users to provide personalized recommendations.

Another healthcare project involved emotion recognition, intended to aid counselors in their conversations with people seeking help. Here, I was investigating the feasibility of leveraging machine learning tools to analyze textual data between counselors and those seeking help. Specifically, we wanted to investigate if machine learning techniques could aid in distinguishing what type of emotion was being expressed, compared to those that might not be explicitly expressed but rather experienced in a latent manner. These use-cases helped me understand the advantages as well as the limitations of machine learning techniques.

From 2017, I started working in the space of explainable AI and AI ethics. Research in this space focuses on a variety of topics such as fairness, transparency, accountability, and AI safety. Broadly speaking, my research studies the societal impact of AI systems.

Some of what I do today has a direct connection with my PhD research. My PhD thesis focused on developing face recognition algorithms to identify sitters depicted in ancient artworks belonging to the Renaissance period. Results from this research provided an independent and complementary source of evidence to art historians, especially in cases where there was some ambiguity regarding the sitter’s identity. It was research that went on to make a significant contribution to art history.

In one of my current research projects, I study the links between creative applications of machine learning and their ethical impact. As with my PhD research, it is something that can also benefit art history. For example, in a recent study, I systematically investigated various types of biases in AI-generated artworks. Biases in generative artworks can have long-standing socio-cultural impacts – historical events and people may be depicted in a manner contrary to their original context, which in turn, can obscure elements of cultural preservation.

This research aims to mitigate such harmful effects by proposing a framework to identify different types of biases in AI-generated art – ranging from those related to problem formulation and dataset curation to those concerning algorithm design and evaluation. To the best of our knowledge, this is the first work to uncover biases in generative artworks along various phases of the AI pipeline.

I also developed a method to quantify a type of bias in AI-generated artworks (*4) (*5), that demonstrates how generative artworks fail to capture the nuances of important art styles of the past, and thereby hinder our understanding of cultural history. Backed up with experimental evidence, this is also one of the first works to come up with a metric to quantify bias in AI-generated artworks.

In another research work, I developed checklists and guidelines towards the responsible development of art datasets, in order to help AI scientists and developers in mitigating harmful downstream impacts.

One of the main challenges in conducting research in this space was the absence of prior art from a comparative perspective. Usually, research takes off from previous works to understand what the limitations are, and thereby suggest improvements. In the case of generative artworks, there wasn't that much prior work to use as a guide, and it was quite a risky venture as we didn't know whether we were headed in the right direction, nor whether it would result in something meaningful or useful.

It is also important to consult domain experts, people who are specialized in the field, to uncover issues of concern. In developing the checklists for art datasets, we consulted legal experts, art historians, and applied AI ethics experts. Consulting experts from different areas can be extremely helpful in AI auditing.

I’ve also investigated the ethical impacts of AI in a finance-related use case (*6). Our research focus was on how to design explanations of AI-based loan decisions in a user-friendly manner – such as designing user-friendly datasets for loan decisions and the generation of explanations. These typically served a number of different purposes such as educating a user about a loan denial or helping a user to take an appropriate action to secure a future loan. In another finance-related AI use case, I proposed a method to quantify existing biases in the datasets used for making credit decisions.

AI regulation is particularly important for finance-related applications. Since different countries have different local contexts and regulatory principles, it is important for us to consider these differences when designing future AI systems. Another challenge concerns issues related to privacy and data security. There are many such factors that need to be considered when making new advances.

With regards to my research vision, I would like to continue to develop technologies that are ethical, sustainable, and beneficial to society as a whole. At a personal level, I want to work towards becoming a better person, as well as exploring new horizons and learning new things.

In future, I hope that I can work on exciting new areas, studying the potential and impact of AI in challenging fields such as climate modeling. As an example, I would like to research on understanding the impact of climate change on food systems. In the context of the global climate crisis, I think it's important to be mindful of whatever we develop. I will not stop striving for value-based research to contribute to a more sustainable world.