Poetic Language Between Humans and AI

Project Started: January 2018
Timeline: 2 Months
Client: Carnegie Mellon University, Environments Design Studio
Skills: Design Research, Speculative Design

 

Poetic Language between Humans and AI is a speculative design exhibit exploring how metaphors, phrases, and poems can be used to communicate with AI, fostering deeper understanding of intangible emotions.

 
 

The Challenge

As AI increasingly integrates into our daily lives, designers must critically examine and provoke discussions on the complex interactions between human and nonhuman intelligences, addressing biases and hidden power structures to thoughtfully shape future environments. In this studio, the challenge was to interpret the topic: Where are the humans in AI?

The Process

I started by conducting exploratory research and brainstorm on social interaction topics regarding humans and artificial intelligence. I categorized insights into four themes:

  • Poetic/Lyricism: Does AI understand metaphors or symbols?

  • Social Status/Hierarchy: Often, AI is called an assistant. Does the AI world have social status or hierarchy?

  • Manners/Traditions: As younger generations spend more time with machines and AI, they might lose contact with human intelligence, culture, and roots. Additionally, they may overlook the importance of daily manners and etiquette.

  • Emotional Quotient/Intelligent Quotient/Adversity Quotient: While humans do not value EQ and IQ equally, how does AI perceive the importance of each?

1*0XEDvpU9m6kz-3XZZuKFgw.jpeg

Discussion

Meanwhile, I joined the Humans and AI discussion on the Carnegie Mellon University campus, focusing on ethics, machine learning, data, and interactions. Below are the speakers' insights that influenced my design:

  • Humans have qualities like multi-dimensional trust, emotional depth, and the ability to attribute mental states—beliefs, intents, desires, feelings, knowledge, etc.—to oneself and others, and to understand that others' mental states are different from one's own (Theory of Mind).

  • Technology should not replace us when it cannot be explained.

  • There is a difference between behavioral data on human actions and mind (e.g., using facial recognition to define mood) versus actual data.

Data and Society - Madeleine Clare Elish

Data and Society - Madeleine Clare Elish

Former Design Manager at Twitter, current Independent Design Consultant - Cennydd

Former Design Manager at Twitter, current Independent Design Consultant - Cennydd

Carnegie Mellon University Human-Computer Interaction in the Machine Learning Department - Min Kyung Lee

Carnegie Mellon University Human-Computer Interaction in the Machine Learning Department - Min Kyung Lee

“Trina, Designing Text Technologies for Critical Interpretation” - Anne Burdick

“Trina, Designing Text Technologies for Critical Interpretation” - Anne Burdick

Carnegie Mellon Universtiy — K & L Gates Conference on Ethics and AI

Speaker MiMi Onuoha — an artist and researcher using data and code to explore new forms of storytelling, social critique, and interaction

1*7EnlmRYBGlxk3Ix8nrIDjQ.png

my design principles

1*fnUkX3pFW4evZ98wFstaPA.jpg

Venn diagram of Humans and AI

1*ID9FXVCf2rtj2S2ar8KomA.png

Pros and Cons of humans and AI

 

Scoping The Problem

I decided to dive into AI topics related to metaphors/lyricism and Emotional Intelligence.

Communicating intangible emotions is challenging between people—even more so with AI. What if humans’ could communicate with AI in humans terms, using metaphors, poems, and phrases to understand one and another contextually?

 

Case Studies on Language and Expression

I researched and explored different expressive terms to communicate with AI. I examined existing expressions and probes, such as:

  • Rorschach Test: Psychologists use this test to examine a person’s personality characteristics and emotional functioning. “By asking the person to tell you what they see in the inkblot, they are telling you about themselves, and how they project meaning onto the real world,” says psychologist Mike Drayton for BBC.

  • Arrival: This sci-fi drama directed by Denis Villeneuve explores language, communication, intention, and time. In the movie, humans create an entire alphabet system by studying the communication patterns of aliens.

  • Human Languages: Human languages create associations for understanding context through the composition of compound characters.

Pages of poetry and art from a German doctor, Justinus Kerner’s Klecksographien (1890), inspired by accidental inkblot.

Rorschach test

A scene from the film, Arrival (2016)

A scene from the film, Arrival (2016)

1*PtLqSwgFLAdE98GtllKuHg.png

Aliens alphabet system and communicate patterns. - the film, Arrival (2016)

0.png

Sited from Teaching Culture through Language: Exploring Metaphor and Metonymy in Chinese Characters by Hu, Ying-Hsueh

Screen Shot 2021-02-13 at 7.43.33 PM.png

Chinese characters: symbols combined to create new character and new meaning

Screen Shot 2021-02-13 at 7.43.11 PM.png
3.png

German Compound words

 

THE RESULT

I created two speculative design experiences that invite visitors to explore how human language is fundamentally built with metaphors that tell stories and provide context, as well as how we express and communicate intangible feelings.I encourage visitors to think about new ways for AI to communicate more humanely?

Poetic Language_Poem0-1.png

Introduction of my research and speculative designs

Poetic Language_Poem0.png

First Prompt

Poetic Language_Poem6.jpg

At the exhibition “Where are the Humans in AI?” 2018

Poetic Language_Inkblot1.png
Poetic Language_Inkblot2.png
Poetic Language_Inkblot7.png
Poetic Language_Inkblot4.png
Poetic Language_Inkblot6.png
Poetic Language_Inkblot5.png
Poetic+Language_Inkblot3.jpg

Inkblots I made

Poetic Language_Poem1.jpg

Google Image Reader’s interpretations of the inkblots

Poetic Language_Poem2.jpg
Poetic Language_Poem3.jpg

Visitors expression of the inkblots

Poetic Language_Form0.png

Examples of how human languages tell stories through compound words

Poetic Language_Form0-1.png

Second Prompt

Poetic-Language_Form28.jpg
Poetic-Language_Form21.jpg
Poetic-Language_Form22.jpg
Poetic-Language_Form23.jpg