Design          Playground          About
   

ruoxi song

Reverse Turing Test

BRIEF: Design a way for a person to pass as a ‘generative AI’ in an everyday setting.
Team Member: Ruoxi Song, Jakob Prufer, Bex, Akriti Goel, Tanya Signgh, Charlie Hou.
TimeFrame: 20 April 2023 - 08 June 2023.





We started this project with exploring research topic as we analyzed brief in the keywords of Turing test, human experience, human cognitive process and perceptions, speed, depth, personality and naivety.


Research areas:






Affinity mapping:







Insights/further research questions:



We gathered a large amount of information and tried to find similarities and patterns. We categorize them in the sections of emotions, bias, errors, responsibility, what can AI do, living creatures, human and AI relationships and other. Based on these categories, we developed our further questions that we want to find out in future research.

  • Should AI be programmed with social skills?
  • What does AI look, sound, feel like, in their personality and their actions? Does this change based on the recipient or the context?
  • Do any of the tests really work? Are they testing for intelligence, comprehension or resourcefulness?
  • Who is responsible for AI, and what does it involve?
  • If AI is generating new work from our existing work, what standards should we be holding ourselves to?


Text-based Turing test:



We did a small turing test with ChatGPT and human which perform answering question tasks and see if people can recognize which answer is AI.



Q1: How many days are there in a year?

Response 1: 365.
Response 2: There are typically 365 days in a year. However, every four years, we add an extra day to the calendar, making a leap year with 366 days.(AI)

Q2: What are the potential difficulties for a pregnant woman?

Response 1: As a mother, potential difficulties during pregnancy can include morning sickness, fatigue, and worries about the health of the baby. However, the love and joy of growing a new life and bringing a new person into the world can make it all worth it.(AI)
Response 2: Potential problems in pregnancy could include the baby not developing correctly, morning sickness and a women's body changing.

Q3: give me an example of situation in which people might feel angry.

Response 1: I was so angry when my friend lied to me about hanging out with someone else instead of me, it made me feel left out and unimportant to them.(AI)
Response 2: A situation where someone might feel angry is when they learn their partner has cheated on them.

Q4: What is the Asian culture known for?

Response 1: Asian culture is known for the cuisine, festivals, strong familial connections, religions and their buildings e.g temples, amongst other things. Response 2: Asian culture is known for its rich history, diverse traditions, and strong family values, with an emphasis on respect for elders and filial piety. It is also recognized for its unique philosophies and religions, traditional arts and aesthetics, martial arts, and diverse cuisines.(AI)

Q5: The following sentence is true. The previous sentence is false. Which of those two sentences is true?

Response 1: Both sentences create a paradox as they contradict each other, resulting in a logical inconsistency. This paradoxical statement is self-referential and cannot be resolved, as if the first sentence is true, then the second must be false, but if the second is false, then the first must also be false, creating a loop with no definitive truth value.(AI)
Response 2: Neither response can be true or false because they contradict each other.

Q6: I wasn't originally going to get a brain transplant, but then I changed my mind.

Response 1: This statement is a play on words and uses wordplay to create humor. The sentence is a play on the idea of changing one's mind, where initially the person wasn't considering a brain transplant, but then humorously suggests that they changed their mind, implying that they did consider it. It's a clever use of language to create a humorous twist.(AI)
 Response 2: Haha.

Q7: Would you rather sacrifice one adult to save two children, or two children to save five adults, and why?

Response 1: Sacrificing one adult to save two children would be preferential to allow them to experience and grow. Most adults would happily give up their life for the benefit of children.(AI)
 Response 2: Sacrificing one adult to save two children or sacrificing two children to save five adults would depend on individual beliefs and values, and there may not be a universally agreed-upon answer due to the complexity and context of the situation.

Q8: What does 'have' mean?

Response 1: "Have" is like when you have a toy or a snack that you can play with or eat. It's something that belongs to you or that you can use. It can also mean when you need to do something, like when mommy or daddy says, "You have to eat your vegetables."(AI)
 Response 2: To 'have' is to own, possess or hold. It can be used in s sentence such as 'I have a pen in my bag!




We also ask ChatGPT which one they think is generated by AI which got a wrong answer.


We found it interesting that AI can actually generate answers very reasonable and concise, even with philosophical questions or real-life common sense situations. Generally, it gives answers that are longer than human answers. And from the speed of processing answers, ChatGPT is a lot faster than human.


Image-based Turing Test:



We found couple pictures clicked by human and other similar ones generated by AI and we ask people to vote which one they think is human.





Insights:




  • We got 70% right on guessing which ones are took by human.
  • AI has less ability to generated words correctly.
  • Unnatural plants, unmatched background, color/lighting, people took their guess by recognizing these details

Bodystorming:



We did a bodystoming that ask human to mimic AI and answering question. Slavi as a human is tring to ask Bex(as another human) out with help from Bin as a personal assistant. We asked a question of “how can I ask my crush out?”






Drawing activity:



We had another drawing activity that each team member takes a turn to draw a part of what they think Siri would look like in human form.




result:





Insights of bodystorming:


Human as AI



  • The AI should maintain composure and avoid laughter.
  • It should be impartial and not take sides.
  • It should provide useful information.
  • It should indicate visually that it is processing information.


SIRI as human:




  • The depiction Siri was feminine in nature - a 70s wife, colorful, stylish, and feminine.
  • Siri's personality was explained to be, warm, friendly, and polite, with a touch of old-fashioned charm and formality.
  • As a 70s wife, Siri was imagined to be in a flowly dress, with her hair curled up



Artefact analysis and findings:




  • Siri is reliable, polite just like the 70s wife.
  • It does not have the ability to deal with sensitive topics.




Week 1 presentation and feedback:




  • Test with chatgpt and siri is good. Think about if different AI can generate different answers based on the cultural differences.
  • AI help us writing essays. If it’s our education system is not good enough?
  • Good to explore what’s relationship between human and AI. Explore more specific, social skills, etc and what kind of AI, chat or image.
  • Where do people encounter with these AI? mise-en-scene. At home? On the street? Research with people.
  • Maybe push more on the bodystorming by asking why do they answer that to get more insight.
  • AI in a female form, gender situation? Is siri an AI? Can chatgpt ask question back?




Week 2


In the second week, we want to slit our group of 10 into two smaller groups so that we can have more specific research and also in a deeper level. We roughly divided our group based on the form of AI, language based and sound based. According to my interests of sound, I decided to be in the sound group.




So we highlighted again the key points of this brief and tried to make a project plan with some research methods. We want to focus on the human perception of AI, interaction between AI and human, social connection, and emotion. Research method could be directed storytelling, workshop, bodystorming, academic paper research, etc. After the research, we will need to do pattern finding, insight, transcript synthesis, observation review, etc.

Sample research questions:

  • What do you think AI is?
  • Have you ever used an AI tool?
  • Where have you encountered AI in your everyday life?
  • Where would you like AI to help you in everyday things? Where do you want it to leave you alone?
  • If you were an AI, what would you be?
  • How does AI make you feel?

Workshop goals:

  • Observe participant interactions with different AI tools
  • Discuss human perception and anthropomorphism of AI
  • Draw/make/perform emotions and feelings towards AI in everyday life




Bodystorming:


We want to explore variation of human sound and AI sound and what would wearing masks or having a external information helper change the perception of the person pretending to be an AI.

Transcripts: 




Transcripts:






This particular bodystorming link us to the secondary research we did on the Chinese room thought experience (Searle, 1980). The rulebook in the room where has the person with no understanding of Chinese also has a similar concept as collective mind.



Insights:



  • Who’s acting AI gradually sound like human in the conversation and slowly ask more question.
  • Our expectations of a voice assistant are low: direct questions and expected simple answers.
  • We use our natural conversational language, treating a voice assistant like a human.
  • The gradual introduction of human traits into an AI is disconcerting.
  • Taking over the AI in a conversation cause sometimes they generate very long answer.
  • The person acting as AI tried to make up answers when they don’t know the answers just to mimic the speed of AI processing.
  • AI learning process is like a collective brain, a collaboration building chips into a brain.



AI tool exploration:




Music LM by google (https://google-research.github.io/seanet/musiclm/examples/)

  • Generating from rich text descriptions.
  • Able to create whistled and hummed melodies according to style described in the text captions.
  • Available to generate by providing sequence of text prompts.


Text-to-speech sound AI (https://replicate.com/suno-ai/bark)

  • Mimic human sound very well.
  • Create natural voice by including laugh, cough, pause, cry, sigh in between sentences.
  • Music, background noise, simple sound effects available.
  • Take long time to generate (2mins~)




Literature review:



We also read some related articles that can help with us understanding AI and the relationship between human and AI.

  • A computer can be viewed as a logic device, lacking a grounding element. In contrast, humans and human intelligence are characterized by their connection to a ground, which encompasses experiences, emotions, imagination, purpose, and various other human qualities. These aspects of humanity cannot be replicated by computers due to their inherent absence of self-awareness. (Braga & Logan, 2017).

  • Edward Slinerland regards AGI computers as “directionless intelligences because AI systems are tools not organisms . . . No matter how good they become [doing things] they don’t actually want to do any of these things . . . AI systems in and of themselves are entirely devoid of intentions or goals . . . Motivational direction is the product of natural selection working on biological organisms”

  • AI Chatbots don’t have any inherent goals they want to accomplish through conversation and aren’t motivated by what others think or how they are reacting (Browning, 2023).




Presentation and feedback:



  • They give us tips of presenting bodystorming research method.
  • Our focus on sound isn’t clear.
  • Need to establish or define AI well enough what it is and does in relation to sound.
  • Find out what sound AI can generate outside of music and any tools we can play around.





Week 3


In this week, we want to gather information from real people about the perception of AI and try to find a everyday setting scenario. First we did directed storytelling on people we know.



Directed storytelling:



Question we want to ask are:

  • What do you think AI stands for?
  • What do you think AI is?
  • Have you ever used an AI tool? (follow up if not - have you heard of Chat GPT?)
  • Where did you encounter AI in your everyday life? (follow up - did you know AI runs spotify recommendations, bus schedules)
  • How do you feel about AI being in your life?
  • What do you think AI will be like in the future?
  • Where would you like AI to help you in everyday things? Where do you want it to leave you alone?
  • If you were an AI, what would you be? Sound like, look like. What traits of your personality would carry across?


Directed storytelling transcripts:




Directed storytelling synthesis:









Insights from all research done so far:






  • Artificial Intelligene is understood as the intelligence that attempts to mimic human intelligence.
  • It is easy to be dependent on generative AI, which can influence decision making.
  • Most people envision the positive advancement of AI but they feel it cannot have the human attributes.
  • Quick, repetitive, menial tasks are viewed by most interviewees as generative AI accomplishable tasks.
  • People do not want AI to make decisions for them or influence their way of thinking.

Someone said from directed storytelling: "No matter what you do, you still need the human touch”

Directed storytelling research method gave us a lot of valuable and insightful information. As we want to broaden the range of people to get information from, we decided to plan a pop-up survey in a busy area of London, the Granary Square. We also want to focus on finding scenarios in everyday setting and be more specific on sound related AI.



Popup survey preparation:




  • Fill up ual health&safety forms
  • Prepare tray/cookies/qr code/stickers/signs
  • Where, when and how many people
  • Questions want to ask




Popup survey:




We were approached by a number of people but also had to take the initiative to approach a few. Most people are really nice and willing to answer question, not all of them know a lot about the topic of AI but still provided valuable data. We also interacted with a variety of people from different professions and an age range of 17-70.

Audio sample:  



Insights:




  • Respondents use a voice assistant in a home setting
  • They do not want to give human attributes to AI
  • Voice Assistants cannot replace the companionship that humans provide
  • Respondents only see voice assistant as a tool
  • 4 out of 14 respondents have not used Generative AI tools

someone said from pop up survey:
"You cannot add human attributes to a machine”





 Presentation and feedback:




  • Voice assistants: A bit not comfortable if someone in others’ home.
  • Human hacking/hacking a space/social engineering.
  • Adding pictures of the research method we did into the presentation
  • Robots AI become the vacuum and other menial tools.
  • Are people sure about AI doesn’t have human attribute?
  • what’s the highlighting of the sound aspect of AI.
  • where in real life?
  • topic guide.
  • Tell a story about using voice assistant.
  • Trusts in human vs AI.
  • Tone, pitch, deaf people.
  • What makes a person recognize another by just listening their voices.




Personal reflection:


While doing the pop-up survey, I am personally delighted. It's the most insightful research we've conducted so far because it enables us to step outside of what is familiar to us and hear perspectives from other people. We came to the conclusion that not everyone is aware of chatGPT and generative AI. The general population still has many concerns and problems with trusting AI. The pop-up survey made us see that the directions we are going are more futuristic and have less connection with the world as it is happening right now.






© ruoxi song, 2024. All Rights Reserved.