Building on our exploration of neural networks, this assignment focuses on the transformative technology of AI chatbots like ChatGPT, Claude, and Gemini, which are powered by sophisticated models such as transformer neural networks, modeled on the brain. These chatbots have garnered significant attention for their ability to emulate human-like language across various tasks, including conversation, essay writing, and coding. It has also been claimed that they can pass the “Turing Test”, in which they can pass for a human in conversation.
In this assignment, you will be designing a version of yourself using two different chatbots of your choice (from ChatGPT, Claude, and Gemini), and then assessing their performance as they chat with people who know you well. You will conduct three trials for each chatbot, where the AI will attempt to pass as you in conversations with your friends or family members.
Successful completion of this assignment is worth 4 points total on your final course grade.
Objective: This extra credit assignment challenges you to apply prompt engineering to two different chatbots, aiming to convincingly simulate your own communication style, knowledge, and personality. You will design and test prompts for each chatbot, then conduct text-based interviews using your phone with at least three people who know you well. Your task is to refine the AI’s responses so that it closely mimics you, thereby attempting to pass as you in a Turing Test-like scenario.
Instructions: Setup: If you do not already have accounts, sign up for the chatbots of your choice (ChatGPT, Claude, and/or Gemini). Once signed up, spend time familiarizing yourself with each chatbot’s capabilities and constraints.
Prompt Engineering: Develop and refine prompts that will guide the chatbots to respond as if they were you, focusing on accurately reflecting your language style, expertise, and mannerisms. Note: The chatbots may often identify themselves as AI during conversations. It is highly recommended that you explain the goal in your prompt and explicitly ask them NOT to identify as AI and to try to pass themselves off as you.
Testing: Test each chatbot’s simulation of you with at least three individuals who know you well, through text-based interviews conducted on your phone. These can be family members, friends, or classmates (but not from this class). Use a premise for the chat, such as that you need to just have a chat with them about some topic for an extra credit assignment but do NOT reveal the fact that it is an AI.
To test your bot, open a chatbot window with your prompt and a separate text thread at the same time. Copy the text from the individual into the chat window and then type out/copy the chatbot’s response back into the text window.
Note: this may be easier on mobile where you can just copy/paste from the chatbot app to the text app, but it may be done across devices. Ensure the subjects are unaware that they are interacting with an AI, maintaining the premise that they are having a regular text conversation with you. If the participant voices any suspicion, simply ask that they complete the conversation and hold all questions until the end.
Evaluation: After each interaction, collect feedback from the subjects, asking them to rate the quality of the conversation and whether they suspect they were conversing with an AI or with you. Observe and record any adjustments needed to improve the AI’s impersonation of you for subsequent trials. Then reveal to them that it was an AI all along and gather feedback from them as to whether they believed it was you, someone else, or an AI.
Report Submission: Compile a report of your findings, including the prompts used, a selection of sample responses, the subjects’ ratings, and any suspicions they voiced. Critically compare the performance of the two chatbots in simulating you, considering factors such as accuracy, coherence, and depth of responses.
Reflection: Conclude with your reflections on the experiment. Discuss the potential and limitations of AI in replicating individual personalities and knowledge. Consider any improvements needed and any areas of cognition where the bots might be lacking. Also consider the ethical implications and potential applications of such technology.
Completion Criteria: To complete the assignment, submit a report using either an attached document or the text field below. You must demonstrate significant effort in completing the tasks as described, thorough documentation, and critical analysis, to receive the extra credit. This is an all-or-none assignment; partial credit will not be awarded.
