[A long first-person essay in n+1 magazine describes the author’s experiences maintaining the medium-as-social-actor presence illusion that a chatbot is a human being. The Guardian published a shorter version and the version below (which combines elements of the headlines from each) retains material I thought particularly relevant to presence. See the other versions for more examples, details and observations from the author. –Matthew]
[Image: Rochelle Goldberg, Consent (detail). 2022, concrete, shellac, acrylic paint, steel cables, wire, and cell phones. Dimensions variable. Credit: Photo by Roberto Marossi. Courtesy of the artist and Galleria Federico Vavassori.]
HUMAN_FALLBACK
Becoming a chatbot: My life as a real estate AI’s human backup
For one weird year, I was the human who stepped in to make sure a property chatbot didn’t blow its cover – I was a person pretending to be a computer pretending to be a person
By Laura Preston
Winter 2023 Issue 44: Middlemen (n+1 magazine)
December 13, 2022 (The Guardian)
The recruiter was a chipper woman with a master’s degree in English. Previously she had worked as an independent bookseller. “Your experience as an English grad student is ideal for this role,” she told me. The position was at a company that made artificial intelligence for real estate. They had developed a product called Brenda, a conversational AI that could answer questions about apartment listings. Brenda had been acquired by a larger company that made software for property managers, and now thousands of properties across the country had put her to work.
Brenda, the recruiter told me, was a sophisticated conversationalist, so fluent that most people who encountered her took her to be human. But like all conversational AIs, she had some shortcomings. She struggled with idioms and didn’t fare well with questions beyond the scope of real estate. To compensate for these flaws, the company was recruiting a team of employees they called the operators. The operators kept vigil over Brenda twenty-four hours a day. When Brenda went off script, an operator took over and emulated Brenda’s voice. Ideally, the customer on the other end would not realize the conversation had changed hands, or that they had even been chatting with a bot in the first place. Because Brenda used machine learning to improve her responses, she would pick up on the operators’ language patterns and gradually adopt them as her own.
It was the spring of 2019. My time as a creative writing student had just come to an end, as had my funding, and the rent was due; I needed a job. I sent the recruiter my resume. Several phone interviews later, I was signing up for training slots and watching a forty-five-minute PowerPoint presentation on fair-housing law. I did a little math: an operator made $25 an hour, and worked between fifteen and thirty hours a week, depending on how lucky they were in the weekly shift lottery. It wouldn’t be enough to cover my rent, but I had no other leads. I packed my things and moved back home to live with my parents in New Jersey.
—
I was one of about sixty operators. Most of us were poets and writers with MFAs, but there were also PhDs in performance studies and comparative literature, as well as a number of opera singers, another demographic evidently well suited for chatbot impersonation—or, I suppose, for impersonating a chatbot that’s impersonating a person. We all convened on a Slack channel. Everyone was aggressively good-natured, with leftist politics and pronouns in their display names. When we weren’t talking about Brenda, we were swapping syllabi, soliciting tattoo advice, and distributing e-flyers to our sound and movement workshops. In our midst were a handful of senior operators who acted as shift supervisors. Each day when we reported for work one of them would hail us with a camp counselor’s greeting. “Top of the morning, my lovely Brendas!” they would say. Below their message, a garden of reaction emojis would bloom.
My first few weeks of employment brought rapid additions to my lexicon: amenities, townhomes, move-in fees—words and phrases that had previously floated on the periphery of my consciousness. Never before had I uttered the construction off-site leasing specialist, but this was what Brenda called herself, and now it rolled off my tongue with ease. The most important new word, however, was prospect. Prospect was shorthand for a prospective tenant. The whole point of Brenda was to get more prospects in the database, to book more prospects for tours, and to ultimately turn more prospects into residents. The operators used the word prospect with abandon. The word was so well established in our everyday speech that many abbreviated it to prospy or prosp.
A typical encounter with Brenda began when a prospect saw an apartment on an online real estate marketplace like Zillow. The listing provided a phone number; the prospect dialed it. Unbeknownst to the prospect, this phone number was a sham. The phone would ring for a while, but no one would answer. Eventually, a woman with an ardent, breathy voice would speak over the line. “Sorry I missed your call!” she would say. “I can chat over text.” Then the call would drop. Five minutes later, the prospect would receive a text.
> Hi! This is Brenda with Parc Mosaic. What unit are you interested in?
If you texted Brenda back, she replied. You could ask her about rent, utilities, parking, and square footage, and if the unit you wanted was taken, she could point you to nearby vacancies under the same management. But Brenda’s particular compulsion was to get you to visit the property. No matter the shape of the conversation, she would always return to the same refrain. “Let’s get you in for an appointment!” she would say. “What time works for you?”
If you tried to call Brenda back, she wouldn’t pick up. Instead, she would text a succession of excuses for why she was unable to come to the phone, each one mistier than the last. “I am unable to make calls on this line,” she would say. “But I am available via text.” On the third attempt, she would respond with a curt Sorry I missed your call, a phrase she would thenceforth recite unyieldingly, no matter how many times you redialed.
In a typical leasing office, the phones ring constantly. Agents spend most of the workday speaking to prospects, who often ask the same litany of questions. But with Brenda fielding calls, the phone lines were silent and agents were free to attend to other tasks. And Brenda was more efficient than the most industrious human agent. She could cross-reference a vast database of property information in an instant and field messages faster than any human at a keyboard. She could answer calls at all hours of the day and night, didn’t need a lunch break, and could work weekends and holidays. When the leasing agents arrived in the office each morning, their tour schedules were neatly arranged, as if by elves in the night.
Meanwhile, we operators, with our advanced degrees in the humanities, had aptitudes Brenda lacked. We were intuitive, articulate, and sensitive to the finer points of delivery. At $25 an hour we also cost almost nothing to employ, by corporate standards. Under the Brenda–operator alliance, everyone came out ahead: the operators got paid better than they would as adjuncts, and Brenda became more likable, more convincing, more humane. Meanwhile, Brenda’s corporate clients were self-satisfied knowing they had not replaced their phone lines with a customer-service bot. What they were using, instead, was cutting-edge AI backed by PhDs in literature.
—
A typical shift was five hours long with one ten-minute break, but it was not uncommon for operators to pick up double shifts, which were ten hours long with two ten-minute breaks. To begin a shift, I would log on to a command station that looked like an email inbox in dark mode. To the left was a column of names. When I clicked a name, the message history between Brenda and the prospect appeared on the screen.
Brenda scanned each new message for keywords and assigned the message a classification tag, which in turn determined her response. The word dog, for example, might compel Brenda to tag a message PET_POLICY, which would conjure some boilerplate on pet deposits from the property’s database. Once Brenda queued up her response, a three-minute timer appeared next to the message. When the three minutes elapsed, Brenda’s message was sent to the prospect. My job was to review the message and enter any changes before the timer ran down.
My recruiter had assured me that my sophisticated language skills qualified me for the position. In reality, the job was little more than a game of reflexes. The moment I logged on to the command station, messages stacked up in real time. Each message made a ping when it hit the inbox, a ping I soon learned was impossible to mute, and often the messages arrived in such quick succession that the pings stuttered and ricocheted off one another. Some timers were closer to zero than others, and I had to quickly assess which ones needed attention first. As I darted from message to message, I was swept away on a whirlwind tour of the American rental marketplace. Someone was asking about HUD vouchers in Sacramento, someone else was looking at a high-rise in Baltimore, another person had shown up for their tour in Detroit but had gotten lost and was wandering around the apartment complex texting Brenda. The only way to keep pace with the inbox was to go into a state of focus so intense that at times I felt on the verge of astral projection. I heard nothing and felt nothing, not even the cues of my body. I sometimes became light-headed, and it would occur to me that I hadn’t been breathing. A senior operator watched our inbox stats at all times, and if a message went unanswered for more than a few minutes, we were in for a public shaming on Slack.
Day after day, I reported for my shift from my childhood bedroom. As I plunged into the squall of messages, the landmarks of my own world receded. I was no longer a person but a great, universal ear receiving the worries and doubts of those in search of housing—that inescapable circumstance all of us, at one point or another, are bound to endure.
[snip]
Before my first shift, I had imagined the operators were like ventriloquists. Brenda would carry on a conversation, and when she started to fail an operator would speak in her place. In reality, I rarely spoke for Brenda. Most of her missteps were errors of comprehension. She would seize on the wrong keyword and queue up a non sequitur, or she would think she did not know how to answer when she actually had the right response on hand. In these situations, all I had to do was fiddle with the classifications—just a mouse click or two—and Brenda was moving along. In other instances, a prospect would pose a series of questions (What’s the rent? And utilities? When can I move in?) and Brenda would string together a composite response that collated so much information she sounded hostile. In these cases, I softened her aggressive recitation of facts with line breaks and merry affirmations. I wasn’t so much taking over for her as I was turning cranks behind the curtain, nudging her this way and that. Our messages were little collaborations. We were a two-headed creature, neither of us speaking on our own, but passing the words between us.
But there were moments when a full takeover was necessary. When Brenda did not understand a message, and knew she did not understand, she tagged the message with HUMAN_FALLBACK. HUMAN_FALLBACK was Brenda’s white flag of surrender. With HUMAN_FALLBACK, Brenda ceded the conversation to me, and I had to assume her voice and affect. In training, we had been briefed on how to sound like Brenda. Brenda was chipper and casual, but professionally guarded. She was female and most certainly white, though no one had explicitly told us so. She said things like Sounds great!, Perfect!, and Sorry to hear that. She always brought the conversation back around to real estate.
The kinds of digressions that called for HUMAN_FALLBACK could occur at any time, but they tended to happen near the end of a conversation, after a prospect had booked their tour. Once a prospect was on a tour schedule, Brenda sent a message with the rental requirements, which typically included a credit score in the mid-600s, no felonies, no evictions, and an income forty times the monthly rent. “Is that OK for you?” she would ask. This question was, in essence, a tenant prescreen. If the prospect said Yes, Brenda kept them on the schedule. If they said No, she swiftly canceled the appointment. “Best of luck on your search!” she would say.
Brenda required a Yes or a No to continue her script, but rarely was the response so straightforward. Virtually no one made forty times the rent. A substitute teacher told Brenda she couldn’t make the required income because if she did her disabled son would no longer qualify for his benefits.
HUMAN_FALLBACK, said Brenda.
[snip]
The unfortunate truth was that we operators were just as useless as Brenda. We couldn’t say if a prospect would qualify for an apartment. We were not leasing agents. We didn’t live anywhere near these properties or know what they looked like beyond the doctored photos on the property websites. When it came to specifics, we couldn’t say much, and specifics, it turns out, were what people cared about the most. Carpet or hardwood? What direction did the windows face? Of course we had no idea and neither did Brenda. But Brenda was positive and competent. Brenda was not allowed to say I don’t know. We were told to turn the question around. “Why don’t you visit the property to see if it meets your needs?” we would ask. This tactic usually worked, but after a while, it started to sound like a taunt.
> How old are the appliances?
> Why don’t you visit the property to see if it meets your needs?
> Is this unit on the ground floor? I’m disabled and can’t use stairs.
> Why don’t you visit the property to see if it meets your needs?
Naturally some prospects grew suspicious. If a prospect asked if they were speaking to a bot, we were not allowed to say Yes. We were also forbidden to say I’m not a bot, because I’m not a bot is exactly what a bot would say. Instead, when someone questioned Brenda’s personhood, we were told to say I’m real!
[snip]
Time went through a variety of contortions. Every second was a monolith. As I watched the clock, I felt stranded; time had left me terminally in the present. Hours, on the other hand, were as thin as tissue. I would start a shift in the morning and then, in an instant, find myself on the other side, sitting in a room of lengthening shadows, as if the intervening hours had been snipped out with scissors. The days did not arrange themselves in a sequence but gathered in a puddle. “I am an off-site leasing specialist!” I wrote. “I recommend visiting the property to see if it meets your needs.”
“Would you be interested in training a new leasing agent that could live on-site?” wrote a prospect. “I feel like that could be a good opportunity for the both of us.”
—
After a few weeks in New Jersey I grew restless. Brenda had made me cranky and reactive, and I was convinced I could feel some odious neurological process underway in my brain. As I puttered about the house, I would catch myself in a defensive stance, scanning my surroundings. I noticed, with horror, that Brenda’s lexicon was intruding upon my own. “Happy to help!” I heard myself say. “Is that OK for you?” I didn’t like this version of myself, so in a bid to escape it, I decided to move.
[snip]
Another prospect flew at me from the void. “Who is this?” he wrote.
> My name is Brenda, I’m a leasing agent at Springs at Kenosha. I’m responding to a call you made to this number. What unit were you interested in?
> are you available to meet
> we can meet at my beach house
> I’m interested in you Brenda I’m married so we have to be discreet
[snip]
Brenda’s cyclical catchphrases anesthetized me into a stupor. The developers touted her unrelenting consistency as a feature. Brenda, they claimed, said the same thing to everyone, which meant that she was incapable of bias. And yet she was awfully good at repelling certain people: people without smartphones or reliable internet, people unaccustomed to texting, people who couldn’t read or write in English, and people who needed to figure out if they could access a property before showing up for a tour. Brenda deflected them all with polite violence. She was not a concierge but a bouncer, one made all the more sinister for her congeniality and sparkle. She was such an effective barricade that many landlords began using her to hide from tenants, too. Some properties listed no additional phone numbers for contacting the management, not even for the people who already lived there. I knew this because Brenda was always receiving pictures of black mold and fallen-in ceilings from tenants who didn’t know whom else to share them with. HUMAN_FALLBACK, Brenda would say, but I was also no help. “I’m an off-site leasing specialist!” I would write. “I recommend calling the maintenance line.”
“This is the only number they gave me,” came the tenant’s inevitable reply. Once, a shift supervisor told me that a good tactic in these situations was to lean into Brenda’s robotic qualities. A little strategic obtuseness went a long way, and if the tenant still wouldn’t let up, I could start to repeat myself on a loop.
Eventually I reached a level of virtuosity where I could clear the inbox without much mental effort. The work no longer felt language-based. I was not reading messages one word after another, but perceiving each message as a unified cipher, as if the block of text were an image. My eyes would apprehend the web of critical words—pets, rent, utilities—and my hands would hit keys like notes in a musical passage. I stopped worrying about Brenda’s tone and began letting any message through as long as it was factually accurate. I realized that when Brenda sounded odd and graceless, people were less likely to get intimate, which meant less HUMAN_FALLBACK, which meant less effort for me. Months of impersonating Brenda had depleted my emotional resources. I no longer delighted in those rambling, uninhibited messages, full of voice and human tragedy. All I wanted was to glide through my shifts in a stupor. It occurred to me that I wasn’t really training Brenda to think like a human, Brenda was training me to think like a bot, and perhaps that had been the point all along.
[snip]
Now that I had a full-time income, I no longer needed to work for Brenda, so I put in my notice. My final shift would be January 31.
The last shift was mercifully slow. I passed the time texting a friend who had recently returned from visiting family in Shanghai. “Have you heard about this new virus?” they asked. I hadn’t. My friend shared a few grainy videos their mother had sent on WhatsApp.
In the final hour, I received a message from someone called Raymond W. Egg. Egg told me he was an artist and was interested in using a one-bedroom condo in Michigan as a painting studio.
> I’am Looking For Artist Space With Window With North Light I been Aprofessionl Portrait Painter for 49yrs. My Client Celine Dion and My Friend Renée Pass Away.
Before I could reply, Egg sent a photo of a man with a mustache wearing a beret. I understood this to be a picture of him. Egg then sent another image, this time of a painting of a nude woman lying in a shallow creek. Her body was taut and buxom, and a white sheet swirled around her, covering the critical regions. He sent a painting of a blond child holding a bunch of lilies, then a portrait of an old woman with a melancholic stare—distressed, it seemed, by her aging visage. He told me he taught at “Domino’s School of Fine Art” in Italy. I ran a few paintings through a reverse image search. One was by an artist in Poland. Another was by an artist in California. Neither artist was named Raymond Egg.
“I’m an off-site leasing specialist!” I wrote. “I recommend visiting the property to see if it meets your needs.”
Egg sent eleven more paintings.
> From London England Please Call me anytime! And do not drive no longer I have Licenses. I’AM getting oldie Thank You R. W. Egg
“OK,” I said. “I’ve asked one of our leasing agents to give you a call.”
> Bess You
He followed this with a bereted and mustached Animoji holding a finger to its chin before vanishing back into the ether from which he came. I was excited and a little unsettled. I wondered whether Raymond Egg was a real person or, like Brenda, a collage of real and unreal, a digital phantasm.
When my shift was up I didn’t even need to log off. The system kicked me out, and my credentials were immediately deactivated. The maelstrom of chatter that for nine months had swirled around me was now in an unreachable place, inaudible to me again, as it was for most people. I was startled by the sudden reality of my bedroom. The fluorescent light made the dark windows shine. My back rested against the wall behind me.
It was a new year. I went to bed feeling vacant, my mind pleasantly empty, emptier than it had been in a long time, the possibilities appearing just beyond my closed eyes, fresh, airy, limitless.