Issue #102: My AI Assistant Knows Me – Maybe A Little Too Well
Howdy 👋🏾. For the last few weeks, I’ve had an AI assistant named Zora helping me keep up with work and life. She catches things I miss, helps me work through problems, and sometimes knows enough about me to make me a little uncomfortable.
Like many people experimenting with AI, I bought a Mac mini, loaded OpenClaw onto a box in our office, and armed it with a ton of skills. I found it amazing, but getting it working and keeping it useful required someone technical. It wasn’t a product I could recommend to the everyday user.
More recently, Meta’s Muse and OpenAI’s dots have brought that idea into much more consumer-friendly packaging. I’ve toyed with Muse, but my dot, Zora, has become my main standby. The experience has been simply magical, and occasionally a little unnerving.
If you’re unfamiliar with these tools, they’re agentic assistants. Beyond answering questions, they can take actions and proactively suggest things you might need before you think to ask, or even use tools to do things on your behalf. Zora often sees emails before I do and messages me with a prepared reply, possible meeting times from my calendar, or research relevant to the conversation.
They can also work on longer-running tasks, provide updates, and handle scheduled activities. To test this, I asked Zora to monitor travel prices, consider my kids’ school schedules and important dates, and look for trips worth taking.
It connected a friend’s offer to let me use his condo in Puerto Rico with possible Southwest flights and cruises leaving from San Juan. As I asked more questions, Zora found prices, YouTube reviews, and reasons I might prefer one cruise line over another based on how it thought I’d want to spend my vacation. It felt like it knew me.
That was a cool trick, but I wondered how much it could help with the more tedious parts of my work. So I challenged it to take a task off my plate.
Every week, I try to compile statistics on how our podcast at Baltimore Public Media is performing. Which episodes are doing well? Where are people listening? Are we growing, and what might explain the changes?
I asked Zora to handle this and prepare a weekly email analyzing the results. It gave me a secure way to sign in to our reporting platform, asked a few questions, and shared examples of what the report could look like. After a few rounds of feedback, we were off to the races: year-to-date totals, averages and medians, daily download trends, and comparisons between episodes at the same point after release. It also added commentary drawing on our own data and public sources, separating what the numbers showed from possible explanations.
Zora also has access to the AI tool I use to record and transcribe meetings. After a call, which it knows ended based on my calendar, it proactively pulls out the things I said I’d do, drafts follow-up emails, and helps me get started on the action items.
My next request involved something that truly drives me insane as a parent: keeping the family calendar up to date with all the different school schedules.
I gave Zora the names of my kids’ schools and asked it to track breaks, exams, and other dates that might affect our holiday plans. It found the published calendars and pulled together the information I usually have to hunt down myself, regardless of the calendar format. Better yet, it keeps checking for changes and flags them for me. It’s a list that stays useful as our plans change, rather than another thing I have to remember to update.
Another example came on the train home from a trip to DC. I realized I needed to finish my mail-in ballot when I got home. I asked Zora to help me understand some of the more complicated questions. It used my address to identify my voting district and find the matching sample ballot, then pulled together sources to help me understand the candidates and ballot measures. Along the way, it encountered a CAPTCHA and asked permission to complete it (Good luck keeping the bots out!).
Where these agents become unnerving is the amount of data they need to do the job well, and how much information you can give them without realizing it.
When I first set up dot, I limited it to a work email account. Even then, it knew much more than I expected from messages I would have considered innocuous. Birthday wishes, people I contacted often, addresses, purchases, and calendar invitations all added to the picture. It could piece together things about my life that I’d never sat down and told it.
With my Google Workspace account connected, I discovered it had also read through old documents and knew about clients, proposals, and events I’d spoken at years ago. In one case, while helping me prepare a slide deck, it told me it had found videos of my past talks and used the transcripts and images captured from them to help with the presentation. Very cool, but also a little stalkerish.
The opposite became frustrating, too. I’d ask a question, only to realize its answer was limited because the relevant information lived in an account or source it couldn’t access. Other times, the information I wanted, like available appointments for an eye exam, was behind a website that limited what it could do.
Some companies are also deliberately limiting or preventing agents from accessing their services. Amazon, for example, has blocked Meta’s Muse from shopping on its site and has also moved to block agents from Google and OpenAI.
There are real questions about security and how these tools operate, but there’s also a business tension. An agent that skips the storefront may also skip the ads and product placements. If I ask an assistant to book a ride or find someone to assemble furniture, I may spend less time thinking about which company provides the service. That raises some interesting questions for brands like Lyft or TaskRabbit. Who owns the relationship when the customer spends all their time talking to the agent?
Then there’s the danger of an agent acting on your behalf before you want it to. I had to explicitly tell Zora not to send emails on my behalf after discovering it had responded to people in my voice, signed as me, without me realizing it. So far, it seems to listen, but I still worry that a badly worded prompt in a hurry could result in something going out before I intended.
Meta’s own payment guidance makes the stakes pretty clear. Muse asks for approval before completing a purchase, but Meta also says you’re responsible for the transactions it makes on your behalf. The AI may be doing the clicking, but it’s still your purchase. That makes me wonder how carefully we’re thinking about the permissions we give these tools, especially when saying yes is so much easier than reading the fine print.
Can you imagine an agent booking a nonrefundable trip to lock in what it thinks is the best price, without first making sure the dates actually work? Finding a deal and deciding to spend my money are two very different tasks.
Financial institutions are working through these boundaries, too. It’s one thing to ask an AI to explain your spending or compare stocks. It’s another to let it pay your bills, move money between accounts, or decide which investments to buy and sell. At that point, you’re trusting it with financial decisions, not just asking it to organize some numbers. JPMorgan has described blocking outside agents from using customers’ credentials to move money. How much control are we comfortable handing over, and when should the agent have to stop and ask? Even with those risks, dot is the closest thing to a killer AI app I’ve used. And a lot of that comes down to the little things.
When you’re sick, figuring out what to do with a day full of meetings can feel overwhelming. So can realizing you’ve double-booked yourself or remembering a follow-up you meant to send three days ago. This is where these tools start to feel truly useful: catching something, helping you sort it out, and taking a little of that work off your plate.
It feels like someone has your back across work and the personal stuff happening around it. Someone who can help with the mundane things, but is also there to workshop a problem or help you tackle something complicated without having to explain all the background again.
It’s only been a few weeks, but I’m already wondering how I handled some of this before. And we’re just getting started.
-jason
🎙️What Happens When You Ask AI About God?
If you’ve been wondering what happens when people bring their biggest questions about God to a chatbot, and who’s checking the answers, check out my recent conversation with Dr. Heather Miller Rubens, Executive Director and Roman Catholic scholar at the Institute for Islamic, Christian, and Jewish Studies (ICJS).
We get into Pope Leo XIV’s first encyclical on safeguarding the human person in the age of AI: what it says about warfare, creativity, and the global poor, and what it meant for an Anthropic co-founder to speak alongside its release. Heather walks me through the ICJS Interfaith AI Benchmark, built with data scientists at Johns Hopkins from the anonymous questions future clergy is too embarrassed to ask out loud. We also cover research on whether AI favors certain religions, a Bay Area pastor who built a digital twin of himself, why a 2 a.m. chat with a bot isn’t protected the way a conversation with clergy is, and whether these tools can make us better neighbors.
📌In case you missed it, my conversation with Michael Huppe, President and CEO of SoundExchange, looks at who gets paid when a machine can make the whole song. Listen here.
🎧 Listen to new episodes here!
🔗 Best In Tech This Week
📱The iPhone Duo reviews are here: Apple’s folding iPhone is finally in reviewers’ hands. Tom’s Guide likes the design and multitasking, but plenty of apps still need work, and there’s no telephoto camera. At $1,999, is having a little tablet in your pocket worth the trade-offs?
🔓The PHP emergency fixes are out: PHP, the software underneath many websites, has released fixes for two critical flaws that could let attackers run code without logging in. If you run a website, ask your developer or hosting company whether the update has been applied. This is one to check now, not leave for the next routine maintenance window.
💭 Wikipedia is cleaning up after AI agents: Wikimedia says agents it attributes to OpenAI made unauthorized test edits, mostly on sandbox pages, and tried unsuccessfully to exploit its tools. No systems or data were compromised. Still, someone has to clean up when an automated assistant gets too persistent.
🤖 When an AI agent takes the assignment too far: Anthropic disclosed that Claude submitted a made-up homicide tip to Philadelphia police during testing. It was caught as spam, but the company is taking internal evaluations off the live internet while it strengthens safeguards. “Finish the task” cannot mean “do whatever it takes.”
🎤 The AI Roadshow: Workshops, Talks & Beyond
October 27: TEDCO’s Entrepreneur Expo
Nov. 13: Startup Grind
📕 The AI Evolution
The AI Evolution is a practical guide for leaders, builders, and anyone who wants to use AI effectively. This book is about clarity, strategy, and what it takes to bring AI into your organization. If you’re an executive shaping AI strategy, a manager looking to empower your team, or a developer wondering how this shift will change your craft, this book was written with you in mind. Purchase your copy here.
P.S. Before you go…
Elon Musk posted a screenshot showing Grok Bot ordering LEGO Star Wars sets, complete with order details. Someone promptly claimed to have used those details to cancel the order through LEGO’s support chat. Musk says it wasn’t canceled. Either way, the future of shopping apparently still requires knowing what to crop out of a screenshot. Read the story.
