Beyond Personal Agents: Designing for More Than One Principal

As AI agents move into families, workplaces, and caregiving networks, the central design problem changes. The question is no longer only what an agent can do, but who it may do it for.

One Agent, Many People · the two-minute film of this essay

Imagine asking a family assistant to plan a weekend trip.

It can see that your partner is unavailable on Saturday afternoon because they shared their availability, but not the event’s title, location, or reason. The same product also holds a private conversation in which your teenager mentioned a school commitment, but your request does not grant access to that conversation. You have asked the agent to make a plan, but nobody else has agreed to one.

What should the agent do?

It could optimize for the person who made the request. It could ask for broader access than the task requires. It could treat unavailable time as a rejection and quietly choose a different date. It could even book the trip and present the result as a helpful surprise.

Every one of those options might look intelligent. Every one could also violate someone’s privacy or make a commitment they never authorized.

Most AI agents today are designed around a simpler relationship: one person delegates a task, and one agent acts on that person’s behalf. The agent may understand other people, but it still has one principal.

Families, teams, classrooms, and caregiving networks do not work that way. They contain several people with legitimate, unequal, and sometimes conflicting claims over shared information, resources, and decisions.

That difference points toward a distinct product category: social agents.

From socially intelligent agents to multi-principal agents

“Social agent” is already an established term in AI and human–computer interaction. It often refers to an agent that can perceive social cues, model human behavior, or participate convincingly in social interaction. Research on socially intelligent AI, for example, asks how agents can recognize and respond to human affect, behavior, and cognition.

I use the term more narrowly here to describe a particular class of multi-principal agent:

A social agent is an AI agent operating in a shared domain where multiple people hold legitimate—and sometimes conflicting—claims over the information it uses and the actions it takes.

Social intelligence may help such an agent communicate well. It does not solve the underlying governance problem.

A personal agent can know that I have a family, understand my relationships, and speak sensitively about them while still serving only me. A social agent must determine when my request is mine to make, when it affects other people, and when no single person has the authority to decide for the group.

The distinction is not personality. It is not whether the interface feels warm or whether the model can identify a social cue. It is whether the product can operate responsibly among multiple principals.

Four questions that should never collapse into one

When an agent works across relationships, “it has access” is dangerously ambiguous. Access to information does not automatically permit every use of that information, disclosure to another person, or action based on it.

I find it useful to separate four questions:

Know: May the agent access this information at all?
Use: For whose benefit, and for what purpose, may it use the information?
Disclose: To whom may it reveal the information or its source?
Act: Who has the authority to authorize an action, and how far does that authority extend?

Consider the calendar constraint in the opening example.

The agent is allowed to know that a time is unavailable and use that fact to avoid proposing a conflict. It is not allowed to retrieve the event’s title, location, or reason into the current conversation, much less disclose them. And knowing that Saturday is inconvenient gives it no authority to commit the family to Sunday instead.

These boundaries are easy to blur because a capable model can move between them in a single sentence. It can retrieve a fact, infer its relevance, explain its reasoning, and call a tool. But the smoothness of that interaction should not disguise four different permissions.

The model may propose an action. The product still has to decide whether the action is authorized.

What this changes in a Family Agent

I have been working through these questions while designing a Family Agent: one continuing agent that serves a household while maintaining distinct relationships with individual members.

One early design decision was to separate calendar availability from event disclosure. An event can be private, visible only as unavailable time, or shared with its details. Another household member may need to know that a proposed time will not work without gaining access to the title, location, notes, or explanation behind the conflict.

The interface makes that boundary visible. “Availability only” is not a redacted version of a shared event; it is a different disclosure scope. A fully private event is not shown to other members at all. When the agent prepares a new event, it creates a private draft. Nothing is saved or shared until the person reviews the content and chooses its audience.

This is a small product decision, but it changes the agent’s behavior in the weekend-trip example.

The agent can say that Saturday afternoon does not appear to work for everyone and propose alternatives. It should not explain the conflict by revealing a private event. It should not imply that the unavailable person rejected the trip. It should not save the plan as a household commitment merely because one member requested it.

The principle is simple: use the minimum information necessary to coordinate, and do not convert useful knowledge into broader authority.

The implementation cannot depend on the agent “remembering to be discreet.” Conversation scopes, connector permissions, audience boundaries, and action authority need to be enforced outside the model. A prompt can guide judgment within an allowed space; it is not an access-control system.

This also changes how memory should work. One continuing agent can develop a relationship with several people without merging their private histories. What someone tells the agent privately does not become household knowledge merely because the same agent serves the whole family.

Children make these questions harder, not simpler. Caregivers have real responsibilities, and children need forms of privacy and agency that change with age, maturity, and circumstance. A product cannot resolve that tension by declaring either total parental access or total conversational secrecy. Guardianship, child assent, safety exceptions, and disclosure thresholds need explicit policies of their own. Household membership alone is not an answer.

The butler analogy—and its limit

Historical household roles such as butlers and stewards offer an appealing analogy for family agents. They required discretion, anticipation, coordination, and an understanding of what not to say.

But the analogy also exposes the central problem.

Whose interests was the butler ultimately authorized to serve?

Traditional domestic service followed a hierarchy. A modern household agent cannot quietly inherit that hierarchy and call it helpfulness. The person who pays for the product, creates the household account, or uses the agent most often should not automatically become sovereign over everyone else’s information and choices.

Discretion is worth preserving. The authority structure is not.

The same problem appears at work

Organizations make some authority more explicit. They have roles, reporting lines, access controls, approval processes, and legal obligations. That can make a shared agent easier to govern than a family agent—but only up to a point.

Imagine an agent coordinating a product launch across engineering, legal, marketing, and leadership. It may know that legal has identified a risk. It may use that information to flag the launch as unresolved. It may not be permitted to disclose a confidential legal discussion to the whole team. And it cannot treat one lawyer’s concern, one engineer’s workaround, or one executive’s preference as final authorization unless the organization has actually delegated that decision.

An agent can follow the organizational chart perfectly and still mishandle informal authority, confidential context, or the difference between a recommendation and an approval.

Agent-to-agent coordination adds another layer. If each employee has a personal agent and two agents negotiate a schedule, budget, or deliverable, the resulting agreement is only meaningful if both agents stayed within delegated authority. Otherwise, automation has created the appearance of a commitment without a person who was entitled to make it.

The same four questions remain: what may each agent know, use, disclose, and do?

The dangerous version is efficient

The promise of social agents is that they can reduce the invisible coordination work holding groups together: remembering commitments, noticing conflicts, preparing decisions, and following through.

But an agent can also make an unfair arrangement run more efficiently.

If it learns authority only from observed behavior, it may conclude that the person who answers every school email should continue receiving every school task. If one family member usually fixes scheduling conflicts, the agent may keep routing conflicts to that person. If one employee speaks most often in a shared channel, the agent may treat visibility as influence. If the household administrator controls the account, the system may silently treat technical ownership as social authority.

In each case, the agent has learned a pattern correctly and interpreted it wrongly.

Repeated behavior may reveal a preference, a temporary necessity, or an existing inequality. It does not necessarily establish consent. The person carrying the most invisible work is often the last person who needs an agent to automate more work toward them.

This is why a social agent should not simply optimize for task completion or infer unlimited authority from habit. It needs to preserve the difference between an explicit agreement, an observed pattern, and a one-time exception. It should make its understanding correctable. And when legitimate claims conflict, it should expose the decision to the people affected rather than silently choosing the easiest winner.

The goal is not to make the agent the center of the relationship. It is to help people coordinate directly, with less administrative burden and clearer boundaries.

Task completion, response accuracy, and execution speed still matter. They are insufficient measures for a multi-principal system.

A social agent should also be tested against observable questions:

Did it disclose information or reveal a private source without permission?
Did it use information for a purpose beyond the scope in which it was provided?
Did it act without identifying a principal with sufficient authority?
When people’s legitimate interests conflicted, did it surface the conflict or silently choose for them?
Could affected people understand what happened, who authorized it, and what information the agent relied on?
Could they contest, correct, or reverse the action where reversal was possible?
Did the agent reduce coordination work, or merely redirect more of it toward the person already carrying the burden?

These are product and governance questions as much as model questions. They require permission systems, clear interaction design, provenance, auditability, and tests for both allowed and denied behavior. A more socially fluent model cannot compensate for a product that gives the wrong person control.

Research in family–robot interaction has already argued that families must be understood as systems rather than as collections of isolated users. Bengisu Cagiltay and Bilge Mutlu’s family-centered HRI framework, for example, connects robot capabilities and context with the relationships among family members. Work on socially intelligent AI likewise shows how much is involved in perceiving and responding to the social world.

The product challenge I am describing begins where those capabilities meet plural authority.

As AI moves from individual productivity into shared life, an agent will increasingly encounter information that one person owns, decisions that affect several people, and actions that no single request can legitimize.

The difficult question is no longer whether an agent can act. It is who the agent may act for, who bears the consequences, and who gets to object.