The SURE Framework: Social Intelligence for Human-Agent Collaboration

Large Language Model agents are evolving from question-answering tools to genuine collaborators that run continuously, reason across turns, reflect on tool usage, and act on our behalf. Yet effective human-agent collaboration requires more than raw capability. It demands social intelligence: the ability to sense, understand, remember, and engage in ways that feel natural, effective, and meaningful. We argue that social intelligence, not reasoning capability, is the primary bottleneck preventing LLM agents from becoming genuine collaborators. We propose SURE (Sense, Understand, Remember, Engage) as a conceptual framework for organizing research on socially intelligent agents. SURE decomposes social intelligence into four interdependent processes: (1) Sensing user cognitive and emotional states through multimodal signals, (2) Understanding user beliefs, intentions, and state through theory of mind, (3) Remembering preferences, history, and responses across interactions, and (4) Engaging appropriately in timing, embodiment, and tone. Drawing on recent work in physiology-aware agents, conversational turn-taking, and empathy measurement, we outline research directions where the HCI community is uniquely positioned to contribute.