Shoulder connects to nothing — no logins, no API keys, no “is my bank supported.” It works with any site or app because it watches the screen, the same way you would. Here’s what’s actually happening between recording and chat.

Projects, recordings, extractions

Shoulder organizes everything around three things:

  • Project — a goal you’re tracking, like “remember how I pay the mortgage” or “collect what I spend on the renovation.”
  • Recording — one screen-capture session inside a project.
  • Extraction — what Shoulder pulled out of that recording, measured against the project’s goal.

Why the goal matters

Shoulder doesn’t transcribe everything on screen — it extracts against the goal you stated when you created the project. A vague goal gets vague extractions. The more specific the goal (“track the confirmation number and due date for every bill I pay,” not just “bills”), the more useful the answers later.

Tip: You can edit a project’s goal at any time from the project settings — it only affects extractions going forward, not ones already saved.

How long a recording can be

A recording stops itself automatically after about 5 minutes, or sooner if it’s captured a lot of visual detail (fast scrolling, video playback) and reaches roughly 60MB — whichever comes first. Either way, nothing is lost: the recording is saved right up to that point and extraction runs on what was captured. If you’re working through something longer, stop and start a new recording rather than trying to capture it all in one pass.

What happens after you stop recording

  1. The recording uploads to the Shoulder relay for extraction — this is the only step that touches the cloud.
  2. A cloud model (Gemini Flash) reads the recording against your project’s goal and pulls out the relevant facts, links, and numbers.
  3. The extraction is written to your project’s on-device knowledge base, and the raw video is discarded from the relay.

Chatting with a project

Once a recording has been extracted, you can ask the project thread about it in plain language. Your project’s extractions are stored and searched on your device — Shoulder finds the ones relevant to your question locally, then sends your question and those extracted notes to a cloud model to write the actual reply. Answers reference the extraction (and, through it, the recording) they came from, so you can always trace an answer back to where it was seen.

See Privacy & Your Data for exactly what leaves your device when you chat, and what stays.