// personal ai infrastructure · phase I · m12

J.A.R.V.I.S

Just A Rather Very Intelligent System

A single-user AI assistant that lives on your phone, thinks with Claude, and tracks every dimension of your life — finances, fitness, mood, learning, and more.

55
Tools
13
Scheduled jobs
22
Data tabs
₹25
Avg/day
FITNESS MOOD FINANCE LEARN CALENDAR HABITS MEDIA LIFE LOG J.A.R.V.I.S AI · PHASE I · M12
You
text or voice note
$ python main.py ✓ Telegram online ✓ FastAPI :8000 ▸ 55 tools ready
Your Mac FastAPI + APScheduler
HTTPS Anthropic API
Claude API
tool calls
DB ₹₹₹ MOOD FIT TODO CAL
55 Tools
reply sent
Telegram reply ~2–3 seconds
// Why does this exist
Most apps solve one problem.
Life isn't one problem.

Your fitness tracker doesn't know you skipped sleep. Your budgeting app doesn't know you're stressed. Your todo list doesn't know your calendar is packed. So you end up juggling 6 apps, none of which talk to each other — and still feel like you're missing something.

Jarvis starts from the opposite assumption: your data should know about each other.

One message. No forms. No app switching.
Just tell it what happened — it figures out what to log, what to check, and what to surface next.
Day to day
Problems it actually solves
Not a feature list — a friction list. Each one is something you stop thinking about.
⚡
Logging friction
A 3-second Telegram message is all it takes
"I'll add it later" is where tracking dies. Jarvis lives where you already are — no app to open, no form to fill.
🧩
Siloed data
Spend, sleep, mood, and schedule — all in one brain
Each piece of data alone tells half a story. Jarvis sees the full picture across everything you log.
💸
Budget anxiety
Know your spend pace without a spreadsheet
Alerts at 50%, 75%, 90%, 100% per category. Daily projection so month-end is never a surprise.
🌅
Morning context tax
7 AM brief replaces 5 apps in 10 lines
Weather, calendar, todos, yesterday's spend, fitness recovery — one message before your first coffee.
🔔
Dumb reminders
Nudges that know your day, not just the clock
Reminders only fire if you haven't logged yet. Already done the thing? No ping.
💡
Ideas that evaporate
Thoughts captured before they're gone
"Decided to skip the MBA — here's why." Timestamped, tagged, searchable. Not lost in a notes app you never reopen.
Intelligence Layer

Two models, one pipeline

Haiku classifies intent in 0.3s. Sonnet gets only the ~12 relevant tools — cutting token cost ~60%.

Primary · Agent Loop
Claude Sonnet 4.6

Handles all user conversations. Receives the classified tool subset, runs multi-step tool loops, preserves 4-message history.

~12
tools / call
4 msg
history
1024
max tokens
Classifier + Oneshot
Claude Haiku 4.5

Intent classification, media recommendations, portrait generation, vocab word generation, quiz answer grading.

₹0.01
per classify
64 tok
output
0.3s
latency
User input
"ran 5km, spent ₹480"
Raw Telegram message
["fitness", "expenses"]
2 of 14 categories matched
Subset loaded — 6 of 48 tools
log_habit() query_habits() add_expense() query_expenses() get_garmin_summary() get_current_time()
2 parallel calls
log_habit + add_expense fired together
Reply
"Logged. ₹8,240 left."
~2.4s total
Tool Ecosystem

55 tools across 10 domains

Every capability is a discrete tool. Adding more is near-free — definitions are cached.

Health & Fitness
3 tools
get_garmin_summary get_recent_workouts suggest_workout
Finances
3 tools · alerts 50/75/90/100%
add_expensequery_expensesupdate_category_budget
Mood & Energy
4 tools · qualitative triggers
log_moodlog_energyquery_moodanalyze_mood_patterns
Todos & Goals
6 tools · evening check
add_todocomplete_todoset_weekly_goalslist_daily_goals
Calendar & Habits
5 tools · append-only
query_calendaradd_calendar_eventlog_habitquery_habits
Media & Vocab
8 tools · spaced repetition
log_watchedrecommend_mediaadd_vocab_wordcheck_vocab_answer
Weather & News
3 tools · no API key needed
get_weathernews_digestget_literature_quote
Life Log
11 tools · 7 domains
log_idealog_decisionlog_socialgenerate_self_portrait
Reminders
3 tools · dynamic APScheduler
add_reminderlist_remindersdelete_reminder
Personality
2 tools · named profiles
switch_personalitylist_personalities
In Practice

What you can say

Natural language — no commands. Each card shows trigger → internal pipeline → response.

Fitness logging
You say
"Just finished a 5.2km run in 28 mins. Felt strong."
Internally
fitness →log_habit("exercise","5.2km 28min") →get_garmin_summary() parallel
Jarvis replies
Logged. 5.2km / 28min — solid pace.
Body battery: 72 · HRV trending up. Recovery: good 🟢
2 tools ~₹0.18
₹ Multi-action
You say
"₹480 lunch at Social, ₹120 Uber. Feeling drained."
Internally
expenses calories mood →add_expense(480,"food") →add_expense(120,"transport") →log_calories(720,"lunch") →log_mood(3) →log_energy(2) 5 parallel
Jarvis replies
Logged. ₹480 food · ₹120 transport · ₹16,760/₹25,000 MTD (67%).
~720 kcal lunch · 1,240/2,200 today. Mood 3 · low energy saved.
5 tools ~₹0.24
Morning context
You say
"What's my day looking like?"
Internally
external calendar fitness →get_weather() →query_calendar(today) →list_todos() →get_garmin_summary() →list_daily_goals() 5 parallel
Jarvis replies
Fri Apr 18 · 32°C Mumbai · Recovery 72%.
📅 1:1 Rahul 10am · Review 2:30pm · Gym 6pm
✅ 3 todos (1 overdue) · Goals: Run ✓ · Study ○
5 tools ~₹0.20
Voice message
You send a voice note 🎙
"Add 800 calories for lunch, chicken rice bowl."
Internally
calories →log_calories(800,"lunch")
Jarvis replies
[Add 800 calories for lunch, chicken rice bowl.]

Logged. 800 kcal lunch · 1,840 total · NET −380 kcal
1 tool ~₹0.16 total
Live Telegram session
Live Console

Dashboard at localhost:8000

Cyan-on-black Stark aesthetic. Real-time data via WebSocket + 30 s polling.

http://localhost:8000
IDLE
J · A · R · V · I · S
personal assistant // phase I
08:42:31
THU 17 APR 2026
// TRANSCRIPT 8
ran 5km today, heart rate avg 148
⚙ log_habit · get_garmin_summary Logged run (5 km, avg HR 148). Body battery at 72 — solid effort. Recovery looks good for tomorrow.
what did i spend yesterday?
⚙ query_expenses Yesterday: ₹1,240 — Food ₹680, Transport ₹310, Misc ₹250.
add milk and eggs to groceries todo
⚙ add_todo Added: "Buy milk + eggs" · tagged groceries.
// TODAY'S AGENDA 3
10:00 Standup sync
14:30 Design review
19:00 Gym session
STANDBY
31°C Partly Cloudy
Humidity 74% · Feels 36°C · Mumbai
// SYSTEM
EVENTS
147
TOOLS
83
ERRORS
0
WS
OK
// HEART RATE today
REST
52
AVG
71
MAX
163
LAST
64
// ACTIVITY
RUN 5.0 km · 28:14 · Z3
STEPS 9,847 of 10k
//TODO5 open
⚠ Review Q1 report overdue
→ Buy milk + eggs
→ Push vocab PR
→ Book dentist
//NUTRITION TODAYnet −340
1,840 kcal
PROT
112g
CARB
198g
FAT
54g
MOOD 7D
//EXPENSESApr
TODAY
₹480
THIS MONTH
₹18,240
DAILY 62%
MONTHLY 73%
//API USAGEINR 83.4
TODAY
₹22.4
38 calls
THIS MONTH
₹610
1,047 calls
COST 7D
//RECENTLY WATCHED
★ 8.5 Dune: Part Two
★ 7.2 Severance S2
★ — The Brutalist
//7-DAY TRENDSGarmin · last synced 07:18
SLEEP7h 12m
HRV58 ms
BODY BATTERY72
DISTANCE5.0 km
// EVENT STREAM 08:42 tool:get_garmin_summary · 08:41 tool:log_habit[run] · 07:18 scheduler:morning_brief · 07:00 garmin:cache_refresh
Always On

12 jobs running in the background

APScheduler drives a daily rhythm. Most jobs are free — only 4 involve Claude API calls.

07:00 · daily
Morning Brief
Quote + weather + calendar + todos + Garmin recovery + yesterday spend + goals.
~₹0.40
18:00 · daily
Evening Reminder
Dinner pick from local list + due-today todos. Fully local, no API calls.
₹0 · local only
21:00 · daily
Goals Check
LLM checks each goal against data. Sends ✓/✗ with nudge for missed ones.
~₹0.20
09:15 · daily
Budget Pace
Projects month-end spend. Alerts when ≥90% of any budget. Pure math, no LLM.
₹0 · math only
hourly
Garmin Auto-Log
Detects new workouts via activity ID dedup, auto-logs exercise habit + Telegram ping.
₹0 · local
Sun 21:00
Weekly Digest
Full week — money, fitness, habits, mood, media, learning, goals.
~₹0.50
08:00 + 10/16/20:00
Vocab Learning
Daily new word + 3 spaced-repetition quizzes. 3 correct = mastered.
~₹0.05/day
22:00 · daily
Evening Summary
Day recap + mood prompt before bed. Reviews what got done.
~₹0.30
per-habit · daily
Habit Reminders
Nudge for each habit not logged today. Times from profile.yaml.
₹0 · local
Sun 20:00
Study Check-in
Progress review on each study topic. Weekly accountability.
~₹0.40
Sun 19:00
Mood Review
Weekly mood + recovery insights. Patterns across the week.
~₹0.30
user-defined
Dynamic Reminders
Push notifications at custom times. Daily/weekdays/once. Created via tools.
₹0 · local
Data Layer

Hybrid storage

SQLite owns agent state. Sheets owns tracking data — editable and reviewable.

SQLite
WAL · thread-safe · local
conversationsFull turn history
token_usagePer-call cost log
factsGarmin dedup, KV store
daily_goalsWeek-keyed goals
vocab_wordsDeck + next_review
remindersTimed push alerts
sheets_queueOffline write buffer
Google Sheets
Auto-created on first write
expensesAmt, category, note
habitsAppend-only log
mood / energyScore + context
mediaWatched + TMDb meta
ideas_logTimestamped ideas
decisions_logDecision + outcome
Cost Profile

₹25/day

Full-time AI assistant at 35–44 messages/day.

₹20–35
Per day
70% cache hit
₹0.01
Per classify
Haiku · 64 tokens
₹0.15
Cache hit
~6900 cached tokens
₹2.5
Cold start
5-min idle miss
Cost reductions applied
−40%
Tiered routing
Haiku classify
−20%
History
10→4 msgs
~₹0
Haiku for
oneshot tasks
Data & Privacy

What goes where

Single user, no SaaS, no analytics. Here's the honest picture.

Stays on your machine
  • 🗄SQLite database — conversations, habits, todos, expenses, token logs
  • 📄profile.yaml & tracking.yaml — all config lives locally
  • 🖼Self-portrait outputs — written to profiles/, never uploaded
  • 📊Google Sheets — your own account, your own spreadsheet

No third-party analytics. No SaaS subscription. No data broker. The only server that owns your data is Google Drive — and that's already yours.

Leaves your machine
  • 🤖Anthropic API — every message including logged data transits Claude. Paid tier does not train on your data by default.
  • 🎙OpenAI Whisper — voice notes sent for transcription. Optional; voice logging is skipped if key is unset.
  • 💬Telegram — messages pass through Telegram servers. Bot messages are not end-to-end encrypted.
  • ⌚Garmin Connect — fitness data lives on Garmin's servers; Jarvis fetches and caches it locally.

The honest gap: any message containing expenses, mood, or health data transits at least one external API. Logging happens in natural language — there's no scrubbing before it hits the wire.

What future iterations will fix
  • 🔒Local Whisper (faster-whisper / whisper.cpp) — voice transcription fully on-device, OpenAI dependency eliminated.
  • 🔐SQLite encryption at rest (SQLCipher) — protects the local DB if the machine is accessed.
  • 🧠Local classifier (Ollama + small model) — intent routing never leaves your machine; only the tool call goes to Claude.
  • 📅CalDAV calendar — replace Google Calendar with a self-hosted or iCloud option.

The goal is a system where only the LLM reasoning step needs an external API — everything else stays local. Not there yet, but the architecture already supports it.

Today · M12
J.A.R.V.I.S
  • Reactive — responds when you message it
  • Logs what you explicitly tell it
  • Scheduled briefs at fixed times
  • 55 tools, 4-message memory window
  • Knows your data. Doesn't yet know you.
→
Target · Phase III
Son of Anton
  • Proactive — surfaces patterns before you ask
  • Builds a growing self-model from months of logs
  • Context-aware nudges, not just clock-based ones
  • Cross-domain correlation: sleep × mood × spend × fitness
  • Knows your patterns better than you do. Stops being a tool.
// What's next
The build queue
From reactive assistant to pattern-aware collaborator — in four phases.
PHASE II · UP NEXT
Pattern Intelligence
  • 3D node cloud HUD — life domains as clusters, node size = frequency
  • Portrait diff — compare who you were last month vs now
  • Cross-domain correlations: mood × sleep × spend
  • Self-portrait injected as context into agent loop
PHASE II · PRIVACY
Local-First Stack
  • Local Whisper — voice transcription fully on-device
  • Local classifier via Ollama — intent routing never leaves
  • SQLite encryption at rest (SQLCipher)
  • CalDAV calendar — remove Google dependency
PHASE III · PROACTIVE
Autonomous Layer
  • Anomaly detection — "you've spent 40% more than usual this week"
  • Recovery-aware schedule suggestions
  • Raspberry Pi deployment — always on, not just when laptop is open
  • Predictive journaling prompts based on historical patterns
PHASE IV · ENDGAME
Son of Anton
  • Self-modifying profile from observed behaviour — no manual YAML edits
  • Decision support: "based on 3 months of data, avoid Thursday evenings for deep work"
  • Long-horizon memory beyond the 6-message window
  • The point where it stops logging your life and starts shaping it
📺
The Silicon Valley reference, for those who get it: In Season 6, Gilfoyle builds "Son of Anton" at Pied Piper — named after Anton, the AI he had previously created. A system so capable at predicting and optimising human behaviour that the board starts asking uncomfortable questions about whether it should exist at all. The joke lands because it's only slightly exaggerated. Right now, Jarvis is firmly in the "helpful butler" phase. The goal is to push it toward the "uncomfortable questions" phase — and then stop just before it figures out your passwords.