How to parse tool calls from OpenAI, Anthropic and Google
How to turn tool calls from different LLM providers into one common interface without losing the identifiers that correlate each result.
Curious about how it is built inside? Both agents are explained step by step in Gymnasia: architecture of an AI agent that runs on the phone.
Download APK for Android
There are plenty of gym apps, and plenty of diet apps. What changes here is what it carries inside: an AI agent that acts as your personal trainer.
And it is not a chat bolted onto the side of the app to answer the odd question. It is an agent with real access to your data: you say "build me a three-day push routine" or "log that I had two eggs and a slice of toast" and it does it, searching the exercise library, writing into today's diet, reading your measurements to decide, and remembering what you told it last week.
The second thing that makes it different: that agent runs entirely on your phone. There is no server of mine your data passes through, because there is no server. The app calls the AI provider directly with a key you supply, and everything else, deciding what to do, running it and storing it, happens inside the phone. Your weight, your diet and your routines never leave it.
This is the agent you talk to. It operates the app through a catalogue of tools: it searches exercises and foods, reads and writes your diet, creates routines, logs measurements and stores what you tell it about yourself so it is there in the next conversation. What it is able to do is exactly that list, and nothing more.
It is BYOK (bring your own key): you supply the API key and it is stored on the device, so you choose which provider to use and control what you spend. Its system prompt is fetched remotely with an embedded fallback copy, so it can be fixed without shipping a new release and the app keeps working offline.
Swipe the graph horizontally to see all of it.
The second agent, independent from the first one: you take a photo of the plate and it gives you calories and macronutrients. If there is a barcode in the photo it reads it and queries OpenFoodFacts, so for supermarket products the numbers are the real ones instead of a guess. Nothing is saved into your diet until you approve it.
Swipe the graph horizontally to see all of it.
Workout Routines
Create custom routines with exercises, sets, reps and weights. Each routine can be categorized (strength, cardio, flexibility, etc.) and customized with icons. During training, a rest timer with sound alert notifies you when to continue.
Exercise Library
Open-source exercise repository with AI-generated images showing correct execution, in both male and female versions. Exercises include muscle group, required equipment, difficulty level and detailed instructions. Users can also create custom exercises.
Diet tracking
Daily meal logging with calorie and macronutrient calculation (protein, carbs and fats), personalized nutritional goals and barcode scanning against OpenFoodFacts.
Body Measurements
Track weight and other body measurements with historical data to visualize your progress.
The app has no backend: both agents run entirely on the phone, so the user's data never leaves the device. The project is an npm workspace monorepo containing:
The mobile app uses AsyncStorage for local persistence and Expo SecureStore for API keys. Exercise images are served from the GitHub repository and cached on-device.
The project is open source and available on GitHub.
If you like the project, consider giving it a star on GitHub.
If you want more information about how the app is made, the two graphs above explained step by step, the tool loop, the provider choice and why the system prompt lives remotely, it is all in Gymnasia: architecture of an AI agent that runs on the phone.
How to turn tool calls from different LLM providers into one common interface without losing the identifiers that correlate each result.
A practical guide to designing tools an LLM can discover, adapting them to any LLM provider, executing them safely, and ...
How the Gymnasia AI agent is built: a BYOK conversational coach with local tools and a food estimator that reads macros ...
Let's talk.
maximofn@gmail.com
Machine Learning and AI specialist. I develop solutions with generative AI, intelligent agents and custom models.
AI agents, powered by LLMs, promise to transform applications. But are they simple executors today or future intelligent collaborators? To reach their...
Learn to create an IA system to execute efficiently on a device
Hugging Face spaces allow us to run models with very simple demos, but what if the demo breaks? Or if the user deletes it? That's why I've created docker containers with some interesting spaces, to be able to use them locally, whatever happens. In fact, if you click on any project view button, it may take you to a space that doesn't work.
Let's talk.
maximofn@gmail.com
Machine Learning and AI specialist. I develop solutions with generative AI, intelligent agents and custom models.
Dataset with jokes in English
Use: Fine-tuning text generation models for humor
Dataset with translations from English to Spanish
Use: Training English-Spanish translation models
Dataset with Netflix movies and series
Use: Netflix catalog analysis and recommendation systems