# PlantAI

> Photograph a sick houseplant, get a species ID, a diagnosis, and what to do about it

A React Native app that chains two vision models, one to identify the plant and one to diagnose it, then returns a treatment plan.

- Year: 2026
- Status: Work in progress
- Tags: React Native, Expo, TypeScript, LLM pipelines
- Page: https://www.rondahan.com/projects/plantai
- Repo: https://github.com/rondahan04/PlantAI

## Numbers

- **2** - chained vision models
- **5** - external APIs orchestrated
- **4** - surfaces sharing one core

## What it is

Photograph a sick houseplant and PlantAI identifies the species, works out what is wrong with it, and writes a treatment plan. A second pipeline finds nearby nurseries and what they actually stock, so the diagnosis leads somewhere. It is a React Native app on Expo; the nursery pipeline runs standalone but is not wired into the app yet.

## How it works

1. **Get the photo off the device** - React Native's WinterCG FormData rejects the blob shapes most upload examples assume. Images stream as expo-file-system File blobs instead.
2. **Identify the species** - PlantNet handles identification, since a purpose-built model beats a general one here. A 404 means the photo is not a plant, which is caught explicitly and surfaced as that rather than as a generic failure.
3. **Diagnose from the species** - The identified species becomes context for a second call to Claude or GPT, which writes the diagnosis and the treatment steps. Splitting the chain means the diagnosis model knows what it is looking at.
4. **Find nearby nurseries** - Google Places text search produces candidate businesses near the user.
5. **Fingerprint each site** - Each nursery site is classified as Shopify, WooCommerce, Wix or unknown, because each needs different handling. Results cache to disk so a site is only fingerprinted once.
6. **Scrape, extract, then verify** - Firecrawl first with Tavily as a fallback, narrowed to a price-focused excerpt, then a model pulls structured JSON and a second critic pass checks it before it is trusted.
7. **One core, four consumers** - A dependency-injection seam lets the mobile app, an offline dataset builder, a dashboard and a small HTTP server all sit on the same core.

## Flow

Two chains. The diagnosis chain is fast enough to sit in front of a user; the nursery chain takes 30 to 60 seconds, which is why it is not wired into the app yet.

- Photo (expo-file-system blob)
- identify → PlantNet (404 = not a plant)
- species as context → Claude / GPT (diagnosis + treatment)
- separately, to act on it → Google Places (nurseries nearby)
- classify each site → Platform fingerprint (shopify / woo / wix, cached)
- scrape → Firecrawl + Tavily (only if Firecrawl fails)
- extract, then check → GPT to JSON + Critic pass (catches invented prices)

## Technical notes

1. **Two specialised models beat one general one** - Asking a single model to identify the species and diagnose the disease at once made both answers worse. Chaining them gives the second model a fact to work from.
2. **The critic pass exists because extraction hallucinated** - Single-pass JSON extraction invented prices often enough that a verification step was not optional.
3. **The DI seam is what makes it testable** - Because the core takes its dependencies rather than importing them, it can be exercised without network access. This is the one project here with a real unit test suite.
4. **Failure paths matter more than expected** - Users photograph the wrong thing constantly, so the not-a-plant case is a designed path rather than an error state.

## Stack

- **Mobile:** React Native 0.85, Expo SDK 56, React Navigation, Reanimated, react-native-maps
- **Backend:** Node.js, TypeScript
- **AI & data:** PlantNet, Anthropic Claude, OpenAI, Firecrawl, Tavily, Google Places

## Repo layout

- `App.tsx` - Expo entry: navigation stack, fonts, theme
- `src/` - Screens, camera diagnosis, nursery service clients
- `scraper/` - Places discovery, then scrape and extract stock
- `server/` - Node API serving the nearby nursery search
- `dashboard/` - Local dashboard for testing scraper queries
- `scripts/` - Offline job rebuilding the nursery dataset
- `design-system/` - Tokens and layout rules for every screen

## Honestly

The app currently ships a static nurseries.json, so the distances and photos you see in it are placeholder data. The live pipeline works standalone but takes 30 to 60 seconds, which is too slow to sit in front of a user without a rethink. Wiring it in properly is the next piece of work.

---

Machine-readable: [llms.txt](https://www.rondahan.com/llms.txt) · [sitemap.xml](https://www.rondahan.com/sitemap.xml) · Markdown variants at [/md](https://www.rondahan.com/md) or via `Accept: text/markdown`.