HikerAPI

Build a local, single-user Instagram data service in an empty folder. Stack: Python 3.11, FastAPI, uvicorn, instagrapi, SQLite via sqlite3 (no ORM), httpx only if needed.

Goal: a small REST API on localhost that returns Instagram profile, media, and follower data for personal research, with heavy caching so I never fetch the same thing twice.

Endpoints:
- GET /user/{username} : profile info (id, full name, bio, counts, is_private, profile pic url)
- GET /user/{username}/media?limit=N : recent posts (id, code, caption, like/comment counts, taken_at, media urls)
- GET /user/{username}/followers?limit=N : follower list, capped at 200 per call
- GET /hashtag/{tag}/top : top media for a hashtag
- GET /cache/stats : row counts and cache hit counters

Rules:
- Credentials and proxy come from .env: IG_USERNAME, IG_PASSWORD, IG_PROXY. Ship a .env.example, never commit secrets.
- Persist the instagrapi session to session.json and reuse it on boot. Only re-login if the session is dead.
- One global request queue with a single worker. Sleep a random 4 to 12 seconds between platform calls. No concurrency, ever.
- Cache every response in SQLite keyed by endpoint plus params, with a fetched_at timestamp and a TTL of 24 hours for profiles and 6 hours for media. Serve cache on hit and mark the response with a "cached": true field.
- On login challenge, 429, or ChallengeRequired, return HTTP 503 with a clear message telling me to log in manually in a browser on the same proxy. Do not retry in a loop.
- Log every platform call to a requests table so I can see exactly how much I hammered them.

Out of scope: no posting, liking, following, or DMs. No web UI beyond FastAPI's built-in docs. No account rotation, no proxy pool, no cloud deploy, no telemetry.

Deliverables: main.py, db.py, client.py, .env.example, requirements.txt, and a README that says plainly that this account will probably get banned and that the TTLs and sleeps are the only thing keeping it alive.

# AGENTS.md — Build guide for HikerAPI

## Project scope
A local FastAPI service that wraps instagrapi with one logged-in session, a proxy, aggressive request pacing, and a SQLite cache so you never fetch the same profile twice.

Catalogue verdict: no. The code here is the easy part: instagrapi and friends are open source, and an agent can wrap them in a REST API in an afternoon. What you cannot one-shot is a farm of aged accounts, residential proxy rotation, fingerprint churn, and a team that patches the client every time Instagram quietly changes an internal endpoint or ships a new challenge flow. Your single account behind your home IP will hit rate limits, then checkpoints, then a permanent ban, usually in that order and usually on the day you need data. A personal build is fine for pulling a few hundred profiles once. It is not fine for anything that has to keep working next month.
Use the implementation prompt below to define the deliverable. Complete each phase's acceptance checks before extending the scope.

## Working agreement
- Inspect the repository and its existing instructions before choosing paths, dependencies or commands. Keep one coherent stack and explain changes to the proposed architecture.
- Plan a vertical slice that accepts a real input and produces the useful output described below. Persist only the state the prompt calls for; respect memory-only and upstream-managed workflows. Use fixtures only when they are clearly labelled.
- After scaffolding, document the actual install, development, check and build commands in README and keep them synchronized with the package or project manifest. Do not report commands as successful unless they ran.
- Work in small steps. At handoff, list implemented flows, checks actually performed, remaining blockers, and any credentials or provider setup the owner must supply.
- Do not publish, spend money, contact customers, delete source data or run irreversible migrations without the project owner's authorization.

## Prerequisites
- A throwaway Instagram account you are willing to lose
- A residential or mobile proxy, datacenter IPs get flagged fast
- Python 3.11 and patience for session challenge prompts
- Acceptance that any of this can break without warning

## Stack and architecture
- Build a local, single-user Instagram data service in an empty folder. Stack: Python 3.11, FastAPI, uvicorn, instagrapi, SQLite via sqlite3 (no ORM), httpx only if needed.
- Persist the instagrapi session to session.json and reuse it on boot. Only re-login if the session is dead.
- Interface for this social data scraping API workflow: a local web page with input, progress, review, and export views.

## Security and data integrity
- On login challenge, 429, or ChallengeRequired, return HTTP 503 with a clear message telling me to log in manually in a browser on the same proxy. Do not retry in a loop.
- Keep API keys and OAuth secrets in .env; redact them from logs, generated files, and error responses.
- State the practical limit: An account pool that absorbs bans so yours does not.

## Agent implementation rules
- Project rule, data: Persist the instagrapi session to session.json and reuse it on boot. Only re-login if the session is dead.
- Project rule, behavior: Goal: a small REST API on localhost that returns Instagram profile, media, and follower data for personal research, with heavy caching so I never fetch the same thing twice.
- Project rule, recovery: On login challenge, 429, or ChallengeRequired, return HTTP 503 with a clear message telling me to log in manually in a browser on the same proxy. Do not retry in a loop.

## Optional agent skills and references
No installable skill is listed for this guide; use the project rules and the stack documentation.

Read the linked SKILL.md and its dependencies before adding a skill. Select only the skills matching this project's runtime and task; their documentation does not supply API access, credentials or approval to perform external actions. Pin the reviewed revision where the tool supports it. Follow the chosen agent's documented project-level installation mechanism.

## Distribution ideas
These are optional planning notes. Obtain the owner's approval before publishing or contacting anyone.
- Demonstrate this social data scraping API operation with sample or user-owned data: Goal: a small REST API on localhost that returns Instagram profile, media, and follower data for personal research, with heavy caching so I never fetch the same thing twice.
- Show a concrete result or export: Deliverables: main.py, db.py, client.py, .env.example, requirements.txt, and a README that says plainly that this account will probably get banned and that the TTLs and sleeps are the only thing keeping it alive.
- State the practical limit: An account pool that absorbs bans so yours does not.

## Engineering roadmap
1. Phase 1, architecture and data: Build a local, single-user Instagram data service in an empty folder. Stack: Python 3.11, FastAPI, uvicorn, instagrapi, SQLite via sqlite3 (no ORM), httpx only if needed. Persist the instagrapi session to session.json and reuse it on boot. Only re-login if the session is dead.
2. Phase 2, implement: Goal: a small REST API on localhost that returns Instagram profile, media, and follower data for personal research, with heavy caching so I never fetch the same thing twice.
3. Phase 3, implement: GET /cache/stats : row counts and cache hit counters.
4. Phase 4, review and output: Deliverables: main.py, db.py, client.py, .env.example, requirements.txt, and a README that says plainly that this account will probably get banned and that the TTLs and sleeps are the only thing keeping it alive.
5. Phase 5, recovery and acceptance: On login challenge, 429, or ChallengeRequired, return HTTP 503 with a clear message telling me to log in manually in a browser on the same proxy. Do not retry in a loop. Verify this invariant with a saved fixture: Invalid input must return a typed error; the same fixture must produce the documented output after restart. State the practical limit: An account pool that absorbs bans so yours does not.

## Paid-product capabilities outside this build
- An account pool that absorbs bans so yours does not
- Residential proxy rotation and device fingerprint management
- Same-day fixes when the platform changes internal endpoints
- Real throughput: concurrent requests instead of one polite call every few seconds
- Coverage beyond Instagram, including TikTok and other networks

## Implementation prompt
Build a local, single-user Instagram data service in an empty folder. Stack: Python 3.11, FastAPI, uvicorn, instagrapi, SQLite via sqlite3 (no ORM), httpx only if needed.

Goal: a small REST API on localhost that returns Instagram profile, media, and follower data for personal research, with heavy caching so I never fetch the same thing twice.

Endpoints:
- GET /user/{username} : profile info (id, full name, bio, counts, is_private, profile pic url)
- GET /user/{username}/media?limit=N : recent posts (id, code, caption, like/comment counts, taken_at, media urls)
- GET /user/{username}/followers?limit=N : follower list, capped at 200 per call
- GET /hashtag/{tag}/top : top media for a hashtag
- GET /cache/stats : row counts and cache hit counters

Rules:
- Credentials and proxy come from .env: IG_USERNAME, IG_PASSWORD, IG_PROXY. Ship a .env.example, never commit secrets.
- Persist the instagrapi session to session.json and reuse it on boot. Only re-login if the session is dead.
- One global request queue with a single worker. Sleep a random 4 to 12 seconds between platform calls. No concurrency, ever.
- Cache every response in SQLite keyed by endpoint plus params, with a fetched_at timestamp and a TTL of 24 hours for profiles and 6 hours for media. Serve cache on hit and mark the response with a "cached": true field.
- On login challenge, 429, or ChallengeRequired, return HTTP 503 with a clear message telling me to log in manually in a browser on the same proxy. Do not retry in a loop.
- Log every platform call to a requests table so I can see exactly how much I hammered them.

Out of scope: no posting, liking, following, or DMs. No web UI beyond FastAPI's built-in docs. No account rotation, no proxy pool, no cloud deploy, no telemetry.

Deliverables: main.py, db.py, client.py, .env.example, requirements.txt, and a README that says plainly that this account will probably get banned and that the TTLs and sleeps are the only thing keeping it alive.

## Completion evidence
Demonstrate the prompt's acceptance scenarios against the scoped workflow. Include setup from a clean checkout and failure recovery. Check persistence across restart and export/restore only for the state the prompt says to store; for memory-only tools, confirm that temporary content is discarded as specified. Record actual results and remaining limitations. A detailed plan alone does not establish a working replacement.