I built a harvest tripwire, then almost let an LLM decide who looks legit

Free API tiers get farmed. What surprised me was how much the farming looked like real evaluation, and how badly a model did at telling the two apart. Companydata: Danish company registry data over HTTP, free monthly quota, and a tripwire that throttles keys looking up hundreds of different companies in an hour. The false story Alerts looked like ordinary builders testing an integration. I vouched a few by hand. That felt expensive, so I almost put a model on the alert to ask: does this look legit? I had been evaluating Jev / TypeSafe System One (https://github.com/typesafe-systems/jev) elsewhere. Identity-only, cheap tokens. Seemed perfect. What the data showed Three "testers" in a couple of days were one client: same egress, same curl, burn free quota to the limit, fresh account. Real people, real company, 1,500 free calls in 48 hours. Quota emails opened. Pricing known. Re-signup was cheaper than a credit pack. The tripwire was right about the pattern. The vouching step was wrong about independence of accounts. Jev Twenty-seven accounts: seven I had vouched, twenty from an earlier ring. Identity-only found every vouched account and waved through several of the ring. No useful threshold. With behaviour, it flipped most vouched accounts to abusive: closer to the truth, useless as a customer-facing gate. Decision: Jev stays out of the tripwire. What I shipped Related accounts on the admin alert: Telegram lists other accounts sharing the client address or exact display name. First live run taught me to exclude my own frontend service identity from that list. An honest 429 for flagged keys: under-review wording, how to buy credits or mail support, instead of the same silent "Rate limit exceeded" as a normal minute limit. I have not auto-withheld free quota by IP yet. That can wait. Harvest detection on breadth and velocity works. The failure mode is the story you tell yourself about who is behind the key. If you solved rotation without nuking real evaluators, I want to hear how. companydata.dk (https://companydata.dk)

Reactions

0

Comments

2

Shares

0

Browse more founder updates