Needle 2 — A 14MB AI model for tool calling and data extraction on small devices like phones and wearables.
Analyzed by Sai Pavan Gopularam · AI · On-Device AI · View on GitHub
- Stars: 10468
- Forks: 673
- Commits last 30 days: 54
- Health: Active (54 commits this month)
- Language: Python
- License: Apache-2.0
What It Is
Needle 2 is a compact, 14MB AI model designed to run directly on small devices like smartphones, wearables, and smart home gadgets. Think of it as a tiny, specialized brain that understands natural language commands and can interact with device functions, similar to how a smart assistant controls your lights, but much smaller and embedded directly into the product.
This matters because traditional large AI models are too big and resource-intensive for most consumer electronics. Needle 2 solves this by providing advanced AI capabilities – like understanding complex commands and extracting structured data – without needing cloud connectivity or powerful processors, opening up new possibilities for intelligent, offline device interactions.
License Verdict
Apache-2.0 License — Build and Sell Freely — Commercial Use Approved • Permissive
The Apache-2.0 license is highly permissive. You can use this software for commercial purposes, distribute modified versions, and even sublicense it. The main requirements are to include the original copyright and license notices, and to state any significant changes you've made. It explicitly grants patent rights, which is a significant benefit for commercial products.
How to Use It
Get Needle 2 running by installing the Python package. The inference engine and weights are fetched automatically and cached, requiring no complex build steps. You can define tools using simple Python decorators and immediately start making tool calls or extracting structured data.
Prerequisites:
- Python 3.8+
Estimated setup time: 5 minutes.
pip install cactus-needle
import needle
@needle.tool
def get_weather(city: str):
"Get the current weather for a city."
return {"city": city, "temp_c": 27, "sky": "clear"}
agent = needle.Needle(tools=[get_weather])
print(agent.run("what's it like in Lagos right now?")["results"])What I'd Build With This
On-Device Data Extractor API (micro-saas)
Build a simple web service that allows users to upload text (e.g., receipts, invoices, sensor logs) and define a Pydantic model. The service uses a fine-tuned Needle 2 instance to extract structured data locally on a user's device (e.g., a Raspberry Pi acting as a gateway) and returns JSON. Users pay for a monthly subscription based on usage or device count. Market to small businesses, researchers, or hobbyists needing privacy-preserving data extraction.
Effort: 1 Week Build Time · Target: Small Businesses, Researchers · Pricing: $19/mo
IoT Agent Builder Platform (saas)
Develop a SaaS platform for IoT device manufacturers to create, fine-tune, and deploy custom on-device AI agents. Provide a UI to define tools (device functions), upload sample data, and fine-tune Needle 2 for specific product lines (e.g., smart ovens, industrial sensors). The platform generates a `.cact` file that manufacturers can embed. Monetize with tiered subscriptions based on the number of deployed devices or agent complexity, targeting mid-sized IoT companies.
Effort: 3 Months Build Time · Target: IoT Manufacturers · Pricing: $99-$999/mo
Custom Edge AI for Industrial Automation (enterprise)
Offer tailored enterprise solutions for industrial clients to embed Needle 2 into their specialized hardware, such as factory robots, monitoring equipment, or heavy machinery. This allows for voice-controlled operations, real-time anomaly detection, or complex sensor data interpretation directly at the edge, without relying on cloud processing. This would involve custom tool definitions, data synthesis, fine-tuning, and integration services, solving critical latency and privacy issues for industrial use cases.
Effort: 6 Months Build Time · Target: Industrial Sector, Manufacturing · Pricing: Custom SOW, $100k+
Sai Pavan Gopularam's Take
This is exactly what I mean by 'AI that fits in your pocket.' Needle 2's small footprint and tool-calling capabilities make it perfect for embedding intelligence directly into hardware, solving real-world latency and privacy issues. I'd target smart appliance manufacturers; imagine charging $75k for a custom, on-device AI agent that lets users naturally control their kitchen or home devices.
Watch Out For
- Fine-tuning Requires OpenRouter API Key: While Needle 2 runs locally, its data synthesis feature for fine-tuning requires an `OPENROUTER_API_KEY` to generate training examples. This introduces an external dependency and cost during the development phase.
- Still Requires Resources: Despite its small size (14MB binary, 28MB RAM), Needle 2 isn't zero-cost. It still needs a device with sufficient RAM and processing power to run efficiently, especially for continuous operation or complex tool chains.
- Telemetry Collection (Opt-Out): Cactus Compute collects anonymous usage telemetry by default. While anonymous, founders should be aware and inform users, providing clear instructions to opt out using `NEEDLE_TELEMETRY=0` or `DO_NOT_TRACK=1` if privacy is a top concern for their product.
I break down trending repos like Needle 2 every week — join the newsletter.