03-27 00:02 · AI,技术,HackerNews,AI
We Rewrote JSONata with AI in a Day, Saved $500K/Year
A few weeks ago, Cloudflare published “How we rebuilt Next.js with AI in one week.” One engineer and an AI model reimplemented the Next.js API surface on Vite. Cost about $1,100 in tokens.
The implementation details didn’t interest me that much (I don’t work on frontend frameworks), but the methodology did. They took the existing Next.js spec and test suite, then pointed AI at it and had it implement code until every test passed. Midway through reading, I realized we had the exact same problem - only in our case, it was with our JSON transformation pipeline.
Long story short, we took the same approach and ran with it. The result is gnata — a pure-Go implementation of JSONata 2.x. Seven hours, $400 in tokens, a 1,000x speedup on common expressions, and the start of a chain of optimizations that ended up saving us $500K/year.
An expensive language boundary
At Reco, we have a policy engine that evaluates JSONata expressions against every message in our data pipeline - billions of events, on thousands of distinct expressions. JSONata is a query and transformation language for JSON (think jq with lambda functions), which makes it ideal for enabling our researchers to write detection rules without having to directly interact with the codebase.
The reference implementation is JavaScript, whereas our pipeline is in Go. So for years we’ve been running a fleet of jsonata-js pods on Kubernetes - Node.js processes that our Go services call over RPC. That meant that for every event (and expression) we had to serialize, send over the network, evaluate, serialize the result, and finally send it back.
This was costing us ~$300K/year in compute, and the number kept growing as more customers and detection rules were added. For example, one of our larger clusters had scaled out to well over 200 replicas just for JSONata expressions, which resulted in some unexpected Kubernetes troubles (like reaching IP allocation limits).
In some respects, the RPC latency overhead was actually worse than the pure dollar cost. An RPC round-trip is ~150 microseconds before any evaluation even starts. For a simple field lookup like user.email = "admin@co.com" something that should take nanoseconds - we’re paying microseconds just for crossing a language boundary. At our scale, those microseconds stack up quickly.
We’d tried a few things over the years - optimizing expressions, output caching, and even embedding V8 directly into Go (to avoid the network hop). They did their part, but it was mostly just incremental improvements. The closest we got was a local evaluator we built using GJSON that handled simple expressions directly on raw bytes. It was fast for what it covered, but anything complex had to fall back to jsonata-js. We were patching around the problem, but the root cause remained unsolved.
Building gnata
During the weekend I built out a plan (using AI) separated into ‘waves’. The approach was the same as Cloudflare’s vinext rewrite: port the official jsonata-js test suite to Go, then implement the evaluator until every test passes. The following day, I pressed play. The plan was straightforward - build out the full JSONata 2.x spec in Go, with a focus on performant streaming and some extra features sprinkled on (localized caching, WASM support, metrics, and fallthrough capabilities back to the jsonata-js RPC).
A few iterations and some 7 hours later - 13,000 lines of Go with 1,778 passing test cases.
Total token cost: $400.
I shared the numbers internally and someone asked about the ROI. Production cost for jsonata-js in the previous month was about $25K - now it was 0. That conversation ended up being pretty short.
Two-tier evaluation
gnata has a two-tier evaluation architecture. At compile time, each expression is analyzed and classified.
The fast path handles simple expressions - field lookups, comparisons, and a set of 21 built-in functions applied to pure paths (things like $exists(a.b) or $lowercase(name)). These are evaluated directly against the raw JSON bytes without ever fully parsing the document. For something like account.status = "active" you get 0 heap allocations.
Everything else goes through the full path - a complete parser and evaluator with full JSONata 2.x semantics. This does parse the JSON, but only the subtrees it actually needs, not the entire document.
On top of this there’s a streaming layer (the StreamEvaluator) designed for our specific workload: evaluate N compiled expressions against each event, where events are structurally similar.
- All field paths from all expressions are merged into a single scan. The number of expressions doesn’t matter - raw event bytes are only read once.
- After warm-up, the hot path is lock-free. Evaluation plans are computed once per event schema and cached immutably, so reads are a single atomic load with no synchronization.
- Memory is bounded. The cache has a configurable capacity and evicts the oldest entries when full.
The fast-path design took a lot of inspiration from t
▸ 展开全文
03-27 00:02 · AI,技术,HackerNews,AI
New York City hospitals drop Palantir as controversial AI firm expands in UK
New York City’s public hospital system announced that it would not be renewing its contract with Palantir as controversy mounts in the UK over the data analytics and AI firm’s government contract.
The president of the US’s largest municipal public healthcare system, Dr Mitchell Katz, testified last week before the New York city council that the agreement with Palantir would expire in October.
He said at the hearing that the contract, which focused on recovering money for insurance claims, was always meant to be short-term, and that there was an “absolute firewall” preventing Palantir from sharing information with US Immigration and Customs Enforcement. He said that the agency had “not had any incidents”.
The contract and related payment documents shared with the Guardian by the American Friends Service Committee and first reported by the Intercept, show that NYC Health + Hospitals has paid Palantir nearly $4m since November 2023. The contract noted that Palantir would be able to review notes about patients’ health and help the hospital claim more money in public benefits through programs such as Medicaid. It also includes a line stating that with permission from the city agency, Palantir can “de-identify” patients’ protected health information and use it for “purposes other than research”.
NYC Health + Hospitals said in an email to the Guardian that it will be transitioning to systems that were made entirely in-house, and there will be no data shared with Palantir or use of the company’s applications after the contract expires. “NYC Health + Hospitals’ use of Palantir technology is strictly limited to revenue cycle optimization, helping the public healthcare system close gaps between services delivered and charges captured, protect critical revenue, and reduce avoidable denials,” the agency said in an emailed statement.
A Palantir spokesperson said in a statement: “Palantir, as a software company, does not own or have any rights to customer data – and each customer environment is individually protected against unauthorized access or misuse via robust security controls which can be fully administered and audited by the customer.”
Palantir presence grows in the UK
As New York City’s hospital system prepares to part ways with Palantir, the company is facing similar scrutiny over privacy issues in its £330m agreement with the UK’s National Health Service (NHS). Health officials in the UK are concerned that the controversy surrounding Palantir may stop the nationwide rollout of the company’s data system, even though Keir Starmer is trying to speed up deployment. As of last summer, not even half of the country’s health authorities had started using Palantir’s technology amid concerns from the community and doctors. A 12 March briefing by Medact, a health justice charity, said Palantir’s software could enable “data-driven state abuses of power”, including US-style ICE raids. Palantir has denied that the data could be used in this way, noting that it would be illegal and a breach of contract.
Palantir, which also contracts with the British government’s Ministry of Defence, is expanding its influence in the country – despite backlash from activists and some lawmakers. The Guardian revealed last week that Palantir is trying to gain access to sensitive national financial regulation data.
The Financial Conduct Authority, a watchdog for thousands of financial bodies from banks to hedge funds, awarded Palantir a contract to investigate internal intelligence data to help root out financial crime. That has sparked outcry from some MPs, who have urged the government to halt this agreement. Liberal Democrats called on Monday for a government investigation into the contract. Starmer has dismissed suggestions that the UK has become “dangerously over-reliant” on American tech companies, including Palantir, but noted he preferred to have more domestic capability.
Medact has raised privacy concerns in the UK about Palantir’s ability to access de-identified patient data. (De-identified data refers to data that has been stripped of characteristics that could indicate who an individual is, such as names and social security numbers.) In a 12 March briefing for health officials, Medact argued that the NHS’s data privacy protections are insufficient; NHS England has said that data is de-identified as it moves through its national software system, the NHS federated data platform (FDP). But Medact cited concerns that this data can be easily re-identified.
An NHS spokesperson said in an emailed statement to the Guardian that the supplier of the FDP “was appointed in line with public contract regulations and must only operate under the instruction of the NHS, with all access to data remaining under NHS control and strict contractual obligations protecting confidentiality”.
Data privacy concerns
Data privacy experts interviewed by the Guardian said that there are risks in Palantir accessing New Yorkers’ de-identified data for purposes other
▸ 展开全文
03-27 00:02 · AI,技术,HackerNews,大模型,AI
My minute-by-minute response to the LiteLLM malware attack
Now I have the full picture. Here's my analysis:
What 2.1.81
is: Claude Code v2.1.81. You had 5 Claude Code instances running at shutdown time.
Shutdown timeline:
- 01:36:33 — Force shutdown initiated
- 01:36:36 — Shutdown stall captured (162 processes still running, 21 of them Python)
- 01:37:11 — System booted
Process tree at shutdown:
Normal Claude Code MCP servers (2 instances):
Claude Code [82193] → python3.13 [88842] → node [88854] (416MB)
Claude Code [35621] → python3.13 [45088] → node [45091] (228MB)
These are MCP server bridges — Python wraps a Node.js MCP server. Normal behavior.
A uv run
chain:
zsh → uv [74172] (deadlocked) → python3.13 [74174] (running a .py file)
→ python3.13 [74180] (python -c "...", reading pipe)
→ python3.13 [86263] (running .py file, sleeping + os.stat)
→ python3.13 [86264] (python -c "...", reading pipe)
14 orphaned python -c
processes (two batches of 7):
- PIDs 89759-89765 (~38 hours old)
- PIDs 67956-67962 (different age)
- All running
_PyRun_SimpleStringFlagsWithName
— this ispython -c "exec(base64.b64decode(...))"
- All orphaned (parent died, reparented to launchd)
- All stuck reading from a dead pipe
What likely happened:
The exec(base64.b64decode('...'))
pattern is not malware — it's how Python tooling (including Claude Code's Bash tool) passes code snippets to python -c
while avoiding shell escaping issues. The base64 encoding is just a safe transport mechanism.
The 11k process storm was likely a runaway spawning loop from one of these sources:
- A Claude Code tool/agent interaction that got stuck in a loop spawning
python -c
processes - A
uv run
script (the 74174 chain) that had a subprocess spawning bug
Good news:
- No persistence mechanisms found — no suspicious crontabs, launch agents, or launchd jobs
- No suspicious Python processes running now — clean after reboot
- All processes used your local uv Python — nothing from unknown paths
- No malicious indicators — all parents trace back to Claude Code, uv, or zsh
Recommendations:
- Check if you had a Claude Code agent running in a loop before the incident
- If it happens again, try
killall python3.13
before force-shutting down - Consider setting a process limit:
ulimit -u 2048
in your.zshrc
to prevent fork bombs
▸ 展开全文