FlashFeed

Full text

A single AI agent conversation can look flawless scored on its own and still point to a broken product. That gap is driving a shift in how enterprises evaluate agents, away from scoring individual traces and toward comparing cohorts of users against a baseline.

At VB Transform 2026 , Harrison Chase, CEO of LangChain; Hui Zhang, CTO and co-founder of Conviva; and Emmanuel Turlay, director of engineering at CoreWeave, described that shift, along with a parallel move toward cheaper, narrower judge models.

Agent-as-judge — judging one AI agent's output with another — hasn't replaced LLM-as-judge, which Chase said remains the default. The larger tension, Zhang said, is between automated judging, whether by LLM or agent, and human review.

"You have scalable but ungrounded, whether it's agents as judge or LLMs as judge, you grade the outcome, you grade the work. It still is very difficult to ground it and then you use humans and that's just not scalable," Zhang said. "The whole industry is facing this, which poison you want to pick."

Evaluation criteria now function as the product spec

That gap — a conversation that scores well but still signals a broken product — is what teams try to close by building an exhaustive evaluation suite before they ship anything. Chase said that doesn't work.

"We sometimes see teams that have almost eval paralysis," Chase said. "They're like, this is an eval set, I can't launch it. The best teams launch and then iterate."

Chase framed evaluation criteria as a living specification, not a one-time test suite: a product requirements document — the standard software-development spec for what an application should do. "Evals are like the new PRD," he said. "They define what your agent should and shouldn't do."

Turlay described hitting the same failure from a different angle. "I was trying to reach 100% coverage for my tests, and I still had bugs in production," he said — a test suite that looked complete but still missed what mattered, the same gap Chase was describing with evals.

Broad, always-on monitoring, he said, catches more real failures than an exhaustive pre-launch test suite. Teams should set up wide online checks first, use those to identify failure classes as they occur, then build a targeted offline evaluation set around the problems that surface.

Why scoring traces one at a time is a mistake

Even a well-built evaluation process can still score the wrong thing. Zhang's objection is to how most teams run evaluation: sampling traces, whether 50 of them or a full population, scoring each in isolation. That approach misses a signal that only shows up when comparing cohorts of users against a baseline, a method Zhang calls contrastive analysis.

Zhang illustrated it with a retail example: a shopper asks an agent for a running shoe ahead of a half marathon, the agent asks qualifying questions, and the shopper buys a shoe. Scored individually, that interaction looks fine. But the clarification ratio, how many follow-up questions an agent asks before completing a task, came in three times higher than baseline for that shoe category across the full user population. A second metric, how often shoppers finished their purchase outside the conversation, was five times higher than baseline for the same category.

Neither number is visible from a single trace. Both point to a debuggable, category-specific problem. Zhang said the industry also lacks a second data source: what happens before, between and after the conversation, not just the trace itself.

Sizing the judge to the job

Once contrastive analysis flags which category is actually broken, the next problem is what watches for it going forward — and at what cost. Turlay's rule was to start with the most capable model available to prove a task is solvable, then work down. If it can't be done with a top-tier model, he said, it won't work with a smaller one. Once a pattern proves viable, teams can sample a fraction of traffic instead of judging every interaction, and move simpler tasks like binary classification to smaller open source models.

LangChain took that further, fine-tuning its own model to detect when a user believes the agent made a mistake, a signal Chase calls perceived error. "The model we fine-tuned was a Qwen model," he said, referring to Alibaba's open source family. Combining hand labeling with distillation, the result performed well. "Same as [Claude]Sonnet, for, depending on how we served it, either 10 to 100x cost reduction," Chase said.

Not every guardrail needs a model. Chase pointed to Claude Code's own guardrails as proof: regexes, the common programming technique for finding and validating patterns in code. "A lot of the guardrails they had were just regexes," he said. "They weren't small LLMs, they were just regexes."

LLM-as-judge doesn't mean human-in-the-loop disappears

The bigger question is whether using LLM as a judge removes the need for a human in the loop.

Turlay pointed to accountability, drawing on his prior work at a self-driving car company. His team compressed data intake and retraining into a two-week cycle for shipping a new model to the car. Even then, someone still had to sign off.

"I felt confident on behalf of the company to say this model should go into the car," he said. The same logic extends to legal, finance and healthcare. "Before we can remove a human to say, I endorse this and I take responsibility legally for it, it's going to be a while before agents can do that on their own."

Zhang agreed a human has to remain the guardian on corner cases, even as automation eventually runs at a scale that beats individual human accuracy — machines can see more at the pattern level.

Chase went further: that human check isn't just a safety net. "Human in the loop is really important for building trust in how these agentic systems work, and also really important for memory and learning from systems," he said. "There has to be interactions in order for the system to learn."

Full text

Bitcoin Magazine

Coinbase Executive Says Clarity Act Has ‘Tremendous Momentum’ in the Senate

Coinbase Vice Chair Ryan VanGrack said the Clarity Act has gained “tremendous momentum” in the Senate, in a CNBC “Squawk Box” appearance that made the case for a federal crypto framework and touched on bitcoin, blockchain, and the industry’s uneasy truce with Wall Street.

VanGrack, a former SEC official, framed the Clarity Act as an overdue set of rules rather than a giveaway. “It’s not about no regulation,” he said. “This is about imposing regulation on the industry for the first time.” He described a “win-win-win” for American investors, innovators, and standards should the measure pass, and said a bipartisan group of senators has kept up work “even in the last few weeks and days.”

Clarity Act updates

The House passed its version of the Clarity Act last year, and attention has shifted to the Senate, where the path to 60 votes remains the central hurdle.

The Senate Banking Committee advanced the bill in a 15-9 vote this spring, with two Democrats crossing over, and House members have urged the Senate to act before the August recess. The measure sits in a narrow window as negotiators work out remaining terms.

President Trump added his voice last week, posting on Truth Social in support of Senator Lindsey Graham and calling on the Senate to pass the bill. Trump framed the stakes in terms of competition with China, a message he has repeated as he presses the chamber to move .

VanGrack said Democrats have won concessions that strengthen the bill’s consumer protections.

JUST IN: Coinbase Vice Chair talks CLARITY ACT on CNBC

"The Democrats have obtained meaningful concessions to make what was already a strong consumer protection bill, THAT much stronger" pic.twitter.com/y3n04dKdRi

— Bitcoin Magazine (@BitcoinMagazine) July 20, 2026

He pointed to an illicit-finance framework, an “FTX loophole” that the text would close, insider-trading safeguards, and added disclosures.

“Across the board, the Democrats have obtained meaningful concessions to make what was already a strong consumer protection bill that much stronger,” he said.

He said the bill would not change how crypto is classified as a commodity or a security in a fundamental sense, and would preserve the registration, examination, and surveillance structure from the House version.

Asked how the industry reconciles with skeptics like JPMorgan chief Jamie Dimon, VanGrack pointed to a wave of bank and institutional deals.

“Not a week goes by,” he said, where a firm fails to announce a new crypto project or investment. He predicted an “inevitable convergence,” a point at which the market stops separating traditional finance from crypto and treats each as a modern financial institution.

That convergence has played out in public, and in conflict. JPMorgan and Coinbase announced a partnership to widen crypto access, and the bank has moved to accept bitcoin as loan collateral and to let clients trade it .

Dimon, for his part, has declared war on the Clarity Act and aimed a crude insult at Coinbase CEO Brian Armstrong, a reminder that the détente carries friction.

Is bitcoin real?

The interview turned to a sharper question from CNBC’s Andrew Ross Sorkin: whether blockchain is real but bitcoin is not. VanGrack called it “a fair question” and said the technology’s benefits stand on their own — faster settlement, more transparency, and round-the-clock transactions.

He argued that no one building a financial system today would recreate the infrastructure of the past century. He cited Citadel Securities, which he said made another large investment in the crypto economy last week, as a sign that major institutions are trending the same course.

Sorkin pressed the harder edge of the design: the technology aims to remove the counterparty a customer might call when something goes wrong. VanGrack conceded the point as fair, then countered with the costs of the current system — days to reconcile trades and the counterparty risk that delay creates.

“I’m not here to tell you it’s the wrong technology,” he said. He acknowledged open questions, including whether crypto accounts should carry interest or loyalty rewards, a debate that bankers have raised and that the law will settle as “a blunt instrument.”

He closed on the case for Clarity Act passage. “In the absence of clarity, you do not have a federal oversight and framework,” he said. “So whether you love crypto or hate crypto, you should want” the Clarity Act.

This post Coinbase Executive Says Clarity Act Has ‘Tremendous Momentum’ in the Senate first appeared on Bitcoin Magazine and is written by Micah Zimmerman .

Full text

After using the James Webb Space Telescope (JWST) to peer back toward the beginning of time, astronomers have proposed a radical new explanation for one of the universe's most flummoxing phenomena.

Little red dots (LRDs) are mysteriously compact, brilliant celestial objects found predominantly when the universe was less than 10% of its current age.

As revealed by JWST's unmatched infrared sensitivity, LRDs emerged incredibly early, only around 600 million years after the Big Bang , and then began disappearing about a billion years later.

Now, in a paper published in The Astrophysical Journal Letters , astronomers have proposed a novel formation mechanism for LRDs: They may be birthed by previously undiscovered celestial companions, whose intense ultraviolet (UV) radiation causes gas clouds to collapse into incredibly dense and exotic objects, like " black hole stars ."

An illustration of a black hole star, or quasi-star, powered by a black hole surrounded by a cocoon of gas. (Image credit: (MPIA/HdA/T. Müller/A. de Graaff) via Wikimedia Commons) "The most surprising aspect… is that these little red dots are not just 'red dots,' but there is a more complex emission nearby and around them," Josephine Baggen , an astronomer at Yale University and first author of the study, told Live Science in an email. "We think those what we call 'companions' are starlight."

Connecting the dots

In the study, the researchers compiled a sample of 83 LRDs, imaged with JWST, from ultradeep surveys. They found that 36 of the 83 LRDs, including over 80% of the brightest ones, hosted at least one companion that shined bright in blue-ish ultraviolet light; little red dots with little blue companions.

The LRDs sampled in this study. (Image credit: (Baggen et al., ApJL, 2026) ) These companions had masses ranging from hundreds of millions to billions that of the sun, suggesting that they may be star clusters or relatively small early galaxies , Baggen told Live Science.

These UV-spewing companions spurred the formation of LRDs from immense gas clouds, the team's new proposal states. Normally, cold molecular gas clouds fragment and condense into stars. But the intense UV irradiation from the companions halted the fragmentation process, potentially squeezing the gas clouds into supermassive stars that directly collapsed into black holes while skipping the explosive supernova stage that normally marks these stars' deaths.

This proposal helps to explain why LRDs appear bright in red optical light and UV wavelengths, with a dip between the two caused by the wavelengths of light absorbed by hydrogen gas. Rather than originating from a single object, the optical red light derives from the LRD, the UV light comes from its nearby companion, and the dip is thought to be caused by the dense cocoon of gas around the LRD.

As a result, the researchers suggested that all LRDs may have such partners but they may be too close together to be distinguished. Conversely, some companions may be separated by greater distances than the researchers accounted for in this study, requiring future observations to zoom out.

Galactic potential?

Related stories

Black hole butterflies? James Webb telescope spots dozens of black hole 'cocoons' in early universe.

The James Webb telescope found hundreds of 'little red dots' in the ancient universe. We still don't know what they are.

'Not so exotic anymore': The James Webb telescope is unraveling the truth about the universe's first black holes

Intriguingly, this work may illuminate a couple of early-universe enigmas. First, the black holes manifesting from these interactions may be between 100,000 and 1 million solar masses, forming the "seeds" necessary to explain how ancient supermassive black holes grew so surprisingly massive so early in cosmic history .

Additionally, because LRDs may merge with the relatively small, UV-spewing galaxies that birthed them, this process may have created the impressively immense galaxies all around us today.

"We do think that this might be the birth of the supermassive black holes around these UV companions, born 'outside the galaxy,' [which] will eventually merge," Baggen told Live Science. "Whether this is what happened to the Milky Way in its earliest phases, we cannot really say, but it is plausible! There is still a lot of debate about how LRDs evolve and what they turn into at later times."

Full text

70.8% of evening instant repliers were promoted in the last two years

Promotion is 1.3x more likely for instant replies than those who wait until next morning

Millennials get the biggest rewards for fast replies in research published by workforce solutions company Careerminds UK

Achieving a work-life balance by disconnecting from work on an evening may be good for your peace of mind and family life, but it might just cause you to miss out on a promotion, a new report has claimed.

Research conducted for Careerminds UK found replying to that late evening message from your manager before you hit the sack can increase your chances of a desirable new role.

Its report found 7 in 10 (70.8%) respondents who admitted to being “instant repliers” got a promotion in the last two years, far more than the cohort that opted for an approach of “let me live my life for a few hours.”

Millennial men message more

Elsewhere, the research is split, with Millennials far more likely to get promoted for evening communication than Gen Xers. This report emerges just months after the British government’s plans for a “Right to Switch Off” was put on hold.

Careerminds’ research found that, essentially, “The slower the reply, the lower the promotion rate.” Replying to out of hours messages immediately is a 70.8% promotion rate; replying within the hour drops the rate to 68.2%, while replying next morning is 55.1%.

There is a gender split, too, with 79.0% of those quick responders in the last two years listed as men, compared with 60.8% marked as women.

Also of interest is the generational difference. The research shows 86.0% of millennials with a fast response for responding to messages out of contracted hours were promoted, compared with 47.2% of Gen X employees (44 to 59). Gen Z employees gained promotion at a rate of 81.2%.

Interestingly, the quick-response-for-promotion effect doesn’t seem to have any significant advantage for entry level or director suite roles. Instead, it appears to be focused around the middle of an organization’s hierarchy.

What happened to the “Right to Switch Off”?

Intended as a workplace reform to feature in the British government’s Employment Rights Bill, the Right to Switch Off was supported by unions, employees, and even business leaders. However, when the bill was announced, this key aspect was omitted, relegated to the government’s “Next Steps” document.

Unfortunately, subsequent reports indicated that the policy would be scrapped, with a government source telling The Sunday Times that the policy was “dead.” The reasoning? Reducing economic pressure on businesses including “unnecessary barriers” in order to make “Britain the best country in the world in which to do business.”

For comparison, Ireland has had a similar rule for employees to ignore out-of-hours work messages since 2021, and France introduced the right to disconnect in 2017.

With the UK welcoming a new Prime Minister as of July 20, 2026, potentially with different priorities, it remains to be seen whether the Right to Switch Off can be switched back on.