Claude Opus 4.8 Costs 57.1x More and Loses All Five Benchmarks.
What Beat It Was Not a Model, but the Harness.

I. $0.175 vs $10 — Winning All Five

Floatboat’s test base this time is DeepSeek-V4-Flash, available to anyone via the official API: $0.14 per million input tokens, $0.28 per million output. At the 3:1 input-to-output ratio typical of agent scenarios, that’s a blended price of $0.175/M.

Across the table sits Claude Opus 4.8 at $5 / $25 — a blended $10/M. 57.1× more expensive.

All five were surpassed.

“Better” is a hard threshold for Agents—if a workstation cannot deliver real tasks, no one will open it every day no matter how cheap it is. In the past, clearing this threshold required paying more: those capable of doing the work were too expensive, and those cheap enough to use freely could not do the work.

What this data is saying is that this trade-off can be eliminated. And what eliminates it is not the model.

II. This Is Not Another Case of “Cheap Models Are Surprisingly Good”

The first reaction to this kind of price contrast is usually the old story of a budget model catching up to a flagship. This time, the data itself refutes that reading.

The same DeepSeek-V4-Flash, running on DeepSeek’s own official Harness, scored just 54.4 on DeepSWE — below Opus 4.8’s 58.0. Put it on Floatboat, and it reached 67.25.

A fuller comparison makes the point even sharper: on Terminal Bench 2.1, DeepSeek’s official Harness tied Opus 4.8 (82.7 : 82.7) — and lost on every one of the other four.

In other words, the same model, inside the model provider’s own system, didn’t beat Opus 4.8 on a single benchmark. Plugged into Floatboat, it won all five.

The model weights didn’t change. The unit cost didn’t change. There is only one variable.

III. A Single-Variable Experiment Deliberately Run on the Cheapest Model

It is worth laying out the methodology in full first, because the legitimacy of the entire conclusion rests on the four words “single variable.”

The control configuration disclosed by Floatboat is as follows:

• Both sides uniformly used the DeepSeek-V4-Flash 0731 model base;
• The Code Agent tasks in DeepSeek’s official public benchmarks used the minimalist mode of DeepSeek Harness (to be released soon), configured at the max setting, top_p=0.95, temperature=1.0;
• On the Floatboat side, execution was uniformly carried out under the Floatboat Evaluation Harness, with model inference for most benchmarks provided by DeepSeek’s official API;
• All tasks ran independently in fully isolated, out-of-the-box physical sandbox environments, with completely identical input parameters.

There’s an easily missed detail here: the control group is not the “raw model API” — it’s the model provider’s own Harness. The five numbers 54.4, 73.2, 82.7, 25.1, 70.7 are the scores of DeepSeek’s official system. The provider’s own engineering capability is in the room.

That makes this comparison far stricter than “product vs raw API” — and makes the gap much harder to explain away as “the other side didn’t tune it properly.”

There is also a deliberately conservative choice: the control uses the 0731 official release, rather than an earlier Preview checkpoint. If Preview were used, the DeepSWE item would be 7.3 → 67.25, an increase of 821%—a much prettier number. But that would mix in iterations from the model’s own post-training, which cannot be attributed to the Harness. This row of data is also in the official master table, but Floatboat did not use it as a promotional framing.

As for why the cheapest model was chosen, the reason works in reverse: running the leaderboard with a top-tier model would produce “high scores + unclear attribution,” and no one could tell whether the scores came from the model or the Harness. Locking in the same cheapest base, connecting one side to the official Harness and the other to Floatboat Harness, means the difference can only be attributed to the Harness. This is a more stringent condition for proof, not an easier one.

(Floatboat also issued another official clarification: the product itself is model-neutral and supports integration with various mainstream models; the single base used this time was only a requirement of the evaluation methodology.)

IV. The Real Information Is in the Ordering

If you only look at “all five items were won,” this is still just a benchmark run. What carries information is the distribution of the gains.

Pull out the same-base Harness deltas on their own:

1.9% → 9.6% → 12.6% → 19.9% → 23.6%.

All five benchmarks have official harness controls. Ordered by task horizon from short to long, the tiers are non-decreasing and the gains rise monotonically — no exceptions. The longer the task horizon, the bigger the Harness gain, climbing all the way to 23.6%.

(One point of framing needs to be made clear: among the five items, BrowseComp is the only one for which the official source did not disclose the average number of steps. Its “medium-to-long-horizon” classification is a conclusion based on the task design—multi-hop retrieval, no direct answer available from the first screen of search-engine results, an a priori unknown retrieval-chain length, and a need for cross-source evidence verification. The number of retrieval and verification rounds in a single task is on the same order as Toolathlon. It and Toolathlon both belong to the medium-to-long-horizon tier; within the same tier, they are ordered by gain, and their sequence does not represent a difference in task horizon.)

The longer the task chain, the more links it has, and the more it depends on cross-step state maintenance and self-correction, the greater the Harness gain; the shorter the task, the more single-point it is, and the closer it is to a one-shot Q&A, the smaller the gain.

And real work happens to be the long-horizon kind. What users deal with every day isn’t a single command; it’s work that spans files, spans apps, spans many steps, with changes of mind along the way. This pattern points precisely at where the Harness operates in real work.

V. None of the Five Benchmarks Were Set by Floatboat

The weight of all the numbers above depends on who wrote the questions.

On BrowseComp—a leaderboard created by OpenAI itself—Floatboat’s 87.80 surpassed GPT-5.6 Terra (87.5, 25.7× more expensive), Claude Sonnet 5 (84.7, 22.9× more expensive), Opus 4.8 (84.3, 57.1× more expensive), and GPT-5.6 Luna (83.3).

“113 completely leak-free code repositories, with an average of 7 files modified and 668 lines added” and “47 SaaS products, nearly 500 API endpoints, with stale data mixed in”—tasks like these cannot be solved with prompt tricks. They require real file access, multi-step debugging, cross-application state maintenance, and the ability to discover and correct errors on their own after something goes wrong.

This also explains why the gains are concentrated in long-horizon tasks: short-horizon tasks test the quality of the model’s single output, while long-horizon tasks test the entire execution system.

VI. HLR: Giving the Harness a Measuring Stick

Floatboat draws the Harness boundary in one sentence:

“Floatboat is turning frontier model capabilities into leading Agent products. All the work involved other than the model itself falls within the scope of Harness.”

The companion metric is called HLR (Harness Leverage Ratio). The question it asks is actually a simple one: for the same budget, is it better spent on switching models or on switching the execution system? (Price is only used to decide which model serves as the reference; it does not enter the formula itself — numerator and denominator are both score differences on the same benchmark.)

Written as a fraction anyone can understand:

The numerator is how much the score rises when the model is kept unchanged and only the Harness is swapped out; the denominator is how much the public score rises when the Harness is kept unchanged and the model is upgraded to a stronger reference system.

So HLR > 1 has a direct meaning: the gain from switching only the Harness already exceeds the entire public performance span represented by upgrading to that reference model.

Substituting DeepSWE:

• DeepSeek official system: 54.4
• Claude Opus 4.8 published score: 58.0, a difference of only 3.6 points between the two public systems
• The same DeepSeek model placed on Floatboat: 67.25, gaining an additional 12.85 points

Without changing the model, and changing only the Harness, it obtained 3.6 times the public performance span represented by “upgrading to the 57.1 times more expensive Opus 4.8.”

For readers who want the precise definition, the full form is

HLR(B) = Δ(Harness) / Δ(Model Leap)

   = [S(B, M, H₁) − S(B, M, H₀)] / [S(B, Mᵣ, Hᵣ) − S(B, M, H₀)]

There is a rule for choosing the reference system that is stated upfront, and this determines whether the metric can be trusted: the primary reference is uniformly Claude Opus 4.8; if it is not higher than the baseline system on a given item, then no model leap has occurred on that item, and the cheapest valid reference for that item is used instead; the cheapest one is used rather than the highest-scoring one because the denominator is meant to measure a real, lowest-cost model upgrade—the path a rational user would actually choose. Set the rule first, then take the numbers.

Note: All scores must come from the same evaluation protocol. HLR is computed per benchmark and never averaged across benchmarks.

Update, August 13 — a new reference model: DeepSeek V4 Pro 0813

The day after this report was published, DeepSeek switched its API to V4 Pro 0813 — same family as Flash, 3× the price, positioned for heavier agent tasks. A new model leap means a new HLR denominator, so we recomputed the comparison against it.

[Figure 5A] 🆕

The pattern matches what we reported against the Opus 4.8 reference: harness leverage is a long-horizon story. Short tasks are close to saturated — the remaining headroom belongs to the model. Long tasks are where the execution system compounds. One honest caveat: the strict monotonic ordering observed against Opus 4.8 (0.78× → 3.57×, no exceptions) does not reproduce at the tail against Pro — Toolathlon (medium-long) edges out DeepSWE (long). Two readings are possible: Toolathlon's official-harness baseline leaves more recoverable headroom, or DeepSWE is precisely the benchmark DeepSeek optimized hardest for this release. We report the numbers as they are.

And the price frame still holds. Pro costs 3× Flash. On DeepSWE, Floatboat-on-Flash scores 67.25 against Pro's 62.7 — better, at one-third of the price, without touching the model.

0.78× → 1.14× → 1.32× → 1.62× → 3.57×. The sequence matches the horizon pattern: ordered from short to long horizon, the tiers are non-decreasing and HLR rises monotonically, no exceptions. The longer the horizon, the higher the Harness leverage.

On this item the reference system was swapped — and the reason for the swap is itself a conclusion: Opus 4.8 scores 82.7 on Terminal Bench, identical to DeepSeek’s official Harness. Switching to a flagship model that costs 57.1× more didn’t move this benchmark’s public score by a single point; keeping the model and switching only the Harness added 1.56 points. Since no model leap occurred, it cannot serve as the reference here. By rule, the cheapest valid reference, GPT-5.6 Luna (84.7, 2.6×), is used instead, giving 1.56 / 2.00 = 0.78×. Using Claude Opus 5 (83.8) would have produced 1.42× — Floatboat did not adopt it, because the rule fixed a single value in advance and cherry-picking among candidates is not allowed. The full HLR range across all valid references for this item is 0.26×–1.42× (Sol 0.26× | Terra 0.33× | Qwen3.8-Max 0.40× | Luna 0.78× | Opus 5 1.42×), and Floatboat published the entire range along with it.

HLR is a new metric proposed in this release, not an industry standard; changing the reference system changes the value. But its numerator and denominator are all public scores from third-party benchmarks, and the reference-selection rules are public too. Anyone can recompute it — or turn it around and audit Floatboat with it.

VII. Where the 12.85 Points Come From: How Long-Horizon Tasks Actually Fail

Techniques like prompt engineering, context compression, and tool reordering typically yield gains on the order of 1–3 points. To explain a gap an order of magnitude larger, you first have to answer a more basic question.

Floatboat’s answer: long-horizon tasks usually fail not because “some step went wrong,” but because the loop doesn’t converge — it drifts further and further from the real bar for delivery.

Drift has two causes, one on the human side, one on the model side.

First: the bar for delivery is never fully stated at the start of a task. Real work is not a textbook problem with a standard answer: human intent often becomes clear only as intermediate artifacts appear, and many critical criteria — whether the tone is right, whether the plan can actually land, whether the structure matches a real decision process, whether the result is ready to hand over — don’t exist at all in the first sentence.

Second: the model’s own drift gets amplified turn by turn in long-horizon tasks. This is an inherent property of probabilistic generation: every turn’s output carries a slight deviation, and the next turn must take the previous turn’s product as its premise. In short tasks this deviation is invisible; twenty steps later it compounds into a directional error — the original instruction in the context is gradually squeezed out, diluted, rewritten — and the model keeps running confidently toward a target that has already deformed. Drift doesn’t need errors to be fatal: every step looks reasonable, but together they are not the thing to be delivered.

Stack these two together and you have the real condition of long-horizon tasks: the target itself is fuzzy and shifting, while the model carrying the target keeps drifting. Without a system layer continuously pulling it back, the longer it runs, the further it ends up from the bar for delivery.

This is where their definition of “Proactive” points.

Today, quite a few products call scheduled wake-ups and automatic execution of preset actions “Proactive Agent.” In Floatboat’s framing, that’s a trigger being proactive, not an agent being proactive. A true Proactive Agent OS is not about whether the system can start working when nobody presses a button. It’s about this: when the user states only a fuzzy goal — when even they don’t fully know what “good” looks like yet — can the agent keep exploring, executing, observing, correcting, and self-reflecting through a long-horizon task, actively recognizing the gap between the result and the goal?

In their own words:

The advanced form of proactivity is not executing on a timer for the user — it’s helping the user discover what they truly want, and getting it done.

This judgment also explains that horizon-ordered sequence of numbers: the longer the task, the more unstated criteria there are and the more model drift gets amplified — so the greater the value of “continuously closing the gap.” When a task is as short as a single Q&A, there is neither room to close any gap nor enough steps for drift to accumulate. The distance between 23.6% and 1.9% measures exactly this.

Why the Entire Stack Has to Be Yours

“Continuously closing in on the delivery bar” sounds like a design philosophy, but landing it requires four layers of capability to hold simultaneously. At a time when the industry is largely building wrapper apps on the Claude Agent SDK and shipping a demo in a few weeks, AOE Tech Labs chose to build its own Runtime, Agent Loop, Tools, and Infra — plus FloatSail (Evolution System). That path is much slower, much heavier, and showed no visible return at the time.

What each of the four layers determines maps exactly onto the four ways long-horizon tasks die:

Runtime determines what the agent can touch — processes, the file system, permissions, sandbox boundaries. DeepSWE requires modifying an average of 7 files and adding 668 lines of code; AutomationBench requires traversing 47 SaaS apps and nearly 500 API endpoints. The premise of these tasks is real read/write and execution privileges — not passing text in and pulling text out.

Agent Loop determines whether a long-horizon task can converge. It doesn’t mean mechanically running the model a few more rounds; it means making every round of exploration shorten the distance between the current result and the delivery bar: gaining new information from execution, spotting deviation in observation, correcting course from failure, reverse-inferring from intermediate artifacts what the user never spelled out; rolling back when it’s time to roll back, asking when it’s time to ask.

Both kinds of drift described above have to be pinned down at this layer. For fuzzy goals, it’s responsible for asking out and testing out the criteria round by round; for model drift, it’s responsible for not letting deviation survive the night. This means the goal anchor must be re-aligned every round rather than left to dilute in the context — what information must stay in the window verbatim, what can be compressed, what should be dropped; judge wrong and drift starts compounding. It also means the system must be able to verify its own intermediate products, pressing deviation back down while it’s still just deviation, instead of waiting for it to grow into a directional error. A loop that only runs forward gets further wrong the longer it runs; a loop that looks back to check can still be heading for the original goal at step 20.

The kind of “proactivity” described above physically lives in this layer.

Tools determine whether the model can correctly consume results. Tool granularity, return structure, error semantics — a tool that wraps a failure into an empty string will let the model make a confident wrong decision at step 15. Problems like this are invisible in single-turn Q&A; in a 20-step task they’re fatal.

Infra determines whether failure can even be found. State persistence, concurrency, observability — if the intermediate state of a long execution can’t be traced back, you don’t even know “which step broke,” and optimization has nowhere to start.

If any one of the four layers belongs to someone else, your ceiling is their ceiling. When the SDK’s Agent Loop loses context at step 20, a wrapper developer can only swap in another prompt and route around it; the team that owns the stack can change the loop itself. That’s the difference between “can fix” and “can’t fix” — not between “fixes well” and “fixes poorly.”

FloatSail (Evolution Engine) solves the time dimension: models rotate through several generations a year, and the Harness can’t be rewritten each time. It lets the Runtime, the Loop, the Tools, and the model-adaptation strategy keep evolving on real-task feedback — and carries accumulated system capability forward when new models arrive.

“Technically, only a system you fully control can harness a highly unpredictable model — and maximize its intelligence.” — AOE Tech Labs team

Back to the opening judgment — why, beneath every product-form innovation, what moved was the Harness: this technical judgment is directly tied to team composition. The founder brings 10 years of OS + AI entrepreneurship and product experience at hundred-million-user scale; of the two technical co-founders, one has model-training experience and the other has deep OS-stack experience. With model training, OS internals, and hundred-million-scale product engineering converging in one team, they naturally treat the agent as a systems-engineering problem, not a prompt problem.

Product strength and technical strength are not two separate things here: a team that can nail “feels great to use” had to hold the Runtime and the Loop in its own hands first.

VIII. Now “More”: The Leaderboard Ran on a Stripped-Down Environment

There’s a counterintuitive footnote to this report card: the environment it ran on is weaker than what users actually get.

Evaluations strip variables out for reproducibility; products stack capabilities on so the work gets done. The two were never the same thing.

The Floatboat Desktop that users actually run integrates the file manager, the file editor, and the browser natively into the product itself, and connects to 3,000+ online services and a range of multimodal models. Nearly every agent product on the market looks the same: you enter a chat box, pick a fixed folder as the project entry, open AI-generated documents somewhere else to edit them, and if the AI needs a browser you have to install a plugin or surrender the browser you’re using. In Floatboat, the overwhelming majority of work happens in one window — no file-picker uploads, no deciding where downloads go; drag and drop.

For an Agent, these are not interfaces, but callable tool surfaces: more available tools, shorter environment round trips, and a more complete execution loop. AOE Tech Labs confirmed that running the same set of benchmarks in the complete client environment produces scores higher than the figures disclosed in this article; the specific scores will be disclosed later.

“In product design, only by making it comfortable for people to use can we maximize human potential.” — AOE Tech Labs team

Viewed along that task-horizon pattern, this is internally consistent: the broader the tool surface and the more comfortable it is for people to use, the more tacit knowledge there is, and the greater the gains on long-horizon tasks. And real work is all long-horizon tasks. So the numbers on the rankings are a conservative lower bound.

The yet-to-be-disclosed difference may be more interesting than the rankings themselves. If HLR answered “how many points the Harness is worth,” this difference will answer the next question: how many points excellent human-computer interaction design is worth.

IX. “Faster” and “Cheaper”: Agents Will Only See Real Use When They Are Cheap Enough to Use Without Hesitation

Finally, returning to the cost side, its significance is not just saving money.

All five benchmarks measure task-completion quality; the materials contain no latency or time-consumption data, so no data claim about inference speed is made here.

“Cheaper” is an open ledger. A blended $0.175/M against $10/M; the beaten rival costs 57.1× more. Scan up the full price list, and the count of participating systems that are more expensive yet lower-scoring is: 9/9 on AutomationBench, 4/4 on Toolathlon, 6/8 on BrowseComp, 5/8 on DeepSWE, 5/9 on Terminal Bench.

And “faster” refers to the change in usage behavior that this price brings. When a long-horizon task costs next to nothing, re-running a run that went off the rails costs next to nothing either. Opus 4.8 costs 57.1× more, and many teams have to weigh whether a run is worth it before using it. When cost drops below the threshold where you need to weigh it, agents get truly used — not sparingly treasured.

The other half of “faster” sits on the human side: the round-trips the client eliminates — finding files, uploading, downloading, switching apps. Total turnaround time was never just time-to-first-token — how long it takes to find a file, whether the agent can read, write, and save directly, whether the browser and online services live in the same execution environment, whether a mid-task failure can roll back instead of starting over, whether a changed request can continue on existing context. Together, these are what “faster” precisely means in this material.

More, faster, better, cheaper — all four land together in this data for the first time: more task types improved, faster environment round-trips, better on all five benchmarks, and 57.1× cheaper.

Only when it’s both fast and good can an AI agent workbench truly enter daily work. Capable-but-too-expensive, affordable-but-incapable — this is exactly where agents have been stuck in the demo stage for so long. What this data shows is that the trade-off can be cancelled. And what cancels it is not the model — it’s the Harness.

X. You Can Run It Yourself Right Now

Every number above can be recomputed.

The entry point is floatboat.ai. AOE Tech Labs has simultaneously launched a DeepSeek-based edition — ¥5 / $1 for the first month — built on the very base model in this leaderboard, DeepSeek V4 Flash. You’re getting the same model.

The verification path is short: take the same model, and compare how it performs elsewhere versus inside Floatboat. Same model, different Harness — the difference is the answer.

Try it with long-horizon tasks, because that’s where the leverage is greatest — and where your daily work actually lives: have it modify a repo that needs to keep running, turn a pile of real source material into a deliverable report, or hand it a fuzzy idea and see whether it can help you think through the bar first.

Appendix I: Complete Overall Results Table

Count of participating systems that are more expensive but have lower scores: AutomationBench 9/9 | Toolathlon 4/4 | BrowseComp 6/8 | DeepSWE 5/8 | Terminal Bench 2.1 5/9.

Appendix II: Notes on Data Definitions

• Surpassing Claude Opus 4.8 on all five items is a comparison against this one system, Opus 4.8, and does not mean being first across all items. The rankings under the full comparison set are: AutomationBench 1/12, Toolathlon 1/7, BrowseComp 3/11, DeepSWE 4/11, Terminal Bench 2.1 5/12.
• GPT-5.6 Sol (64.3× more expensive) has higher scores on BrowseComp, DeepSWE, and Terminal Bench; Claude Opus 5 ($5/$25, 57.1× more expensive) is higher on BrowseComp (92.4) and DeepSWE (74.0), and lower than Floatboat on Terminal Bench (83.8) and AutomationBench (26.2). The main comparison arena in this article is Opus 4.8, and this does not represent a comprehensive surpassing of the entire Claude family.
• Terminal Bench 2.1 is the item with the lowest same-base increment among the five items (+1.9%), with HLR 0.78×; on this item, GPT-5.6 Luna (84.7, 2.6× more expensive) and Qwen3.8-Max (86.6, 17.1× more expensive) score higher than Floatboat, and no price-advantage claim is made for this item.
• For Toolathlon, the comparison set contains only 7 participating systems, so the persuasiveness of the ranking is limited; the evidentiary value of this item lies in the same-base increment of +12.6% (70.7 → 79.62) and the fact that all 4 more expensive participating systems have lower scores.
• The DeepSeek control results come from DeepSeek-V4-Flash-0731 + DeepSeek official Harness minimalist mode (54.4 / 73.2 / 82.7 / 25.1 / 70.7), not the raw model API and not the Preview checkpoint.
• AutomationBench uses the Public subset. The overall difficulty is extremely high, and all participating systems score below 31, so the statistical significance of the lead in the low-score range is limited.
• Prices were checked against officially visible public pricing as of 2026-08-07: DeepSeek V4-Flash 0731 is $0.14/$0.28, and V4-Pro Preview is $0.435/$0.87 (DeepSeek pricing page); Qwen3.8-Max is $2.00/$6.00 (Alibaba Cloud Bailian model page); GLM-5.2 is $1.40/$4.40 (Z.ai pricing page); Claude Opus 5 and Opus 4.8 are both $5.00/$25.00, Sonnet 5 is $2.00/$10.00, and Fable 5 is $10.00/$50.00; GPT-5.6 Sol/Terra/Luna are respectively $5.00/$30.00, $2.00/$12.00, and $0.20/$1.20 (OpenAI pricing page). The blended price is converted according to a typical 3:1 input-output ratio in Agent scenarios; the actual ratio varies with the task’s token structure.
• HLR is a new metric proposed here, not an industry standard; the reference system must be higher than the baseline system on that benchmark, changing the reference will change the value, and no averages are taken across benchmarks; price enters only the reference selection rule and does not enter the HLR formula. On Terminal Bench 2.1, the HLR range across all valid references is 0.26×~1.42×.
• The relationship between task horizon and gains is a consistent observation within the five samples in this study, and should not be directly extrapolated into a proven universal causal law. Among the five items, only BrowseComp has not officially disclosed the average number of steps, and its “medium-to-long-horizon” grouping is judged based on task design; it belongs to the same medium-to-long-horizon tier as Toolathlon, and in charts and tables the items in the same tier are ordered by gain / HLR, so their sequence does not constitute a claim of a difference in task horizon.
• Floatboat is a model-neutral product. The single base is used only to isolate pure Harness gains; gains on other model bases have not been tested, and no claims are made.
• The results in this article are taken from the evaluation environment. Results in the client environment are higher, but the specific scores have not been disclosed, and this article does not cite any client numbers.
• “Fast” is not a speed-benchmark definition: all data in this article concerns task-completion quality and does not include latency, throughput, or time-consumption benchmarks; no data claims are made about inference speed.
• In the results table, “—” indicates that the model was not evaluated on that item, not a zero score.
• 10. The August 13 update uses V4 Pro 0813 scores from DeepSeek's official model chart (Terminal Bench 2.1 87.9, AutomationBench 31.8, Toolathlon-Verified 74.1, DeepSWE 62.7); its BrowseComp score is not disclosed. Public harnesses / run configurations may differ across sources. HLR values against the Pro reference are computed with the same formula and are reported per benchmark, never averaged.

Appendix III: Task Horizon Reference

• Terminal-Bench 2.1: execution in complex terminal environments, about 7–35 interaction rounds.
• AutomationBench: cross-application business processes, about 13–22 steps / 30–44 calls.
• BrowseComp: continuous web retrieval, with the official source not disclosing the average number of steps (medium-to-long-horizon grouping judged based on task design).
• Toolathlon: multi-application long-horizon tool execution, about 20–26 rounds / 36–49 calls.
• DeepSWE: repository-level long-horizon engineering, about 61–268 steps.

Appendix IV: About AOE Tech Labs

AOE Tech Labs Limited was founded at the end of 2025, with HongShan and VLight Capital as seed-round investors.

• January 2026 — Released the Floatboat Desktop, pioneering the Agent desktop workstation category
• April 2026 — Released FloatIM, building a collaborative office network between people and Agents and among Agents
• May 2026 — Released FloatSchedule, enabling Agents to work proactively according to schedules
• August 2026 — Released the complete Floatboat Harness evaluation results across five benchmarks: once connected to Floatboat, the same DeepSeek-V4-Flash surpassed Claude Opus 4.8 on all five benchmarks and outperformed most top-tier models; at the same time, AOE Tech Labs proposed the Harness Leverage Ratio (HLR)

See the Full Report: https://floatboat.ai/news/harness-benchmark

Website: floatboat.ai

Contact: [email protected]