Your privacy, your choices

We use necessary features to run the website and platform. With your permission, we also measure usage and performance and use marketing cookies for advertising and remarketing. Your choice applies in this browser across the website, signup and platform. Cookie policy

Closing this notice or pressing Esc lets you continue with necessary features only.

Skip to content

Resources · Answer quality

1st in answer completeness. 2nd overall.

Ahead of the evaluated configurations from OpenAI, Amazon, Microsoft, Google and NVIDIA, in a public test with over 500,000 documents. Leaderboard of September 15, 2026.

Talk to a specialistTry it free
See how it works
Skyller · Freight contract · Vale Sul Transport
Skyller - AI AssistantFreight contract · Vale Sul Transport

What are the terms of our contract with Vale Sul Transport? I need the term, the late-delivery penalty and the price adjustment.

…

Transport Contract no. 2025-118

Clause 4: 24-month term. Clause 7: annual adjustment by the inflation index in April. Clause 11: termination with 90 days' notice.

Amendment no. 2 · June 2026

The late-delivery penalty becomes 2%, capped at 10% of the monthly invoice.

Skyller is an AI and can make mistakes. Remember to check.

Responding

Demonstration · illustrative data

1ST IN ANSWER COMPLETENESS

87.39, the highest completeness score among the 22 systems

2ND OVERALL

79.30 points, 1.04 behind first place, on September 15, 2026

2ND IN RIGHT DOCUMENTS

86.50% found, practically tied with the best mark (86.55)

80 UNSEEN QUESTIONS

Onyx checked the result with them before publishing

The leaderboard

The most complete answers on the leaderboard.

Answer completeness measures whether the answer brings every fact the question asks for, checked one by one. None of the 22 systems brought as much as Skyller.

Answer completeness, 0 to 100 — positions on September 15, 2026
  1. Position 1Skyller1st87.39
  2. Position 2metor.com86.22
  3. Position 3Causal Dynamics Lab85.16
  4. Position 4Troml81.84
  5. Position 8OpenAI File Search67.87
  6. Position 9Amazon Q (Kendra)60.65
  7. Position 11Azure AI Search57.63
  8. Position 12Vertex AI Search55.45
  9. Position 16NVIDIA AI Blueprints45.20

The overall leaderboard, with every score.

The overall score combines correctness and completeness; there, Skyller ranked 2nd, 1.04 points behind first place. The 1st tag marks who leads each column.

  1. Position 1metor.com

    Overall score
    1st80.34
    Correctness
    82.0
    Completeness
    86.22
    Right documents
    85.53

  2. Position 2Skyller

    Overall score
    79.30
    Correctness
    81.6
    Completeness
    1st87.39
    Right documents
    86.50

  3. Position 3Causal Dynamics Lab

    Overall score
    78.95
    Correctness
    82.0
    Completeness
    85.16
    Right documents
    80.54

  4. Position 4Troml

    Overall score
    76.79
    Correctness
    1st83.8
    Completeness
    81.84
    Right documents
    1st86.55

  5. Position 8OpenAI File Search

    Overall score
    61.03
    Correctness
    69.8
    Completeness
    67.87
    Right documents
    71.65

  6. Position 10Amazon Q (Kendra)

    Overall score
    48.96
    Correctness
    55.4
    Completeness
    60.65
    Right documents
    70.38

  7. Position 11Azure AI Search

    Overall score
    48.42
    Correctness
    56.4
    Completeness
    57.63
    Right documents
    64.25

  8. Position 14Vertex AI Search

    Overall score
    41.87
    Correctness
    49.2
    Completeness
    55.45
    Right documents
    61.76

  9. Position 16NVIDIA AI Blueprints

    Overall score
    37.73
    Correctness
    59.6
    Completeness
    45.20
    Right documents
    72.61

A selection of the top four entries and five enterprise search systems, out of the 22 listed on September 15, 2026. Each row is the submitted configuration, with its models and settings. New submissions may change rankings; the full list is on Onyx's site.

Completeness · 87.39
1st place: nearly nine out of ten expected facts showed up.
Correctness · 81.6
Four out of five answers say the same as the expected answer.
Right documents · 86.50
2nd place: 86.50% of the needed sources; the best mark is 86.55.

How to interpret the ranking

The ranking compares published submissions to this test, which brings together products, technical tools and research projects — not the whole market — and does not certify commercial terms. Each column is a measurement of its own, published side by side by Onyx.

Day to day

What a complete answer solves.

An incomplete answer looks right and leaves out the detail that changes the decision.

Illustrative examples, with the rules and documents of a fictional company.

Finance

“What are the terms of our contract with the carrier?”

Incomplete answer

24-month term and a 1% penalty per late delivery.

Complete answer

  • 24-month term, until March 31, 2027
  • 2% penalty per late delivery, capped at 10% of the monthly invoice — the June amendment replaced the original 1%
  • Annual adjustment by the inflation index, every April
  • 90 days' notice to end it without a penalty

Sources: Transport Contract no. 2025-118 · Amendment no. 2 · June 2026

What it prevents: charging the wrong penalty — or failing to charge the right one.

Sales

“What discount can I give a wholesale customer?”

Incomplete answer

Up to 8%.

Complete answer

  • Up to 8%, under the Sales Policy in force since June 2
  • The 12% cap is still awaiting approval and applies only once published
  • Above 8%, only with the sales manager's approval

Sources: Sales Policy · version 4, published

What it prevents: closing a deal outside the rules, or losing it without knowing the exception.

People

“What do I need to do to take time off in December?”

Incomplete answer

Request it 45 days in advance.

Complete answer

  • Request it 45 days in advance, in the HR system
  • People working on the accounting close cannot take leave in the last week of the year
  • The request only counts once your manager approves it

Sources: Leave Policy · version 2 · 2026 Closing Calendar

What it prevents: the request coming back denied after the trip is booked.

Operations

“A shipment arrived damaged. What do I do?”

Incomplete answer

File a complaint with the carrier.

Complete answer

  • Photograph the load before unloading
  • Note the damage on the delivery receipt, in front of the driver
  • Notify the carrier within 24 hours
  • Open the incident card on the Kanban, with an owner and a deadline

Sources: Goods Receiving Procedure · SOP-031

What it prevents: missing the deadline and losing the right to charge the carrier.

The evaluation

An entrance exam for company assistants.

Maintained by Onyx, which sells a product of the same kind and excludes itself from its own leaderboard. The corpus, questions, grader and leaderboard history are public.

500K

A whole company, invented

Slightly over 500,000 documents from nine origins: messages, email, tickets, files, CRM, meetings, code, support and wiki.

The volume of a real company

ON PURPOSE

Messy like real life

Misfiled documents, near-duplicates across systems, outdated versions and documents that contradict each other.

It measures retrieval in everyday disorder

500

Ten types of question

From a single-source question to one that pulls a whole project together, with conflicting information and twenty with no answer in the corpus.

Including the question with no answer

INDEPENDENT

Onyx does the grading

Each answer is compared with the expected one and with its list of facts; three independent evaluations classify the documents.

No step of the grading goes through us

What was tested

The submission ran on the live product.

The 500 answers came from Skyller's production conversation, the same one that serves companies. Models and settings are in the submission record; product changes call for a new evaluation.

  1. Starting point

    The literal question, in a new conversation, with no category, no expected answer, no correct documents and no prior context.

  2. What happened on each question

    1. 01The model that wrote the answer is Claude, by Anthropic; the evaluation measures Skyller's work around it: finding the documents that hold the answer.
    2. 02Retrieval searched 510,646 documents through the same path as a customer conversation, and Onyx can rerun any question at the public address.
    3. 03Onyx re-graded the 500 answers and checked the result with 80 unseen questions before publishing. The check passed.
Skyller · Wholesale discount · policy in force
Wholesale discount · policy in force

What is the maximum discount I can give a wholesale customer?

…

Sales Policy · version 4, published

Maximum discount for wholesale customers: 8%. Above that, sales manager approval.

Skyller is an AI and can make mistakes. Remember to check.

Responding

Demonstration · illustrative data

Where the test stops

  • The corpus is in English and belongs to a fictional company; the proof that counts for yours is a test with your documents.
  • The environment was tuned for this corpus; with other documents, languages and settings, quality needs to be measured.

Values from the submission's technical record, stored with the answers.

When it's time to decide

How to use this in your evaluation.

  1. 01Who will use it

    Ask each tool the same hard question, ideally one whose answer sits in two documents that contradict each other.

  2. 02Who decides

    Ask for the date and source of every quality figure; ours is on Onyx's leaderboard.

  3. 03Who runs IT

    Ask whether the test ran on the live product; ours did, and whoever published it checked.

Where to check

Every number on this page has an address.

The four addresses are public and maintained by Onyx.

Questions

What people usually ask.

Get started

The proof that decides is with your documents.

The leaderboard shows retrieval working in a large, messy corpus, judged by someone who does not sell Skyller. With your own documents, the test is free.

Talk to a specialistTry it free