Skip to content
Pioneer Page PP markPPPIONEERPAGE

Pages from the advancing edge of computing.

Foundation models

Experts struggle to measure AI progress as capabilities reportedly double

Rapid AI development outpaces current measurement methods, leaving researchers unsure how to track the speed of capability gains.

Experts struggle to measure AI progress as capabilities reportedly double
Depth
0 outlets · newest source 416h ago

Key points

  • AI capabilities may be doubling every few months according to some researchers.
  • Current measurement tools are struggling to keep pace with development speed.
  • Experts building the technology find it difficult to quantify progress accurately.

Artificial intelligence development is accelerating so rapidly that even the engineers creating these systems are having trouble measuring their actual progress, according to a report by Bing News.reported This lack of clear metrics complicates efforts to understand the true scale of recent advancements.Computer experts say AI is getting better so fast that they can't keep track of how much it is improving.The discrepancy between development velocity and evaluation granularity is creating significant uncertainty in capability tracking.

How we got here

The speed of artificial intelligence advancement has reached a point where traditional evaluation methods appear insufficient, as noted in the Bing News report. Researchers in the field are observing changes that outstrip standard benchmarking approaches.

What is new

Some researchers believe that AI capabilities are currently doubling every few months, a claim highlighted in the recent coverage. This rapid growth rate suggests a significant shift in how quickly models can acquire new skills or improve existing ones.

Reading between the lines

The inability to accurately measure progress implies that public understanding of AI risks may be lagging behind reality. If experts cannot quantify the speed of improvement, it becomes harder to predict future milestones or potential safety concerns.

The open questions

The central issue is whether AI capabilities are truly doubling every few months or if this perception stems from improved reporting. The sources acknowledge that experts are struggling to measure progress, meaning the "doubling" claim remains an estimate rather than a verified metric. There is no independent evidence cited to confirm the specific timeline of capability gains.

Why it matters

  • Model evaluators need to understand that current benchmarks may not capture the true rate of AI improvement.
  • The lack of standardized measurement tools could lead to underestimating the speed of capability gains.
  • Researchers must account for the possibility that AI progress is faster than officially reported metrics suggest.
  • Uncertainty in measurement affects how risks associated with rapid development are assessed and communicated.

Topics: AI development, measurement, benchmarks, research, progress

Updated

The strongest doubt

Counterpoint from the sources

The report does not provide counter-evidence to the claim that capabilities are doubling; it only notes the difficulty in measuring this trend.

Ask the article

Local keyword search of this article; no generated answers.

Did this article give you the evidence you needed?

No vote saved on this device.
Your choice stays on this device. No public tally.

Questions readers ask

Are AI capabilities doubling every few months?

Some researchers believe this is the case, but experts are struggling to accurately measure the progress to confirm it.

Who is reporting on the speed of AI development?

A report by Bing News highlights the concerns of researchers and experts in the field.

Can experts currently measure AI progress accurately?

No, the sources state that even the experts building AI are struggling to measure its progress effectively.

Connected stories

This storyEngadget review says Surface Laptop Ultra lags AppleEngadget review says Surface LapResearchAndMarkets Report Projects AI Clinical Trial Market Growth to $2.8 Billion by 2030ResearchAndMarkets Report ProjecStudy Finds Autistic Users Trade AI Support for Risk of Losing Personal VoiceStudy Finds Autistic Users TradeTom's Hardware benchmarks show Nvidia Vera CPU leads in Python and memory bandwidthTom's Hardware benchmarks show N

Reading receipt

  • AI capabilities may be doubling every few months according to some researchers.
  • Current measurement tools are struggling to keep pace with development speed.
  • Experts building the technology find it difficult to quantify progress accurately.
Time spent: 0:00 · Depth: Full

Sources

    Related stories

    Background reading