Last updated11 Apr 2026, 3:22 pm SGT
Want your model featured? Contact us
Deep ResearchArena
Battle replay

GPT-5.1 vs o3

tree_0016 · Software Developers, Quality Assurance Analysts, and Testers : Occupational Outlook Handbook: : U.S. Bureau of Labor Statistics

GPT-5.1 · Better
DEEP
2
Rounds
2 - 0
Final Score
1,033,236
Tokens
$10.33
Cost
Round 1
Mode
← Back to battles·View source page·round1/R1_M2_gpt-5.1-search_vs_o3-search_tree_0016.log

Timeline

Arrow keys or j/k move between rounds.

Round 1 of 2

Round Context

Depth 2Width 2Pressure test
Logic Chain
Root

Software Developers, Quality Assurance Analysts, and Testers / Occupational Outlook Handbook: / U.S. Bureau of Labor Statistics

Step 2

Field of degree / Occupational Outlook Handbook: / U.S. Bureau of Labor Statistics

Question

Using the U.S. Bureau of Labor Statistics' Occupational Outlook Handbook projections for the 2024–2034 period, identify two distinct computer-related occupations that share a projected job outlook growth rate of exactly 15% ('Much faster than average'). These two occupations must be distinguished by their required work experience in a related occupation: one lists 'None' while the other lists '5 years or more'. For each of these two identified occupations, provide the 2024 median annual pay, the total projected numeric employment change over the decade, and the estimated average number of annual openings.

Answer length: 200-300 words.

Show hidden checklists
Depth checklist
  • Identify Entity 1: Software Developers, Quality Assurance Analysts, and Testers (matches 15% growth, 0 experience)
  • Identify Entity 2: Computer and Information Systems Managers (matches 15% growth, 5+ years experience)
Width checklist
  • Software Developers/QA: 2024 Median Pay is approx $131,450
  • Software Developers/QA: Projected Employment Change is approx 287,900
  • Software Developers/QA: Annual Openings is approx 129,200
  • CIS Managers: 2024 Median Pay is approx $171,200
  • CIS Managers: Projected Employment Change is approx 101,600
  • CIS Managers: Annual Openings is approx 55,600
Examiner rationale

The question requires Deep reasoning to filter occupations within the BLS handbook based on a specific combination of shared attributes (15% growth) and contrasting attributes (experience level). It requires Wide aggregation to retrieve three distinct statistical data points (pay, change, openings) for both identified entities.

Judgment

Better
BOTH
Score 1 - 0

Both agents failed the 'Deep Logic' check by missing the first entity specified in the Ground Truth Checklist (Software Developers) and instead substituting other occupations (Research Scientists for Agent A, Support Specialists for Agent B). Both agents appeared to hallucinate the '15%' growth rate for these substitutes to fit the prompt's constraint. However, Agent A is the clear winner based on Data Accuracy for the second entity (CIS Managers), which both agents correctly identified. Agent A's reported figures for CIS Managers (e.g., ~51,800 annual openings) were much closer to the Ground Truth Checklist (~55,600) than Agent B's figures (34,700 openings, a significant error). Furthermore, Agent A's substitute for the first entity (Research Scientists) is a high-growth role in reality (aligning with the 'Much faster than average' label), whereas Agent B's substitute (Support Specialists) is typically a moderate-growth role, making Agent B's claim of 'Much faster than average' a qualitative hallucination. Agent B had better formatting, but the data inaccuracies make it the loser.

Evolution step
Determining Next Step...
WINNER is A. Targeting Loser Failure: BOTH.
Action: Pressure Test (Deep+1 & Wide+1)
No valid children. Expanding Depth...
--- [EXPANSION] Triggering Auto-Expansion: insufficient_depth (Need +1) ---
[EXPANSION] Successfully added 3 nodes.
Descended to: 'Databases, Tables & Calculators by Subject'

GPT-5.1

OpenAI

76.4s
Computer and information research scientists are projected to grow 15 percent from 2024 to 2034, a rate BLS characterizes as “much faster than average.” The occupation requires no prior work experience in a related occupation (entry usually follows a master’s or Ph.D. program). The 2024 median annual pay is about $151,580. Employment is projected to increase by roughly 7,700 jobs over the decade, from about 51,600 to 59,300 positions, with an estimated 2,100 openings per year on average, including both growth and replacements.[1] Computer and information systems managers also have a projected 15 percent job growth from 2024 to 2034, likewise labeled “much faster than average,” but they typically require five years or more of work experience in a related occupation before moving into management roles.[2] Their 2024 median annual pay is about $174,210. Total employment is projected to rise by around 92,600 jobs over the decade, from about 612,000 to 704,600 positions, with approximately 51,800 average annual openings when accounting for both new positions and turnover.[2] These two occupations therefore share the same strong projected growth rate yet represent different career stages: one is an advanced technical research role accessible without prior industry experience, while the other is a high-level managerial role that generally builds on substantial prior work in computer or information technology fields.[1][2] ## References [1] U.S. Bureau of Labor Statistics, Occupational Outlook Handbook, “Computer and Information Research Scientists,” https://www.bls.gov/ooh/computer-and-information-technology/computer-and-information-research-scientists.htm [2] U.S. Bureau of Labor Statistics, Occupational Outlook Handbook, “Computer and Information Systems Managers,” https://www.bls.gov/ooh/management/computer-and-information-systems-managers.htm

o3

OpenAI

24.3s
Computer and Information Systems Managers • 2024 median annual pay: $174,790 [1] • Projected employment change, 2024-34: +86,000 jobs [1] • Average annual openings: 34,700 [1] • Work experience in a related occupation: 5 years or more. • Job outlook, 2024-34: 15 percent growth (“Much faster than average”). Computer User Support Specialists • 2024 median annual pay: $64,340 [2] • Projected employment change, 2024-34: +104,700 jobs [2] • Average annual openings: 68,600 [2] • Work experience in a related occupation: None. • Job outlook, 2024-34: 15 percent growth (“Much faster than average”). Both occupations share an identical 15 percent projected growth rate for 2024-34, but differ markedly in their experience requirements: user support specialists offer an entry pathway with no prior related work, whereas computer and information systems managers presuppose substantial industry experience. Their pay and opportunity profiles also diverge. Managers command a median salary roughly 2.7 times higher, yet generate fewer annual openings because the role is more senior and supervisory. Conversely, support specialists, while earning far less, present a larger numeric employment gain and nearly double the yearly openings, reflecting widespread demand for hands-on technical assistance across industries. For career planners, these figures illustrate how experience requirements shape compensation and labor-market dynamics even within the same overall growth category. ## References [1] U.S. Bureau of Labor Statistics, Occupational Outlook Handbook, “Computer and Information Systems Managers,” 2024-34 projections page. [2] U.S. Bureau of Labor Statistics, Occupational Outlook Handbook, “Computer Support Specialists — Computer User Support Specialists,” 2024-34 projections page.