COBOL Migration in the AI Era: What OpenAI Changes for Legacy Systems

COBOL Migration in the AI Era: What OpenAI Changes for Legacy Systems

10 min read

Authors :

Igor Omelianchuk

COBOL Migration in the AI Era: What OpenAI Changes for Legacy Systems

COBOL migration after OpenAI’s latest results

According to OpenAI, its newest model, GPT-6.1 Sol, scored high in COBOL program migration. For owners of legacy systems planning a transfer to modern environments, the result raises a practical question: is better code translation enough to migrate safely?

In OpenAI’s own evaluation, GPT-6.1 Sol correctly migrated 86.6% of COBOL programs at low reasoning effort, compared with 5.4% for GPT-5.6 Sol. The evaluation came with ready-made hidden tests that defined the correct output for each program. A real migration project may have no such checks until the team builds them. 

So the short answer is no: AI-assisted COBOL migration speeds up conversion, but a safe launch depends on behavior preservation: recovered business rules, tests, and integration checks. 

For CTOs and CIOs considering migration, OpenAI’s results determine a starting point. They help understand what was tested, what cannot be applied to a real system, and how to check a migrated module before replacing the original.   

What OpenAI’s COBOL migration benchmark actually shows

OpenAI’s COBOL migration evaluation tests whether AI can translate existing COBOL programs into C++, Java, Kotlin, Python, or Rust while preserving their original behavior. The task requires the model to understand the program’s logic and reproduce its results in another language.

The OpenAI COBOL benchmark compares models on AI code translation at different reasoning effort levels. OpenAI published a GPT-5.6 Sol score only for low effort, so the comparison with GPT-6.1 Sol applies to that setting. 

ModelReasoning effortPrograms migrated correctly
GPT-5.6 SolLow5.4%
GPT-6.1 SolLow86.6%
GPT-6.1 SolHigher97.9%

Share of COBOL programs migrated correctly in OpenAI’s evaluation. A program counted only when it passed all hidden tests. Source: OpenAI. 

COBOL still runs a large share of banking and other financial systems [→ The Decoder], so the success criterion matters as much as the score. A program that compiles and runs without errors can still produce different financial results. OpenAI counted a migration as successful only when it passed all hidden tests and its observable outputs matched the original byte for byte.

This evaluation is different from MultiLangBench. 

  • MultiLangBench tests whether a model can build a new SQLite-compatible database engine in different programming languages. 
  • The COBOL benchmark addresses the challenge of translating an existing program without changing what it does.

Why benchmark accuracy is not production readiness

OpenAI’s benchmark shows what AI COBOL migration can achieve when the programs and tests are prepared in advance. 

OpenAI built and ran the evaluation itself and has not published its programs or tests, so outside teams cannot repeat it. The 86.6% figure describes programs in that test set. It says nothing about the share of your code that will migrate correctly, or about the odds that your whole migration succeeds. 

Real legacy systems offer less to check against. They may hide undocumented business rules and have no tests that could confirm a new implementation behaves the same. These benchmark limitations matter when you are assessing production readiness.

In modernization projects, address the following areas:

  • Business rules. The results of financial calculations and exception handling must be the same. Business-rule validation confirms that the logic recovered from COBOL code matches what the business expects.
  • Legacy dependencies. External databases, batch jobs, interfaces, and connected applications can affect system behavior. Code translation alone does not account for them.
  • Verification. If existing tests are insufficient, the team writes new ones according to business requirements and historical data, as well as observations of the running system.

Before switching off the original application, engineers check how the new implementation behaves under production-like load and across its integrations. OpenAI itself notes that production use “still requires expert review and testing against the system’s business rules.” The same applies when developers translate the code by hand. 

Code conversion is only part of COBOL modernization 

Teams often use three terms interchangeably, though each covers a different scope of work:

  • Conversion translates source code into another language, such as Java. 
  • Migration moves the application to a new environment, with its runtime and data. It also covers job execution and integrations. 
  • Modernization addresses architecture, maintainability, and connections with other systems.

This is what a COBOL application typically depends on:

  • Copybooks contain reusable code and data definitions included during compilation. Map their structures correctly in the new implementation.
  • JCL (Job Control Language) tells an IBM mainframe which programs a job runs, in what order, and which input and output datasets each step uses. Any COBOL modernization has to reproduce that sequence in the new environment. 
  • Batch processing runs large volumes of records in scheduled jobs, often overnight. The schedule and job order decide when data reaches databases and downstream systems.
  • Databases and external systems, such as DB2 tables, VSAM files, and partner interfaces, receive what the programs write. The new version has to keep the same tables, file layouts, and message formats. 

Consider a hypothetical monthly statement run at a bank. A team converts the COBOL program that builds the statements to Java. That alone does not complete the COBOL application migration. The new version has to run after the same upstream jobs, update the same database tables, and feed the downstream reporting systems. Otherwise, the statement totals may be right while the wider business process fails. 

  • The source code and the operational environment in which the application runs are the parts to consider in mainframe modernization.

What must stay the same in COBOL to Java conversion:

Code that translates without errors can still behave differently from the original. Numbers can easily confirm that. 

In OpenAI’s example, COBOL drops extra decimal digits unless rounding is requested. A rewrite that rounds runs without errors and still produces different financial results. 

For instance, store 10.239 in a field with two decimal places: COBOL gives 10.23 by default and 10.24 with ROUNDED. Only the business can say which of the two results is right.

When you convert COBOL to Java, exact decimal types such as BigDecimal are the usual choice for fixed-point arithmetic, because double can’t represent many decimal values exactly. 

But BigDecimal doesn’t guarantee a match. You must explicitly set rounding rules: setScale() without a mode throws an ArithmeticException when digits would be lost. RoundingMode.DOWN reproduces COBOL’s default truncation, and RoundingMode.HALF_UP matches what ROUNDED does by default, rounding half away from zero.

COBOL copybook to Java conversion is a separate data mapping step. A COBOL copybook defines field lengths, signs, implied decimal points, and storage formats such as packed decimal (COMP-3). If you map any of them incorrectly, the Java code will read or write records differently from the original. 

Two more differences show up often. COBOL silently drops high-order digits when a result is too large for its field unless the code checks ON SIZE ERROR, and EBCDIC data sorts in a different order than ASCII once it leaves the mainframe. 

  • Java is not your only possible destination. For COBOL to Java conversion and COBOL modernisation, the target depends on the system’s architecture, its integrations, and who will maintain it.

How to verify that migrated code preserves business behavior

The old program is often the most reliable description of what your system does. So, COBOL migration verification has to begin with what the legacy code currently produces. 

The order we recommend: 

  1. Determine the observable outputs. Amounts, database updates, files, status codes, messages to other systems.
  2. Record reference runs. Choose inputs that look like your real workload and save what the COBOL program returns. These saved runs are characterization tests: they record current behavior, even if nobody remembers why it works that way. Together, these saved outputs are the approved reference for golden master testing: each new run is compared against them.
  3. Run the migrated code on the same inputs, with identical starting data and the same system date.
  4. Compare. Running both versions on one input set and comparing the results is differential testing. This checks output parity: values, record layouts, and side effects.
  5. Go through each mismatch with the business process owner. Distinguish between the bugs in the new code and the old workarounds the business wants to keep or fix.

Typical transactions rarely show where two versions differ, so test edge cases:

Let’s consider a nightly batch that posts interest. Test an account with no record, a zero balance, a negative one, the last day of February, an invalid code, and a run that fails halfway. Then look at the report as well as in the database and output files.

Compare byte for byte when another system reads the output, like a fixed-width file. If some differences are fine, such as a timestamp or run ID, write the acceptance criteria down before the first run. Otherwise, you’ll argue over every mismatch.

  • Passing tests show that the chosen scenarios behave the same. They don’t establish behavioral equivalence for every possible input. Keep the tests too, since you’ll need regression testing after future changes.

Where to start when documentation and tests are missing

In some systems, the documentation and tests are out of date. In others, they may be absent. 

In either case, you start with software archaeology: finding out what the legacy COBOL code does, which business rules it applies, and what depends on it.

Then comes reverse engineering: the team traces the code and reads execution logs and reports to see what the system does. 

Business rules extraction takes those findings, together with input from the people who use the system, and turns them into confirmed rules. It gets harder when one rule is spread across several modules. 

For example: 

A program calculates the commission correctly only for a particular account type. No specification mentions it, just a conditional in the code. Historical transactions or a business user can confirm it. You should find and record that rule, because if you miss that undocumented behavior, the migrated system will deliver the wrong commission.

Categorize your findings:

  • Dependency map: what each program reads, writes, and calls, gathered through dependency mapping.
  • Rules catalog: a full list of rules, where each rule came from, and who confirmed it.
  • Control scenarios: inputs with their expected outputs, for later comparison.
  • Open assumptions: behavior that hasn’t been confirmed yet.

AI can read unfamiliar code and draft documentation, tests, and an initial test harness. 

  • The reliable sources of expected results are the legacy system’s real output and business users who confirm the rules. The generated code shouldn’t be one of them, since in this case, the same mistake would get into the code and its tests. 

A practical COBOL to Java migration approach with AI

Plan the work as a series of stages, each ending with a result the team can check, and start with one module as a migration pilot.

StageResult
1DiscoveryA map of dependencies, data flows, batch jobs, and business rules.
2Pilot moduleOne limited but representative module.
3Contracts and testsDocumented interfaces, expected outputs, and critical scenarios, prepared before you migrate COBOL to Java.
4AI-assisted conversionDraft Java code for the pilot module. AI speeds up converting COBOL to Java, but engineers own the outcome.
5Review and verificationReviewed code with behavioral equivalence checked against the tests.
6Integration and load testsInterfaces, performance, and stability confirmed.
7Parallel runBoth versions compared on the same inputs.
8Gradual releaseWorkload or traffic moved in steps, with a rollback plan.

Responsibilities:

  • COBOL engineer: explains the original logic.
  • Java engineer: reviews the target code and design.
  • Architect: checks integration and deployment choices.
  • QA: builds and runs the tests.
  • Business rules owner: confirms rules and expected outcomes.

In a parallel run, only one version executes real transactions. The other works in shadow mode, and its output is compared with production, avoiding duplicated payments or updates. 

Pilot results guide the next steps of the incremental migration. Timelines are determined by what the pilot shows, as well as by system complexity and dependencies.

Migration pilot readiness checklist

Critical scenariosResults match the COBOL version; the business owner has approved any differences
Parallel runMismatches are explained, and no real transaction was duplicated
OperationsPerformance, stability, and error handling meet agreed thresholds
Remaining risksKnown issues are documented and accepted
Rollback planThe return to the original system has been tested

When every row of the checklist is met, the team can extend the same process to the next module.

What GitHub’s AI-assisted Rust migration can teach COBOL teams

GitHub’s migration of the Copilot runtime from TypeScript/Node.js to Rust involves a different pair of technologies than COBOL to Java, but the way GitHub organized the work carries over to COBOL teams.

According to GitHub, the AI-assisted rewrite was an incremental migration. Most pull requests replaced one component or slice of the runtime. A temporary compatibility layer let Rust and the remaining TypeScript call each other, and existing end-to-end tests had to pass before a pull request could merge.

The key lessons:

  • Implement small changes. Replace one component at a time. GitHub started with self-contained helpers that had no side effects and left the most tightly coupled part, session orchestration, for the end.
  • Use automated validation on every change. Check the contracts where components meet, and don’t let the AI edit the tests without expert approval. GitHub, adopted this rule for its end-to-end tests after one regression.
  • Keep human code review. Engineers check design, contracts, and risk, so behavior preservation doesn’t rest only on generated code.

Organizations in the financial and banking sectors shouldn’t expect GitHub’s speed, performance, or effort gains. GitHub rewrote the runtime behind Copilot CLI, the Copilot app, and the Copilot SDK. A bank, in turn, has to get transaction rules and data integrity right and work within strict operational limits.

How AI changes the cost of COBOL migration

AI can cut the effort of translating code, yet the total project budget may reduce by a much smaller share  

  • To estimate COBOL migration cost, separate the cost of generating code from the total migration cost, which covers everything needed to get a verified implementation running in production.

OpenAI’s figures cover only generation. As Mixed states, GPT-6.1 Sol’s low-effort run had a 39% lower API cost than GPT-5.6 Sol at the same setting. The best accuracy score came from a higher-effort run, and no cost is indicated for it, so that saving can’t be expected to be reproduced. 

In AI-assisted modernization, API cost is one line of the budget:

Budget itemWhat it covers
DiscoveryCode analysis, dependencies, business rules
Test preparationReference data, test scenarios
AI generationAPI calls, prompts, repeat attempts
VerificationAutomated checks, QA, manual review
IntegrationInterfaces, performance and load testing
Delivery and supportDeployment, monitoring, maintenance

Verification effort stays, since the team has to show the new code behaves the same as the old. 

On a pilot, we suggest measuring time per accepted module, the share of manual corrections, the number of behavioral mismatches, and cost per accepted module: total pilot spend divided by accepted modules. This gives you a better basis for the next stages than only API cost.

In conclusion: Better translation still needs verified migration

Better models assist in COBOL migration, but they don’t tell you when a system is safe to launch. A verified migration depends on confirmed business rules, tested interfaces, known dependencies, and the results of a representative pilot. 

In our view, COBOL conversion gets cheaper and more accurate, while more of the effort and risk moves to proving that the new code behaves like the old one:  recovering rules, building reference tests, and running the pilot in parallel. That work is what protects business continuity, making the switch less painful for a company. 

  • Transform in small steps and start with one module. Build control tests for it, and only then migrate more. 

That test set is the starting point of behavior-preserving modernization. The same sequence then applies to a full COBOL to Java migration. 

Frequently Asked Questions

Yes. Recent models can convert COBOL to Java, and in OpenAI’s evaluation, GPT-6.1 Sol migrated most test programs correctly. But the result is only a draft until it passes tests built from the legacy system’s recorded outputs, including details such as rounding and field formats.

No. What is COBOL modernization? It implies the whole change: code, data, interfaces, operations, and tests. Conversion is one step inside it.

In a COBOL application migration, run the old and new programs on the same inputs and compare the outputs, including edge cases and side effects in databases and files. This legacy migration testing starts from saved COBOL outputs, and every difference goes to the business owner.

You can migrate COBOL without documentation, if the team rebuilds the missing knowledge first. Recover the rules from the code, logs, reports, and user interviews, then record them as tests. Keep what you can’t confirm on a list of open assumptions.

No. Decide for each application separately, using four criteria. Business goals: does it need to change? Dependencies: what relies on it? Risk: what breaks if the change is wrong? Support cost: what does it cost to keep it? Some can stay as they are, sit behind an API, or move gradually, module by module. Java is just one of the options.

About autors

Igor Omelianchuk
Igor Omelianchuk

Igor Omelianchuk is the Co-Founder & CEO at Corsac Technologies. Igor has led 30+ modernization projects, helping companies move from fragile legacy systems to scalable, secure, and modern platforms.

Modernizing the past. Empowering the future.

Let us help you rebuild what’s holding you back.

Connect with Experts //