Engineering
Replacing an AS/400 (IBM i) while the plant keeps running
The production schedule is a bigger obstacle than the code. Cut over one process at a time and budget for four failures.
Computing America. Applications, integrations, data analysis and AI/ML engineering.
Describe the cutover
In short
- Cut over per process, not per system, and settle how each cutover rolls back before its date. The journal feed is one-way, so a rollback needs a write-back path.
- A trained operator finishes a green-screen transaction without looking at the display. Count keystrokes per transaction before designing the new interface, so the replacement doesn't add seconds to every one.
- The nightly batch chain is an undocumented dependency graph, and the dangerous jobs fail quietly. Log a full month of its starts, counts and messages from the history log, job logs and journals before replacing any of it.
- For files that are journaled and whose receivers you keep, a journal reader gives you a change feed without modifying a line of RPG. Check both before relying on it.
- The person who can still read the RPG is a project role. If they are only available for questions, the archaeology phase is missing from the estimate.
The only real cutover window may be a holiday shutdown. A failed cutover on a shop floor means a line that isn't making product, so the schedule sets the plan before the code does. The RPG still works and the business rules are all in it, though perhaps only one person can read it.
What's in there¶
Inventory what the system does, not what it stores. Look for five parts:
- Interactive programs behind the green screens, holding business rules written down nowhere else.
- A batch chain whose dependencies live in the order a CL program submits jobs and in one person's memory.
- Physical files and the logical files over them, which encode assumptions about keys and uniqueness that a modern schema won't reproduce by accident.
- Device integrations: label printers, scales, scanners, a CNC handoff, a customer EDI feed. They surface late because they aren't in any repository.
- Spool-file reports that end up as a customer document, an auditor's evidence or a regulatory submission, in a layout someone downstream depends on.
Four things that break¶
1. The new screen is slower for the person using it¶
An operator who has run a 5250 (green-screen) session for a decade finishes a transaction without looking at the display, in gloves. A replacement that needs a mouse, a page transition and a confirmation dialog adds seconds to every transaction, and shift volume multiplies them.
Count keystrokes at the terminal for the ten highest-volume transactions and make that count a design requirement: keyboard-first entry, function-key equivalents, no dialog between the operator and the next scan.
2. The batch chain is a dependency graph that was never drawn¶
Job order matters, and some jobs are safe to rerun and some aren't. The dangerous jobs fail quietly: a job processes zero records because an earlier step didn't deliver a file, and the gap shows up later as an inventory discrepancy.
For a full month before replacing any of it, log every job's start and end times, record counts, and completion and escape messages. Take starts and ends from the history log (QHST, shown with DSPLOG), which carries job-start and job-end messages, and messages from each job's job log. For record counts, first find the files a job writes: run DSPPGMREF (opens in a new tab) on the job's programs to see how each file is used (input, output, update). Where a file is journaled and its receivers are kept, count its DISPLAY_JOURNAL entries (below) per JOB_NAME and JOB_NUMBER, split by JOURNAL_ENTRY_TYPE, so adds, updates and deletes stay apart. For a file that is not journaled, COUNT(*) before and after a job shows only adds minus deletes. A job that updated every row then looks like one that touched none.
Then confirm the history log still holds the month. It is kept as separate log versions, and IBM says they “accumulate on a system” and old ones are deleted periodically (opens in a new tab). List them with DSPOBJD OBJ(QSYS/QHST*) OBJTYPE(*FILE), check that the oldest predates your month, and save the versions you need. The QHSTLOGSIZ system value (opens in a new tab) sets how many records a version holds. That log is the specification, and the baseline you compare the new system against.
3. The data types carry meaning your new schema drops¶
Look for packed decimals, six-digit dates, and zero and blank used as different sentinel values. A customer number may be zero-padded in one file and not in another.
Treat migration as reconciliation: move the data, run the same queries against both systems, and compare totals per period, per plant and per part.
4. The edges are physical¶
A label that prints two millimeters off is a customer receiving failure. A scale that returns weight in a fixed-width field with an implied decimal is silently off by a factor of ten. These integrations surface late and can't be tested from a laptop. Budget bench time with the real hardware on the real floor.
Read before you write, and check the journal first¶
A plant won't accept a big-bang replacement, so take one process at a time and route it to the new system while the old one keeps running: the strangler fig (opens in a new tab) approach. Choose the seam carefully, and start with reads: a reporting surface built on the existing data changes nothing operationally, delivers value in weeks, and teaches the team the data model.
Side by side, the new system must see changes as they happen. Editing the existing programs to publish events is the obvious route and puts production at risk. Journaling is the alternative: a journal records changes to the objects you specify, and a reader over it can turn them into a change feed. Don't assume the journal is there. Check three things first:
- 1.Which files are journaled. Journaling is started per physical file, with STRJRNPF (opens in a new tab). Starting it on a file is a change to production, so schedule it.
- 2.How receivers are retained. With system-managed receivers and automatic deletion (opens in a new tab), the system deletes detached receivers. Make sure the reader has consumed a receiver before it is deleted, or keep receivers until it has.
- 3.Whether the programs use commitment control. A commit writes a C CM entry to the journal, so only changes made under commitment control come with transaction boundaries. For the rest, the feed carries record changes, not committed transactions.
The reader can use QSYS2.DISPLAY_JOURNAL (opens in a new tab), an SQL table function that returns information similar to the DSPJRN command and the QjoRetrieveJournalEntries API. You can call that API directly instead, or use the RCVJRNE command, which passes journal entries to a user exit program continuously. A reader adds load to the IBM i, so measure it off-peak before the plant depends on it.
Group: Existing IBM i
RPG programs
Screens and batch jobs, unchanged
- To Files · writes
Group: Existing IBM i
Files
Journaling is started per physical file
- To Journal · journaled files only
- To Comparison · the old files
Group: Existing IBM i
Journal
Only the files that are journaled
- To Journal reader
Journal reader
Reads journal entries; adds load
- To New system · feed
New system
Fed continuously, checked on real traffic
- To Comparison
Comparison
Same queries on both, totals per period, plant and part
The new system is fed only from the journal, so a change the journal does not hold never reaches it, and only the comparison against the old files can show the gap.
Writes come later, one process at a time, behind an anti-corruption layer (opens in a new tab) that translates between the old vocabulary and the new so the new system doesn't inherit thirty years of encoding decisions. Give that layer an owner and a test suite, because it may stay for years.
Cutover is a schedule problem¶
Settle three rules with the plant manager before the first phase is designed, not during the week it happens.
- 1.Cut over per process, not per system. Receiving can move while production reporting stays. One date on which everything changes is a date on which everything can fail.
- 2.Decide how each cutover rolls back before its date, and keep that path open until the next business day. While the new system only reads, rollback means pointing users back at the old screens. Once writes move, the journal feed is one-way, so run dual writes for that process or build a reverse feed that carries the new system's writes back into the IBM i files. If you can afford neither, the answer to “can we roll this back by first shift tomorrow?” is no, whatever the tests say. Treat the cutover as one-way, and do not call it ready until the plant manager accepts that in writing.
- 3.Cut over into a low-volume window with the old system still warm, and keep it warm until a full month-end close has run on the new system.
Don't fix the process while you replace the system, however bad the process looks. When behavior and technology change together, no one can tell whether a discrepancy is a defect or the new intent.
Start the batch log today, and save the history-log versions you still have, because the history log keeps only so much of the month.
Sources
- 1.Strangler Fig (opens in a new tab), martinfowler.com,
- 2.Anti-Corruption Layer pattern (opens in a new tab), Microsoft Azure Architecture Center
Next step
Tell us what cannot stop.
Which parts of the floor cannot pause for a weekend, and what the old system still owns outright. We will say where the first seam goes and what has to run in parallel while it moves.
- Phone
- (214) 723-2510
- Reply
- A person replies, not a sequence: within one business day, from someone who would be on the engagement.
Packages
Start with a package
Four packages, each with its price on the page. Pick one and buy it online.
$199/month
Billed monthly
$182.42/month
Billed yearly
$499/month
Billed monthly
$457.42/month
Billed yearly
- Scammers stopped from sending email as you
- Your website and systems scanned every week
- Your email settings checked every week
- Everything in the 1 to 10 plan
- Covers up to 50 people
$199/month
after a one-time setup fee
- A site planned, designed and built for you
- Hosting and updates every month
$499/month
Billed monthly
$457.42/month
Billed yearly
$999/month
Billed monthly
$915.75/month
Billed yearly
- Your Google profile, fully set up
- A Google post every week
- 2 social posts a week
- Everything in Get found
- 4 social posts a week
- Your top 10 Google searches tracked
from$500/month
after a one-time setup fee
- Your data gathered from the tools you use
- A first analysis that answers your questions
- New findings every month
Related reading
Engineering3 min readLegacy software modernization: why rewrites get canceled, and a safer bet
A big-bang rewrite is a bad bet, not a bad pitch. Replacing the system one slice at a time changes the bet and gives the budget review something it can approve.
Strategy3 min readBuild vs buy software: sort each process step before you write requirements
Sort every process step as commodity, differentiating or contextual before anyone writes a requirements document.
