Start Today

← What's new

Faster legacy imports, and large IMS segments that finish

Two changes to how an import reads its source, each measured rather than assumed.

VSAM datasets can be paged by record key instead of by counting from the start of the dataset. Paging by count makes every page pay again for the records ahead of it, so a scan gets slower the further it goes; paging by key does not. On a dataset of 1,095,753 records the run went from 2h 04m to 1h 04m, and the cost of a page stopped rising with depth.

IMS segments can be read on a cursor that only moves forwards and holds nothing open behind it. That removes a ceiling around 260,000 records where a large segment load used to fail — sometimes as a hang, sometimes as an I/O or authentication error, all of them the same source running out of room. A 612,500-record segment now reads whole in 35 minutes, costing the same per page at half a million records as at fifty thousand. It stays off by default because it holds one connection and one database schedule open for the whole scan, and it needs a source that supports it; without it, keep a segment load under about 250,000 records per step.

Both settings, and the page size each reader uses, travel with the exported job rather than with the service, so one installation can serve sources of different vintages. Administrators set them through environment variables, and the read metrics page shows what each one was worth.

Read more