We’re pleased to introduce a new capability for Dynamics 365 Finance and Operations archive with Dataverse long-term retention: parallel processing for archive jobs. This enhancement allows the Archive job scheduler to run multiple archive jobs at the same time, dramatically reducing the time required to archive high volumes of transaction data across legal entities.
The challenge: Sequential bottlenecks
Previously, archive jobs within the same scenario were processed sequentially. For organizations operating across dozens of legal entities—each with millions of transaction records in General Ledger, Sales Orders, or other scenarios—this approach created a bottleneck, and archiving could take days or even weeks to complete.
For example, a multinational organization may have 50–200+ legal entities, each containing one fiscal year of General Ledger transactions. Archiving one legal entity data at a time delays storage optimization, increases SQL Server load, and slows the movement of data into long-term retention in Dataverse.
The solution: Use Job criteria key partition to enable parallel processing
Parallel processing introduces the Job Criteria Key—a partition identifier you set when you build the archive job contract. The job criteria key tells the archive job scheduler which archive jobs operate on independent data sets. This allows them to run simultaneously without conflict.
How it works
- Define the partition key — When you build the archive job contract, set the criteria key (typically the legal entity) that represents the data partition.
- The scheduler identifies parallel candidates — The archive job scheduler detects that jobs with different criteria keys target non-overlapping records.
- Jobs run concurrently — Rather than waiting in a queue, archive jobs for different partitions execute in parallel.
The zero-overlap guarantee
The job criteria key depends on one critical invariant: when multiple archive jobs run within the same scenario, each job with a different job criteria key must process a completely distinct set of records with zero overlap.
For example, if you run two Sales Order archive jobs at the same time—one with job criteria key “USMF” and another with “DEMF”—the records archived by the USMF job must not overlap with those archived by the DEMF job. This is why DataAreaId is a natural choice for many scenarios: it inherently partitions data by legal entity.
Monitoring parallel jobs
You can monitor archive jobs running in parallel from the Archive with Dataverse long term retention workspace in Dynamics 365 Finance and Operations. Each job shows its criteria key value, making it easy to confirm which partitions are being processed concurrently.
Join the private preview
Parallel processing for archive jobs is currently available in private preview. If you’d like to try this capability in your environment, we’d be happy to have you participate.
Submit your request to join the private preview
By joining the preview, you’ll get early access to parallel archive job execution. You’ll also have an opportunity to provide feedback that helps shape the final release. The preview is open to all Dynamics 365 Finance and Operations customers and partners.
