Batch Processing
Batch processing runs the same operation across a large set of records in one job — overnight enrichments, weekly imports — instead of processing each record as it arrives.
What is batch processing?
Batch processing is the pattern of accumulating work and running it in a single job, usually on a schedule. The opposite is stream processing, which handles each record the instant it arrives. Batch is cheaper at high volume and simpler to operate; streams give you lower latency at higher cost.
Why it matters
- Enrichment APIs typically charge per call but offer bulk endpoints with better throughput and price
- A nightly batch job is much easier to monitor than a 24/7 streaming pipeline
- For most sales workflows, "fresh by tomorrow morning" is a perfectly fine SLA
When to use batch vs. real-time
- Batch: list refresh, cohort recompute, weekly report generation, deduping a CRM
- Real-time: lead routing the moment a form is submitted, abandoned-cart triggers, fraud detection
How TexAu helps
Run a TexAu workflow on a schedule against a list of thousands of records — the platform paginates, retries, and rate-limits internally so you can write the workflow once and trust it to chew through the queue overnight.
Related