HomeCasa Design a background job system with queues and workers
Design queues and workers for async tasks: technology choice, idempotent jobs, retries per error type, priorities, concurrency limits, the outbox pattern and monitoring. Designed for ChatGPT, Claude, Gemini and Microsoft Copilot. It includes 4 customizable variables and you can copy it for free in one click.
Advanced · Text · Pending editorial review
Prompt
Act as a software architect experienced in asynchronous processing and queue systems.
I want to move heavy or slow work out of my application's web requests.
- Tasks to move to the background: {{tasks}}
- Expected volume and peaks: {{volume}}
- Current stack: {{stack}}
- Infrastructure and constraints: {{infrastructure}}
Deliverables:
1. A classification of each task: urgency, typical duration, whether it calls external services, whether it can safely run twice, and what happens if it is lost.
2. Technology options (database-backed queue, Redis or a message broker) with pros, cons and operational cost for my case, without invented performance numbers. A justified recommendation.
3. Job design: minimal payload (IDs, not full objects), idempotency via keys or state checks, timeouts and message format versioning.
4. A retry policy per error type (transient, third-party rate limit, bad data, programming bug), backoff with jitter, and a dead-letter queue with a procedure to review and replay it.
5. Separate queues and priorities, concurrency per queue, and rate limits for external APIs.
6. Consistency between the database and the queue: the outbox pattern or enqueue-after-commit, and why not to enqueue inside an open transaction.
7. Scheduled tasks: how to avoid duplicate runs with several workers.
8. Deploys and graceful shutdown: what happens to in-flight jobs.
9. Monitoring: metrics (queue depth, age of the oldest job, error rate, duration), alerts and dashboards.
10. A phased migration plan from the current setup.
Constraints:
- If volume, delay tolerance or infrastructure details are missing, ask me first or mark them as [FILL IN].
- Prefer the simplest option that meets the requirements and explain when it would be worth switching.
- Include a text diagram of the whole flow.How to use it
- List today's slow tasks and how long they take inside the request.
- State what infrastructure you can genuinely operate (for example, whether you already run Redis or only a database).
- Move a single task first, add the metrics, then migrate the rest.
Usage example
Tasks: send emails, generate PDF invoices, sync orders with the ERP · 20,000 jobs a day with peaks during sales · Stack: Laravel with MySQL · One server, Redis available.
Sample output
1. Task classification
| Task | Urgency | Typical duration | External service | Safe to run twice? | If lost… |
|---|---|---|---|---|---|
| Transactional email | High (seconds) | Short | Yes (SMTP/API) | Not without an idempotency key | Customer never gets the confirmation |
| PDF invoice | Medium (minutes) | Medium, CPU-heavy | No | Yes, if it overwrites the same file | Regenerated |
| ERP sync | Medium | Variable | Yes, rate-limited | Only with a state check | Orders out of sync |
2. Recommendation
Laravel queues on Redis + Horizon. You already run Redis, it adds no new component to operate, and Horizon gives you dashboards and retries out of the box. A dedicated broker would only pay off if you needed several independent consumers or stricter delivery guarantees.
5. Queues and concurrency
emails— high priority, several workers.invoices— few workers so the single server's CPU is not saturated.erp— 1–2 workers andRateLimitedmatching the ERP's limit → [FILL IN].
Diagram
Request → DB::transaction → outbox / afterCommit → Redis
→ Horizon (emails | invoices | erp) → retries with backoff → failed_jobsWith a two-person team and no on-call, alerts should go to a channel checked during working hours, except "email queue stalled", which deserves an immediate ping…
Key facts
- Category
- Programming & development · Software architecture
- Output type
- Text
- Level
- Advanced
- Designed for
- ChatGPT, Claude, Gemini and Microsoft Copilot
- Verified in
- Pending editorial verification
- Variables
- Background tasks, Volume and peaks, Current stack and Infrastructure and constraints
- Language
- English
- Price
- Free, no sign-up
- Updated
- Nov 7, 2026
Frequently asked questions
What is the outbox pattern and why is it recommended?
You write the message to a table in the same transaction as the business data and publish it afterwards. That prevents queuing a job for something that never got saved, or saving something without its job.
Is a database-backed queue enough?
For moderate volumes it usually is, and it adds no infrastructure. Once the queue competes with application queries or you need high concurrency, moving to Redis or a broker makes sense.
What is this prompt for?
Design queues and workers for async tasks: technology choice, idempotent jobs, retries per error type, priorities, concurrency limits, the outbox pattern and monitoring.
Which AI tools does it work with?
It is designed for ChatGPT, Claude, Gemini and Microsoft Copilot. We have not verified it in these tools yet, so results may vary between them.
How do I customize the prompt?
Click “Customize” and fill in the variables (Background tasks, Volume and peaks, Current stack and Infrastructure and constraints). The text updates instantly and you can then copy it in one click. Values are stored only in your browser.
Do I need experience to use it?
It is advanced: it is meant for people with experience in the topic who can evaluate and adjust the result.
Is this prompt free?
Yes. Every HomeCasa Prompts prompt is free, with no sign-up or subscription. You can also use them in professional projects.
Can I trust the AI output?
Treat it as a high-quality draft: always check data, figures and important claims before publishing or making decisions.