- Knowledge
- technology
- OOP
- Tips
- Programming
- Tips
- Tutorial
- SEO
- Ranking
- Knowledge
- Special Day
- Seo
- Bug
- Data science
- Seo
- artificial intelligence
- Machine Learning
- Robotics
- happyNewYear2021
- newYearEve
- 2021
- Automation
- Smart Home
- Career
- Best Practices
- Git
- Logging
- Web Fundamentals
- DNS
- HTTPS
- Performance
- AI Tools
- ChatGPT
- Claude
- Gemini
- Laravel
- Eloquent
- MySQL
- HTTPS
- TLS
- Web Security
- Certificates
- Developer Life
- Debugging
- Docker
- DevOps
- Transactions
- Queues
- LLMs
- AI
- AI Coding
- Developer Tools
- React Native
- Expo
- Kate PMS
- Mobile Apps
- Laravel
- Authentication
- Sanctum
- Cookies
- API Design
- Payments
- Idempotency
- DeepSeek
- Open Source AI
- LLMs
- AI News
- Git
- Version Control
- AI Coding
- Prompting
- PHP
- Checklist
- MCP
- AI Agents
- OpenAI
- Architecture
- Microservices
- Modular Monolith
- Estimation
- Developer Life
- Project Planning
- Humour
- OAuth
- OpenID Connect
- Authentication
- Embeddings
- Vector Search
- RAG
- pgvector
- OpenAI
- GPT-4.1
- Codex CLI
- Events
- Testing
- Clean Code
- Maintainability
- Code Review
- Webhooks
- API
- Security
- Claude Code
- Workflow
- AI
- LLM
- Prompt Injection
- Mobile
- React
- Networking
- TCP
- UDP
- HTTP/3
- CLAUDE.md
- AWS
- Cloud Security
- Backups
- PHPUnit
- Software Engineering
- Leadership
- Communication
- RAG
- Embeddings
- AI Engineering
- IT Infrastructure
- Networking
- Access Control
- CI/CD
- GitHub Actions
- Gemini CLI
- Claude Code
- JavaScript
- Async/Await
- Node.js
- Promises
- Security
- Cryptography
- Passwords
- MySQL
- Database
- Vibe Coding
- Software Quality
- DNS
- Code Reading
- Onboarding
- Productivity
- Background Jobs
- Developer Humour
- Estimates
- Dev Life
- JWT
- o3-mini
- DeepSeek R1
- Rate Limiting
- Kate PMS
- E-Signing
- Audit Trail
- REST
- GraphQL
- API Design
- Laravel 12
- Upgrade Guide
- Open Source
- Self-Hosting
- Task Scheduling
- Cron
- Secrets
- CORS
- PHP
- PHP-FPM
- OPcache
- GitHub Copilot
- Software Architecture
- Engineering
- TypeScript
- JavaScript
- Type Safety
- AI Security
- React Native
- Product Design
- AI Agents
- Kiro
- Queues
- Redis
- RabbitMQ
- AWS SQS
- Nginx
- Apache
- GPT-5
- gpt-oss
- Clean Code
- Architecture
- Naming
- Documentation
- Career
- ADR
- Teamwork
- Supply Chain
- Kate HRM
- HR Software
- Permissions
- System Design
- Pagination
- SSH
- Linux
- Big O
- Databases
- Laravel Boost
- MCP
- Developer Skills
- Validation
- Databases
- Indexes
- Code Quality
- Deployment
- Developer Humour
- Feature Flags
- Code Review
- Pull Requests
- Docker
- Cursor
- Authorization
- RBAC
- Gemini
- Long Context
- PHP 8.4
- Caching
- Dependency Injection
- Web Performance
- Browser
- CSS
- Database
- Migrations
- ChatGPT
- AI for Developers
- Monitoring
- On-Call
- REST
- Backend
- SQL
- NoSQL
- Database Design
- Coding Agents
- Claude 4
- API Resources
- REST API
- Load Balancing
- Scaling
- AWS
- AI Tools
- Claude
- Sora 2
- CTE
- 2FA
- TOTP
- Programming Languages
- Prompts
- Developer Workflow
- API Gateway
- APIs
- Passport
- API Auth
- Learning
- Burnout
- Developer Growth
- Web Development
- SEO
- Kate Mall
- ChatGPT Atlas
- Agent Skills
- Middleware
- Laravel 12
- Collections
- Context Window
- Monitoring
- Commit Messages
- Self Review
- Growth
- Regex
- Programming Basics
- Text Processing
- Database Design
- Normalization
- Linux
- Server Security
- Linux Foundation
- Open Standards
- Legacy Code
- Documentation
- AI Workflow
- File Uploads
- Test Data
- Hashing
- Performance
- Caching
- Enums
- Scope Creep
- Estimation
- Codex
- Gemini CLI
- Timezones
- Carbon
- Bugs
- PHP 8.5
- Gemini 3
- GPT-5.1
- Data Integrity
- Event Loop
- Async
- Opus 4.5
- AI Models
- React
- Forms
- Frontend
- Backups
- AI Images
- DALL-E
- Midjourney
- Race Conditions
- Concurrency
- Legacy Code
- Refactoring
- Senior Engineer
- Scope
- LLM
- CDN
- Web
- Sub-Agents
- Soft Deletes
- Audit Log
- Concurrency
- AI Learning
- NestJS
- AI Evals
- Policies
- SPF DKIM DMARC
- Unicode
- UTF-8
- Knowledge Graph
- Value Objects
- Technical Debt
- Feature Flags
- Laravel Pennant
- Deployment
- Copilot
- Composer
- Dependencies
- Artisan
- Automation
- AWS S3
- Object Storage
- Cloud
- Small Language Models
- Ollama
- Production
- Sessions
- HTTP
- Mentoring
- SQL
- Virtual Machines
- Web Development
- HTTP/2
- QUIC
- Web Performance
- AI Integration
- LLM API
- SOLID
- OOP
- Hosting
- Serverless
- Merge Conflicts
- Temperature
- AI Development
- Reverse Proxy
- Nginx
- Infrastructure
- Verification
- Passkeys
- WebAuthn
- Teams
- Communication
- Stakeholders
- Monorepo
- CI/CD
- Versioning
- JSON Schema
- Livewire
- Inertia
- Meetings
- Distributed Systems
- Privacy
- Full-Stack
- T-Shaped Skills
- Money
- Notifications
- Web Security
- HTTP Headers
- CSP
- Function Calling
- Load Testing
- k6
- Data Extraction
- Debugging
- WebSockets
- SSE
- Real-Time
- Laravel Reverb
- Infrastructure as Code
- Terraform
- Side Projects
- Laravel Pint
- OpenAPI
- Swagger
- UX
- Multimodal
- Jest
- Pair Programming
- APIs
- Rate Limiting
- Resilience
- Dev Humour
- Design Tokens
- JWT
- API Keys
- Sessions
- PHPStan
- Rector
- Incidents
- Reporting
- Dashboards
- Zero Trust
- IAM
- Search
- Laravel Scout
- Junior Developers
- Mentoring
- Images
- WebP
- AVIF
- Bug Reports
- Let's Encrypt
- Design Docs
- Software Design
- Observers
- Replication
- Accountability
- Data Structures
- Reliability
- LLM Memory
- Error Handling
- Payments
- Payment Gateway
- Webhooks
- PCI DSS
- Observability
- OpenTelemetry
- Personal Brand
- Writing
- Conventions
- Dates
- Scheduling
- Disaster Recovery
- Compression
- Brotli
- Deadlines
- Developer Habits
- State Machines
- Tech Roles
- UUID
- ULID
- Horizon
- Planning
- Engineering Culture
- Ownership
- Soft Skills
- Socialite
- Cost Control
- Collations
- Unicode
- Octane
- PostgreSQL
Queues and Message Brokers Explained: Redis, RabbitMQ and SQS Without the Jargon
About Post
A user uploads a signed contract. Your controller saves it, generates a PDF, emails three people, pushes a notification to the mobile app and updates a report. The user stares at a spinner for eight seconds, gives up, and taps the button again.
Nothing in that list was slow on its own. The mistake was doing all of it while the user waited. That's the problem queues solve, and once you understand them properly, a lot of "mysterious" production behaviour starts to make sense.
Why queue anything at all?
A queue is a to-do list that sits between two parts of your system. One side writes tasks onto it, the other side picks them up and does the work later, usually within milliseconds or seconds.
Think of a busy restaurant. The waiter doesn't walk into the kitchen and cook your meal. They write the order on a ticket, pin it to the rail, and go back to the tables. The kitchen works through the tickets at its own pace. If ten orders arrive at once, the rail gets longer, but nobody at the tables is kept waiting at the door.
That gives you three things:
- Fast responses. The request only has to save the important bit and drop a ticket on the rail.
- Absorbed spikes. A burst of traffic becomes a longer queue, not a crashed server.
- Retries for free. If the email provider is down for a minute, the job fails, waits, and tries again. The user never sees it.
Producers, consumers and the broker in the middle
The vocabulary is simpler than it sounds:
- The producer puts a message on the queue. In Laravel, that's your code calling
SendContractEmail::dispatch($contract). - The consumer (or worker) takes messages off and processes them. In Laravel, that's
php artisan queue:workrunning under Supervisor or systemd. - The broker is the thing that stores the messages in between: Redis, RabbitMQ, Amazon SQS, or even your database.
The producer and the consumer never talk to each other directly. That's the whole point. You can restart workers, add more of them, or deploy new code, and the messages wait patiently on the broker.
The part people skip: at-least-once delivery
Here's the thing that bites people in production. Almost every queue you'll use promises at-least-once delivery. Not exactly once. At least once.
Why? Picture a worker that picks up a job, sends the email, and then crashes before it can tell the broker "done". The broker never got the acknowledgement, so after a timeout it assumes the worker died and hands the message to someone else. The email goes out twice.
No broker can fully fix that, because the crash happened in your code, between the side effect and the acknowledgement. So the fix has to live in your code too: make your jobs idempotent. Running the job twice should have the same effect as running it once.
public function handle(): void
{
// Already done? Then this is a duplicate delivery. Skip it.
if ($this->invoice->receipt_sent_at !== null) {
return;
}
Mail::to($this->invoice->tenant)->send(new RentReceipt($this->invoice));
$this->invoice->update(['receipt_sent_at' => now()]);
}
This is simplified (there's still a tiny window between sending and saving), but it turns "duplicate on every retry" into "duplicate only if the worker dies at exactly the wrong moment". For payments, use a unique constraint or an idempotency key so the database itself refuses the second attempt.
One Laravel-specific trap: the retry_after value in config/queue.php must be longer than your longest job's timeout. If a job takes 120 seconds and retry_after is 90, the broker decides the job is lost and gives it to a second worker while the first one is still busy. Now you have two workers doing the same job, and no crash was involved at all.
Dead letter queues: where bad messages go
Some messages will never succeed. The record was deleted, the payload is malformed, or the code has a bug. Without a limit, they'd be retried forever, clogging the queue and filling your logs.
A dead letter queue (DLQ) is a separate queue where a message goes after it has failed a set number of times. It stops the retry loop and keeps the message so a human can look at it, fix the cause and replay it.
- In SQS, you attach a redrive policy with a
maxReceiveCount. - In RabbitMQ, you configure a dead letter exchange on the queue.
- In Laravel, the
failed_jobstable plays this role. Set$triesand$backoffon the job, and usephp artisan queue:failedandqueue:retryto inspect and replay.
The rule: a DLQ nobody looks at is just a slower way of losing data. Alert on it. A failed job count that's growing is one of the most useful signals you can have.
Redis, RabbitMQ or SQS?
All three move messages from A to B. They differ in what they're good at and what they cost you to run.
| Redis | RabbitMQ | Amazon SQS | |
|---|---|---|---|
| What it is | In-memory data store that also does queues | A dedicated message broker (AMQP) | A fully managed queue service |
| Best at | Simple, fast job queues, especially if you already run Redis for cache | Routing: one message to many queues, topic patterns, priorities | Zero maintenance, scales without you thinking about it |
| You manage | The Redis server, memory and persistence settings | The broker, clustering, upgrades | Almost nothing, but you're tied to AWS |
| Watch out for | Memory limits; persistence settings decide what survives a restart | More concepts to learn (exchanges, bindings, acks) | Visibility timeouts; standard queues can deliver out of order |
| Laravel support | Built in, plus Horizon for monitoring | Via a community package | Built in |
How I'd choose
For a typical Laravel app, my default is simple:
- Start with the database driver if you have low volume and want zero new infrastructure. It's slower, but it's honest and easy to debug.
- Move to Redis when you want speed and visibility. Laravel Horizon gives you a dashboard, metrics and failed-job views with almost no setup.
- Pick SQS if you're already on AWS and don't want to babysit another server. It's a very good fit for spiky workloads.
- Reach for RabbitMQ when the problem is routing: several services that each need their own copy of an event, or messages that go to different consumers based on type. That's where it really shines.
Kafka deserves a mention too, but it's a different animal: a distributed log you can replay, not a to-do list. If you need event streaming and history, look there. If you need "send this email in the background", you don't.
If you remember one thing
- Queue anything the user doesn't need to wait for.
- Assume every message can arrive twice, and write jobs that don't care.
- Keep
retry_afterlonger than your slowest job. - Give failures somewhere to go, and watch that place.
- Choose the broker for the problem: Redis for simple speed, SQS for no ops, RabbitMQ for routing.
The Laravel queues documentation is worth a full read once; most of the settings above are covered there.
What's the first job you moved to a queue in your app, and what made you finally do it?

Be first to comment it...