Linux process limits: understand ulimit, systemd and cgroup ceilings

Read the failure and effective limit

“Too many open files,” fork failures, thread-creation errors, and service start failures point to different limits. Check the process’s effective limits through /proc, prlimit or the service manager. The interactive shell value may not apply to the service.

systemd units, containers, PAM sessions, and application runtimes can each introduce another boundary.

Measure actual usage before raising ceilings

Count file descriptors, processes, threads, sockets, and queue depth. A leak can consume any larger limit you grant, so the growth pattern matters more than the moment it hits the ceiling.

For connection-heavy services, include upstream/downstream pool limits and database connection limits in the same analysis.

Change the right configuration layer

A ulimit command in one shell will not persist for a systemd service. Update the controlling layer, validate syntax, reload configuration correctly, and restart only the affected service when a restart is required.

Avoid setting everything to extremely high values; limits also constrain blast radius.

Retest under realistic concurrency

Reproduce the workload that reached the limit and verify both usage and latency after the change. If resource use continues to grow without returning, fix the application behavior if resource use keeps growing; a larger ceiling is not the final solution.

• Identify the exact exhausted resource.
• Inspect the process-effective limit.
• Measure growth before increasing it.
• Retest under the same concurrency.