The process keeps growing until the server kills it: what to check before you buy memory

You can see the shape without a profiler. The site answers. Over the day one process grows. The server kills it. Someone restarts it, the site is quiet, and the growth starts again. The proposal that arrives with the incident is another stick of memory, or a rewrite of the service that happened to be the one running. Both spend money before anyone has named the action that keeps the memory.

I have chased this on long-running applications, next to slow queries, and the two are not the same finding. A query can be slow and give the memory back. A process can be fast and still keep a little more after every job until the box runs out. Buying memory is reasonable when the process returns to its usual size and the remaining size is honest work. It is a postponement when a restart is the thing currently keeping the site up.

1. Write down the shape before you touch the box

Three facts get collapsed into "we need more RAM". The process name. Whether a restart is what made it recover. How long the quiet lasted. Split them. A process that grows for an hour and is killed is not the same incident as a single request that is large and then gone. A restart that fixes the site until lunch is a climb. A restart that fixes nothing is a different bug, and more memory will not explain it either.

Write the process, not "the server". A PHP request that dies at the end of the work usually hands the memory back, which is why this symptom hides in whatever stays up: a queue process, a Node process, a scheduler, anything you started and forgot. If you do not know which process the killer is shooting, you are about to buy memory for the whole machine because one of them misbehaves.

Do the watching on a copy. On the live site you learn the size at the moment the process is already dead, and you learn it in front of customers. A copy with a realistic amount of data is enough. An empty database hides both a heavy read and a leak, because there is nothing to hold. If the only running copy is production, say that out loud and take a restore before you experiment. The experiment is not a purchase.

• The process name, and the signal that killed it if you have it.
• Whether a restart is the current fix, and how long the quiet lasts.
• A copy with real volume. Not the live site, and not an empty schema.
• The action you suspect, named as a screen or a job, not as "traffic".

2. Repeat one action and watch the size come back

Pick one action. A screen someone uses, or one job the worker runs. Note the process size before it, run the action, note the size at the peak, then note the size again when the action is finished and the process is idle. Repeat that twenty times. You are looking for a step, not for a graph that impresses anyone.

If the size rises for the action and falls back when the action is done, the action is heavy. It held a large set and let go. That can still be a problem, and it can still be why a single request dies, but it is not a leak. The fix is to stop holding the whole set: a narrower read, a stream, a limit. More memory lets a larger set succeed once and leaves the shape in place.

If the size steps up and stays up after the work is finished, something is keeping it. Twenty repeats make the stair obvious. A cache that only inserts. A list on the process that appends and never drops. A client, a buffer or a log line retained on a long-lived object. The restart "fixes" this because it throws the stair away. The twenty-first repeat builds the first step again.

You do not need a week of metrics to see this. You need a baseline, a peak and an after, on one action, repeated. If the size wanders with no relation to the action, you picked the wrong action or the wrong process. That is a finding. Change the name on the note. Do not average it into a recommendation to buy hardware.

• Size before, peak, and size after the process is idle. Three numbers, one action.
• Twenty repeats. A single run cannot show a stair.
• A heavy action falls back. A leak does not. Write which one you saw.
• If nothing correlates, the process or the action on the note is wrong. Correct it.

3. Where the climb usually sits

A typical PHP request is a poor place for an overnight leak, because the request ends and the memory goes with it. Look at the processes that outlive a request before you spend a day profiling one controller. Queue workers and Node processes are the usual residents. They are also the processes a team adds when they are trying to make a slow click feel fast, which is how a climb gets introduced beside a fix.

A request can still be the heavy action. The classic shape is a loop that pulls rows into the process, one query at a time or one giant list, and holds them until the response is built. That shows up as time as well as size. Read it as a query problem first. How to group the slow log, and why a loop of statements is easy to under-count, is in reading the slow log before you split the service. Do not restate that work as a memory purchase. If the size falls back after the response, you do not have a leak. You have a read that holds too much, and the log is the next tool.

If the size does not fall back, stay on the long-lived process. A worker that stores a result "for next time" in a map with no cap will look healthy at request twenty and dead at request twenty thousand. A Node process that keeps a buffer, a socket or a growing array across jobs does the same. The boundary for when work should leave the request at all, and when a retry must not repeat a side effect, is a separate decision, covered in when a Node process belongs beside Laravel. A queue does not forgive a map that never forgets. It gives the map a process to live in.

4. What more memory does not buy

It does not name the action. It moves the kill later, which on a quiet product can look like a cure for weeks, until the stair reaches the new ceiling. On a busier product it looks like a cure until tomorrow afternoon. Either way you have changed the clock, not the step.

It does not justify a rewrite of the service. The stair is often a small retention in a process you meant to keep: the cache, the client you open per job and never close, the array you append to for a report nobody reads. A new service that copies those objects copies the stair. Split the service when the deploy boundary is the reason. Not because the process died and the name on the unit file was embarrassing.

It also does not replace the copy. Adding memory on the live box, then watching customers while you guess, is how the next kill arrives during the guess. Take the restore. Repeat the action. Write the three numbers. Then decide whether the remaining size, after the process has given back what it can, is actually more than the machine should hold. That remaining size is the only honest argument for a larger box.

• No purchase until you have seen the size come back, or named what keeps it.
• No service split whose only evidence is a killed process.
• The uncapped map, the unclosed client, the appended array: looked for by name.
• A larger box is argued from the size after the process goes idle, not from the size at the kill.

5. An order you can repeat the next time a restart "fixes" it

Name the process the killer is shooting. Confirm a restart is what recovered the site, and write how long the quiet lasted. Restore a copy with real rows. Pick one action. Record size before, peak, and after, twenty times. If it falls back, treat it as a heavy read and go to the queries. If it steps and stays, find the object that outlives the action: the cache with no cap, the client opened per job, the array that only grows. Fix that. Measure the same twenty again. Only then talk about the size of the machine.

Keep the note next to the incident, in words an on-call person can follow at night. "Restart it" is how this becomes a ritual. The three numbers are how it becomes a change. If the action is a screen a person waits on, the screen still wants its own timing. Memory that comes back can still be a slow answer. Those are two lines on the same note, not one purchase.

When to ask for a second look

You can name the process and run the twenty repeats without a second person. Ask when the size stays up and you cannot see what holds it, when the climb sits in a worker you are afraid to restart because it also sends mail or charges, or when the only environment is production and the copy does not exist yet.

That is the work I take: find out whether the process gives the memory back, on the system you already run, and change the retention before anyone resizes the box. Services cover that kind of production performance work. Selected work keeps each result next to the operation that produced it, rather than turning one incident into a story about the whole platform. If you want the second look, message me on LinkedIn. Send the process name, the action you repeated, and whether the size came back down. Leave the passwords out.