Eight Minutes In, Eight Minutes Out: Anatomy of a Crypto-Miner Intrusion

There's a temptation, when you write about catching an intruder, to make yourself the hero of every sentence. We're going to resist it, because the honest version is more useful — and frankly more interesting. An attacker got root on one of our servers and lived there for over a week before we caught it — and not because we were careless. That box had been hardened, more than once, and was running antivirus the whole time. That's the uncomfortable truth underneath this whole story: capable attackers get into well-tended servers, and most of the time nobody ever finds out. We found out. What happened after is the part we're proud of.

The page that wouldn't load

It started the way these things always do: something unrelated broke. Our internal dashboard went dark. The machine in the closet was fine, the brain was healthy — but a forward-auth dependency was pointing at a server that had gone completely unreachable. 100% packet loss, public and private.

The reason it was unreachable: the host had been suspended for crypto mining. Not by us. By the provider, who detected the mining load and pulled the plug before we did. Not a flattering sentence, but a true one — their abuse detection beat our own monitoring to the punch, and the honest version of this story includes that.

The instruction at that point was exactly four things, and the order mattered: bring it back, find the miner, stop it, and do not delete the forensics. That last clause is the whole discipline. The instinct under attack is to scrub — kill everything, wipe the user, rebuild clean. That instinct destroys the evidence you need to understand what actually happened and whether it can happen again. Preserve first, remediate second.

Eight minutes, the second time

Here's the response clock, from the system's own logs:

• 23:39:59 — the server comes back online. Load average: 32.83 on a freshly booted box. Something is pinning every core.
• 23:41:39 — the miner is identified: a MoneroOcean xmrig service, installed as a systemd unit with Restart=always so it would survive every reboot.
• 23:42:36 — the bigger find: a privileged user account that shouldn't exist, sitting in the sudo group, with a home directory full of attack tooling — a vulnerability scanner, a kernel exploit, custom mass-scanning binaries.
• 23:47:55 — miner dead, account locked. The service ignored a polite SIGTERM, so it got SIGKILL. The user got password-locked and nologin-shelled rather than deleted — locked, not erased, because erasing is destroying evidence.

From box-online to contained: about eight minutes. Not because anyone was heroic, but because the response was systematic — health sweep, identify persistence, identify the account, kill the live threat, preserve everything else.

By 23:51 containment was verified. By midnight — roughly twenty-three minutes after the first login — the full forensic report was written: timeline, indicators of compromise, root-cause analysis, and a list of evidence marked do-not-delete. Incident response of this shape is normally measured in weeks. This one was measured in minutes because the work was done by an agent that doesn't get tired, doesn't lose the thread, and reads a 400-line shell history faster than you can scroll it.

Eight minutes, the first time

Then we ran the attacker's clock — reconstructed from the auth logs and his own bash history, which he left sitting on disk. It's a tidy eight minutes of its own:

• 03:14:41 — a PHP webshell (82KB, full file-manager-and-RCE) is uploaded through an unauthenticated upload endpoint on a WordPress site we host for a client.
• 03:22:09 — through that shell, a new user is created and given a password.
• 03:22:21 — the user is added to sudo.
• 03:22:31 — a remote SSH login succeeds with that password, twenty-two seconds after the account was born.
• 03:22:36 — sudo su. Full root.

Webshell to root in eight minutes. Then he stayed eight days — defacing a page for SEO cloaking, installing a vuln scanner, building custom Go mass-scanners, running a WordPress vulnerability sweep against four million targets from our box, compiling and running a kernel privilege-escalation exploit, and finally — on day eight — dropping the Monero miner that got him caught.

Where the doors were

First, some context, because it matters: this was not a naked, default box. By the time of this incident we'd been fighting crypto-miner intrusions on our infrastructure since March — this was an ongoing campaign, not a one-off. The server had been through four separate hardening rounds. It had antivirus running. We even had the hosting provider's own AI assistant helping us audit it — and it missed this too. That's not an excuse; it's the point. The opening that got used was subtle enough to survive four hardening passes, a running AV daemon, and a second pair of automated eyes.

Two gaps let this happen, and only one of them was the attacker being clever.

Gap one — the SSH config that lied. We had turned password authentication off. The main SSH config said PasswordAuthentication no, exactly as we'd set it. But a cloud-init drop-in file — shipped by the provider, in a directory most people never think to check — said yes, and .d/ include files load after the main config and override it. So the server swore password login was off while quietly accepting passwords: our setting silently defeated by one we didn't know was there. Without it, the attacker's freshly-created account would have been stuck in the webshell with no way to pivot to a real shell. The lock we set was real; a provider default we couldn't see unlocked it behind our backs.

Gap two — the upload endpoint. A client WordPress site had an upload handler that accepted PHP and wrote it somewhere executable. That's the front door. The webshell walked right in.

So where did the two doors come from? One was a real gap on our side: a client's site had an upload endpoint that accepted PHP and wrote it somewhere executable. The other wasn't something we left open at all — it was a provider default, tucked in a config directory we'd have had no reason to check, quietly flipping a setting we'd correctly turned off. They're worth not blurring together: one is the kind of hole you close by being more careful, the other is the kind you don't even know exists until it's used against you.

What he left behind became a tool

Here's the turn. An attacker's home directory, captured intact, is a curriculum. His bash history was a step-by-step replay of exactly how a real, financially-motivated intruder operates on a Linux box — which binaries, which order, which persistence mechanism, what a miner's config and systemd unit actually look like in the wild.

We didn't throw that away. We studied it, and it became the seed of a behavioral crypto-mining defense — a detector built not on signatures but on the behavior this incident taught us: the load signature, the systemd persistence pattern, the process shape of a hidden miner. The intrusion that got past us trained the system that now watches for the next one.

What we'd tell you to take from this

1. Preserve before you scrub. Lock accounts, don't delete them. Kill processes, keep their files. The evidence is worth more than the few minutes of "clean" feeling you get from rm -rf.
2. Audit your defaults. The most dangerous line in your config is the one you never wrote — the vendor default that silently overrides the setting you did write. Your SSH config can say one thing and do another. Check what it actually does.
3. Your upload endpoints are your front door. Every place a user can put a file is a place an attacker can put a shell. Treat them accordingly.
4. Detection speed is a force multiplier — in both directions. The provider caught the mining before we did, which stings. But once we were looking, automated, evidence-preserving response turned a three-week engagement into a twenty-three-minute one. The lesson isn't "we're fast." It's that the combination of a tireless responder and a preserve-first discipline changes what incident response costs.