SSH Security: How to Lock Down Remote Access to Your Server

The moment a server goes online, it starts getting knocked on. Within minutes of a fresh machine getting a public IP, automated bots somewhere in the world will try to log in over SSH — guessing usernames, hammering passwords, probing for anything left open. This isn’t paranoia; it’s just the background noise of the internet. If your server runs on Linux, SSH is the single most important door to secure, because it’s the one that gives someone a shell.

The good news is that locking it down well doesn’t take much. A handful of settings, done once and done right, turns your SSH from a soft target into something the automated attacks simply bounce off. Let’s walk through what SSH security actually means and how to set it up properly, starting from the basics.

Why SSH is the door that matters most

SSH stands for Secure Shell. It’s the protocol you use to connect to a remote server and run commands on it as if you were sitting in front of it. If you’ve already met it in the basics, you know the everyday shape of it: SSH basics covers how the connection and the everyday commands work. This article is about the layer on top of that — how to make sure only the right people can get through.

Here’s why SSH gets so much attention from attackers. Most other services on a server do one narrow thing: a web server hands out pages, a database answers queries. But SSH hands out a shell — full command-line control of the machine. If someone breaks in through SSH, they don’t get one page or one record; they get the whole server. That’s why it’s the first thing worth hardening, and why getting it right matters more than almost anything else you’ll do.

The encryption in SSH is already strong — nobody is realistically breaking the math. The weak point is almost always how you log in: a guessable password, an account that shouldn’t be reachable, a default left in place. Those are the doors attackers actually walk through, and those are exactly what we’re going to close.

How SSH authentication works

To secure SSH, you first need to understand the two ways it lets you prove who you are. When you connect, the server has to be convinced you’re allowed in. There are two main methods for that.

Password authentication is the obvious one: you type a username and a password, and if they match, you’re in. Simple, but it has a fatal flaw on a public server — a password is something that can be guessed. Bots try millions of common combinations, and if your password is weak or reused, eventually one lands.

Key-based authentication works differently and is far stronger. Instead of a shared secret you type in, you have a key pair: two related files. One is the private key, which stays on your own computer and never leaves it. The other is the public key, which you place on the server. They’re mathematically linked: anything proven with the private key can be verified with the public one, but you can’t work backwards from the public key to the private key.

   YOUR LAPTOP                          SERVER
  ┌───────────────┐                ┌───────────────┐
  │  private key  │                │  public key   │
  │  (stays here, │                │ (placed here, │
  │   secret)     │                │  shareable)   │
  └───────┬───────┘                └───────┬───────┘
          │                                │
          │  "prove you hold the private   │
          │   key matching this public one"│
          │ ─────────────────────────────► │
          │                                │  checks the proof
          │  ◄───────────── access granted │

When you connect, the server asks your client to prove it holds the private key that matches the public key on file. Your client does that without ever sending the private key across the network. No secret travels over the wire, and there’s nothing to guess — a private key has so many possible values that brute-forcing it is hopeless. This is the foundation of good SSH security, and almost everything else builds on it.

Step one: use SSH keys instead of passwords

The first and biggest improvement you can make is to stop logging in with a password and use a key pair instead. This single change eliminates the entire category of password-guessing attacks, because there’s simply no password to guess.

Generating a key pair takes one command on your own machine:

# Create a modern, strong key pair
ssh-keygen -t ed25519 -C "john.doe@example.com"

This creates two files, usually in a hidden .ssh folder in your home directory: the private key (e.g. id_ed25519) and the public key (id_ed25519.pub). The -t ed25519 picks a modern, fast, very secure key type — it’s the recommended default today. The comment after -C is just a label to help you remember which key is which.

To use the key, you copy the public half to the server. The easy way is a helper command:

# Copy your public key to the server's authorized list
ssh-copy-id john@your-server-ip

This appends your public key to a file on the server called ~/.ssh/authorized_keys. From then on, when you run ssh john@your-server-ip, your client proves it holds the matching private key and you’re let in — no password prompt at all.

Step two: disable password login entirely

Setting up keys is great, but if password login is still allowed, attackers can keep trying passwords against your server all day long. The bots don’t know or care that you personally use a key — the door is still there for them to rattle. So once your key works, the next move is to turn password authentication off completely.

SSH’s behavior is controlled by a configuration file on the server, almost always at /etc/ssh/sshd_config. You edit it with an editor (using sudo, since it’s a system file) and look for the relevant setting:

# In /etc/ssh/sshd_config
PasswordAuthentication no

With that one line, the server stops accepting passwords for SSH entirely. From now on, the only way in is with a valid key. The endless stream of password-guessing bots now hits a wall — there’s no password to guess, so they fail instantly, every time.

After changing the config, you tell SSH to reload it so the change takes effect:

# Reload the SSH service to apply config changes
sudo systemctl restart ssh

Step three: stop root from logging in directly

The root account is the all-powerful administrator on a Linux server — it can do literally anything, with no guardrails. That makes it the most valuable possible target, and attackers know the username root exists on essentially every Linux box, so it’s the first thing they try. Letting root log in over SSH means a bot only has to get one account right, and it’s the most dangerous one.

The fix is to forbid direct root login and instead log in as a normal user, then elevate to root privileges with sudo only when you need them. In the same config file:

# In /etc/ssh/sshd_config
PermitRootLogin no

Now an attacker can’t target root directly at all. They’d first have to break into a regular account and then break sudo on top of it — two locks instead of one. This pairs naturally with the idea of giving each person their own ordinary account, which connects to how users and groups are managed on Linux: real people get named accounts, and root stays a destination you step up to, never a door you walk straight through.

Step four: tighten the rest of the config

With keys in place, passwords off, and root locked out, you’ve already closed the biggest holes. A few more settings sand down the remaining edges. None of these is dramatic on its own, but together they shrink your exposure.

Limit who can log in. By default, any user account on the system can SSH in. You can restrict it to an explicit short list, so even a valid account that’s not on the list is refused:

# Only these users may connect over SSH
AllowUsers john jane

Change the default port (optional). SSH listens on port 22 by default, and bots scan that port relentlessly. Moving SSH to a non-standard port won’t stop a determined attacker — it’s not real security on its own — but it dramatically cuts the volume of automated noise hitting your logs, because most bulk scanners only check the default.

# Move SSH off the well-known port (defense in depth, not a substitute)
Port 2222

Set a login grace timeout and limit attempts. These cap how long an unauthenticated connection can sit open and how many tries it gets, so a stuck or malicious connection can’t linger:

LoginGraceTime 30
MaxAuthTries 3

How key access maps to files

It helps to picture where all of this physically lives, because SSH key security is really just careful management of a few files and their permissions. On the server side, each user’s authorized keys sit in their own home directory:

/home/john/
└── .ssh/
    ├── authorized_keys     ← public keys allowed to log in as john
    └── (permissions matter: 700 on .ssh, 600 on the file)

SSH is deliberately strict about these permissions. If the .ssh folder or the authorized_keys file is readable or writable by other users, SSH will often refuse to use the key at all as a safety measure — because a key file anyone can edit is a key file anyone can add themselves to. This is exactly why understanding file permissions pays off here: the right permissions aren’t bureaucracy, they’re part of the security model. The folder should be 700 (only the owner can touch it) and the file 600 (only the owner can read or write it).

A sensible baseline, all together

You don’t have to invent your own policy. Here’s a solid, conventional starting configuration that covers everything above — the kind of baseline that keeps the automated flood out while staying perfectly usable:

# /etc/ssh/sshd_config — a hardened baseline
PasswordAuthentication no       # keys only, no password guessing
PermitRootLogin no              # no direct root; use sudo
AllowUsers john jane            # only these accounts may connect
MaxAuthTries 3                  # few tries per connection
LoginGraceTime 30               # don't let connections linger
Port 2222                       # off the default (optional, reduces noise)

Apply it, reload SSH, and confirm you can still log in from a fresh session before you close your existing one. That cautious habit — change, test in a new window, then trust it — is the difference between a hardened server and a self-inflicted lockout. With this in place, the constant background attacks against your server hit nothing but a closed, key-only door.

Why this matters beyond your own server

It’s tempting to think hardening SSH only matters for big, important machines. It doesn’t. A small personal VPS with nothing valuable on it is still a prize to an attacker — not for your data, but as a foothold to launch attacks on others, mine cryptocurrency, or join a botnet. Every reachable server is a target, full stop. Locking down SSH isn’t about how important you think your server is; it’s basic hygiene for being a responsible part of the internet.

And the payoff is enormous relative to the effort. The settings in this article take maybe ten minutes to apply once, and they shut the door on the overwhelming majority of real-world attacks — the automated, opportunistic ones that make up almost all of the noise. There’s no cleverness required; just close the easy doors, and the bots move on to softer targets.

Wrapping up

Here’s the whole picture in one place:

• SSH is the highest-value door on a server because it hands out a full shell — control of the entire machine — so it’s the first thing worth hardening.
• The encryption is strong; the weak point is how you log in. Almost all attacks are automated password guessing, so the goal is to remove anything guessable.
• Use key-based authentication instead of passwords: a private key stays on your machine, a public key lives on the server, and nothing secret travels the network.
• Disable password login (PasswordAuthentication no) once your key works, so there’s no password left to guess — but always test the key in a fresh session first.
• Forbid direct root login (PermitRootLogin no) and use a normal account plus sudo, turning one lock into two.
• Tighten the rest: limit who can connect (AllowUsers), optionally move off port 22, cap attempts, and keep your key-file permissions strict (700/600).

Once SSH is locked down, the natural next step is the layer in front of it: deciding which ports are even reachable from the outside in the first place. That’s the job of the firewall — the wall that controls who can knock on the door at all.