The complete guide to SSH operations: from key management to hardening a server without locking yourself out

• Introduction
• 1. Creating keys — which type to choose
• 2. Distributing and revoking keys — working with authorized_keys
• 3. Client configuration — ~/.ssh/config is the standard document
• 4. Host key verification — do not dismiss the warning
• 5. Hardening sshd — in an order that does not lock you out
• 6. Port forwarding and tunnels
• 7. When the connection fails — a layered diagnostic order
• 8. Auditing and operational practice
• Quiz: check your understanding
• Closing
• References
• Further reading

Introduction

SSH is easy to learn and hard to operate. Back when getting a shell was the whole job, ssh user@host was enough. The moment you run dozens of servers and an audit requirement lands on you, the questions change. Who owns the keys, how do you cut off access for someone who left, how do you standardise the route through the bastion, and what do you do when a bad configuration change locks you out.

This blog already has an in-depth analysis of the SSH protocol. That post covers the internals: the transport layer, authentication methods, channels, certificates, the Terrapin attack. This post goes in the opposite direction. The subject here is configuration and procedure for someone who does not need to know the protocol but does need to run a hundred servers safely.

The baseline is OpenSSH 9.x or newer. Option availability varies by version, so if a directive mentioned here does not work, check the version installed on the server first.

ssh -V
sshd -V

1. Creating keys — which type to choose

Two choices are current. Ed25519 is the default pick, and RSA 4096 is only for when organisational policy such as FIPS compliance requires it.

ssh-keygen -t ed25519 -C 'youngju@laptop-2026' -f ~/.ssh/id_ed25519
ssh-keygen -t rsa -b 4096 -C 'youngju@laptop-2026' -f ~/.ssh/id_rsa
ssh-keygen -t ecdsa-sk -f ~/.ssh/id_ecdsa_sk
ssh-keygen -t ed25519-sk -f ~/.ssh/id_ed25519_sk

• -t is the key type, -b the bit length (meaningful for RSA only), -C the comment, -f the output file.
• The types carrying the -sk suffix require a FIDO2 hardware security key. The private key itself never leaves the hardware, so the risk of leaking it drops sharply. Supported from OpenSSH 8.2 onward.

Getting into the habit of naming both the person and the machine in the comment matters. Once authorized_keys has twenty lines in it, that comment is the only clue to which line belongs to whom.

Always put a passphrase on a key. If automation makes that hard, use an agent.

ssh-keygen -p -f ~/.ssh/id_ed25519
eval "$(ssh-agent -s)"
ssh-add ~/.ssh/id_ed25519
ssh-add -l
ssh-add -D

ssh-add -D removes every key loaded into the agent. Make running it a habit when you walk away from a shared workstation.

Fingerprint checks are what you use for key distribution and for audits.

ssh-keygen -lf ~/.ssh/id_ed25519.pub
ssh-keygen -lf /etc/ssh/ssh_host_ed25519_key.pub

2. Distributing and revoking keys — working with authorized_keys

ssh-copy-id -i ~/.ssh/id_ed25519.pub deploy@10.0.3.14

ssh-copy-id is convenient, but once the server count grows it stops being a management method. You need to move to a configuration management tool (Ansible, Salt, and so on) or to SSH certificates. If you manage the files directly, the permissions have to be exact. When permissions are too loose, sshd silently rejects the key, and that is the single most common cause of "I installed the key and it still asks for a password".

chmod 700 ~/.ssh
chmod 600 ~/.ssh/authorized_keys
chmod 600 ~/.ssh/id_ed25519
chmod 644 ~/.ssh/id_ed25519.pub

A home directory that is group-writable gets rejected too.

chmod 755 ~

Each line in authorized_keys can carry restrictions. That is especially useful for keys with a fixed purpose, such as deployment keys.

restrict,from="10.0.0.0/8",command="/usr/local/bin/deploy-only" ssh-ed25519 AAAAC3Nza... deploy@ci

• restrict is the safe default that turns every feature off. You then turn back on only what you need.
• from= limits where the connection may originate.
• command= runs only the specified command no matter what the client asks for.

Revocation does not end with deleting a line from a file. Sessions that are already open stay alive. A complete revocation goes in this order.

sudo -u deploy sed -i '/deploy@ci/d' /home/deploy/.ssh/authorized_keys
who
sudo pkill -TERM -u deploy sshd

Destructive command warning: the last command kills every SSH session belonging to that user. Your own session may be among them, so double-check the user and the target account.

3. Client configuration — ~/.ssh/config is the standard document

Writing down how to connect in a file rather than in somebody's memory reduces mistakes across the whole team.

Host bastion
  HostName bastion.example.com
  User youngju
  IdentityFile ~/.ssh/id_ed25519
  IdentitiesOnly yes
  ServerAliveInterval 30
  ServerAliveCountMax 3

Host prod-*
  User deploy
  ProxyJump bastion
  IdentityFile ~/.ssh/id_ed25519_deploy
  IdentitiesOnly yes
  StrictHostKeyChecking yes

Host prod-web-01
  HostName 10.0.3.14

The key directives mean the following.

• ProxyJump (-J) routes through the bastion. It is available from OpenSSH 7.3 onward and is far safer and shorter than the ProxyCommand combinations people used before.
• IdentitiesOnly yes makes the client try only the key you named. Without it, the client tries every key loaded in the agent one after another and trips the server side MaxAuthTries (default 6), so authentication fails. This is the classic problem for anyone who holds several keys.
• ServerAliveInterval and ServerAliveCountMax stop an idle connection from being dropped silently by NAT or a firewall.

The same thing works from the command line.

ssh -J bastion deploy@10.0.3.14
ssh -o IdentitiesOnly=yes -i ~/.ssh/id_ed25519_deploy deploy@10.0.3.14

Connection reuse makes repeated connections dramatically faster.

Host *
  ControlMaster auto
  ControlPath ~/.ssh/cm-%r@%h:%p
  ControlPersist 10m

ControlPersist is how long the master connection is kept after the last session exits. Note, though, that on a shared workstation another user could reuse the socket, so check the permissions on the socket path.

4. Host key verification — do not dismiss the warning

The host key warning is the only defence you have against a man-in-the-middle attack. But it also appears every time a server is reinstalled, so people learn to dismiss it reflexively. That habit is the danger.

ssh-keygen -F 10.0.3.14
ssh-keygen -R 10.0.3.14
ssh-keyscan -t ed25519 10.0.3.14

• -F finds an entry in known_hosts and -R removes it.
• ssh-keyscan fetches the host key from the server. Trusting what it fetched without verifying it is pointless, so you have to compare the fingerprint against the one shown on the server console.

Once you reach a certain scale, moving to SSH certificates is the right answer. If host keys are signed by a CA, the client only has to trust the one CA.

@cert-authority *.example.com ssh-ed25519 AAAAC3Nza...

User keys can be signed the same way. That removes the need to manage authorized_keys on every server, and a short validity period solves the revocation problem structurally. The issuing procedure and the important options depend on how your organisation implements its CA, so check the certificate-related options of ssh-keygen in the man page of the version you have installed.

5. Hardening sshd — in an order that does not lock you out

When you change server configuration, the order is the most important thing. Get it wrong and you lock yourself out.

The safe procedure is this.

1. Keep the current session open and open one more terminal.
2. Edit the configuration file.
3. Check the syntax with sshd -t.
4. Reload the service.
5. Do not close the existing session; confirm you can connect from the new terminal.
6. Only after that confirmation, close the original session.

sudo cp /etc/ssh/sshd_config /etc/ssh/sshd_config.bak-2026-08-15
sudo vi /etc/ssh/sshd_config
sudo sshd -t
sudo systemctl reload sshd

sshd -t checks syntax only. To see the values that will actually take effect, use extended test mode.

sudo sshd -T | sort | head -40
sudo sshd -T -C user=deploy,host=10.0.3.14,addr=10.0.3.14 | grep -i -E 'passwordauth|pubkey|permitroot'

-T prints the entire effective configuration and -C shows how the Match blocks resolve for a specific connection. If you use Match blocks, verify them with this command without exception. Conditions match differently from what people expect surprisingly often.

Here is a recommended configuration. The default value quoted for each directive is the one from the man page.

PermitRootLogin no
PasswordAuthentication no
KbdInteractiveAuthentication no
PubkeyAuthentication yes
AuthorizedKeysFile .ssh/authorized_keys
MaxAuthTries 3
LoginGraceTime 30
ClientAliveInterval 300
ClientAliveCountMax 2
AllowGroups sshusers
X11Forwarding no
PermitEmptyPasswords no
LogLevel VERBOSE

What each value means.

• The default for PermitRootLogin is prohibit-password. In other words even out of the box, password login for root is blocked while key login is allowed. Set it to no to block it entirely.
• The default for PasswordAuthentication is yes. Once you have moved to key authentication you must set it explicitly to no.
• The default for KbdInteractiveAuthentication is also yes. People sometimes think they blocked passwords and traffic still comes in through this path, so turn it off as well. Its old name, ChallengeResponseAuthentication, is a deprecated alias.
• The default for MaxAuthTries is 6, and failures start being written to the log once they reach half of that value.
• The default for LoginGraceTime is 120 seconds. It is generally better to shorten it so that connections which never log in are not held open for long.
• With AllowGroups, only users in the named groups may log in. DenyGroups is processed first, then AllowGroups.
• LogLevel VERBOSE records the fingerprint of the key used for authentication. In an environment with audit requirements it is effectively mandatory.

Use Match blocks for exceptions.

Match Group sftponly
  ChrootDirectory /srv/sftp/%u
  ForceCommand internal-sftp
  AllowTcpForwarding no
  PermitTunnel no

When its condition matches, Match overrides the settings up to the next Match or the end of the file. If the same keyword is satisfied by several Match blocks, only the first one applies. Order changes the result, so be careful.

Configuration is split up differently depending on the distribution. Recent Debian/Ubuntu and RHEL-family systems read fragment files via Include /etc/ssh/sshd_config.d/*.conf. If an included file defines a value earlier on, settings further down can be ignored, which is why it matters to make checking effective values with sshd -T a habit before you change anything.

6. Port forwarding and tunnels

ssh -L 15432:db.internal:5432 bastion
ssh -R 8080:localhost:3000 relay.example.com
ssh -D 1080 bastion
ssh -N -f -L 15432:db.internal:5432 bastion

• -L is local forwarding. A connection arriving on port 15432 of my machine is sent through the SSH server to db.internal:5432. You use it to reach a database on an internal network.
• -R is remote forwarding. It pulls a port on the server back to my machine. By default it binds only to the loopback address on the server; to expose it externally you have to enable GatewayPorts on the server.
• -D is dynamic forwarding and creates a SOCKS proxy.
• -N runs no remote command and -f sends the client to the background. You use them together for tunnel-only connections.

Forwarding is powerful, which is exactly why it needs to be controlled. Restrict it on the server side with these directives.

AllowTcpForwarding no
AllowAgentForwarding no
GatewayPorts no
PermitOpen 10.0.5.20:5432

Agent forwarding (-A) deserves particular caution. Anyone with root on the intermediate server can use your agent socket to authenticate to other servers as you. If the goal is going through a bastion, use ProxyJump. You get the same result without exposing the agent.

PermitOpen restricts destinations to a whitelist without blocking forwarding outright. It fits the situation where a developer has to reach the production database but must not reach other internal services. You can list several destinations separated by spaces, and setting it to none refuses every forwarding request.

File transfer does not use a separate tool; it uses the same channel. Recent OpenSSH changed scp to use the SFTP protocol internally, so some path expansion behaviour that used to work has changed. For bulk synchronisation, running rsync over SSH is better for resuming and for partial transfers.

scp -i ~/.ssh/id_ed25519 ./app.tar.gz deploy@10.0.3.14:/tmp/
sftp -J bastion deploy@10.0.3.14
rsync -avz -e 'ssh -J bastion' ./dist/ deploy@10.0.3.14:/srv/app/

7. When the connection fails — a layered diagnostic order

Splitting the problem into layers narrows the cause quickly.

Step 1 — can the network reach it.

nc -vz 10.0.3.14 22
ss -tlnp | grep ':22'

Step 2 — what does the server offer.

ssh -vvv deploy@10.0.3.14 2>&1 | head -60

-v gives one level, and -vv and -vvv get progressively more detailed. Here is how to read the output.

• If you never get as far as debug1: Connecting to ..., it is a network or firewall problem.
• If Authentications that can continue repeats after debug1: Offering public key: ..., the server rejected that key.
• Permission denied (publickey) is an authentication failure, Connection refused means the daemon is not running, and Connection timed out means the path is blocked. Telling those three messages apart is half of the diagnosis.

Step 3 — read the server-side log.

sudo journalctl -u sshd -n 100 --no-pager
sudo journalctl -u sshd --since '10 min ago' -g 'Failed|Invalid|Accepted'

On RHEL-family systems the unit is usually named sshd, while on Debian/Ubuntu it is often ssh. Use the name that matches your system.

Step 4 — permissions and SELinux.

sudo ls -ld /home/deploy /home/deploy/.ssh
sudo ls -l /home/deploy/.ssh/authorized_keys
sudo ausearch -m avc -ts recent | tail -20
sudo restorecon -Rv /home/deploy/.ssh

On RHEL-family systems, if the home directory was created by hand or the files were copied in from another path, the SELinux context is wrong and the key is rejected. restorecon is the standard fix. This symptom often does not show up clearly in the log as a permission problem, so it eats a lot of time.

Step 5 — account state.

sudo passwd -S deploy
sudo chage -l deploy
getent group sshusers

An account whose password is locked can, depending on the distribution settings, be blocked even for key authentication. An expired account produces the same result.

8. Auditing and operational practice

You have to be able to trace who came in and when.

last -a | head -20
lastb -a | head -20
sudo journalctl -u sshd -g 'Accepted' --since '7 days ago' | tail -40

With LogLevel VERBOSE enabled, the fingerprint of the key used for authentication is recorded alongside the event, so you can answer which key was this connection made with after the fact. Having that one line or not having it makes an enormous difference during an incident investigation.

There are three directions for dealing with brute force. First, turning off password authentication makes most automated attacks pointless. Second, restrict access itself by source address. Third, use a tool that blocks based on a failure threshold. It is worth stating plainly that moving off port 22 only reduces log noise and is not a security measure.

Keeping a list of periodic review items stops things slipping through.

• Quarterly confirmation that the keys in each server's authorized_keys belong to current staff and current systems
• Host key fingerprint lists kept in a separate repository
• sshd -T output pinned by configuration management, with changes tracked
• Bastion logs included in central collection
• A plan for moving to hardware security keys or SSH certificates

Once the server count grows, human discipline will not hold the line. It helps to fix in advance the criteria for deciding when to move to the next stage. Past ten servers, distribute authorized_keys through configuration management; past twenty people, move to SSH certificates; once audit requirements appear, force the bastion as the single entry point and record sessions. Defer these three steps and you end up doing them all at once, by which point nobody knows which key is on which server any more.

Restricting SSH access with a firewall is covered in this series in the firewall and access control guide. Read the procedure there for avoiding the accident of locking yourself out while changing rules remotely.

Quiz: check your understanding

chmod 755 ~
chmod 700 ~/.ssh
chmod 600 ~/.ssh/authorized_keys

ssh -vvv deploy@10.0.3.14 2>&1 | grep -i -E 'offering|authentications that can continue'
sudo journalctl -u sshd -n 50 --no-pager

ssh -o IdentitiesOnly=yes -i ~/.ssh/id_ed25519_deploy deploy@10.0.3.14

Host prod-*
  IdentityFile ~/.ssh/id_ed25519_deploy
  IdentitiesOnly yes

sudo cp /etc/ssh/sshd_config /etc/ssh/sshd_config.bak-2026-08-15
sudo sshd -t
sudo systemctl reload sshd
sudo sshd -T | grep -i -E 'permitrootlogin|passwordauthentication|allowgroups'

ssh -J bastion deploy@10.0.3.14

Host prod-*
  ProxyJump bastion

who
sudo pkill -TERM -u leaver sshd
sudo passwd -l leaver

nc -vz 10.0.3.14 22
ss -tlnp | grep ':22'
sudo systemctl status sshd

Closing

Incidents in SSH operations arrive in essentially two shapes. Either somebody who needs access cannot get in, or somebody whose access should have been cut is still getting in. The first problem announces itself loudly; the second sits there quietly. That is why auditing and revocation procedures come before convenience of access.

Boiled down to three things you can apply today: turn off password authentication and keyboard-interactive authentication, record key fingerprints with LogLevel VERBOSE, and always validate a configuration change from a new session while the old one is still alive. Those three alone prevent most incidents.

References

• sshd_config(5) — OpenSSH manual (verified 2026-08-15)
• ssh_config(5) — OpenSSH manual (verified 2026-08-15)
• ssh(1) — OpenSSH manual (verified 2026-08-15)
• ssh-keygen(1) — OpenSSH manual (verified 2026-08-15)
• OpenSSH official site (verified 2026-08-15)

Further reading

• Previous: The complete guide to processes and signals
• Next: The complete guide to undoing things in Git
• An in-depth analysis of the SSH protocol — how the transport layer and certificates work internally
• SSH escape sequences and remote operations — how to get out when a session hangs
• Linux terminal — practise the commands
• chmod calculator — check key file permissions