nohup — keep a command running after you log out — garron.me

This post was originally published on go2linux.org. The domain is no longer mine, but I am the original author. I am republishing it here on garron.me with corrections and improvements.

When you close a terminal or an SSH connection drops, the kernel sends the hangup signal (SIGHUP) to the processes started from that session, and most of them exit. nohup runs a command that ignores that signal, so a long job survives the logout.

Basic usage

nohup command &

nohup makes the command immune to SIGHUP. The & sends it to the background so you get your prompt back. The two do different jobs: without & the command still ignores the hangup, but it keeps your terminal busy.

Example — a long backup you do not want to babysit:

nohup tar -czf /backup/home.tar.gz /home &

The shell prints the job number and the process ID:

[1] 48213
nohup: ignoring input and appending output to 'nohup.out'

You can now log out. The backup keeps running.

Where the output goes

If you do not redirect anything, nohup appends both stdout and stderr to nohup.out in the current directory. If that directory is not writable, it uses $HOME/nohup.out.

In practice it is better to choose the file yourself:

nohup ./long-job.sh > long-job.log 2>&1 &

> long-job.log redirects stdout, and 2>&1 sends stderr to the same place. Without 2>&1 the error messages end up somewhere else and you lose them when you need them most.

To discard the output completely:

nohup ./long-job.sh > /dev/null 2>&1 &

Launching a job over SSH

When you start a background job in a one-line SSH command, ssh waits until every file descriptor attached to the session is closed. Redirect all three, including stdin, or the command will appear to hang:

ssh user@server 'nohup ./long-job.sh > long-job.log 2>&1 < /dev/null &'

Check on the job later

Follow the log:

tail -f long-job.log

Find the process:

pgrep -af long-job.sh

Stop it:

kill 48213

nohup only blocks SIGHUP. A normal kill (which sends SIGTERM) still stops the process.

Lower the priority

A long job can share the machine more politely if you combine nohup with nice:

nohup nice -n 10 ./long-job.sh > long-job.log 2>&1 &

You forgot nohup — use disown

If the command is already running, you do not have to start over. Suspend it, send it to the background, and detach it from the shell:

Ctrl+Z
bg
disown -h %1

disown -h tells Bash not to send SIGHUP to that job when the shell exits. The output is still connected to the terminal, though, so a job that writes a lot after the terminal is gone may fail. For anything important, nohup with a log file is safer.

When nohup is not enough

The process dies anyway. On systems where KillUserProcesses=yes is set in /etc/systemd/logind.conf, systemd kills every process of the user at logout, nohup or not. Most distributions ship it disabled, but if your jobs vanish when you log out, check that setting. In that case use systemd-run (below) or enable lingering for the user with loginctl enable-linger.

The program handles SIGHUP itself. Some programs install their own handler for SIGHUP (many daemons reload their configuration on it), which overrides what nohup set up.

You need to interact with it again. nohup detaches for good. You cannot reattach to the process to type something into it.

Alternatives

Tool	Use it when
`nohup cmd &`	A one-off job that needs no input, and a log file is enough
`disown -h`	The job is already running and you forgot `nohup`
`setsid cmd`	You want the command in a new session, fully detached from the terminal
`tmux` / `screen`	You want to come back later and see or control the program
`systemd-run`	You want resource limits, logging in the journal, and a proper unit

With tmux:

tmux new -s backup
./long-job.sh

Detach with Ctrl+B then D, log out, and later reattach with tmux attach -t backup.

With systemd-run, as a transient user unit:

systemd-run --user --unit=backup ./long-job.sh
journalctl --user -u backup -f

nohup — keep a command running after you log out