Hackers are often seen as computer burglars. Hackers can gain access to computer using exploit program, either local or remote. Remote exploits allow hacker to gain access to server without having local account.
Vulnerable services can be detected using nmap tool (port scanning - to see open ports and services running on them), telnet (to talk with service and learn about it).
Once we get past firewall, we can attack application via open port. Succesful attack might result in privileges escalation for example.
That is, after using nmap, once we know service name and version we can look in internet for it's vulnerabilities, for example on Bug Traq mailing list (available under address: http://www.securityfocus.com/archive/1).
Computer system is penetrated and we have root privileges.
Probably install something on computer, hide processess in kernel, perhaps hack firewall module in the kernel to allow for transmission via ports that open as desired (for example in reaction for specially prepared packet sequence), anything else.
User does not know he's hacked.
See also: Hiding processes, files and folders in Linux using kernel module (for kernel version 2.4), How to implement Firewall, Shellcode, Assembler & Machine Code, Buffer Overflow Attack, Heap Overflow type Attacks.