Debian Reference Osamu Aoki Copyright © 2007-2009 Osamu Aoki This Debian Reference (v2) (2009-10-24 04:04:08 UTC) is intended to provide a broad overview of the Debian system as a post-installation user's guide. It covers many aspects of system administration through shell-command examples for non-developers. Abstract This book is free; you may redistribute it and/or modify it under the terms of the GNU General Public License of any version compliant to the Debian Free Software Guidelines (DFSG). ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ Table of Contents Preface 1. Disclaimer 2. What is Debian 3. About this document 3.1. Guiding rules 3.2. Prerequisites 3.3. Conventions 3.4. Debian BTS 3.5. The popcon 3.6. The package size 3.7. Bug reports on this document 4. Some quotes for new users 1. GNU/Linux tutorials 1.1. Console basics 1.1.1. The shell prompt 1.1.2. The shell prompt under X 1.1.3. The root account 1.1.4. The root shell prompt 1.1.5. GUI system administration tools 1.1.6. Virtual consoles 1.1.7. How to leave the command prompt 1.1.8. How to shutdown the system 1.1.9. Recovering a sane console 1.1.10. Additional package suggestions for the newbie 1.1.11. An extra user account 1.1.12. sudo configuration 1.1.13. Play time 1.2. Unix-like filesystem 1.2.1. Unix file basics 1.2.2. Filesystem internals 1.2.3. Filesystem permissions 1.2.4. Control of permissions for newly created files: umask 1.2.5. Permissions for groups of users (group) 1.2.6. Timestamps 1.2.7. Links 1.2.8. Named pipes (FIFOs) 1.2.9. Sockets 1.2.10. Device files 1.2.11. Special device files 1.2.12. procfs and sysfs 1.3. Midnight Commander (MC) 1.3.1. Customization of MC 1.3.2. Starting MC 1.3.3. File manager in MC 1.3.4. Command-line tricks in MC 1.3.5. The internal editor in MC 1.3.6. The internal viewer in MC 1.3.7. Auto-start features of MC 1.3.8. FTP virtual filesystem of MC 1.4. The basic Unix-like work environment 1.4.1. The login shell 1.4.2. Customizing bash 1.4.3. Special key strokes 1.4.4. Unix style mouse operations 1.4.5. The pager 1.4.6. The text editor 1.4.7. Setting a default text editor 1.4.8. Customizing vim 1.4.9. Recording the shell activities 1.4.10. Basic Unix commands 1.5. The simple shell command 1.5.1. Command execution and environment variable 1.5.2. "$LANG" variable 1.5.3. "$PATH" variable 1.5.4. "$HOME" variable 1.5.5. Command line options 1.5.6. Shell glob 1.5.7. Return value of the command 1.5.8. Typical command sequences and shell redirection 1.5.9. Command alias 1.6. Unix-like text processing 1.6.1. Unix text tools 1.6.2. Regular expressions 1.6.3. Replacement expressions 1.6.4. Global substitution with regular expressions 1.6.5. Extracting data from text file table 1.6.6. Script snippets for piping commands 2. Debian package management 2.1. Debian package management prerequisites 2.1.1. Package configuration 2.1.2. Basic precautions 2.1.3. Life with eternal upgrades 2.1.4. Debian archive basics 2.1.5. Package dependencies 2.1.6. The event flow of the package management 2.1.7. First response to package management troubles 2.2. Basic package management operations 2.2.1. Basic package management operations with commandline 2.2.2. Interactive use of aptitude 2.2.3. Key bindings of aptitude 2.2.4. Package views under aptitude 2.2.5. Search method options with aptitude 2.2.6. The aptitude regex formula 2.2.7. Dependency resolution of aptitude 2.2.8. Package activity logs 2.2.9. Aptitude advantages 2.3. Examples of aptitude operations 2.3.1. Listing packages with regex matching on package names 2.3.2. Browsing with the regex matching 2.3.3. Purging removed packages for good 2.3.4. Tidying auto/manual install status 2.3.5. System wide upgrade with aptitude 2.4. Advanced package management operations 2.4.1. Advanced package management operations with commandline 2.4.2. Verification of installed package files 2.4.3. Safeguarding for package problems 2.4.4. Searching on the package meta data 2.5. Debian package management internals 2.5.1. Archive meta data 2.5.2. Top level "Release" file and authenticity 2.5.3. Archive level "Release" files 2.5.4. Fetching of the meta data for the package 2.5.5. The package state for APT 2.5.6. The package state for aptitude 2.5.7. Local copies of the fetched packages 2.5.8. Debian package file names 2.5.9. The dpkg command 2.5.10. The update-alternative command 2.5.11. The dpkg-statoverride command 2.5.12. The dpkg-divert command 2.6. Recovery from a broken system 2.6.1. Incompatibility with old user configuration 2.6.2. Different packages with overlapped files 2.6.3. Fixing broken package script 2.6.4. Rescue with the dpkg command 2.6.5. Recovering package selection data 2.7. Tips for the package management 2.7.1. How to pick Debian packages 2.7.2. Packages from mixed source of archives 2.7.3. Tweaking candidate version 2.7.4. Volatile and Backports.org 2.7.5. Automatic download and upgrade of packages 2.7.6. Limiting download bandwidth for APT 2.7.7. Emergency downgrading 2.7.8. Who uploaded the package? 2.7.9. The equivs package 2.7.10. Porting a package to the stable system 2.7.11. Proxy server for APT 2.7.12. Small public package archive 2.7.13. Recording and copying system configuration 2.7.14. Converting or installing an alien binary package 2.7.15. Extracting package without dpkg 2.7.16. More readings for the package management 3. The system initialization 3.1. An overview of the boot strap process 3.2. Stage 1: the BIOS 3.3. Stage 2: the boot loader 3.4. Stage 3: the mini-Debian system 3.5. Stage 4: the normal Debian system 3.5.1. The meaning of the runlevel 3.5.2. The configuration of the runlevel 3.5.3. The runlevel management example 3.5.4. The default parameter for each init script 3.5.5. The hostname 3.5.6. The filesystem 3.5.7. Network interface initialization 3.5.8. Network service initialization 3.5.9. The system message 3.5.10. The kernel message 3.5.11. The udev system 3.5.12. The kernel module initialization 4. Authentication 4.1. Normal Unix authentication 4.2. Managing account and password information 4.3. Good password 4.4. Creating encrypted password 4.5. PAM and NSS 4.5.1. Configuration files accessed by the PAM and NSS 4.5.2. The modern centralized system management 4.5.3. "Why GNU su does not support the wheel group" 4.5.4. Stricter password rule 4.6. Other access controls 4.6.1. sudo 4.6.2. SELinux 4.6.3. Restricting access to some server services 4.7. Security of authentication 4.7.1. Secure password over the Internet 4.7.2. Secure Shell 4.7.3. Extra security measures for the Internet 4.7.4. Securing the root password 5. Network setup 5.1. The basic network infrastructure 5.1.1. The domain name 5.1.2. The hostname resolution 5.1.3. The network interface name 5.1.4. The network address range for the LAN 5.1.5. The network configuration infrastructure 5.1.6. The network device support 5.2. The network connection method 5.2.1. The DHCP connection with the Ethernet 5.2.2. The static IP connection with the Ethernet 5.2.3. The PPP connection with pppconfig 5.2.4. The alternative PPP connection with wvdialconf 5.2.5. The PPPoE connection with pppoeconf 5.3. The basic network configuration with ifupdown 5.3.1. The command syntax simplified 5.3.2. The basic syntax of "/etc/network/interfaces" 5.3.3. The loopback network interface 5.3.4. The network interface served by the DHCP 5.3.5. The network interface with the static IP 5.3.6. The basics of wireless LAN interface 5.3.7. The wireless LAN interface with WPA/WPA2 5.3.8. The wireless LAN interface with WEP 5.3.9. The PPP connection 5.3.10. The alternative PPP connection 5.3.11. The PPPoE connection 5.3.12. The network configuration state of ifupdown 5.3.13. The basic network reconfiguration 5.3.14. The ifupdown-extra package 5.4. The advanced network configuration with ifupdown 5.4.1. The ifplugd package 5.4.2. The ifmetric package 5.4.3. The virtual interface 5.4.4. The advanced command syntax 5.4.5. The mapping stanza 5.4.6. The manually switchable network configuration 5.4.7. Scripting with the ifupdown system 5.4.8. Mapping with guessnet 5.5. The network configuration for desktop 5.5.1. GUI network configuration tools 5.5.2. Automatic network configuration 5.6. The low level network configuration 5.6.1. Iproute2 commands 5.6.2. Safe low level network operations 5.7. Network optimization 5.7.1. Finding optimal MTU 5.7.2. Setting MTU 5.7.3. WAN TCP optimization 5.8. Netfilter infrastructure 6. Network applications 6.1. Web browsers 6.1.1. Browser configuration 6.2. The mail system 6.2.1. Modern mail service basics 6.2.2. The mail configuration strategy for workstation 6.3. Mail transport agent (MTA) 6.3.1. The configuration of exim4 6.3.2. The configuration of postfix with SASL 6.3.3. The mail address configuration 6.3.4. Basic MTA operations 6.4. Mail user agent (MUA) 6.4.1. Basic MUA — Mutt 6.5. The remote mail retrieval and forward utility 6.5.1. getmail configuration 6.5.2. fetchmail configuration 6.6. Mail delivery agent (MDA) with filter 6.6.1. maildrop configuration 6.6.2. procmail configuration 6.6.3. Redeliver mbox contents 6.7. POP3/IMAP4 server 6.8. The print server and utility 6.9. The remote access server and utility (SSH) 6.9.1. Basics of SSH 6.9.2. Port forwarding for SMTP/POP3 tunneling 6.9.3. Connecting without remote passwords 6.9.4. Dealing with alien SSH clients 6.9.5. Setting up ssh-agent 6.9.6. How to shutdown the remote system on SSH 6.9.7. Troubleshooting SSH 6.10. Other network application servers 6.11. Other network application clients 6.12. The diagnosis of the system daemons 7. The X Window System 7.1. Key packages 7.2. Setting up desktop environment 7.2.1. Debian menu 7.2.2. Freedesktop.org menu 7.2.3. Debian menu under GNOME desktop environment 7.3. The server/client relationship 7.4. The X server 7.4.1. The (re)configuration of the X server 7.4.2. The connection methods to the X server 7.5. Starting the X Window System 7.5.1. Starting X session with gdm 7.5.2. Customizing the X session (classic method) 7.5.3. Customizing the X session (new method) 7.5.4. Connecting a remote X client via SSH 7.5.5. Secure X terminal via the Internet 7.6. Fonts in the X Window 7.6.1. Basic fonts 7.6.2. Additional fonts 7.6.3. CJK fonts 7.7. X applications 7.7.1. X office applications 7.7.2. X utility applications 7.8. The X trivia 7.8.1. Keymaps and pointer button mappings in X 7.8.2. Classic X clients 7.8.3. The X terminal emulator — xterm 7.8.4. Running X clients as root 8. I18N and L10N 8.1. The keyboard input 8.1.1. The input method support with SCIM 8.1.2. An example for Japanese 8.1.3. Disabling the input method 8.2. The display output 8.3. The locale 8.3.1. Basics of encoding 8.3.2. Rationale for UTF-8 locale 8.3.3. The reconfiguration of the locale 8.3.4. The value of the "$LANG" environment variable 8.3.5. Specific locale only under X Window 8.3.6. Filename encoding 8.3.7. Localized messages and translated documentation 8.3.8. Effects of the locale 9. System tips 9.1. The screen program 9.1.1. The use scenario for screen(1) 9.1.2. Key bindings for the screen command 9.2. Data recording and presentation 9.2.1. The log daemon 9.2.2. Log analyzer 9.2.3. Recording the shell activities cleanly 9.2.4. Customized display of text data 9.2.5. Customized display of time and date 9.2.6. Colorized shell echo 9.2.7. Colorized commands 9.2.8. Recording the graphic image of an X application 9.2.9. Recording changes in configuration files 9.3. Data storage tips 9.3.1. Disk partition configuration 9.3.2. Accessing partition using UUID 9.3.3. Filesystem configuration 9.3.4. Filesystem creation and integrity check 9.3.5. Optimization of filesystem by mount options 9.3.6. Optimization of filesystem via superblock 9.3.7. Optimization of hard disk 9.3.8. Using SMART to predict hard disk failure 9.3.9. Expansion of usable storage space via LVM 9.3.10. Expansion of usable storage space by mounting another partition 9.3.11. Expansion of usable storage space using symlink 9.3.12. Expansion of usable storage space using aufs 9.4. Data encryption tips 9.4.1. Removable disk encryption with dm-crypt/LUKS 9.4.2. Encrypted swap partition with dm-crypt 9.4.3. Automatically encrypting files with eCryptfs 9.4.4. Automatically mounting eCryptfs 9.5. Monitoring, controlling, and starting program activities 9.5.1. Timing a process 9.5.2. The scheduling priority 9.5.3. The ps command 9.5.4. The top command 9.5.5. Listing files opened by a process 9.5.6. Tracing program activities 9.5.7. Identification of processes using files or sockets 9.5.8. Repeating a command with a constant interval 9.5.9. Repeating a command looping over files 9.5.10. Starting a program from GUI 9.5.11. Customizing program to be started 9.5.12. Killing a process 9.5.13. Scheduling tasks once 9.5.14. Scheduling tasks regularly 9.5.15. Alt-SysRq key 9.6. System maintenance tips 9.6.1. Who is on the system? 9.6.2. Warning everyone 9.6.3. Hardware identification 9.6.4. Hardware configuration 9.6.5. System and hardware time 9.6.6. The terminal configuration 9.6.7. The sound infrastructure 9.6.8. Disabling the screen saver 9.6.9. Disabling beep sounds 9.6.10. Memory usage 9.6.11. System security and integrity check 9.7. The kernel 9.7.1. Linux kernel 2.6 9.7.2. Kernel headers 9.7.3. Compiling the kernel and related modules 9.7.4. Compiling the kernel source: Debian standard method 9.7.5. Compiling the module source: Debian standard method 9.7.6. Compiling the kernel source: classic method 9.7.7. Non-free hardware drivers 9.8. Virtualized system 9.8.1. Virtualization tools 9.8.2. Chroot system 9.8.3. Setting up login for chroot 10. Data management 10.1. Sharing, copying, and archiving 10.1.1. Archive and compression tools 10.1.2. Copy and synchronization tools 10.1.3. Idioms for the archive 10.1.4. Idioms for the copy 10.1.5. Idioms for the selection of files 10.1.6. Backup and recovery 10.1.7. Backup utility suites 10.1.8. An example script for the system backup 10.1.9. A copy script for the data backup 10.1.10. Removable storage device 10.1.11. Sharing data via network 10.1.12. Archive media 10.2. The binary data 10.2.1. Making the disk image file 10.2.2. Writing directly to the disk 10.2.3. Mounting the disk image file 10.2.4. Making an empty disk image file 10.2.5. Viewing and editing binary data 10.2.6. Manipulating files without mounting disk 10.2.7. Data redundancy 10.2.8. Data file recovery and forensic analysis 10.2.9. Making the ISO9660 image file 10.2.10. Writing directly to the CD/DVD-R/RW 10.2.11. Mounting the ISO9660 image file 10.2.12. Splitting a large file into small files 10.2.13. Clearing file contents 10.2.14. Dummy files 10.2.15. Erasing an entire hard disk 10.2.16. Undeleting deleted but still open files 10.2.17. Searching all hardlinks 10.2.18. Invisible disk space consumption 10.3. Data security infrastructure 10.3.1. Key management for GnuPG 10.3.2. Using GnuPG on files 10.3.3. Using GnuPG with Mutt 10.3.4. Using GnuPG with Vim 10.3.5. The MD5 sum 10.4. Source code merge tools 10.4.1. Extracting differences for source files 10.4.2. Merging updates for source files 10.4.3. Updating via 3-way-merge 10.5. Version control systems 10.5.1. Comparison of VCS commands 10.6. CVS 10.6.1. Configuration of CVS repository 10.6.2. Local access to CVS 10.6.3. Remote access to CVS with pserver 10.6.4. Remote access to CVS with ssh 10.6.5. Importing a new source to CVS 10.6.6. File permissions in CVS repository 10.6.7. Work flow of CVS 10.6.8. Latest files from CVS 10.6.9. Administration of CVS 10.6.10. Execution bit for CVS checkout 10.7. Subversion 10.7.1. Configuration of Subversion repository 10.7.2. Access to Subversion via Apache2 server 10.7.3. Local access to Subversion by group 10.7.4. Remote access to Subversion via SSH 10.7.5. Subversion directory structure 10.7.6. Importing a new source to Subversion 10.7.7. Work flow of Subversion 10.8. Git 10.8.1. Configuration of Git client 10.8.2. Git references 10.8.3. Git commands 10.8.4. Git for recording configuration history 11. Data conversion 11.1. Text data conversion tools 11.1.1. Converting a text file with iconv 11.1.2. Converting file names with iconv 11.1.3. EOL conversion 11.1.4. TAB conversion 11.1.5. Editors with auto-conversion 11.1.6. Plain text extraction 11.1.7. Highlighting and formatting plain text data 11.2. XML data 11.2.1. Basic hints for XML 11.2.2. XML processing 11.2.3. The XML data extraction 11.3. Printable data 11.3.1. Ghostscript 11.3.2. Merge two PS or PDF files 11.3.3. Printable data utilities 11.3.4. Printing with CUPS 11.4. Type setting 11.4.1. roff typesetting 11.4.2. TeX/LaTeX 11.4.3. Pretty print a manual page 11.4.4. Creating a manual page 11.5. The mail data conversion 11.5.1. Mail data basics 11.6. Graphic data tools 11.7. Miscellaneous data conversion 12. Programming 12.1. The shell script 12.1.1. POSIX shell compatibility 12.1.2. Shell parameters 12.1.3. Shell conditionals 12.1.4. Shell loops 12.1.5. The shell command-line processing sequence 12.1.6. Utility programs for shell script 12.1.7. Shell script dialog 12.1.8. Shell script example with zenity 12.2. Make 12.3. C 12.3.1. Simple C program (gcc) 12.4. Debug 12.4.1. Basic gdb execution 12.4.2. Debugging the Debian package 12.4.3. Obtaining backtrace 12.4.4. Advanced gdb commands 12.4.5. Debugging X Errors 12.4.6. Check dependency on libraries 12.4.7. Memory leak detection tools 12.4.8. Static code analysis tools 12.4.9. Disassemble binary 12.5. Flex — a better Lex 12.6. Bison — a better Yacc 12.7. Autoconf 12.7.1. Compile and install a program 12.7.2. Uninstall program 12.8. Perl short script madness 12.9. Web 12.10. The source code translation 12.11. Making Debian package A. Appendix A.1. The Debian maze A.2. Copyright history A.3. Document format List of Tables 1.1. List of interesting text-mode program packages 1.2. List of informative documentation packages 1.3. List of usage of key directories 1.4. List of the first character of "ls -l" output 1.5. The numeric mode for file permissions in chmod(1) commands 1.6. The umask value examples 1.7. List of notable system-provided groups for file access 1.8. List of notable system provided groups for particular command executions 1.9. List of types of timestamps 1.10. List of special device files 1.11. The key bindings of MC 1.12. The reaction to the enter key in MC 1.13. List of shell programs 1.14. List of key bindings for bash 1.15. List of Unix style mouse operations 1.16. List of basic Unix commands 1.17. 3 parts of locale value 1.18. List of locale recommendations 1.19. List of "$HOME" values 1.20. Shell glob patterns 1.21. Command exit codes 1.22. Shell command idioms 1.23. Predefined file descriptors 1.24. Metacharacters for BRE and ERE 1.25. The replacement expression 1.26. List of script snippets for piping commands 2.1. List of Debian package management tools 2.2. List of Debian archive sites 2.3. List of Debian archive components 2.4. The relationship between suite and codename 2.5. List of key web site to resolving problems with a specific package 2.6. Basic package management operations with commandline using aptitude(8) and apt-get(8) /apt-cache(8) 2.7. Notable command options for aptitude(8) 2.8. List of key bindings for aptitude 2.9. List of views for aptitude 2.10. The categorization of standard package views 2.11. List of the aptitude regex formula 2.12. The log files for package activities 2.13. List of advanced package management operations 2.14. The content of the Debian archive meta data 2.15. The name structure of Debian packages 2.16. The usable characters for each component in the Debian package names 2.17. The notable files created by dpkg 2.18. List of the default Pin-Priority value for each package source type 2.19. List of the proxy tools specially for Debian archive 3.1. List of boot loaders 3.2. The meaning of GRUB parameters 3.3. List of runlevels and description of their usage 3.4. List of kernel error levels 4.1. 3 important configuration files for pam_unix(8) 4.2. The second entry content of "/etc/passwd" 4.3. List of commands to manage account information 4.4. List of tools to generate password 4.5. List of notable PAM and NSS systems 4.6. List of configuration files accessed by the PAM 4.7. List of insecure and secure services and ports 4.8. List of tools to provide extra security measures 5.1. List of network address ranges 5.2. List of network configuration tools 5.3. List of network connection methods and connection paths 5.4. List of network connection configurations 5.5. List of network connection acronyms 5.6. List of configuration files for the PPP connection with pppconfig 5.7. List of configuration files for the PPP connection with wvdialconf 5.8. List of configuration files for the PPPoE connection with pppoeconf 5.9. List of basic network configuration commands with ifupdown 5.10. List of stanzas in "/etc/network/interfaces" 5.11. List of acronyms for WLAN 5.12. List of terminology for network devices 5.13. List of advanced network configuration commands with ifupdown 5.14. List of environment variables passed by the ifupdown system 5.15. Translation table from obsolete net-tools commands to new iproute2 commands 5.16. List of low level network commands 5.17. List of network optimization tools 5.18. Basic guide lines of the optimal MTU value 5.19. List of firewall tools 6.1. List of web browsers 6.2. List of browser plugin packages 6.3. List of basic mail transport agent related packages for workstation 6.4. List of choices for mail transport agent (MTA) packages in Debian archive 6.5. List of important postfix manual pages 6.6. List of mail address related configuration files 6.7. List of basic MTA operation 6.8. List of mail user agent (MUA) 6.9. List of remote mail retrieval and forward utilities 6.10. List of MDA with filter 6.11. List of POP3/IMAP4 servers 6.12. List of print servers and utilities 6.13. List of remote access server and utilities 6.14. List of SSH authentication protocols and methods 6.15. List of SSH configuration files 6.16. List of SSH client startup examples 6.17. List of free SSH clients for other platforms 6.18. List of other network application servers 6.19. List of network application clients 6.20. List of popular RFCs 7.1. List of key (meta)packages for X Window 7.2. List of server/client terminology 7.3. List of connection methods to the X server 7.4. Table of packages to support X Window font systems 7.5. Table of corresponding PostScript Type 1 fonts 7.6. Table of corresponding TrueType fonts 7.7. Table of key words used in CJK font names to indicate font types 7.8. List of basic X office applications 7.9. List of basic X utility applications 8.1. List of keyboard reconfiguration methods 8.2. List of input method supports with SCIM 9.1. List of programs to support interrupted network connections 9.2. List of key bindings for screen 9.3. List of system log analyzers 9.4. Display examples of time and date for the "ls -l" command for lenny 9.5. List of graphic image manipulation tools 9.6. List of packages to record configuration history in VCS 9.7. List of disk partition management packages 9.8. List of filesystem management packages 9.9. List of data encryption utilities 9.10. List of tools for monitoring and controlling program activities 9.11. List of nice values for the scheduling priority 9.12. List of ps command styles 9.13. List of commands for top 9.14. List of frequently used signals for kill command 9.15. List of SAK command keys 9.16. List of hardware identification tools 9.17. List of hardware configuration tools 9.18. List of sound packages 9.19. List of commands for disabling the screen saver 9.20. List of memory sizes reported 9.21. List of tools for system security and integrity check 9.22. List of key packages to be installed for the kernel recompilation on the Debian system 9.23. List of virtualization tools 10.1. List of archive and compression tools 10.2. List of copy and synchronization tools 10.3. List of backup suite utilities 10.4. List of packages which permit normal users to mount removable devices without a matching "/etc/fstab" entry 10.5. List of filesystem choices for removable storage devices with typical usage scenarios 10.6. List of the network service to chose with the typical usage scenario 10.7. List of packages which view and edit binary data 10.8. List of packages to manipulate files without mounting disk 10.9. List of tools to add data redundancy to files 10.10. List of packages for data file recovery and forensic analysis 10.11. List of data security infrastructure tools 10.12. List of GNU Privacy Guard commands for the key management 10.13. List of the meaning of the trust code 10.14. List of GNU Privacy Guard commands on files 10.15. List of source code merge tools 10.16. List of version control system tools 10.17. Comparison of native VCS commands 10.18. Notable options for CVS commands (use as first argument (s) to cvs(1)) 10.19. Notable options for Subversion commands (use as first argument(s) to svn(1)) 10.20. List of git related packages and commands 11.1. List of text data conversion tools 11.2. List of encoding values and their usage 11.3. List of EOL styles for different platforms 11.4. List of TAB conversion commands from bsdmainutils and coreutils packages 11.5. List of tools to extract plain text data 11.6. List of tools to highlight plain text data 11.7. List of predefined entities for XML 11.8. List of XML tools 11.9. List of DSSL tools 11.10. List of XML data extraction tools 11.11. List of XML pretty print tools 11.12. List of Ghostscript PostScript interpreters 11.13. List of printable data utilities 11.14. List of type setting tools 11.15. List of packages to help creating the manpage 11.16. List of packages to help mail data conversion 11.17. List of graphic data tools 11.18. List of miscellaneous data conversion tools 12.1. List of packages to help programing 12.2. List of typical bashisms 12.3. List of shell parameters 12.4. List of shell parameter expansions 12.5. List of key shell parameter substitutions 12.6. List of file comparison operators in the conditional expression 12.7. List of string comparison operators in the conditional expression 12.8. List of packages containing small utility programs for shell scripts 12.9. List of user interface programs 12.10. List of make automatic variables 12.11. List of make variable expansions 12.12. List of advanced gdb commands 12.13. List of memory leak detection tools 12.14. List of tools for static code analysis 12.15. List of Yacc-compatible LALR parser generators 12.16. List of source code translation tools Preface This Debian Reference (version 2) (2009-10-24 04:04:08 UTC) is intended to provide a broad overview of Debian system administration as a post-installation user guide. The target reader is someone who is willing to learn shell scripts but who is not ready to read all the C sources to figure out how the GNU/Linux system works. 1. Disclaimer All warranties are disclaimed. All trademarks are property of their respective trademark owners. The Debian system itself is a moving target. This makes its documentation difficult to be current and correct. Although the current unstable version of Debian system was used as the basis for writing this, some contents may be already outdated by the time you read this. Please treat this document as the secondary reference. This document does not replace any authoritative guides. The author and contributors do not take responsibility for consequences of errors, omissions or ambiguity in this document. 2. What is Debian The Debian Project is an association of individuals who have made common cause to create a free operating system. It's distribution is characterized by the following. ● Commitment to the software freedom: Debian Social Contract and Debian Free Software Guidelines (DFSG) ● Internet based distributed unpaid volunteer effort: http:// www.debian.org ● Large number of pre-compiled high quality softwares ● Focus on stability and security with easy access to the security updates ● Focus on smooth upgrade to latest softwares with unstable and testing archives ● Large number of supported hardware architectures Free Software pieces in Debian come from GNU, Linux, BSD, X, ISC , Apache, Ghostscript, Common Unix Printing System , Samba, GNOME, KDE, Mozilla, OpenOffice.org, Vim, TeX, LaTeX, DocBook, Perl, Python, Tcl, Java, Ruby, PHP, Berkeley DB, MySQL, PostgreSQL, Exim, Postfix, Mutt, FreeBSD, OpenBSD, Plan 9 and many more independent free software projects. Debian integrates this diversity of Free Software into one system. 3. About this document 3.1. Guiding rules Following guiding rules are followed while compiling this document. ● Provide overview and skip corner cases. (Big Picture) ● Keep It Short and Simple. (KISS) ● Do not reinvent the wheel. (Use pointers to the existing references) ● Focus on non-GUI tools and consoles. (Use shell examples) ● Be objective. (Use popcon etc.) Tip I tried to elucidate hierarchical aspects and lower levels of the system. 3.2. Prerequisites Warning You are expected to make good efforts to seek answers by yourself beyond this documentation. This document only gives efficient starting points. You must seek solution by yourself from primary sources. ● The Debian site at http://www.debian.org for the general information ● The documentation under the "/usr/share/doc/" directory ● The Unix style manpage: "dpkg -L |grep '/man/ man.*/'" ● The GNU style info page: "dpkg -L |grep '/ info/'" ● The bug report: http://bugs.debian.org/ ● The Debian Wiki at http://wiki.debian.org/ for the moving and specific topics ● The HOWTOs from The Linux Documentation Project (TLDP) at http://tldp.org/ ● The Single UNIX Specification from the Open Group's The UNIX System Home Page at http://www.unix.org/ ● The free encyclopedia from Wikipedia at http://wikipedia.org / Note For detailed documentation, you may need to install the corresponding documentation package named with "-doc" as its suffix. 3.3. Conventions This document provides information through the following simplified presentation style with bash(1) shell command examples. # $ These shell prompts distinguish account used and correspond to set environment variables as: "PS1='\$'" and "PS2=' '". These values are chosen for the sake of readability of this document and are not typical on actual installed system. Note See the meaning of the "$PS1" and "$PS2" environment variables in bash(1). Action required by the system administrator is written in the imperative sentence, e.g. "Type Enter-key after typing each command string to the shell." The description column and similar ones in the table may contain a noun phrase following the package short description convention which drops leading articles such as "a" and "the". They may alternatively contain an infinitive phrase as a noun phrase without leading "to" following the short command description convention in manpages. These may look funny to some people but are my intentional choices of style to keep this documentation as simple as possible. These Noun phrases do not capitalize their starting nor end with periods following these short description convention. Note Proper nouns including command names keeps their case irrespective of their location. A command snippet quoted in a text paragraph is referred by the typewriter font between double quotation marks, such as "aptitude safe-upgrade". A text data from a configuration file quoted in a text paragraph is referred by the typewriter font between double quotation marks, such as "deb-src". A command is referred by its name in the typewriter font optionally followed by its manpage section number in parenthesis, such as bash(1). You are encouraged to obtain information by typing the following. $ man 1 bash A manpage is referred by its name in the typewriter font followed by its manpage section number in parenthesis, such as sources.list(5). You are encouraged to obtain information by typing the following. $ man 5 sources.list An info page is referred by its command snippet in the typewriter font between double quotation marks, such as "info make". You are encouraged to obtain information by typing the following. $ info make A filename is referred by the typewriter font between double quotation marks, such as "/etc/passwd". For configuration files, you are encouraged to obtain information by typing the following. $ sensible-pager "/etc/passwd" A directory name is referred by the typewriter font between double quotation marks, such as "/etc/init.d/". You are encouraged to explore its contents by typing the following. $ mc "/etc/init.d/" A package name is referred by its name in the typewriter font, such as vim. You are encouraged to obtain information by typing the following. $ dpkg -L vim $ apt-cache show vim $ aptitude show vim A documentation may indicate its location by the filename in the typewriter font between double quotation marks, such as "/usr/ share/doc/sysv-rc/README.runlevels.gz" and "/usr/share/doc/ base-passwd/users-and-groups.html"; or by its URL, such as http: //www.debian.org. You are encouraged to read the documentation by typing the following. $ zcat "/usr/share/doc/sysv-rc/README.runlevels.gz" | sensible-pager $ sensible-browser "/usr/share/doc/base-passwd/users-and-groups.html" $ sensible-browse "http://www.debian.org" An environment variable is referred by its name with leading "$" in the typewriter font between double quotation marks, such as "$TERM". You are encouraged to obtain its current value by typing the following. $ echo "$TERM" 3.4. Debian BTS Astarisk "*" placed right after each package name is linked to Debian bug tracking system (BTS) of each package. 3.5. The popcon The popcon data is presented as the objective measure for the popularity of each package. It was downloaded on 2009-10-02 17:49:56 UTC and contains the total submission of 86828 reports over 98248 binary packages and 19 architectures. Note Please note that the amd64 unstable archive contains only 26849 packages currently. The popcon data contains reports from many old system installations. The popcon number preceded with "V:" for "votes" is calculated by "100 * (the popcon submissions for the package executed recently on the PC)/(the total popcon submissions)". The popcon number preceded with "I:" for "installs" is calculated by "100 * (the popcon submissions for the package installed on the PC)/(the total popcon submissions)". Note The popcon figures should not be considered as absolute measures of the importance of packages. There are many factors which can skew statistics. For example, some system participating popcon may have mounted directories such as "/bin" with "noatime" option for system performance improvement and effectively disabled "vote" from such system. 3.6. The package size The package size data is also presented as the objective measure for each package. It is based on the "Installed-Size:" reported by "apt-cache show" or "aptitude show" command (currently on amd64 architecture for the unstable release). The reported size is in KiB (Kibibyte = unit for 1024 bytes). Note A package with a small numerical package size may indicate that the package in the unstable release is a dummy package which installs other packages with significant contents by the dependency. The dummy package enables a smooth transition or split of the package. Note A package size followed by "(*)" indicates that the package in the unstable release is missing and the package size for the experimental release is used instead. 3.7. Bug reports on this document Please file bug reports on the debian-reference package using reportbug(1) if you find any issues on this document. Please include correction suggestion by "diff -u" to the plain text version or to the source. 4. Some quotes for new users Here are some interesting quotes from the Debian mailing list which may help enlighten new users. ● "This is Unix. It gives you enough rope to hang yourself." --- Miquel van Smoorenburg ● "Unix IS user friendly… It's just selective about who its friends are." --- Tollef Fog Heen Chapter 1. GNU/Linux tutorials I think learning a computer system is like learning a new foreign language. Although tutorial books and documentation are helpful, you have to practice it yourself. In order to help you get started smoothly, I elaborate a few basic points. The powerful design of Debian GNU/Linux comes from the Unix operating system, i.e., a multiuser, multitasking operating system. You must learn to take advantage of the power of these features and similarities between Unix and GNU/Linux. Don't shy away from Unix oriented texts and don't rely solely on GNU/Linux texts, as this robs you of much useful information. "Rute User's Tutorial and Exposition", in the Debian non-free archive as the rutebook package (popcon: I:0.2), provides a good online resource to the generic system administration. Note If you have been using any Unix-like system for a while with command line tools, you probably know everything I explain here. Please use this as a reality check and refresher. 1.1. Console basics 1.1.1. The shell prompt Upon starting the system, you are presented with the character based login screen if you did not install X Window System with the display manager such as gdm. Suppose your hostname is foo, the login prompt looks as follows. foo login: If you did install a GUI environment such as GNOME or KDE, then you can get to a login prompt by Ctrl-Alt-F1, and you can return to the GUI environment via Alt-F7 (see Section 1.1.6, “Virtual consoles” below for more). At the login prompt, you type your username, e.g. penguin, and press the Enter-key, then type your password and press the Enter-key again. Note Following the Unix tradition, the username and password of the Debian system are case sensitive. The username is usually chosen only from the lowercase. The first user account is usually created during the installation. Additional user accounts can be created with adduser(8) by root. The system starts with the greeting message stored in "/etc/ motd" (Message Of The Day) and presents a command prompt. Debian GNU/Linux lenny/sid foo tty1 foo login: penguin Password: Last login: Sun Apr 22 09:29:34 2007 on tty1 Linux snoopy 2.6.20-1-amd64 #1 SMP Sun Apr 15 20:25:49 UTC 2007 x86_64 The programs included with the Debian GNU/Linux system are free software; the exact distribution terms for each program are described in the individual files in /usr/share/doc/*/copyright. Debian GNU/Linux comes with ABSOLUTELY NO WARRANTY, to the extent permitted by applicable law. foo:~$ Here, the main part of the greeting message can be customized by editing the "/etc/motd.tail" file. The first line is generated from the system information using "uname -snrvm". Now you are in the shell. The shell interprets your commands. 1.1.2. The shell prompt under X If you installed X Window System with a display manager such as GNOME's gdm by selecting "Desktop environment" task during the installation, you are presented with the graphical login screen upon starting your system. You type your username and your password to login to the non-privileged user account. Use tab to navigate between username and password, or use the mouse and primary click. You can gain the shell prompt under X by starting a x-terminal-emulator program such as gnome-terminal(1), rxvt(1) or xterm(1). Under the GNOME Desktop environment, clicking "Applications" → "Accessories" → "Terminal" does the trick. You can also see the section below Section 1.1.6, “Virtual consoles”. Under some other Desktop systems (like fluxbox), there may be no obvious starting point for the menu. If this happens, just try (right) clicking the center of the screen and hope for a menu to pop-up. 1.1.3. The root account The root account is also called superuser or privileged user. From this account, you can perform the following system administration tasks. ● Read, write, and remove any files on the system irrespective of their file permissions ● Set file ownership and permissions of any files on the system ● Set the password of any non-privileged users on the system ● Login to any accounts without their passwords This unlimited power of root account requires you to be considerate and responsible when using it. Warning Never share the root password with others. Note File permissions of a file (including hardware devices such as CD-ROM etc. which are just another file for the Debian system) may render it unusable or inaccessible by non-root users. Although the use of root account is a quick way to test this kind of situation, its resolution should be done through proper setting of file permissions and user's group membership (see Section 1.2.3, “Filesystem permissions”). 1.1.4. The root shell prompt Here are a few basic methods to gain the root shell prompt by using the root password. ● Type root at the character based login prompt. ● Click "Applications" → "Accessories" → "Root Terminal", under the GNOME Desktop environment. ● Type "su -l" from any user shell prompt. ○ This does not preserve the environment of the current user. ● Type "su" from any user shell prompt. ○ This preserves some of the environment of the current user. 1.1.5. GUI system administration tools When your desktop menu does not start GUI system administration tools automatically with the appropriate privilege, you can start them from the root shell prompt of the X terminal emulator, such as gnome-terminal(1), rxvt(1), or xterm(1). See Section 1.1.4, “The root shell prompt” and Section 7.8.4, “Running X clients as root”. Warning Never start the X display/session manager under the root account by typing in root to the prompt of the display manager such as gdm(1). Warning Never run untrusted remote GUI program under X Window when critical information is displayed since it may eavesdrop your X screen. 1.1.6. Virtual consoles In the default Debian system, there are six switchable VT100-like character consoles available to start the command shell directly on the Linux host. Unless you are in a GUI environment, you can switch between the virtual consoles by pressing the Left-Alt-key and one of the F1 — F6 keys simultaneously. Each character console allows independent login to the account and offers the multiuser environment. This multiuser environment is a great Unix feature, and very addictive. If you are under the X Window System, you gain access to the character console 1 by pressing Ctrl-Alt-F1 key, i.e., the left-Ctrl-key, the left-Alt-key, and the F1-key are pressed together. You can get back to the X Window System, normally running on the virtual console 7, by pressing Alt-F7. You can alternatively change to another virtual console, e.g. to the console 1, from the commandline. # chvt 1 1.1.7. How to leave the command prompt You type Ctrl-D, i.e., the left-Ctrl-key and the d-key pressed together, at the command prompt to close the shell activity. If you are at the character console, you return to the login prompt with this. Even though these control characters are referred as "control D" with the upper case, you do not need to press the Shift-key. The short hand expression, ^D, is also used for Ctrl-D. Alternately, you can type "exit". If you are at x-terminal-emulator(1), you can close x-terminal-emulator window with this. 1.1.8. How to shutdown the system Just like any other modern OS where the file operation involves caching data in memory for improved performance, the Debian system needs the proper shutdown procedure before power can safely be turned off. This is to maintain the integrity of files, by forcing all changes in memory to be written to disk. If the software power control is available, the shutdown procedure automatically turns off power of the system. (Otherwise, you may have to press power button for few seconds after the shutdown procedure.) You can shutdown the system under the normal multiuser mode from the commandline. # shutdown -h now You can shutdown the system under the single-user mode from the commandline. # poweroff -i -f Alternatively, you may type Ctrl-Alt-Delete (The left-Ctrl-key, the left-Alt-Key, and the Delete are pressed together) to shutdown if "/etc/inittab" contains "ca:12345:ctrlaltdel:/sbin/ shutdown -t1 -a -h now" in it. See inittab(5) for details. See Section 6.9.6, “How to shutdown the remote system on SSH”. 1.1.9. Recovering a sane console When the screen goes berserk after doing some funny things such as "cat ", type "reset" at the command prompt. You may not be able to see the command echoed as you type. You may also issue "clear" to clean up the screen. 1.1.10. Additional package suggestions for the newbie Although even the minimal installation of the Debian system without any desktop environment tasks provides the basic Unix functionality, it is a good idea to install few additional commandline and curses based character terminal packages such as mc and vim with aptitude(8) for beginners to get started by the following. # aptitude update ... # aptitude install mc vim sudo ... If you already had these packages installed, no new packages are installed. Table 1.1. List of interesting text-mode program packages ┌────────┬──────┬─────┬────────────────────────────────────────┐ │package │popcon│size │description │ ├────────┼──────┼─────┼────────────────────────────────────────┤ │mc * │V:12, │6452 │A text-mode full-screen file manager │ │ │I:27 │ │ │ ├────────┼──────┼─────┼────────────────────────────────────────┤ │sudo * │V:43, │612 │A program to allow limited root │ │ │I:73 │ │privileges to users │ ├────────┼──────┼─────┼────────────────────────────────────────┤ │ │V:14, │ │Unix text editor Vi IMproved, a │ │vim * │I:30 │1684 │programmers text editor (standard │ │ │ │ │version) │ ├────────┼──────┼─────┼────────────────────────────────────────┤ │vim-tiny│V:18, │ │Unix text editor Vi IMproved, a │ │* │I:91 │760 │programmers text editor (compact │ │ │ │ │version) │ ├────────┼──────┼─────┼────────────────────────────────────────┤ │emacs22 │V:4, │11020│GNU project Emacs, the Lisp based │ │* │I:7 │ │extensible text editor (version 22) │ ├────────┼──────┼─────┼────────────────────────────────────────┤ │emacs23 │V:1.0,│13272│GNU project Emacs, the Lisp based │ │* │I:1.8 │ │extensible text editor (version 23) │ ├────────┼──────┼─────┼────────────────────────────────────────┤ │w3m * │V:29, │1964 │Text-mode WWW browsers │ │ │I:84 │ │ │ ├────────┼──────┼─────┼────────────────────────────────────────┤ │gpm * │V:3, │564 │The Unix style cut-and-paste on the text│ │ │I:4 │ │console (daemon) │ └────────┴──────┴─────┴────────────────────────────────────────┘ It may be a good idea to read some informative documentations. Table 1.2. List of informative documentation packages ┌────────────────────┬──────┬─────┬────────────────────────────┐ │package │popcon│size │description │ ├────────────────────┼──────┼─────┼────────────────────────────┤ │ │ │ │Debian Project │ │doc-debian * │I:83 │376 │documentation, (Debian FAQ) │ │ │ │ │and other documents │ ├────────────────────┼──────┼─────┼────────────────────────────┤ │debian-policy * │I:3 │2748 │Debian Policy Manual and │ │ │ │ │related documents │ ├────────────────────┼──────┼─────┼────────────────────────────┤ │developers-reference│I:1.2 │1352 │Guidelines and information │ │* │ │ │for Debian developers │ ├────────────────────┼──────┼─────┼────────────────────────────┤ │maint-guide * │I:0.8 │644 │Debian New Maintainers' │ │ │ │ │Guide │ ├────────────────────┼──────┼─────┼────────────────────────────┤ │debian-history * │I:0.4 │2544 │History of the Debian │ │ │ │ │Project │ ├────────────────────┼──────┼─────┼────────────────────────────┤ │debian-faq * │I:52 │1190 │Debian FAQ │ ├────────────────────┼──────┼─────┼────────────────────────────┤ │doc-linux-text * │I:82 │8616 │Linux HOWTOs and FAQ (text) │ ├────────────────────┼──────┼─────┼────────────────────────────┤ │doc-linux-html * │I:0.9 │62564│Linux HOWTOs and FAQ (html) │ ├────────────────────┼──────┼─────┼────────────────────────────┤ │sysadmin-guide * │I:0.3 │964 │The Linux System │ │ │ │ │Administrators' Guide │ ├────────────────────┼──────┼─────┼────────────────────────────┤ │rutebook * │I:0.2 │8264 │Linux: Rute User's Tutorial │ │ │ │ │and Exposition (non-free) │ └────────────────────┴──────┴─────┴────────────────────────────┘ You can install some of these packages by the following. # aptitude install package_name 1.1.11. An extra user account If you do not want to use your main user account for the following training activities, you can create a training user account, e.g. fish by the following. # adduser fish Answer all questions. This creates a new account named as fish. After your practice, you can remove this user account and its home directory by the following. # deluser --remove-home fish 1.1.12. sudo configuration For the typical single user workstation such as the desktop Debian system on the laptop PC, it is common to deploy simple configuration of sudo(8) as follows to let the non-privileged user, e.g. penguin, to gain administrative privilege just with his user password but without the root password. # echo "penguin ALL=(ALL) ALL" >> /etc/sudoers This trick should only be used for the single user workstation which you administer and where you are the only user. Warning Do not set up accounts of regular users on multiuser workstation like this because it would be very bad for system security. Caution The password and the account of the penguin in the above example requires as much protection as the root password and the root account. Caution Administrative privilege in this context belongs to someone authorized to perform the system administration task on the workstation. Never give some manager in the Admin department of your company or your boss such privilege unless they are authorized and capable. Note For providing access privilege to limited devices and limited files, you should consider to use group to provide limited access instead of using the root privilege via sudo(8). Note With more thoughtful and careful configuration, sudo(8) can grant limited administrative privileges to other users on a shared system without sharing the root password. This can help with accountability with hosts with multiple administrators so you can tell who did what. On the other hand, you might not want anyone else to have such privileges. 1.1.13. Play time Now you are ready to play with the Debian system without risks as long as you use the non-privileged user account. This is because the Debian system is, even after the default installation, configured with proper file permissions which prevent non-privileged users from damaging the system. Of course, there may still be some holes which can be exploited but those who worry about these issues should not be reading this section but should be reading Securing Debian Manual. We learn the Debian system as a Unix-like system with the following. ● Section 1.2, “Unix-like filesystem” (basic concept) ● Section 1.3, “Midnight Commander (MC)” (survival method) ● Section 1.4, “The basic Unix-like work environment” (basic method) ● Section 1.5, “The simple shell command” (shell mechanism) ● Section 1.6, “Unix-like text processing” (text processing method) 1.2. Unix-like filesystem In GNU/Linux and other Unix-like operating systems, files are organized into directories. All files and directories are arranged in one big tree rooted at "/". It's called a tree because if you draw the filesystem, it looks like a tree but it is upside down. These files and directories can be spread out over several devices. mount(8) serves to attach the filesystem found on some device to the big file tree. Conversely, umount(8) detaches it again. On recent Linux kernels, mount(8) with some options can bind part of a file tree somewhere else or can mount filesystem as shared, private, slave, or unbindable. Supported mount options for each filesystem are available in "/share/doc/ linux-doc-2.6.*/Documentation/filesystems/". Directories on Unix systems are called folders on some other systems. Please also note that there is no concept for drive such as "A:" on any Unix system. There is one filesystem, and everything is included. This is a huge advantage compared to Windows. 1.2.1. Unix file basics Here are some Unix file basics. ● Filenames are case sensitive. That is, "MYFILE" and "MyFile" are different files. ● The root directory means root of the filesystem referred as simply "/". Don't confuse this with the home directory for the root user: "/root". ● Every directory has a name which can contain any letters or symbols except "/". The root directory is an exception; its name is "/" (pronounced "slash" or "the root directory") and it cannot be renamed. ● Each file or directory is designated by a fully-qualified filename, absolute filename, or path, giving the sequence of directories which must be passed through to reach it. The three terms are synonymous. ● All fully-qualified filenames begin with the "/" directory, and there's a "/" between each directory or file in the filename. The first "/" is the top level directory, and the other "/"'s separate successive subdirectories, until we reach the last entry which is the name of the actual file. The words used here can be confusing. Take the following fully-qualified filename as an example: "/usr/share/ keytables/us.map.gz". However, people also refers to its basename "us.map.gz" alone as a filename. ● The root directory has a number of branches, such as "/etc/" and "/usr/". These subdirectories in turn branch into still more subdirectories, such as "/etc/init.d/" and "/usr/local/ ". The whole thing viewed collectively is called the directory tree. You can think of an absolute filename as a route from the base of the tree ("/") to the end of some branch (a file). You also hear people talk about the directory tree as if it were a family tree: thus subdirectories have parents, and a path shows the complete ancestry of a file. There are also relative paths that begin somewhere other than the root directory. You should remember that the directory "../" refers to the parent directory. This terminology also applies to other directory like structures, such as hierarchical data structures. ● There's no special directory path name component that corresponds to a physical device, such as your hard disk. This differs from RT-11, CP/M, OpenVMS, MS-DOS, AmigaOS, and Microsoft Windows, where the path contains a device name such as "C:\". (However, directory entries do exist that refer to physical devices as a part of the normal filesystem. See Section 1.2.2, “Filesystem internals”.) Note While you can use almost any letters or symbols in a file name, in practice it is a bad idea to do so. It is better to avoid any characters that often have special meanings on the command line, including spaces, tabs, newlines, and other special characters: { } ( ) [ ] ' ` " \ / > < | ; ! # & ^ * % @ $ . If you want to separate words in a name, good choices are the period, hyphen, and underscore. You could also capitalize each word, "LikeThis". Experienced Linux users tend to avoid spaces in filenames. Note The word "root" can mean either "root user" or "root directory". The context of their usage should make it clear. Note The word path is used not only for fully-qualified filename as above but also for the command search path. The intended meaning is usually clear from the context. The detailed best practices for the file hierarchy are described in the Filesystem Hierarchy Standard ("/usr/share/doc/ debian-policy/fhs/fhs-2.3.txt.gz" and hier(7)). You should remember the following facts as the starter. Table 1.3. List of usage of key directories ┌─────────┬────────────────────────────────────────────────────┐ │directory│usage of the directory │ ├─────────┼────────────────────────────────────────────────────┤ │/ │the root directory │ ├─────────┼────────────────────────────────────────────────────┤ │/etc/ │system wide configuration files │ ├─────────┼────────────────────────────────────────────────────┤ │/var/log/│system log files │ ├─────────┼────────────────────────────────────────────────────┤ │/home/ │all the home directories for all non-privileged │ │ │users │ └─────────┴────────────────────────────────────────────────────┘ 1.2.2. Filesystem internals Following the Unix tradition, the Debian GNU/Linux system provides the filesystem under which physical data on hard disks and other storage devices reside, and the interaction with the hardware devices such as console screens and remote serial consoles are represented in an unified manner under "/dev/". Each file, directory, named pipe (a way two programs can share data), or physical device on a Debian GNU/Linux system has a data structure called an inode which describes its associated attributes such as the user who owns it (owner), the group that it belongs to, the time last accessed, etc. If you are really interested, see "/usr/include/linux/fs.h" for the exact definition of "struct inode" in the Debian GNU/Linux system. The idea of representing just about everything in the filesystem was a Unix innovation, and modern Linux kernels have developed this idea ever further. Now, even information about processes running in the computer can be found in the filesystem. This abstract and unified representation of physical entities and internal processes is very powerful since this allows us to use the same command for the same kind of operation on many totally different devices. It is even possible to change the way the kernel works by writing data to special files that are linked to running processes. Tip If you need to identify the correspondence between the file tree and the physical entity, execute mount(8) with no arguments. 1.2.3. Filesystem permissions Filesystem permissions of Unix-like system are defined for three categories of affected users. ● The user who owns the file (u) ● Other users in the group which the file belongs to (g) ● All other users (o) also referred to as "world" and "everyone" For the file, each corresponding permission allows following actions. ● The read (r) permission allows owner to examine contents of the file. ● The write (w) permission allows owner to modify the file. ● The execute (x) permission allows owner to run the file as a command. For the directory, each corresponding permission allows following actions. ● The read (r) permission allows owner to list contents of the directory. ● The write (w) permission allows owner to add or remove files in the directory. ● The execute (x) permission allows owner to access files in the directory. Here, the execute permission on a directory means not only to allow reading of files in that directory but also to allow viewing their attributes, such as the size and the modification time. ls(1) is used to display permission information (and more) for files and directories. When it is invoked with the "-l" option, it displays the following information in the order given. ● Type of file (first character) ● Access permission of the file (nine characters, consisting of three characters each for user, group, and other in this order) ● Number of hard links to the file ● Name of the user who owns the file ● Name of the group which the file belongs to ● Size of the file in characters (bytes) ● Date and time of the file (mtime) ● Name of the file Table 1.4. List of the first character of "ls -l" output ┌─────────┬─────────────────────┐ │character│meaning │ ├─────────┼─────────────────────┤ │- │normal file │ ├─────────┼─────────────────────┤ │d │directory │ ├─────────┼─────────────────────┤ │l │symlink │ ├─────────┼─────────────────────┤ │c │character device node│ ├─────────┼─────────────────────┤ │b │block device node │ ├─────────┼─────────────────────┤ │p │named pipe │ ├─────────┼─────────────────────┤ │s │socket │ └─────────┴─────────────────────┘ chown(1) is used from the root account to change the owner of the file. chgrp(1) is used from the file's owner or root account to change the group of the file. chmod(1) is used from the file's owner or root account to change file and directory access permissions. Basic syntax to manipulate a foo file is the following. # chown foo # chgrp foo # chmod [ugoa][+-=][rwxXst][,...] foo For example, you can make a directory tree to be owned by a user foo and shared by a group bar by the following. # cd /some/location/ # chown -R foo:bar . # chmod -R ug+rwX,o=rX . There are three more special permission bits. ● The set user ID bit (s or S instead of user's x) ● The set group ID bit (s or S instead of group's x) ● The sticky bit (t or T instead of other's x) Here the output of "ls -l" for these bits is capitalized if execution bits hidden by these outputs are unset. Setting set user ID on an executable file allows a user to execute the executable file with the owner ID of the file (for example root). Similarly, setting set group ID on an executable file allows a user to execute the executable file with the group ID of the file (for example root). Because these settings can cause security risks, enabling them requires extra caution. Setting set group ID on a directory enables the BSD-like file creation scheme where all files created in the directory belong to the group of the directory. Setting the sticky bit on a directory prevents a file in the directory from being removed by a user who is not the owner of the file. In order to secure contents of a file in world-writable directories such as "/tmp" or in group-writable directories, one must not only reset the write permission for the file but also set the sticky bit on the directory. Otherwise, the file can be removed and a new file can be created with the same name by any user who has write access to the directory. Here are a few interesting examples of file permissions. $ ls -l /etc/passwd /etc/shadow /dev/ppp /usr/sbin/exim4 crw------- 1 root root 108, 0 2007-04-29 07:00 /dev/ppp -rw-r--r-- 1 root root 1427 2007-04-16 00:19 /etc/passwd -rw-r----- 1 root shadow 943 2007-04-16 00:19 /etc/shadow -rwsr-xr-x 1 root root 700056 2007-04-22 05:29 /usr/sbin/exim4 $ ls -ld /tmp /var/tmp /usr/local /var/mail /usr/src drwxrwxrwt 10 root root 4096 2007-04-29 07:59 /tmp drwxrwsr-x 10 root staff 4096 2007-03-24 18:48 /usr/local drwxrwsr-x 4 root src 4096 2007-04-27 00:31 /usr/src drwxrwsr-x 2 root mail 4096 2007-03-28 23:33 /var/mail drwxrwxrwt 2 root root 4096 2007-04-29 07:11 /var/tmp There is an alternative numeric mode to describe file permissions with chmod(1). This numeric mode uses 3 to 4 digit wide octal (radix=8) numbers. Table 1.5. The numeric mode for file permissions in chmod(1) commands ┌─────────────┬────────────────────────────────────────────────┐ │digit │meaning │ ├─────────────┼────────────────────────────────────────────────┤ │1st optional │sum of set user ID (=4), set group ID (=2), and │ │digit │sticky bit (=1) │ ├─────────────┼────────────────────────────────────────────────┤ │2nd digit │sum of read (=4), write (=2), and execute (=1) │ │ │permissions for user │ ├─────────────┼────────────────────────────────────────────────┤ │3rd digit │ditto for group │ ├─────────────┼────────────────────────────────────────────────┤ │4th digit │ditto for other │ └─────────────┴────────────────────────────────────────────────┘ This sounds complicated but it is actually quite simple. If you look at the first few (2-10) columns from "ls -l" command output and read it as a binary (radix=2) representation of file permissions ("-" being "0" and "rwx" being "1"), the last 3 digit of the numeric mode value should make sense as an octal (radix=8) representation of file permissions to you. For example, try the following $ touch foo bar $ chmod u=rw,go=r foo $ chmod 644 bar $ ls -l foo bar -rw-r--r-- 1 penguin penguin 17 2007-04-29 08:22 bar -rw-r--r-- 1 penguin penguin 12 2007-04-29 08:22 foo Tip If you need to access information displayed by "ls -l" in shell script, you should use pertinent commands such as test(1), stat (1) and readlink(1). The shell builtin such as "[" or "test" may be used too. 1.2.4. Control of permissions for newly created files: umask What permissions are applied to a newly created file or directory is restricted by the umask shell builtin command. See dash(1), bash(1), and builtins(7). (file permissions) = (requested file permissions) & ~(umask value) Table 1.6. The umask value examples ┌─────┬─────────────────┬────────────────────┬─────────────────┐ │umask│file permissions │directory │usage │ │ │created │permissions created │ │ ├─────┼─────────────────┼────────────────────┼─────────────────┤ │0022 │-rw-r--r-- │-rwxr-xr-x │writable only by │ │ │ │ │the user │ ├─────┼─────────────────┼────────────────────┼─────────────────┤ │0002 │-rw-rw-r-- │-rwxrwxr-x │writable by the │ │ │ │ │group │ └─────┴─────────────────┴────────────────────┴─────────────────┘ The Debian system uses a user private group (UPG) scheme as its default. A UPG is created whenever a new user is added to the system. A UPG has the same name as the user for which it was created and that user is the only member of the UPG. UPG scheme makes it is safe to set umask to 0002 since every user has their own private group. (In some Unix variants, it is quite common to setup all normal users belonging to a single users group and is good idea to set umask to 0022 for security in such cases.) 1.2.5. Permissions for groups of users (group) In order to make group permissions to be applied to a particular user, that user needs to be made a member of the group using "sudo vigr". Note Alternatively, you may dynamically add users to groups during the authentication process by adding "auth optional pam_group.so" line to "/etc/pam.d/common-auth" and setting "/etc /security/group.conf". (See Chapter 4, Authentication.) The hardware devices are just another kind of file on the Debian system. If you have problems accessing devices such as CD-ROM and USB memory stick from a user account, you should make that user a member of the relevant group. Some notable system-provided groups allow their members to access particular files and devices without root privilege. Table 1.7. List of notable system-provided groups for file access ┌───────┬──────────────────────────────────────────────────────┐ │group │description for accessible files and devices │ ├───────┼──────────────────────────────────────────────────────┤ │dialout│full and direct access to serial ports ("/dev/ttyS │ │ │[0-3]") │ ├───────┼──────────────────────────────────────────────────────┤ │dip │limited access to serial ports for Dialup IP │ │ │connection to trusted peers │ ├───────┼──────────────────────────────────────────────────────┤ │cdrom │CD-ROM, DVD+/-RW drives │ ├───────┼──────────────────────────────────────────────────────┤ │audio │audio device │ ├───────┼──────────────────────────────────────────────────────┤ │video │video device │ ├───────┼──────────────────────────────────────────────────────┤ │scanner│scanner(s) │ ├───────┼──────────────────────────────────────────────────────┤ │adm │system monitoring logs │ ├───────┼──────────────────────────────────────────────────────┤ │staff │some directories for junior administrative work: "/usr│ │ │/local", "/home" │ └───────┴──────────────────────────────────────────────────────┘ Tip You need to belong to the dialout group to reconfigure modem, dial anywhere, etc. But if root creates pre-defined configuration files for trusted peers in "/etc/ppp/peers/", you only need to belong to the dip group to create Dialup IP connection to those trusted peers using pppd(8), pon(1), and poff(1) commands. Some notable system-provided groups allow their members to execute particular commands without root privilege. Table 1.8. List of notable system provided groups for particular command executions ┌───────┬──────────────────────────────────────────────────────┐ │group │accessible commands │ ├───────┼──────────────────────────────────────────────────────┤ │sudo │execute sudo without their password │ ├───────┼──────────────────────────────────────────────────────┤ │lpadmin│execute commands to add, modify, and remove printers │ │ │from printer databases │ ├───────┼──────────────────────────────────────────────────────┤ │plugdev│execute pmount(1) for removable devices such as USB │ │ │memories │ └───────┴──────────────────────────────────────────────────────┘ For the full listing of the system provided users and groups, see the recent version of the "Users and Groups" document in "/ usr/share/doc/base-passwd/users-and-groups.html" provided by the base-passwd package. See passwd(5), group(5), shadow(5), newgrp(1), vipw(8), vigr(8), and pam_group(8) for management commands of the user and group system. 1.2.6. Timestamps There are three types of timestamps for a GNU/Linux file. Table 1.9. List of types of timestamps ┌─────┬────────────────────────────────────┐ │type │meaning │ ├─────┼────────────────────────────────────┤ │mtime│the file modification time (ls -l) │ ├─────┼────────────────────────────────────┤ │ctime│the file status change time (ls -lc)│ ├─────┼────────────────────────────────────┤ │atime│the last file access time (ls -lu) │ └─────┴────────────────────────────────────┘ Note ctime is not file creation time. ● Overwriting a file changes all of the mtime, ctime, and atime attributes of the file. ● Changing ownership or permission of a file changes the ctime and atime attributes of the file. ● Reading a file changes the atime of the file. Note Even simply reading a file on the Debian system normally causes a file write operation to update atime information in the inode. Mounting a filesystem with "noatime" or "relatime" option makes the system skip this operation and results in faster file access for the read. This is often recommended for laptops, because it reduces hard drive activity and saves power. See mount(8). Use touch(1) command to change timestamps of existing files. For timestamps, the ls command outputs different strings under the modern English locale ("en_US.UTF-8") from under the old one ("C"). $ LANG=en_US.UTF-8 ls -l foo -rw-r--r-- 1 penguin penguin 3 2008-03-05 00:47 foo $ LANG=C ls -l foo -rw-r--r-- 1 penguin penguin 3 Mar 5 00:47 foo Tip See Section 9.2.5, “Customized display of time and date” to customize "ls -l" output. 1.2.7. Links There are two methods of associating a file "foo" with a different filename "bar". ● Hard link ○ Duplicate name for an existing file ○ "ln foo bar" ● Symbolic link or symlink ○ Special file that points to another file by name ○ "ln -s foo bar" See the following example for changes in link counts and the subtle differences in the result of the rm command. $ echo "Original Content" > foo $ ls -li foo 2398521 -rw-r--r-- 1 penguin penguin 17 2007-04-29 08:15 foo $ ln foo bar # hard link $ ln -s foo baz # symlink $ ls -li foo bar baz 2398521 -rw-r--r-- 2 penguin penguin 17 2007-04-29 08:15 bar 2398538 lrwxrwxrwx 1 penguin penguin 3 2007-04-29 08:16 baz -> foo 2398521 -rw-r--r-- 2 penguin penguin 17 2007-04-29 08:15 foo $ rm foo $ echo "New Content" > foo $ ls -li foo bar baz 2398521 -rw-r--r-- 1 penguin penguin 17 2007-04-29 08:15 bar 2398538 lrwxrwxrwx 1 penguin penguin 3 2007-04-29 08:16 baz -> foo 2398540 -rw-r--r-- 1 penguin penguin 12 2007-04-29 08:17 foo $ cat bar Original Content $ cat baz New Content The hardlink can be made within the same filesystem and shares the same inode number which the "-i" option with ls(1) reveals. The symlink always has nominal file access permissions of "rwxrwxrwx", as shown in the above example, with the effective access permissions dictated by permissions of the file that it points to. Caution It is generally good idea not to create complicated symbolic links or hardlinks at all unless you have a very good reason. It may cause nightmares where the logical combination of the symbolic links results in loops in the filesystem. Note It is generally preferable to use symbolic links rather than hardlinks unless you have a good reason for using a hardlink. The "." directory links to the directory that it appears in, thus the link count of any new directory starts at 2. The ".." directory links to the parent directory, thus the link count of the directory increases with the addition of new subdirectories. If you are just moving to Linux from Windows, it soon becomes clear how well-designed the filename linking of Unix is, compared with the nearest Windows equivalent of "shortcuts". Because it is implemented in the filesystem, applications can't see any difference between a linked file and the original. In the case of hardlinks, there really is no difference. 1.2.8. Named pipes (FIFOs) A named pipe is a file that acts like a pipe. You put something into the file, and it comes out the other end. Thus it's called a FIFO, or First-In-First-Out: the first thing you put in the pipe is the first thing to come out the other end. If you write to a named pipe, the process which is writing to the pipe doesn't terminate until the information being written is read from the pipe. If you read from a named pipe, the reading process waits until there is nothing to read before terminating. The size of the pipe is always zero --- it does not store data, it just links two processes like the shell "|". However, since this pipe has a name, the two processes don't have to be on the same command line or even be run by the same user. Pipes were a very influential innovation of Unix. For example, try the following $ cd; mkfifo mypipe $ echo "hello" >mypipe & # put into background [1] 8022 $ ls -l mypipe prw-r--r-- 1 penguin penguin 0 2007-04-29 08:25 mypipe $ cat mypipe hello [1]+ Done echo "hello" >mypipe $ ls mypipe mypipe $ rm mypipe 1.2.9. Sockets Sockets are used extensively by all the Internet communication, databases, and the operating system itself. It is similar to the named pipe (FIFO) and allows processes to exchange information even between different computers. For the socket, those processes do not need to be running at the same time nor to be running as the children of the same ancestor process. This is the endpoint for the inter process communication (IPC). The exchange of information may occur over the network between different hosts. The two most common ones are the Internet socket and the Unix domain socket. Tip "netstat -an" provides a very useful overview of sockets that are open on a given system. 1.2.10. Device files Device files refer to physical or virtual devices on your system, such as your hard disk, video card, screen, or keyboard. An example of a virtual device is the console, represented by "/ dev/console". There are 2 types of device files. ● Character device ○ Accessed one character at a time ○ 1 character = 1 byte ○ E.g. keyboard device, serial port, … ● Block device ○ accessed in larger units called blocks ○ 1 block > 1 byte ○ E.g. hard disk, … You can read and write device files, though the file may well contain binary data which may be an incomprehensible-to-humans gibberish. Writing data directly to these files is sometimes useful for the troubleshooting of hardware connections. For example, you can dump a text file to the printer device "/dev/ lp0" or send modem commands to the appropriate serial port "/dev /ttyS0". But, unless this is done carefully, it may cause a major disaster. So be cautious. Note For the normal access to a printer, use lp(1). The device node number are displayed by executing ls(1) as the following. $ ls -l /dev/hda /dev/ttyS0 /dev/zero brw-rw---- 1 root cdrom 3, 0 2007-04-29 07:00 /dev/hda crw-rw---- 1 root dialout 4, 64 2007-04-29 07:00 /dev/ttyS0 crw-rw-rw- 1 root root 1, 5 2007-04-29 07:00 /dev/zero ● "/dev/hda" has the major device number 3 and the minor device number 0. This is read/write accessible by the user who belongs to cdrom group. ● "/dev/ttyS0" has the major device number 4 and the minor device number 64. This is read/write accessible by the user who belongs to dialout group. ● "/dev/zero" has the major device number 1 and the minor device number 5. This is read/write accessible by anyone. In the Linux 2.6 system, the filesystem under "/dev/" is automatically populated by the udev(7) mechanism. 1.2.11. Special device files There are some special device files. Table 1.10. List of special device files ┌───────┬──────┬───────────────────────────────────────────────┐ │device │action│description of response │ │file │ │ │ ├───────┼──────┼───────────────────────────────────────────────┤ │/dev/ │read │return "end-of-file (EOF) character" │ │null │ │ │ ├───────┼──────┼───────────────────────────────────────────────┤ │/dev/ │write │return nothing (a bottomless data dump pit) │ │null │ │ │ ├───────┼──────┼───────────────────────────────────────────────┤ │/dev/ │read │return "the \0 (NUL) character" (not the same │ │zero │ │as the number zero ASCII) │ ├───────┼──────┼───────────────────────────────────────────────┤ │/dev/ │ │return random characters from a true random │ │random │read │number generator, delivering real entropy │ │ │ │(slow) │ ├───────┼──────┼───────────────────────────────────────────────┤ │/dev/ │ │return random characters from a │ │urandom│read │cryptographically secure pseudorandom number │ │ │ │generator │ ├───────┼──────┼───────────────────────────────────────────────┤ │/dev/ │write │return the disk-full (ENOSPC) error │ │full │ │ │ └───────┴──────┴───────────────────────────────────────────────┘ These are frequently used in conjunction with the shell redirection (see Section 1.5.8, “Typical command sequences and shell redirection”). 1.2.12. procfs and sysfs The procfs and sysfs mounted on "/proc" and "/sys" are the pseudo-filesystem and expose internal data structures of the kernel to the userspace. In other word, these entries are virtual, meaning that they act as a convenient window into the operation of the operating system. The directory "/proc" contains (among other things) one subdirectory for each process running on the system, which is named after the process ID (PID). System utilities that access process information, such as ps(1), get their information from this directory structure. The directories under "/proc/sys/" contain interface to change certain kernel parameters at run time. (You may do the same through specialized sysctl(8) command or its preload/ configuration file "/etc/sysctrl.conf".) Note The Linux kernel may complain "Too many open files". You can fix this by increasing "file-max" value to a larger value from the root shell, e.g., "echo "65536" > /proc/sys/fs/file-max" (This was needed on older kernels). People frequently panic when they notice one file in particular - "/proc/kcore" - which is generally huge. This is (more or less) a copy of the content of your computer's memory. It's used to debug the kernel. It is a virtual file that points to computer memory, so don't worry about its size. The directory under "/sys" contains exported kernel data structures, their attributes, and their linkages between them. It also contains interface to change certain kernel parameters at run time. See "proc.txt(.gz)", "sysfs.txt(.gz)" and other related documents in the Linux kernel documentation ("/usr/share/doc/ linux-doc-2.6.*/Documentation/filesystems/*") provided by the linux-doc-2.6.* package. 1.3. Midnight Commander (MC) Midnight Commander (MC) is a GNU "Swiss army knife" for the Linux console and other terminal environments. This gives newbie a menu driven console experience which is much easier to learn than standard Unix commands. You may need to install the Midnight Commander package which is titled "mc" by the following. $ sudo aptitude install mc Use the mc(1) command to explore the Debian system. This is the best way to learn. Please explore few interesting locations just using the cursor keys and Enter key. ● "/etc" and its subdirectories ● "/var/log" and its subdirectories ● "/usr/share/doc" and its subdirectories ● "/sbin" and "/bin" 1.3.1. Customization of MC In order to make MC to change working directory upon exit and cd to the directory, I suggest to modify "~/.bashrc" to include a script provided by the mc package. . /usr/share/mc/bin/mc.sh See mc(1) (under the "-P" option) for the reason. (If you do not understand what exactly I am talking here, you can do this later.) 1.3.2. Starting MC MC can be started by the following. $ mc MC takes care of all file operations through its menu, requiring minimal user effort. Just press F1 to get the help screen. You can play with MC just by pressing cursor-keys and function-keys. Note In some consoles such as gnome-terminal(1), key strokes of function-keys may be stolen by the console program. You can disable these features by "Edit" → "Keyboard Shortcuts" for gnome-terminal. If you encounter character encoding problem which displays garbage characters, adding "-a" to MC's command line may help prevent problems. If this doesn't clear up your display problems with MC, see Section 9.6.6, “The terminal configuration”. 1.3.3. File manager in MC The default is two directory panels containing file lists. Another useful mode is to set the right window to "information" to see file access privilege information, etc. Following are some essential keystrokes. With the gpm(8) daemon running, one can use a mouse on Linux character consoles, too. (Make sure to press the shift-key to obtain the normal behavior of cut and paste in MC.) Table 1.11. The key bindings of MC ┌──────────────┬───────────────────────────────────────────────┐ │key │key binding │ ├──────────────┼───────────────────────────────────────────────┤ │F1 │help menu │ ├──────────────┼───────────────────────────────────────────────┤ │F3 │internal file viewer │ ├──────────────┼───────────────────────────────────────────────┤ │F4 │internal editor │ ├──────────────┼───────────────────────────────────────────────┤ │F9 │activate pull down menu │ ├──────────────┼───────────────────────────────────────────────┤ │F10 │exit Midnight Commander │ ├──────────────┼───────────────────────────────────────────────┤ │Tab │move between two windows │ ├──────────────┼───────────────────────────────────────────────┤ │Insert or │mark file for a multiple-file operation such as│ │Ctrl-T │copy │ ├──────────────┼───────────────────────────────────────────────┤ │Del │delete file (be careful---set MC to safe delete│ │ │mode) │ ├──────────────┼───────────────────────────────────────────────┤ │Cursor keys │self-explanatory │ └──────────────┴───────────────────────────────────────────────┘ 1.3.4. Command-line tricks in MC ● cd command changes the directory shown on the selected screen. ● Ctrl-Enter or Alt-Enter copies a filename to the command line. Use this with cp(1) and mv(1) commands together with command-line editing. ● Alt-Tab shows shell filename expansion choices. ● One can specify the starting directory for both windows as arguments to MC; for example, "mc /etc /root". ● Esc + n-key → Fn (i.e., Esc + 1 → F1, etc.; Esc + 0 → F10) ● Pressing Esc before the key has the same effect as pressing the Alt and the key together.; i.e., type Esc + c for Alt-C. Esc is called meta-key and sometimes noted as "M-". 1.3.5. The internal editor in MC The internal editor has an interesting cut-and-paste scheme. Pressing F3 marks the start of a selection, a second F3 marks the end of selection and highlights the selection. Then you can move your cursor. If you press F6, the selected area is moved to the cursor location. If you press F5, the selected area is copied and inserted at the cursor location. F2 saves the file. F10 gets you out. Most cursor keys work intuitively. This editor can be directly started on a file using one of the following commands. $ mc -e filename_to_edit $ mcedit filename_to_edit This is not a multi-window editor, but one can use multiple Linux consoles to achieve the same effect. To copy between windows, use Alt-F keys to switch virtual consoles and use "File→Insert file" or "File→Copy to file" to move a portion of a file to another file. This internal editor can be replaced with any external editor of choice. Also, many programs use the environment variables "$EDITOR" or "$VISUAL" to decide which editor to use. If you are uncomfortable with vim(1) or nano(1) initially, you may set these to "mcedit" by adding the following lines to "~/.bashrc". export EDITOR=mcedit export VISUAL=mcedit I do recommend setting these to "vim" if possible. If you are uncomfortable with vim(1), you can keep using mcedit (1) for most system maintenance tasks. 1.3.6. The internal viewer in MC MC is a very smart viewer. This is a great tool for searching words in documents. I always use this for files in the "/usr/ share/doc" directory. This is the fastest way to browse through masses of Linux information. This viewer can be directly started using one of the following commands. $ mc -v path/to/filename_to_view $ mcview path/to/filename_to_view 1.3.7. Auto-start features of MC Press Enter on a file, and the appropriate program handles the content of the file (see Section 9.5.11, “Customizing program to be started”). This is a very convenient MC feature. Table 1.12. The reaction to the enter key in MC ┌─────────────────────────┬────────────────────────────────────┐ │file type │reaction to enter key │ ├─────────────────────────┼────────────────────────────────────┤ │executable file │execute command │ ├─────────────────────────┼────────────────────────────────────┤ │man file │pipe content to viewer software │ ├─────────────────────────┼────────────────────────────────────┤ │html file │pipe content to web browser │ ├─────────────────────────┼────────────────────────────────────┤ │"*.tar.gz" and "*.deb" │browse its contents as if │ │file │subdirectory │ └─────────────────────────┴────────────────────────────────────┘ In order to allow these viewer and virtual file features to function, viewable files should not be set as executable. Change their status using chmod(1) or via the MC file menu. 1.3.8. FTP virtual filesystem of MC MC can be used to access files over the Internet using FTP. Go to the menu by pressing F9, then type "p" to activate the FTP virtual filesystem. Enter a URL in the form "username:passwd@hostname.domainname", which retrieves a remote directory that appears like a local one. Try "[http.us.debian.org/debian]" as the URL and browse the Debian archive. 1.4. The basic Unix-like work environment Although MC enables you to do almost everything, it is very important for you to learn how to use the command line tools invoked from the shell prompt and become familiar with the Unix-like work environment. 1.4.1. The login shell You can select your login shell with chsh(1). Table 1.13. List of shell programs ┌───────┬───────┬─────┬─────┬──────────────────────────────────┐ │package│popcon │size │POSIX│description │ │ │ │ │shell│ │ ├───────┼───────┼─────┼─────┼──────────────────────────────────┤ │bash * │V:91, │3100 │Yes │Bash: the GNU Bourne Again SHell │ │ │I:99 │ │ │(de facto standard) │ ├───────┼───────┼─────┼─────┼──────────────────────────────────┤ │tcsh * │V:7, │736 │No │TENEX C Shell: an enhanced version│ │ │I:46 │ │ │of Berkeley csh │ ├───────┼───────┼─────┼─────┼──────────────────────────────────┤ │dash * │V:13, │236 │Yes │Debian Almquist Shell, good for │ │ │I:21 │ │ │shell script │ ├───────┼───────┼─────┼─────┼──────────────────────────────────┤ │zsh * │V:2, │12960│Yes │Z shell: the standard shell with │ │ │I:5 │ │ │many enhancements │ ├───────┼───────┼─────┼─────┼──────────────────────────────────┤ │pdksh *│V:0.2, │468 │Yes │public domain version of the Korn │ │ │I:1.2 │ │ │shell │ ├───────┼───────┼─────┼─────┼──────────────────────────────────┤ │csh * │V:0.5, │404 │No │OpenBSD C Shell, a version of │ │ │I:1.8 │ │ │Berkeley csh │ ├───────┼───────┼─────┼─────┼──────────────────────────────────┤ │ │V:0.2, │ │ │Stand-alone shell with builtin │ │sash * │I:1.0 │836 │Yes │commands (Not meant for standard "│ │ │ │ │ │/bin/sh") │ ├───────┼───────┼─────┼─────┼──────────────────────────────────┤ │ksh * │V:0.4, │2860 │Yes │the real, AT&T version of the Korn│ │ │I:1.5 │ │ │shell │ ├───────┼───────┼─────┼─────┼──────────────────────────────────┤ │rc * │V:0.11,│204 │No │implementation of the AT&T Plan 9 │ │ │I:0.8 │ │ │rc shell │ ├───────┼───────┼─────┼─────┼──────────────────────────────────┤ │posh * │V:0.01,│232 │Yes │Policy-compliant Ordinary SHell │ │ │I:0.13 │ │ │(pdksh derivative) │ └───────┴───────┴─────┴─────┴──────────────────────────────────┘ In this tutorial chapter, the interactive shell always means bash. 1.4.2. Customizing bash You can customize bash(1) behavior by "~/.bashrc". For example, try the following. # CD upon exiting MC . /usr/share/mc/bin/mc.sh # set CDPATH to good one CDPATH=.:/usr/share/doc:~/Desktop/src:~/Desktop:~ export CDPATH PATH="${PATH}":/usr/sbin:/sbin # set PATH so it includes user's private bin if it exists if [ -d ~/bin ] ; then PATH=~/bin:"${PATH}" fi export PATH EDITOR=vim export EDITOR Tip You can find more bash customization tips, such as Section 9.2.7, “Colorized commands”, in Chapter 9, System tips. 1.4.3. Special key strokes In the Unix-like environment, there are few key strokes which have special meanings. Please note that on a normal Linux character console, only the left-hand Ctrl and Alt keys work as expected. Here are few notable key strokes to remember. Table 1.14. List of key bindings for bash ┌───────────────────────┬──────────────────────────────────────┐ │key │description of key binding │ ├───────────────────────┼──────────────────────────────────────┤ │Ctrl-U │erase line before cursor │ ├───────────────────────┼──────────────────────────────────────┤ │Ctrl-H │erase a character before cursor │ ├───────────────────────┼──────────────────────────────────────┤ │Ctrl-D │terminate input (exit shell if you are│ │ │using shell) │ ├───────────────────────┼──────────────────────────────────────┤ │Ctrl-C │terminate a running program │ ├───────────────────────┼──────────────────────────────────────┤ │Ctrl-Z │temporarily stop program by moving it │ │ │to the background job │ ├───────────────────────┼──────────────────────────────────────┤ │Ctrl-S │halt output to screen │ ├───────────────────────┼──────────────────────────────────────┤ │Ctrl-Q │reactivate output to screen │ ├───────────────────────┼──────────────────────────────────────┤ │Ctrl-Alt-Del │reboot/halt the system, see inittab(5)│ ├───────────────────────┼──────────────────────────────────────┤ │Left-Alt-key │ │ │(optionally, │meta-key for Emacs and the similar UI │ │Windows-key) │ │ ├───────────────────────┼──────────────────────────────────────┤ │Up-arrow │start command history search under │ │ │bash │ ├───────────────────────┼──────────────────────────────────────┤ │Ctrl-R │start incremental command history │ │ │search under bash │ ├───────────────────────┼──────────────────────────────────────┤ │Tab │complete input of the filename to the │ │ │command line under bash │ ├───────────────────────┼──────────────────────────────────────┤ │Ctrl-V Tab │input Tab without expansion to the │ │ │command line under bash │ └───────────────────────┴──────────────────────────────────────┘ Tip The terminal feature of Ctrl-S can be disabled using stty(1). 1.4.4. Unix style mouse operations Unix style mouse operations are based on the 3 button mouse system. Table 1.15. List of Unix style mouse operations ┌────────────────────┬─────────────────────────────────────────┐ │action │response │ ├────────────────────┼─────────────────────────────────────────┤ │Left-click-and-drag │select and copy to the clipboard │ │mouse │ │ ├────────────────────┼─────────────────────────────────────────┤ │Left-click │select the start of selection │ ├────────────────────┼─────────────────────────────────────────┤ │Right-click │select the end of selection and copy to │ │ │the clipboard │ ├────────────────────┼─────────────────────────────────────────┤ │Middle-click │paste clipboard at the cursor │ └────────────────────┴─────────────────────────────────────────┘ The center wheel on the modern wheel mouse is considered middle mouse button and can be used for middle-click. Clicking left and right mouse buttons together serves as the middle-click under the 2 button mouse system situation. In order to use a mouse in Linux character consoles, you need to have gpm(8) running as daemon. 1.4.5. The pager less(1) is the enhanced pager (file content browser). Hit "h" for help. It can do much more than more(1) and can be supercharged by executing "eval $(lesspipe)" or "eval $ (lessfile)" in the shell startup script. See more in "/usr/share /doc/lessf/LESSOPEN". The "-R" option allows raw character output and enables ANSI color escape sequences. See less(1). 1.4.6. The text editor You should become proficient in one of variants of Vim or Emacs programs which are popular in the Unix-like system. I think getting used to Vim commands is the right thing to do, since Vi-editor is always there in the Linux/Unix world. (Actually, original vi or new nvi are programs you find everywhere. I chose Vim instead for newbie since it offers you help through F1 key while it is similar enough and more powerful.) If you chose either Emacs or XEmacs instead as your choice of the editor, that is another good choice indeed, particularly for programming. Emacs has a plethora of other features as well, including functioning as a newsreader, directory editor, mail program, etc. When used for programming or editing shell scripts, it intelligently recognizes the format of what you are working on, and tries to provide assistance. Some people maintain that the only program they need on Linux is Emacs. Ten minutes learning Emacs now can save hours later. Having the GNU Emacs manual for reference when learning Emacs is highly recommended. All these programs usually come with tutoring program for you to learn them by practice. Start Vim by typing "vim" and press F1-key. You should at least read the first 35 lines. Then do the online training course by moving cursor to "|tutor|" and pressing Ctrl-]. Note Good editors, such as Vim and Emacs, can be used to handle UTF-8 and other exotic encoding texts correctly with proper option in the x-terminal-emulator on X under UTF-8 locale with proper font settings. Please refer to their documentation on multibyte text. 1.4.7. Setting a default text editor Debian comes with a number of different editors. We recommend to install the vim package, as mentioned above. Debian provides unified access to the system default editor via command "/usr/bin/editor" so other programs (e.g., reportbug(1)) can invoke it. You can change it by the following. $ sudo update-alternatives --config editor The choice "/usr/bin/vim.basic" over "/usr/bin/vim.tiny" is my recommendation for newbies since it supports syntax highlighting. Tip Many programs use the environment variables "$EDITOR" or "$VISUAL" to decide which editor to use (see Section 1.3.5, “The internal editor in MC” and Section 9.5.11, “Customizing program to be started”). For the consistency on Debian system, set these to "/usr/bin/editor". (Historically, "$EDITOR" was "ed" and "$VISUAL" was "vi".) 1.4.8. Customizing vim You can customize vim(1) behavior by "~/.vimrc". For example, try the following " ------------------------------- " Local configuration " set nocompatible set nopaste set pastetoggle= syn on if $USER == "root" set nomodeline set noswapfile else set modeline set swapfile endif " filler to avoid the line above being recognized as a modeline " filler " filler 1.4.9. Recording the shell activities The output of the shell command may roll off your screen and may be lost forever. It is good practice to log shell activities into the file for you to review them later. This kind of record is essential when you perform any system administration tasks. The basic method of recording the shell activity is to run it under script(1). For example, try the following $ script Script started, file is typescript Do whatever shell commands under script. Press Ctrl-D to exit script. $ vim typescript See Section 9.2.3, “Recording the shell activities cleanly” . 1.4.10. Basic Unix commands Let's learn basic Unix commands. Here I use "Unix" in its generic sense. Any Unix clone OSs usually offer equivalent commands. The Debian system is no exception. Do not worry if some commands do not work as you wish now. If alias is used in the shell, its corresponding command outputs are different. These examples are not meant to be executed in this order. Try all following commands from the non-privileged user account. Table 1.16. List of basic Unix commands ┌─────────────┬────────────────────────────────────────────────┐ │command │description │ ├─────────────┼────────────────────────────────────────────────┤ │pwd │display name of current/working directory │ ├─────────────┼────────────────────────────────────────────────┤ │whoami │display current user name │ ├─────────────┼────────────────────────────────────────────────┤ │id │display current user identity (name, uid, gid, │ │ │and associated groups) │ ├─────────────┼────────────────────────────────────────────────┤ │file │display a type of file for the file "" │ ├─────────────┼────────────────────────────────────────────────┤ │type -p │display a file location of command " │ │" │ ├─────────────┼────────────────────────────────────────────────┤ │which │, , │ ││ │ ├─────────────┼────────────────────────────────────────────────┤ │type │display information on command "" │ ││ │ ├─────────────┼────────────────────────────────────────────────┤ │apropos │find commands related to "" │ │ │ │ ├─────────────┼────────────────────────────────────────────────┤ │man -k │, , │ │ │ │ ├─────────────┼────────────────────────────────────────────────┤ │whatis │display one line explanation on command " │ │" │ ├─────────────┼────────────────────────────────────────────────┤ │man -a │display explanation on command "" │ ││(Unix style) │ ├─────────────┼────────────────────────────────────────────────┤ │info │display rather long explanation on command " │ │" (GNU style) │ ├─────────────┼────────────────────────────────────────────────┤ │ls │list contents of directory (non-dot files and │ │ │directories) │ ├─────────────┼────────────────────────────────────────────────┤ │ls -a │list contents of directory (all files and │ │ │directories) │ ├─────────────┼────────────────────────────────────────────────┤ │ls -A │list contents of directory (almost all files and│ │ │directories, i.e., skip ".." and ".") │ ├─────────────┼────────────────────────────────────────────────┤ │ls -la │list all contents of directory with detail │ │ │information │ ├─────────────┼────────────────────────────────────────────────┤ │ls -lai │list all contents of directory with inode number│ │ │and detail information │ ├─────────────┼────────────────────────────────────────────────┤ │ls -d │list all directories under the current directory│ ├─────────────┼────────────────────────────────────────────────┤ │tree │display file tree contents │ ├─────────────┼────────────────────────────────────────────────┤ │lsof │list open status of file "" │ ├─────────────┼────────────────────────────────────────────────┤ │lsof -p │list files opened by the process ID: "" │ ├─────────────┼────────────────────────────────────────────────┤ │mkdir │make a new directory "" in the current │ │ │directory │ ├─────────────┼────────────────────────────────────────────────┤ │rmdir │remove a directory "" in the current │ │ │directory │ ├─────────────┼────────────────────────────────────────────────┤ │ │change directory to the directory "" in the│ │cd │current directory or in the directory listed in │ │ │the variable "$CDPATH" │ ├─────────────┼────────────────────────────────────────────────┤ │cd / │change directory to the root directory │ ├─────────────┼────────────────────────────────────────────────┤ │cd │change directory to the current user's home │ │ │directory │ ├─────────────┼────────────────────────────────────────────────┤ │cd / │change directory to the absolute path directory │ │ │"/" │ ├─────────────┼────────────────────────────────────────────────┤ │cd .. │change directory to the parent directory │ ├─────────────┼────────────────────────────────────────────────┤ │cd ~ │change directory to the home directory of the │ │ │user "" │ ├─────────────┼────────────────────────────────────────────────┤ │cd - │change directory to the previous directory │ ├─────────────┼────────────────────────────────────────────────┤ │" │ │ │ │ ├─────────────┼────────────────────────────────────────────────┤ │cp │copy a existing file "" to a new file " │ │" │ ├─────────────┼────────────────────────────────────────────────┤ │rm │remove a file "" │ ├─────────────┼────────────────────────────────────────────────┤ │mv │rename an existing file "" to a new name " │ │" ("" must not exist) │ ├─────────────┼────────────────────────────────────────────────┤ │mv │move an existing file "" to a new location │ │ │"/" (the directory "" must exist)│ ├─────────────┼────────────────────────────────────────────────┤ │ │move an existing file "" to a new location │ │mv │with a new name "/" (the directory " │ │/" must exist but the directory "/│ │ │" must not exist) │ ├─────────────┼────────────────────────────────────────────────┤ │chmod 600 │make an existing file "" to be non-readable│ │ │and non-writable by the other people │ │ │(non-executable for all) │ ├─────────────┼────────────────────────────────────────────────┤ │chmod 644 │make an existing file "" to be readable but│ │ │non-writable by the other people (non-executable│ │ │for all) │ ├─────────────┼────────────────────────────────────────────────┤ │chmod 755 │make an existing file "" to be readable but│ │ │non-writable by the other people (executable for│ │ │all) │ ├─────────────┼────────────────────────────────────────────────┤ │find . -name │find matching filenames using shell "" │ │ │(slower) │ ├─────────────┼────────────────────────────────────────────────┤ │locate -d . │find matching filenames using shell "" │ │ │(quicker using regularly generated database) │ ├─────────────┼────────────────────────────────────────────────┤ │grep -e " │find a "" in all files ending with │ │" │".html" in current directory and display them │ │*.html │all │ ├─────────────┼────────────────────────────────────────────────┤ │top │display process information using full screen, │ │ │type "q" to quit │ ├─────────────┼────────────────────────────────────────────────┤ │ps aux | │display information on all the running processes│ │pager │using BSD style output │ ├─────────────┼────────────────────────────────────────────────┤ │ps -ef | │display information on all the running processes│ │pager │using Unix system-V style output │ ├─────────────┼────────────────────────────────────────────────┤ │ps aux | grep│display all processes running "exim" and "exim4"│ │-e "[e]xim4*"│ │ ├─────────────┼────────────────────────────────────────────────┤ │ps axf | │display information on all the running processes│ │pager │with ASCII art output │ ├─────────────┼────────────────────────────────────────────────┤ │kill <1234> │kill a process identified by the process ID: " │ │ │<1234>" │ ├─────────────┼────────────────────────────────────────────────┤ │gzip │compress "" to create ".gz" using the │ │ │Lempel-Ziv coding (LZ77) │ ├─────────────┼────────────────────────────────────────────────┤ │gunzip │decompress ".gz" to create "" │ │.gz │ │ ├─────────────┼────────────────────────────────────────────────┤ │ │compress "" to create ".bz2" using the│ │bzip2 │Burrows-Wheeler block sorting text compression │ │ │algorithm, and Huffman coding (better │ │ │compression than gzip) │ ├─────────────┼────────────────────────────────────────────────┤ │bunzip2 │decompress ".bz2" to create "" │ │.bz2 │ │ ├─────────────┼────────────────────────────────────────────────┤ │tar -xvf │extract files from ".tar" archive │ │.tar │ │ ├─────────────┼────────────────────────────────────────────────┤ │tar -xvzf │extract files from gzipped ".tar.gz" │ │.tar.gz │archive │ ├─────────────┼────────────────────────────────────────────────┤ │tar -xvf -j │extract files from ".tar.bz2" archive │ │.tar.bz2│ │ ├─────────────┼────────────────────────────────────────────────┤ │tar -cvf │archive contents of folder "/" in " │ │.tar │.tar" archive │ │/ │ │ ├─────────────┼────────────────────────────────────────────────┤ │tar -cvzf │archive contents of folder "/" in │ │.tar.gz │compressed ".tar.gz" archive │ │/ │ │ ├─────────────┼────────────────────────────────────────────────┤ │tar -cvjf │archive contents of folder "/" in " │ │.tar.bz2│.tar.bz2" archive │ │/ │ │ ├─────────────┼────────────────────────────────────────────────┤ │zcat │display contents of compressed "README.gz" using│ │README.gz | │the default pager │ │pager │ │ ├─────────────┼────────────────────────────────────────────────┤ │zcat │create a file "foo" with the decompressed │ │README.gz > │content of "README.gz" │ │foo │ │ ├─────────────┼────────────────────────────────────────────────┤ │zcat │append the decompressed content of "README.gz" │ │README.gz >> │to the end of the file "foo" (if it does not │ │foo │exist, create it first) │ └─────────────┴────────────────────────────────────────────────┘ Note Unix has a tradition to hide filenames which start with ".". They are traditionally files that contain configuration information and user preferences. Note For cd command, see builtins(7). Note The default pager of the bare bone Debian system is more(1) which cannot scroll back. By installing the less package using command line "aptitude install less", less(1) becomes default pager and you can scroll back with cursor keys. Note The "[" and "]" in the regular expression of the "ps aux | grep -e "[e]xim4*"" command above enable grep to avoid matching itself. The "4*" in the regular expression means 0 or more repeats of character "4" thus enables grep to match both "exim" and "exim4". Although "*" is used in the shell filename glob and the regular expression, their meanings are different. Learn the regular expression from grep(1). Please traverse directories and peek into the system using the above commands as training. If you have questions on any of console commands, please make sure to read the manual page. For example, try the following $ man man $ man bash $ man builtins $ man grep $ man ls The style of man pages may be a little hard to get used to, because they are rather terse, particularly the older, very traditional ones. But once you get used to it, you come to appreciate their succinctness. Please note that many Unix-like commands including ones from GNU and BSD display brief help information if you invoke them in one of the following ways (or without any arguments in some cases). $ --help $ -h 1.5. The simple shell command Now you have some feel on how to use the Debian system. Let's look deep into the mechanism of the command execution in the Debian system. Here, I have simplified reality for the newbie. See bash(1) for the exact explanation. A simple command is a sequence of components. 1. Variable assignments (optional) 2. Command name 3. Arguments (optional) 4. Redirections (optional: > , >> , < , << , etc.) 5. Control operator (optional: && , || , , ; , & , ( , ) ) 1.5.1. Command execution and environment variable Values of some environment variables change the behavior of some Unix commands. Default values of environment variables are initially set by the PAM system and then some of them may be reset by some application programs. ● The display manager such as gdm resets environment variables. ● The shell in its start up codes resets environment variables in "~/bash_profile" and "~/.bashrc". 1.5.2. "$LANG" variable The full locale value given to "$LANG" variable consists of 3 parts: "xx_YY.ZZZZ". Table 1.17. 3 parts of locale value ┌────────────┬────────────────────────────────────────────────┐ │locale value│meaning │ ├────────────┼────────────────────────────────────────────────┤ │xx │ISO 639 language codes (lower case) such as "en"│ ├────────────┼────────────────────────────────────────────────┤ │YY │ISO 3166 country codes (upper case) such as "US"│ ├────────────┼────────────────────────────────────────────────┤ │ZZZZ │codeset, always set to "UTF-8" │ └────────────┴────────────────────────────────────────────────┘ For language codes and country codes, see pertinent description in the "info gettext". For the codeset on the modern Debian system, you should always set it to UTF-8 unless you specifically want to use the historic one with good reason and background knowledge. For fine details of the locale configuration, see Section 8.3, “The locale”. Note The "LANG=en_US" is not "LANG=C" nor "LANG=en_US.UTF-8". It is "LANG=en_US.ISO-8859-1" (see Section 8.3.1, “Basics of encoding” ). Table 1.18. List of locale recommendations ┌─────────────────────┬─────────────────────────┐ │locale recommendation│Language (area) │ ├─────────────────────┼─────────────────────────┤ │en_US.UTF-8 │English(USA) │ ├─────────────────────┼─────────────────────────┤ │en_GB.UTF-8 │English(Great_Britain) │ ├─────────────────────┼─────────────────────────┤ │fr_FR.UTF-8 │French(France) │ ├─────────────────────┼─────────────────────────┤ │de_DE.UTF-8 │German(Germany) │ ├─────────────────────┼─────────────────────────┤ │it_IT.UTF-8 │Italian(Italy) │ ├─────────────────────┼─────────────────────────┤ │es_ES.UTF-8 │Spanish(Spain) │ ├─────────────────────┼─────────────────────────┤ │ca_ES.UTF-8 │Catalan(Spain) │ ├─────────────────────┼─────────────────────────┤ │sv_SE.UTF-8 │Swedish(Sweden) │ ├─────────────────────┼─────────────────────────┤ │pt_BR.UTF-8 │Portuguese(Brazil) │ ├─────────────────────┼─────────────────────────┤ │ru_RU.UTF-8 │Russian(Russia) │ ├─────────────────────┼─────────────────────────┤ │zh_CN.UTF-8 │Chinese(P.R._of_China) │ ├─────────────────────┼─────────────────────────┤ │zh_TW.UTF-8 │Chinese(Taiwan_R.O.C.) │ ├─────────────────────┼─────────────────────────┤ │ja_JP.UTF-8 │Japanese(Japan) │ ├─────────────────────┼─────────────────────────┤ │ko_KR.UTF-8 │Korean(Republic_of_Korea)│ ├─────────────────────┼─────────────────────────┤ │vi_VN.UTF-8 │Vietnamese(Vietnam) │ └─────────────────────┴─────────────────────────┘ Typical command execution uses a shell line sequence as the following. $ date Sun Jun 3 10:27:39 JST 2007 $ LANG=fr_FR.UTF-8 date dimanche 3 juin 2007, 10:27:33 (UTC+0900) Here, the program date(1) is executed with different values of the environment variable "$LANG". ● For the first command, "$LANG" is set to the system default locale value "en_US.UTF-8". ● For the second command, "$LANG" is set to the French UTF-8 locale value "fr_FR.UTF-8". Most command executions usually do not have preceding environment variable definition. For the above example, you can alternatively execute as the following. $ LANG=fr_FR.UTF-8 $ date dimanche 3 juin 2007, 10:27:33 (UTC+0900) As you can see here, the output of command is affected by the environment variable to produce French output. If you want the environment variable to be inherited to subprocesses (e.g., when calling shell script), you need to export it instead by the following. $ export LANG Tip When filing a bug report, running and checking the command under "LANG=en_US.UTF-8" is good idea if you use non-English environment. See locale(5) and locale(7) for "$LANG" and related environment variables. Note I recommend you to configure the system environment just by the "$LANG" variable and to stay away from "$LC_*" variables unless it is absolutely needed. 1.5.3. "$PATH" variable When you type a command into the shell, the shell searches the command in the list of directories contained in the "$PATH" environment variable. The value of the "$PATH" environment variable is also called the shell's search path. In the default Debian installation, the "$PATH" environment variable of user accounts may not include "/sbin" and "/usr/ sbin". For example, the ifconfig command needs to be issued with full path as "/sbin/ifconfig". (Similar ip command is located in "/bin".) You can change the "$PATH" environment variable of Bash shell by "~/.bash_profile" or "~/.bashrc" files. 1.5.4. "$HOME" variable Many commands stores user specific configuration in the home directory and changes their behavior by their contents. The home directory is identified by the environment variable "$HOME". Table 1.19. List of "$HOME" values ┌─────────────────┬────────────────────────────────────────────┐ │value of "$HOME" │program execution situation │ ├─────────────────┼────────────────────────────────────────────┤ │/ │program run by the init process (daemon) │ ├─────────────────┼────────────────────────────────────────────┤ │/root │program run from the normal root shell │ ├─────────────────┼────────────────────────────────────────────┤ │/home/ │program run from the normal user shell │ │ │ │ ├─────────────────┼────────────────────────────────────────────┤ │/home/ │program run from the normal user GUI desktop│ │ │menu │ ├─────────────────┼────────────────────────────────────────────┤ │/home/ │program run as root with "sudo program" │ │ │ │ ├─────────────────┼────────────────────────────────────────────┤ │/root │program run as root with "sudo -H program" │ └─────────────────┴────────────────────────────────────────────┘ Tip Shell expands "~/" to current user’s home directory, i.e., "$HOME/". Shell expands "~foo/" to foo's home directory, i.e., " /home/foo/". 1.5.5. Command line options Some commands take arguments. Arguments starting with "-" or "--" are called options and control the behavior of the command. $ date Mon Oct 27 23:02:09 CET 2003 $ date -R Mon, 27 Oct 2003 23:02:40 +0100 Here the command-line argument "-R" changes date(1) behavior to output RFC2822 compliant date string. 1.5.6. Shell glob Often you want a command to work with a group of files without typing all of them. The filename expansion pattern using the shell glob, (sometimes referred as wildcards), facilitate this need. Table 1.20. Shell glob patterns ┌───────────┬──────────────────────────────────────────────────┐ │shell glob │description of match rule │ │pattern │ │ ├───────────┼──────────────────────────────────────────────────┤ │* │filename (segment) not started with "." │ ├───────────┼──────────────────────────────────────────────────┤ │.* │filename (segment) started with "." │ ├───────────┼──────────────────────────────────────────────────┤ │? │exactly one character │ ├───────────┼──────────────────────────────────────────────────┤ │[…] │exactly one character with any character enclosed │ │ │in brackets │ ├───────────┼──────────────────────────────────────────────────┤ │[a-z] │exactly one character with any character between │ │ │"a" and "z" │ ├───────────┼──────────────────────────────────────────────────┤ │[^…] │exactly one character other than any character │ │ │enclosed in brackets (excluding "^") │ └───────────┴──────────────────────────────────────────────────┘ For example, try the following $ mkdir junk; cd junk; touch 1.txt 2.txt 3.c 4.h .5.txt ..6.txt $ echo *.txt 1.txt 2.txt $ echo * 1.txt 2.txt 3.c 4.h $ echo *.[hc] 3.c 4.h $ echo .* . .. .5.txt ..6.txt $ echo .*[^.]* .5.txt ..6.txt $ echo [^1-3]* 4.h $ cd ..; rm -rf junk See glob(7). Note Unlike normal filename expansion by the shell, the shell pattern "*" tested in find(1) with "-name" test etc., matches the initial "." of the filename. (New POSIX feature) Note BASH can be tweaked to change its glob behavior with its shopt builtin options such as "dotglob", "noglob", "nocaseglob", "nullglob", "nocaseglob", "extglob", etc. See bash(1). 1.5.7. Return value of the command Each command returns its exit status (variable: "$?") as the return value. Table 1.21. Command exit codes ┌───────────────────┬────────────────────┬────────────────────┐ │command exit status│numeric return value│logical return value│ ├───────────────────┼────────────────────┼────────────────────┤ │success │zero, 0 │TRUE │ ├───────────────────┼────────────────────┼────────────────────┤ │error │non-zero, -1 │FALSE │ └───────────────────┴────────────────────┴────────────────────┘ For example, try the following. $ [ 1 = 1 ] ; echo $? 0 $ [ 1 = 2 ] ; echo $? 1 Note Please note that, in the logical context for the shell, success is treated as the logical TRUE which has 0 (zero) as its value. This is somewhat non-intuitive and needs to be reminded here. 1.5.8. Typical command sequences and shell redirection Let's try to remember following shell command idioms typed in one line as a part of shell command. Table 1.22. Shell command idioms ┌──────────┬───────────────────────────────────────────────────┐ │command │description │ │idiom │ │ ├──────────┼───────────────────────────────────────────────────┤ │command & │background execution of command in the subshell │ ├──────────┼───────────────────────────────────────────────────┤ │command1 |│pipe the standard output of command1 to the │ │command2 │standard input of command2 (concurrent execution) │ ├──────────┼───────────────────────────────────────────────────┤ │command1 │pipe both standard output and standard error of │ │2>&1 | │command1 to the standard input of command2 ( │ │command2 │concurrent execution) │ ├──────────┼───────────────────────────────────────────────────┤ │command1 ;│execute command1 and command2 sequentially │ │command2 │ │ ├──────────┼───────────────────────────────────────────────────┤ │command1 &│execute command1; if successful, execute command2 │ │& command2│sequentially (return success if both command1 and │ │ │command2 are successful) │ ├──────────┼───────────────────────────────────────────────────┤ │command1 |│execute command1; if not successful, execute │ │| command2│command2 sequentially (return success if command1 │ │ │or command2 are successful) │ ├──────────┼───────────────────────────────────────────────────┤ │command > │redirect standard output of command to a file foo │ │foo │(overwrite) │ ├──────────┼───────────────────────────────────────────────────┤ │command 2>│redirect standard error of command to a file foo │ │foo │(overwrite) │ ├──────────┼───────────────────────────────────────────────────┤ │command >>│redirect standard output of command to a file foo │ │foo │(append) │ ├──────────┼───────────────────────────────────────────────────┤ │command │redirect standard error of command to a file foo │ │2>> foo │(append) │ ├──────────┼───────────────────────────────────────────────────┤ │command > │redirect both standard output and standard error of│ │foo 2>&1 │command to a file "foo" │ ├──────────┼───────────────────────────────────────────────────┤ │command < │redirect standard input of command to a file foo │ │foo │ │ ├──────────┼───────────────────────────────────────────────────┤ │command <<│redirect standard input of command to the following│ │delimiter │lines until "delimiter" is met (here document) │ ├──────────┼───────────────────────────────────────────────────┤ │command │redirect standard input of command to the following│ │<<- │lines until "delimiter" is met (here document, the │ │delimiter │leading tab characters are stripped from input │ │ │lines) │ └──────────┴───────────────────────────────────────────────────┘ The Debian system is a multi-tasking system. Background jobs allow users to run multiple programs in a single shell. The management of the background process involves the shell builtins: jobs, fg, bg, and kill. Please read sections of bash (1) under "SIGNALS", and "JOB CONTROL", and builtins(1). For example, try the following $ foo $ exec 3bar # open files $ cat <&3 >&4 # redirect stdin to 3, stdout to 4 $ exec 3<&- 4>&- # close files $ cat bar Hello Here, "n<&-" and "n>&-" mean to close the file descriptor "n". The file descriptor 0-2 are predefined. Table 1.23. Predefined file descriptors ┌──────┬───────────────┬───────────────┐ │device│description │file descriptor│ ├──────┼───────────────┼───────────────┤ │stdin │standard input │0 │ ├──────┼───────────────┼───────────────┤ │stdout│standard output│1 │ ├──────┼───────────────┼───────────────┤ │stderr│standard error │2 │ └──────┴───────────────┴───────────────┘ 1.5.9. Command alias You can set an alias for the frequently used command. For example, try the following $ alias la='ls -la' Now, "la" works as a short hand for "ls -la" which lists all files in the long listing format. You can list any existing aliases by alias (see bash(1) under "SHELL BUILTIN COMMANDS"). $ alias ... alias la='ls -la' You can identity exact path or identity of the command by type (see bash(1) under "SHELL BUILTIN COMMANDS"). For example, try the following $ type ls ls is hashed (/bin/ls) $ type la la is aliased to ls -la $ type echo echo is a shell builtin $ type file file is /usr/bin/file Here ls was recently searched while "file" was not, thus "ls" is "hashed", i.e., the shell has an internal record for the quick access to the location of the "ls" command. Tip See Section 9.2.7, “Colorized commands”. 1.6. Unix-like text processing In Unix-like work environment, text processing is done by piping text through chains of standard text processing tools. This was another crucial Unix innovation. 1.6.1. Unix text tools There are few standard text processing tools which are used very often on the Unix-like system. ● No regular expression is used: ○ cat(1) concatenates files and outputs the whole content. ○ tac(1) concatenates files and outputs in reverse. ○ cut(1) selects parts of lines and outputs. ○ head(1) outputs the first part of files. ○ tail(1) outputs the last part of files. ○ sort(1) sorts lines of text files. ○ uniq(1) removes duplicate lines from a sorted file. ○ tr(1) translates or deletes characters. ○ diff(1) compares files line by line. ● Basic regular expression (BRE) is used: ○ grep(1) matches text with patterns. ○ ed(1) is a primitive line editor. ○ sed(1) is a stream editor. ○ vim(1) is a screen editor. ○ emacs(1) is a screen editor. (somewhat extended BRE) ● Extended regular expression (ERE) is used: ○ egrep(1) matches text with patterns. ○ awk(1) does simple text processing. ○ tcl(3tcl) can do every conceivable text processing: re_syntax(3). Often used with tk(3tk). ○ perl(1) can do every conceivable text processing. perlre (1). ○ pcregrep(1) from the pcregrep package matches text with Perl Compatible Regular Expressions (PCRE) pattern. ○ python(1) with the re module can do every conceivable text processing. See "/usr/share/doc/python/html/ index.html". If you are not sure what exactly these commands do, please use "man command" to figure it out by yourself. Note Sort order and range expression are locale dependent. If you wish to obtain traditional behavior for a command, use C locale instead of UTF-8 ones by prepnding command with "LANG=C" (see Section 1.5.2, “"$LANG" variable” and Section 8.3, “The locale” ). Note Perl regular expressions (perlre(1)), Perl Compatible Regular Expressions (PCRE), and Python regular expressions offered by the re module have many common extensions to the normal ERE. 1.6.2. Regular expressions Regular expressions are used in many text processing tools. They are analogous to the shell globs, but they are more complicated and powerful. The regular expression describes the matching pattern and is made up of text characters and metacharacters. The metacharacter is just a character with a special meaning. There are 2 major styles, BRE and ERE, depending on the text tools as described above. Table 1.24. Metacharacters for BRE and ERE ┌─────────┬──────┬─────────────────────────────────────────────┐ │BRE │ERE │description of the regular expression │ ├─────────┼──────┼─────────────────────────────────────────────┤ │\ . [ ] ^│\ . [ │ │ │$ * │] ^ $ │common metacharacters │ │ │* │ │ ├─────────┼──────┼─────────────────────────────────────────────┤ │\+ \? \( │ │ │ │\) \{ \} │  │BRE only "\" escaped metacharacters │ │\| │ │ │ ├─────────┼──────┼─────────────────────────────────────────────┤ │ │+ ? ( │ │ │  │) { } │ERE only non-"\" escaped metacharacters │ │ │| │ │ ├─────────┼──────┼─────────────────────────────────────────────┤ │c │c │match non-metacharacter "c" │ ├─────────┼──────┼─────────────────────────────────────────────┤ │\c │\c │match a literal character "c" even if "c" is │ │ │ │metacharacter by itself │ ├─────────┼──────┼─────────────────────────────────────────────┤ │. │. │match any character including newline │ ├─────────┼──────┼─────────────────────────────────────────────┤ │^ │^ │position at the beginning of a string │ ├─────────┼──────┼─────────────────────────────────────────────┤ │$ │$ │position at the end of a string │ ├─────────┼──────┼─────────────────────────────────────────────┤ │\< │\< │position at the beginning of a word │ ├─────────┼──────┼─────────────────────────────────────────────┤ │\> │\> │position at the end of a word │ ├─────────┼──────┼─────────────────────────────────────────────┤ │\[abc…\] │[abc…]│match any characters in "abc…" │ ├─────────┼──────┼─────────────────────────────────────────────┤ │\[^abc…\]│[^ │match any characters except in "abc…" │ │ │abc…] │ │ ├─────────┼──────┼─────────────────────────────────────────────┤ │r* │r* │match zero or more regular expressions │ │ │ │identified by "r" │ ├─────────┼──────┼─────────────────────────────────────────────┤ │r\+ │r+ │match one or more regular expressions │ │ │ │identified by "r" │ ├─────────┼──────┼─────────────────────────────────────────────┤ │r\? │r? │match zero or one regular expressions │ │ │ │identified by "r" │ ├─────────┼──────┼─────────────────────────────────────────────┤ │r1\|r2 │r1|r2 │match one of the regular expressions │ │ │ │identified by "r1" or "r2" │ ├─────────┼──────┼─────────────────────────────────────────────┤ │\(r1\|r2 │(r1| │match one of the regular expressions │ │\) │r2) │identified by "r1" or "r2" and treat it as a │ │ │ │bracketed regular expression │ └─────────┴──────┴─────────────────────────────────────────────┘ The regular expression of emacs is basically BRE but has been extended to treat "+"and "?" as the metacharacters as in ERE. Thus, there are no needs to escape them with "\" in the regular expression of emacs. grep(1) can be used to perform the text search using the regular expression. For example, try the following $ egrep 'GNU.*LICENSE|Yoyodyne' /usr/share/common-licenses/GPL GNU GENERAL PUBLIC LICENSE GNU GENERAL PUBLIC LICENSE Yoyodyne, Inc., hereby disclaims all copyright interest in the program Tip See Section 9.2.7, “Colorized commands”. 1.6.3. Replacement expressions For the replacement expression, some characters have special meanings. Table 1.25. The replacement expression ┌───────────────┬──────────────────────────────────────────────┐ │replacement │description of the text to replace the │ │expression │replacement expression │ ├───────────────┼──────────────────────────────────────────────┤ │& │what the regular expression matched (use \& in│ │ │emacs) │ ├───────────────┼──────────────────────────────────────────────┤ │\n │what the n-th bracketed regular expression │ │ │matched ("n" being number) │ └───────────────┴──────────────────────────────────────────────┘ For Perl replacement string, "$n" is used instead of "\n" and "& " has no special meaning. For example, try the following $ echo zzz1abc2efg3hij4 | \ sed -e 's/\(1[a-z]*\)[0-9]*\(.*\)$/=&=/' zzz=1abc2efg3hij4= $ echo zzz1abc2efg3hij4 | \ sed -e 's/\(1[a-z]*\)[0-9]*\(.*\)$/\2===\1/' zzzefg3hij4===1abc $ echo zzz1abc2efg3hij4 | \ perl -pe 's/(1[a-z]*)[0-9]*(.*)$/$2===$1/' zzzefg3hij4===1abc $ echo zzz1abc2efg3hij4 | \ perl -pe 's/(1[a-z]*)[0-9]*(.*)$/=&=/' zzz=&= Here please pay extra attention to the style of the bracketed regular expression and how the matched strings are used in the text replacement process on different tools. These regular expressions can be used for cursor movements and text replacement actions in some editors too. The back slash "\" at the end of line in the shell commandline escapes newline as a white space character and continues shell command line input to the next line. Please read all the related manual pages to learn these commands. 1.6.4. Global substitution with regular expressions The ed(1) command can replace all instances of "FROM_REGEX" with "TO_TEXT" in "file". $ ed file <, , and combined. Be careful about using this shell IFS tricks. Strange things may happen, when shell interprets some parts of the script as its input. $ IFS=":," # use ":" and "," as IFS $ echo IFS=$IFS, IFS="$IFS" # echo is a Bash builtin IFS= , IFS=:, $ date -R # just a command output Sat, 23 Aug 2003 08:30:15 +0200 $ echo $(date -R) # sub shell --> input to main shell Sat 23 Aug 2003 08 30 36 +0200 $ unset IFS # reset IFS to the default $ echo $(date -R) Sat, 23 Aug 2003 08:30:50 +0200 1.6.6. Script snippets for piping commands The following scripts do nice things as a part of a pipe. Table 1.26. List of script snippets for piping commands ┌─────────────────────┬────────────────────────────────────────┐ │script snippet (type │effect of command │ │in one line) │ │ ├─────────────────────┼────────────────────────────────────────┤ │find /usr -print │find all files under "/usr" │ ├─────────────────────┼────────────────────────────────────────┤ │seq 1 100 │print 1 to 100 │ ├─────────────────────┼────────────────────────────────────────┤ │| xargs -n 1 │run command repeatedly with each item │ │ │from pipe as its argument │ ├─────────────────────┼────────────────────────────────────────┤ │| xargs -n 1 echo │split white-space-separated items from │ │ │pipe into lines │ ├─────────────────────┼────────────────────────────────────────┤ │| xargs echo │merge all lines from pipe into a line │ ├─────────────────────┼────────────────────────────────────────┤ │| grep -e │extract lines from pipe containing │ │ │ ├─────────────────────┼────────────────────────────────────────┤ │| grep -v -e │extract lines from pipe not containing │ │ │ ├─────────────────────┼────────────────────────────────────────┤ │| cut -d: -f3 - │extract third field from pipe separated │ │ │by ":" (passwd file etc.) │ ├─────────────────────┼────────────────────────────────────────┤ │| awk '{ print $3 }' │extract third field from pipe separated │ │ │by whitespaces │ ├─────────────────────┼────────────────────────────────────────┤ │| awk -F'\t' '{ print│extract third field from pipe separated │ │$3 }' │by tab │ ├─────────────────────┼────────────────────────────────────────┤ │| col -bx │remove backspace and expand tabs to │ │ │spaces │ ├─────────────────────┼────────────────────────────────────────┤ │| expand - │expand tabs │ ├─────────────────────┼────────────────────────────────────────┤ │| sort| uniq │sort and remove duplicates │ ├─────────────────────┼────────────────────────────────────────┤ │| tr 'A-Z' 'a-z' │convert uppercase to lowercase │ ├─────────────────────┼────────────────────────────────────────┤ │| tr -d '\n' │concatenate lines into one line │ ├─────────────────────┼────────────────────────────────────────┤ │| tr -d '\r' │remove CR │ ├─────────────────────┼────────────────────────────────────────┤ │| sed 's/^/# /' │add "#" to the start of each line │ ├─────────────────────┼────────────────────────────────────────┤ │| sed 's/\.ext//g' │remove ".ext" │ ├─────────────────────┼────────────────────────────────────────┤ │| sed -n -e 2p │print the second line │ ├─────────────────────┼────────────────────────────────────────┤ │| head -n 2 - │print the first 2 lines │ ├─────────────────────┼────────────────────────────────────────┤ │| tail -n 2 - │print the last 2 lines │ └─────────────────────┴────────────────────────────────────────┘ One-line shell script can loop over many files using find(1) and xargs(1) to perform quite complicated tasks. See Section 10.1.5, “Idioms for the selection of files” and Section 9.5.9, “Repeating a command looping over files”. When using the shell interactive mode becomes too complicated, please consider to write a shell script (see Section 12.1, “The shell script”). Chapter 2. Debian package management Note This chapter is written assuming the latest stable release is codename: lenny. Debian is a volunteer organization which builds consistent distributions of pre-compiled binary packages of free software and distributes them from its archive. The Debian archive is offered by many remote mirror sites for access through HTTP and FTP methods. It is also available as CD-ROM/DVD. The Debian package management system, when used properly, offers the user to install consistent sets of binary packages to the system from the archive. Currently, there are 26849 packages available for the amd64 architecture. The Debian package management system has a rich history and many choices for the front end user program and back end archive access method to be used. Currently, we recommend aptitude(8) as the main front end program for the Debian package management activity. Table 2.1. List of Debian package management tools ┌───────────────────┬───────┬────┬─────────────────────────────┐ │package │popcon │size│description │ ├───────────────────┼───────┼────┼─────────────────────────────┤ │ │V:27, │ │terminal-based package │ │aptitude * │I:98 │9812│manager (current standard, │ │ │ │ │front-end for apt) │ ├───────────────────┼───────┼────┼─────────────────────────────┤ │ │ │ │Advanced Packaging Tool │ │ │ │ │(APT), front-end for dpkg │ │apt * │V:89, │5228│providing "http", "ftp", and │ │ │I:99 │ │"file" archive access methods│ │ │ │ │(apt-get/apt-cache commands │ │ │ │ │included) │ ├───────────────────┼───────┼────┼─────────────────────────────┤ │ │V:7, │ │tool for selecting tasks for │ │tasksel * │I:93 │904 │installation on Debian system│ │ │ │ │(front-end for APT) │ ├───────────────────┼───────┼────┼─────────────────────────────┤ │ │ │ │enhancement package for APT │ │unattended-upgrades│V:4, │216 │to enable automatic │ │* │I:23 │ │installation of security │ │ │ │ │upgrades │ ├───────────────────┼───────┼────┼─────────────────────────────┤ │ │ │ │terminal-based package │ │dselect * │V:4, │2100│manager (previous standard, │ │ │I:49 │ │front-end for APT and other │ │ │ │ │old access methods) │ ├───────────────────┼───────┼────┼─────────────────────────────┤ │dpkg * │V:92, │6636│package management system for│ │ │I:99 │ │Debian │ ├───────────────────┼───────┼────┼─────────────────────────────┤ │dpkg-ftp * │V:0.08,│136 │older ftp method for dselect │ │ │I:0.4 │ │ │ ├───────────────────┼───────┼────┼─────────────────────────────┤ │synaptic * │V:18, │6124│graphical package manager │ │ │I:45 │ │(GNOME front-end for APT) │ ├───────────────────┼───────┼────┼─────────────────────────────┤ │kpackage * │V:6, │1068│graphical package manager │ │ │I:13 │ │(KDE front-end for APT) │ ├───────────────────┼───────┼────┼─────────────────────────────┤ │ │ │ │APT utility programs: │ │apt-utils * │V:52, │496 │apt-extracttemplates(1), │ │ │I:99 │ │apt-ftparchive(1), and │ │ │ │ │apt-sortpkgs(1) │ ├───────────────────┼───────┼────┼─────────────────────────────┤ │apt-listchanges * │V:4, │504 │package change history │ │ │I:6 │ │notification tool │ ├───────────────────┼───────┼────┼─────────────────────────────┤ │apt-listbugs * │V:1.5, │512 │lists critical bugs before │ │ │I:2 │ │each APT installation │ ├───────────────────┼───────┼────┼─────────────────────────────┤ │apt-file * │V:2, │184 │APT package searching utility│ │ │I:9 │ │— command-line interface │ ├───────────────────┼───────┼────┼─────────────────────────────┤ │apt-rdepends * │V:0.16,│92 │recursively lists package │ │ │I:0.9 │ │dependencies │ └───────────────────┴───────┴────┴─────────────────────────────┘ Note The annoying bug #411123 for the mixed use of aptitude(8) and apt-get(8) commands has been resolved. If this kept you from using aptitude, please reconsider. 2.1. Debian package management prerequisites 2.1.1. Package configuration Here are some key points for package configuration on the Debian system. ● The manual configuration by the system administrator is respected. In other words, the package configuration system makes no intrusive configuration for the sake of convenience. ● Each package comes with its own configuration script with standardized user interface called debconf(7) to help initial installation process of the package. ● Debian Developers try their best to make your upgrade experience flawless with package configuration scripts. ● Full functionalities of packaged software are available to the system administrator. But ones with security risks are disabled in the default installation. ● If you manually activate a service with some security risks, you are responsible for the risk containment. ● Esoteric configuration may be manually enabled by the system administrator. This may creates interference with popular generic helper programs for the system configuration. 2.1.2. Basic precautions Warning Do not install packages from random mixture of suites. It probably breaks the package consistency which requires deep system management knowledge, such as compiler ABI, library version, interpreter features, etc. The newbie Debian system administrator should stay with the stable release of Debian while applying only security updates. I mean that some of the following valid actions are better avoided, as a precaution, until you understand the Debian system very well. Here are some reminders. ● Do not include testing or unstable in "/etc/apt/ sources.list". ● Do not mix standard Debian with other non-Debian archives such as Ubuntu in "/etc/apt/sources.list". ● Do not create "/etc/apt/preferences". ● Do not change default behavior of package management tools through configuration files without knowing their full impacts. ● Do not install random packages by "dpkg -i ". ● Do not ever install random packages by "dpkg --force-all -i ". ● Do not erase or alter files in "/var/lib/dpkg/". ● Do not overwrite system files by installing software programs directly compiled from source. ○ Install them into "/usr/local" or "/opt", if needed. The non-compatible effects caused by above actions to the Debian package management system may leave your system unusable. The serious Debian system administrator who runs mission critical servers, should use extra precautions. ● Do not install any packages including security updates from Debian without thoroughly testing them with your particular configuration under safe conditions. ○ You as the system administrator are responsible for your system in the end. ○ The long stability history of Debian system is no guarantee by itself. 2.1.3. Life with eternal upgrades Despite my warnings above, I know many readers of this document wish to run the testing or unstable suites of Debian as their main system for self-administered Desktop environments. This is because they work very well, are updated frequently, and offer the latest features. Caution For your production server, the stable suite with the security updates is recommended. The same can be said for desktop PCs on which you can spend limited administration efforts, e.g. for your mother's PC. It takes no more than simply setting the distribution string in the "/etc/apt/sources.list" to the suite name: "testing" or "unstable"; or the codename: "squeeze" or "sid". This makes you live the life of eternal upgrades. The use of testing or unstable is a lot of fun but comes with some risks. Even though the unstable suite of Debian system looks very stable for most of the times, there have been some package problems on the testing and unstable suite of Debian system and a few of them were not so trivial to resolve. It may be quite painful for you. Sometimes, you may have a broken package or missing functionality for a few weeks. Here are some ideas to ensure quick and easy recovery from bugs in Debian packages. ● Make the system dual bootable by installing the stable suite of Debian system to another partition ● Make the installation CD handy for the rescue boot ● Consider installing apt-listbugs to check the Debian Bug Tracking System (BTS) information before the upgrade ● Learn the package system infrastructure enough to work around the problem ● Create a chroot or similar environment and run the latest system in it in advance (optional) (If you can not do any one of these precautionary actions, you are probably not ready for the testing and unstable suites.) Enlightenment with the following saves a person from the eternal karmic struggle of upgrade hell and let him reach Debian nirvana . 2.1.4. Debian archive basics Let's look into the Debian archive from a system user's perspective. Tip Official policy of the Debian archive is defined at Debian Policy Manual, Chapter 2 - The Debian Archive. For the typical HTTP access, the archive is specified in the "/ etc/apt/sources.list" file as the following, e.g. for the current stable = lenny system. deb http://ftp.XX.debian.org/debian/ lenny main contrib non-free deb-src http://ftp.XX.debian.org/debian/ lenny main contrib non-free deb http://security.debian.org/ lenny/updates main contrib deb-src http://security.debian.org/ lenny/updates main contrib Please note "ftp.XX.debian.org" must be replaced with appropriate mirror site URL for your location, for USA "ftp.us.debian.org", which can be found in the list of Debian worldwide mirror sites. The status of these servers can be checked at Debian Mirror Checker site. Here, I tend to use codename "lenny" instead of suite name "stable" to avoid surprises when the next stable is released. The meaning of "/etc/apt/sources.list" is described in sources.list(5) and key points are followings. ● The "deb" line defines for the binary packages. ● The "deb-src" line defines for the source packages. ● The 1st argument is the root URL of the Debian archive. ● The 2nd argument is the distribution name: either the suite name or the codename. ● The 3rd and following arguments are the list of valid archive component names of the Debian archive. The "deb-src" lines can safely be omitted (or commented out by placing "#" at the start of the line) if it is just for aptitude which does not access source related meta data. It speeds up the updates of the archive meta data. The URL can be "http://", "ftp://", "file://", …. Tip If "sid" is used in the above example instead of "lenny", the "deb: http://security.debian.org/ …" line for security updates in the "/etc/apt/sources.list" is not required. Security updates are only available for stable and testing (i.e., lenny and squeeze). Here is the list of URL of the Debian archive sites and suite name or codename used in the configuration file. Table 2.2. List of Debian archive sites ┌───────────────────┬───────────────────────┬──────────────────┐ │archive URL │suite name (codename) │purpose │ ├───────────────────┼───────────────────────┼──────────────────┤ │http:// │ │stable (lenny) │ │ftp.XX.debian.org/ │stable (lenny) │release │ │debian/ │ │ │ ├───────────────────┼───────────────────────┼──────────────────┤ │http:// │ │testing (squeeze) │ │ftp.XX.debian.org/ │testing (squeeze) │release │ │debian/ │ │ │ ├───────────────────┼───────────────────────┼──────────────────┤ │http:// │ │unstable (sid) │ │ftp.XX.debian.org/ │unstable (sid) │release │ │debian/ │ │ │ ├───────────────────┼───────────────────────┼──────────────────┤ │http:// │ │experimental │ │ftp.XX.debian.org/ │experimental │pre-release │ │debian/ │ │(optional, only │ │ │ │for developer) │ ├───────────────────┼───────────────────────┼──────────────────┤ │http:// │ │Updates for the │ │ftp.XX.debian.org/ │stable-proposed-updates│next stable point │ │debian/ │ │release (optional)│ ├───────────────────┼───────────────────────┼──────────────────┤ │http:// │ │security updates │ │security.debian.org│stable/updates │for stable release│ │/ │ │(important) │ ├───────────────────┼───────────────────────┼──────────────────┤ │http:// │ │security updates │ │security.debian.org│testing/updates │for testing │ │/ │ │release │ │ │ │(important) │ ├───────────────────┼───────────────────────┼──────────────────┤ │http:// │ │compatible updates│ │volatile.debian.org│volatile │for spam filter, │ │/debian-volatile/ │ │IM clients, etc. │ ├───────────────────┼───────────────────────┼──────────────────┤ │http:// │ │non-compatible │ │volatile.debian.org│volatile-sloppy │updates for spam │ │/debian-volatile/ │ │filter, IM │ │ │ │clients, etc. │ ├───────────────────┼───────────────────────┼──────────────────┤ │http:// │ │newer backported │ │backports.org/ │lenny-backports │packages for lenny│ │debian/ │ │(non-official, │ │ │ │optional) │ └───────────────────┴───────────────────────┴──────────────────┘ Caution Only pure stable release with security updates provides the best stability. Running mostly stable release mixed with some packages from testing or unstable release is riskier than running pure unstable release. If you really need the latest version of some programs under stable release, please use packages from the debian-volatile project and backports.org (see Section 2.7.4, “Volatile and Backports.org”) services. These services must be used with extra care. Caution You should basically list only one of stable, testing, or unstable suites in the "deb" line. If you list any combination of stable, testing, and unstable suites in the "deb" line, APT programs slow down while only the latest archive is effective. Multiple listing makes sense for these when the "/etc/apt/ preferences" file is used with clear objectives (see Section 2.7.3, “Tweaking candidate version”). Note For the Debian system with the stable and testing suites, it is a good idea to include lines with "http://security.debian.org/" in the "/etc/apt/sources.list" to enable security updates as in the example above. Each Debian archive consists of 3 components. Components are alternatively called categories in "Debian Policy" or areas in "Debian Social Contract". The component is grouped by the compliance to "The Debian Free Software Guidelines" (DFSG). Table 2.3. List of Debian archive components ┌─────────┬──────────────┬─────────────────────────────────────┐ │component│number of │criteria of package │ │ │packages │ │ ├─────────┼──────────────┼─────────────────────────────────────┤ │main │26261 │DSFG compliant and no dependency to │ │ │ │non-free │ ├─────────┼──────────────┼─────────────────────────────────────┤ │contrib │190 │DSFG compliant but having dependency │ │ │ │to non-free │ ├─────────┼──────────────┼─────────────────────────────────────┤ │non-free │398 │not DSFG compliant │ └─────────┴──────────────┴─────────────────────────────────────┘ Here the number of packages in the above is for the amd64 architecture. Strictly speaking, only the main component archive shall be considered as the Debian system. The Debian archive organization can be studied best by pointing your browser to the each archive URL appended with dists or pool. The distribution is referred by two ways, the suite or codename. The word distribution is alternatively used as the synonym to the suite in many documentations. The relationship between the suite and the codename can be summarized as the following. Table 2.4. The relationship between suite and codename ┌───────────────────┬─────────────┬───────────────┬────────────┐ │Timing │suite = │suite = testing│suite = │ │ │stable │ │unstable │ ├───────────────────┼─────────────┼───────────────┼────────────┤ │after the lenny │codename = │codename = │codename = │ │release │lenny │squeeze │sid │ ├───────────────────┼─────────────┼───────────────┼────────────┤ │after the squeeze │codename = │codename = │codename = │ │release │squeeze │squeeze+1 │sid │ └───────────────────┴─────────────┴───────────────┴────────────┘ The history of codenames are described in Debian FAQ: 6.3.1 Which other codenames have been used in the past? In the stricter Debian archive terminology, the word "section" is specifically used for the categorization of packages by the application area. (Although, the word "main section" may sometimes be used to describe the Debian archive section which provides the main component.) Every time a new upload is done by the Debian developer (DD) to the unstable archive (via incoming processing), DD is required to ensure uploaded packages to be compatible with the latest set of packages in the latest unstable archive. If DD breaks this compatibility intentionally for important library upgrade etc, there is usually announcement to the debian-devel mailing list etc. Before a set of packages are moved by the Debian archive maintenance script from the unstable archive to the testing archive, the archive maintenance script not only checks the maturity (about 10 days old) and the status of the RC bug reports for the packages but also tries to ensure them to be compatible with the latest set of packages in the testing archive. This process makes the testing archive very current and usable. Through the gradual archive freeze process led by the release team, the testing archive is matured to make it completely consistent and bug free with some manual interventions. Then the new stable release is created by assigning the codename for the old testing archive to the new stable archive and creating the new codename for the new testing archive. The initial contents of the new testing archive is exactly the same as that of the newly released stable archive. Both the unstable and the testing archives may suffer temporary glitches due to several factors. ● Broken package upload to the archive (mostly for unstable) ● Delay of accepting the new packages to the archive (mostly for unstable) ● Archive synchronization timing issue (both for testing and unstable) ● Manual intervention to the archive such as package removal (more for testing) etc. So if you ever decide to use these archives, you should be able to fix or work around these kinds of glitches. Caution For about few months after a new stable release, most desktop users should use the stable archive with its security updates even if they usually use unstable or testing archives. For this transition period, both unstable and testing archives are not good for most people. Your system is difficult to keep in good working condition with the unstable archive since it suffers surges of major upgrades for core packages. The testing archive is not useful either since it contains mostly the same content as the stable archive without its security support (Debian testing-security-announce 2008-12). After a month or so, the unstable archive may be usable if you are careful. Tip When tracking the testing archive, problem caused by a removed package is usually worked around by installing corresponding package from the unstable archive which is uploaded for bug fix. See Debian Policy Manual for archive definitions. ● "Sections" ● "Priorities" ● "Base system" ● "Essential packages" 2.1.5. Package dependencies The Debian system offers a consistent set of binary packages through its versioned binary dependency declaration mechanism in the control file fields. Here is a bit over simplified definition for them. ● "Depends" ○ This declares an absolute dependency and all of the packages listed in this field must be installed at the same time or in advance. ● "Pre-Depends" ○ This is like Depends, except that it requires completed installation of the listed packages in advance. ● "Recommends" ○ This declares a strong, but not absolute, dependency. Most users would not want the package unless all of the packages listed in this field are installed. ● "Suggests" ○ This declares a weak dependency. Many users of this package may benefit from installing packages listed in this field but can have reasonable functions without them. ● "Enhances" ○ This declares a week dependency like Suggests but works in the opposite direction. ● "Conflicts" ○ This declares an absolute incompatibility. All of the packages listed in this field must be removed to install this package. ● "Replaces" ○ This is declared when files installed by this package replace files in the listed packages. ● "Provides" ○ This is declared when this package provide all of the files and functionality in the listed packages. Note Please note that defining, "Provides", "Conflicts" and "Replaces" simultaneously to an virtual package is the sane configuration. This ensures that only one real package providing this virtual package can be installed at any one time. The official definition including source dependency can be found in the Policy Manual: Chapter 7 - Declaring relationships between packages. 2.1.6. The event flow of the package management Here is a summary of the simplified event flow of the package management by APT. ● Update ("aptitude update" or "apt-get update"): 1. Fetch archive metadata from remote archive 2. Reconstruct and update local metadata for use by APT ● Upgrade ("aptitude safe-upgrade" and "aptitude full-upgrade", or "apt-get upgrade" and "apt-get dist-upgrade"): 1. Chose candidate version which is usually the latest available version for all installed packages (see Section 2.7.3, “Tweaking candidate version” for exception) 2. Make package dependency resolution 3. Fetch selected binary packages from remote archive if candidate version is different from installed version 4. Unpack fetched binary packages 5. Run preinst script 6. Install binary files 7. Run postinst script ● Install ("aptitude install …" or "apt-get install …"): 1. Chose packages listed on the command line 2. Make package dependency resolution 3. Fetch selected binary packages from remote archive 4. Unpack fetched binary packages 5. Run preinst script 6. Install binary files 7. Run postinst script ● Remove ("aptitude remove …" or "apt-get remove …"): 1. Chose packages listed on the command line 2. Make package dependency resolution 3. Run prerm script 4. Remove installed files except configuration files 5. Run postrm script ● Purge ("aptitude purge …" or "apt-get purge …"): 1. Chose packages listed on the command line 2. Make package dependency resolution 3. Run prerm script 4. Remove installed files including configuration files 5. Run postrm script Here, I intentionally skipped technical details for the sake of big picture. 2.1.7. First response to package management troubles You should read the fine official documentation. The first document to read is the Debian specific "/usr/share/doc/ /README.Debian". Other documentation in "/usr/ share/doc//" should be consulted too. If you set shell as Section 1.4.2, “Customizing bash”, type the following. $ cd $ pager README.Debian $ mc You may need to install the corresponding documentation package named with "-doc" suffix for detailed information. If you are experiencing problems with a specific package, make sure to check out the Debian bug tracking system (BTS) sites, first. Table 2.5. List of key web site to resolving problems with a specific package ┌────────────────────────────┬─────────────────────────────────┐ │web site │command │ ├────────────────────────────┼─────────────────────────────────┤ │Home page of the Debian bug │sensible-browser "http:// │ │tracking system (BTS) │bugs.debian.org/" │ ├────────────────────────────┼─────────────────────────────────┤ │The bug report of a known │sensible-browser "http:// │ │package name │bugs.debian.org/" │ ├────────────────────────────┼─────────────────────────────────┤ │The bug report of known bug │sensible-browser "http:// │ │number │bugs.debian.org/" │ └────────────────────────────┴─────────────────────────────────┘ Search Google with search words including "site:debian.org", "site:wiki.debian.org", "site:lists.debian.org", etc. When you file a bug report, please use reportbug(1) command. 2.2. Basic package management operations Aptitude is the current preferred package management tool for the Debian system. It can be used as the commandline alternative to apt-get / apt-cache and also as the full screen interactive package management tool. For the package management operation which involves package installation or updates package metadata, you need to have root privilege. 2.2.1. Basic package management operations with commandline Here are basic package management operations with commandline using aptitude(8) and apt-get(8) /apt-cache(8). Table 2.6. Basic package management operations with commandline using aptitude(8) and apt-get(8) /apt-cache(8) ┌────────────┬─────────────┬───────────────────────────────────┐ │aptitude │apt-get/ │ │ │syntax │apt-cache │description │ │ │syntax │ │ ├────────────┼─────────────┼───────────────────────────────────┤ │aptitude │apt-get │update package archive metadata │ │update │update │ │ ├────────────┼─────────────┼───────────────────────────────────┤ │aptitude │apt-get │install candidate version of "foo" │ │install foo │install foo │package with its dependencies │ ├────────────┼─────────────┼───────────────────────────────────┤ │aptitude │apt-get │install candidate version of │ │safe-upgrade│upgrade │installed packages without removing│ │ │ │any other packages │ ├────────────┼─────────────┼───────────────────────────────────┤ │aptitude │apt-get │install candidate version of │ │full-upgrade│dist-upgrade │installed packages while removing │ │ │ │other packages if needed │ ├────────────┼─────────────┼───────────────────────────────────┤ │aptitude │apt-get │remove "foo" package while leaving │ │remove foo │remove foo │its configuration files │ ├────────────┼─────────────┼───────────────────────────────────┤ │N/A │apt-get │remove auto-installed packages │ │ │autoremove │which is no longer required │ ├────────────┼─────────────┼───────────────────────────────────┤ │aptitude │apt-get purge│purge "foo" package with its │ │purge foo │foo │configuration files │ ├────────────┼─────────────┼───────────────────────────────────┤ │aptitude │apt-get clean│clear out the local repository of │ │clean │ │retrieved package files completely │ ├────────────┼─────────────┼───────────────────────────────────┤ │aptitude │apt-get │clear out the local repository of │ │autoclean │autoclean │retrieved package files for │ │ │ │outdated packages │ ├────────────┼─────────────┼───────────────────────────────────┤ │aptitude │apt-cache │display detailed information about │ │show foo │show │"foo" package │ │ │ │ │ ├────────────┼─────────────┼───────────────────────────────────┤ │aptitude │apt-cache │ │ │search │search │search packages which match │ │ │ │ ├────────────┼─────────────┼───────────────────────────────────┤ │aptitude why│ │explain the reason why │ │ │N/A │matching packages should be │ │ │ │installed │ ├────────────┼─────────────┼───────────────────────────────────┤ │aptitude │ │explain the reason why │ │why-not │N/A │matching packages can not be │ │ │ │installed │ └────────────┴─────────────┴───────────────────────────────────┘ Although it is now safe to mix different package tools on the Debian system, it is best to continue using aptitude as much as possible. The difference between "safe-upgrade"/"upgrade" and "full-upgrade"/"dist-upgrade" only appears when new versions of packages stand in different dependency relationships from old versions of those packages. The "aptitude safe-upgrade" command does not install new packages nor remove installed packages. The "aptitude why " can list more information by "aptitude -v why ". Similar information can be obtained by "apt-cache rdepends ". When aptitude command is started in the commandline mode and faces some issues such as package conflicts, you can switch to the full screen interactive mode by pressing "e"-key later at the prompt. You may provide command options right after "aptitude". Table 2.7. Notable command options for aptitude(8) ┌──────────┬───────────────────────────────────────────────────┐ │command │description │ │option │ │ ├──────────┼───────────────────────────────────────────────────┤ │-s │simulate the result of the command │ ├──────────┼───────────────────────────────────────────────────┤ │-d │download only but no install/upgrade │ ├──────────┼───────────────────────────────────────────────────┤ │-D │show brief explanations before the automatic │ │ │installations and removals │ └──────────┴───────────────────────────────────────────────────┘ See aptitude(8) and "aptitude user's manual" at "/usr/share/doc/ aptitude/README" for more. Tip The dselect package is still available and was the preferred full screen interactive package management tool in previous releases. 2.2.2. Interactive use of aptitude For the interactive package management, you start aptitude in interactive mode from the console shell prompt as follows. $ sudo aptitude -u Password: This updates the local copy of the archive information and display the package list in the full screen with menu. Aptitude places its configuration at "~/.aptitude/config". Tip If you want to use root's configuration instead of user's one, use "sudo -H aptitude …" instead of "sudo aptitude …" in the above expression. Tip Aptitude automatically sets pending actions as it is started interactively. If you do not like it, you can reset it from menu: "Action" → "Cancel pending actions". 2.2.3. Key bindings of aptitude Notable key strokes to browse status of packages and to set "planned action" on them in this full screen mode are the following. Table 2.8. List of key bindings for aptitude ┌───────────────────┬──────────────────────────────────────────┐ │key │key binding │ ├───────────────────┼──────────────────────────────────────────┤ │F10 or Ctrl-t │menu │ ├───────────────────┼──────────────────────────────────────────┤ │? │display help for keystroke (more complete │ │ │listing) │ ├───────────────────┼──────────────────────────────────────────┤ │F10 → Help → User's│display User's Manual │ │Manual │ │ ├───────────────────┼──────────────────────────────────────────┤ │u │update package archive information │ ├───────────────────┼──────────────────────────────────────────┤ │+ │mark the package for the upgrade or the │ │ │install │ ├───────────────────┼──────────────────────────────────────────┤ │- │mark the package for the remove (keep │ │ │configuration files) │ ├───────────────────┼──────────────────────────────────────────┤ │_ │mark the package for the purge (remove │ │ │configuration files) │ ├───────────────────┼──────────────────────────────────────────┤ │= │place the package on hold │ ├───────────────────┼──────────────────────────────────────────┤ │U │mark all upgradable packages (function as │ │ │full-upgrade) │ ├───────────────────┼──────────────────────────────────────────┤ │g │start downloading and installing selected │ │ │packages │ ├───────────────────┼──────────────────────────────────────────┤ │q │quit current screen and save changes │ ├───────────────────┼──────────────────────────────────────────┤ │x │quit current screen and discard changes │ ├───────────────────┼──────────────────────────────────────────┤ │Enter │view information about a package │ ├───────────────────┼──────────────────────────────────────────┤ │C │view a package's changelog │ ├───────────────────┼──────────────────────────────────────────┤ │l │change the limit for the displayed │ │ │packages │ ├───────────────────┼──────────────────────────────────────────┤ │/ │search for the first match │ ├───────────────────┼──────────────────────────────────────────┤ │\ │repeat the last search │ └───────────────────┴──────────────────────────────────────────┘ The file name specification of the command line and the menu prompt after pressing "l" and "//" take the aptitude regex as described below. Aptitude regex can explicitly match a package name using a string started by "~n and followed by the package name. Tip You need to press "U" to get all the installed packages upgraded to the candidate version in the visual interface. Otherwise only the selected packages and certain packages with versioned dependency to them are upgraded to the candidate version. 2.2.4. Package views under aptitude In the interactive full screen mode of aptitude(8), packages in the package list are displayed as the next example. idA libsmbclient -2220kB 3.0.25a-1 3.0.25a-2 Here, this line means from the left as the following. ● The "current state" flag (the first letter) ● The "planned action" flag (the second letter) ● The "automatic" flag (the third letter) ● The Package name ● The change in disk space usage attributed to "planned action" ● The current version of the package ● The candidate version of the package Tip The full list of flags are given at the bottom of Help screen shown by pressing "?". The candidate version is chosen according to the current local preferences (see apt_preferences(5) and Section 2.7.3, “Tweaking candidate version”). Several types of package views are available under the menu "Views". Table 2.9. List of views for aptitude ┌───────────────┬──────────┬───────────────────────────────────┐ │view │status │description of view │ ├───────────────┼──────────┼───────────────────────────────────┤ │ │ │see Table 2.10, “The categorization│ │Package View │Good │of standard package views” │ │ │ │(default) │ ├───────────────┼──────────┼───────────────────────────────────┤ │Audit │ │list packages which are recommended│ │Recommendations│Good │by some installed packages but not │ │ │ │yet installed are listed │ ├───────────────┼──────────┼───────────────────────────────────┤ │Flat Package │Good │list packages without │ │List │ │categorization (for use with regex)│ ├───────────────┼──────────┼───────────────────────────────────┤ │Debtags Browser│Very │list packages categorized according│ │ │usable │to their debtags entries │ ├───────────────┼──────────┼───────────────────────────────────┤ │Categorical │ │list packages categorized according│ │Browser │Deprecated│to their category (use Debtags │ │ │ │Browser, instead) │ └───────────────┴──────────┴───────────────────────────────────┘ Note Please help us improving tagging packages with debtags! The standard "Package View" categorizes packages somewhat like dselect with few extra features. Table 2.10. The categorization of standard package views ┌──────────────────────┬───────────────────────────────────────┐ │category │description of view │ ├──────────────────────┼───────────────────────────────────────┤ │Upgradable Packages │list packages organized as section → │ │ │component → package │ ├──────────────────────┼───────────────────────────────────────┤ │New Packages │, , │ ├──────────────────────┼───────────────────────────────────────┤ │Installed Packages │, , │ ├──────────────────────┼───────────────────────────────────────┤ │Not Installed Packages│, , │ ├──────────────────────┼───────────────────────────────────────┤ │Obsolete and Locally │, , │ │Created Packages │ │ ├──────────────────────┼───────────────────────────────────────┤ │Virtual Packages │list packages with the same function │ ├──────────────────────┼───────────────────────────────────────┤ │Tasks │list packages with different functions │ │ │generally needed for a task │ └──────────────────────┴───────────────────────────────────────┘ Tip Tasks view can be used to cherry pick packages for your task. 2.2.5. Search method options with aptitude Aptitude offers several options for you to search packages using its regex formula. ● Shell commandline: ○ "aptitude search ''" to list installation status, package name and short description of matching packages ○ "aptitude show ''" to list detailed description of the package ● Interactive full screen mode: ○ "l" to limit package view to matching packages ○ "/" for search to a matching package ○ "\" for backward search to a matching package ○ "n" for find-next ○ "N" for find-next (backward) Tip The string for is treated as the exact string match to the package name unless it is started explicitly with "~" to be the regex formula. 2.2.6. The aptitude regex formula The aptitude regex formula is mutt-like extended ERE (see Section 1.6.2, “Regular expressions”) and the meanings of the aptitude specific special match rule extensions are as follows. Table 2.11. List of the aptitude regex formula ┌──────────────┬──────────────────────────────────────────────────┐ │description of│ │ │the extended │regex formula │ │match rule │ │ ├──────────────┼──────────────────────────────────────────────────┤ │match on │~n │ │package name │ │ ├──────────────┼──────────────────────────────────────────────────┤ │match on │~d │ │description │ │ ├──────────────┼──────────────────────────────────────────────────┤ │match on task │~t │ │name │ │ ├──────────────┼──────────────────────────────────────────────────┤ │match on │~G │ │debtag │ │ ├──────────────┼──────────────────────────────────────────────────┤ │match on │~m │ │maintainer │ │ ├──────────────┼──────────────────────────────────────────────────┤ │match on │ │ │package │~s │ │section │ │ ├──────────────┼──────────────────────────────────────────────────┤ │match on │ │ │package │~V │ │version │ │ ├──────────────┼──────────────────────────────────────────────────┤ │match archive │~A{sarge,etch,sid} │ ├──────────────┼──────────────────────────────────────────────────┤ │match origin │~O{debian,…} │ ├──────────────┼──────────────────────────────────────────────────┤ │match priority│~p{extra,important,optional,required,standard} │ ├──────────────┼──────────────────────────────────────────────────┤ │match │ │ │essential │~E │ │packages │ │ ├──────────────┼──────────────────────────────────────────────────┤ │match virtual │~v │ │packages │ │ ├──────────────┼──────────────────────────────────────────────────┤ │match new │~N │ │packages │ │ ├──────────────┼──────────────────────────────────────────────────┤ │match with │~a │ │pending action│{install,upgrade,downgrade,remove,purge,hold,keep}│ ├──────────────┼──────────────────────────────────────────────────┤ │match │ │ │installed │~i │ │packages │ │ ├──────────────┼──────────────────────────────────────────────────┤ │match │ │ │installed │ │ │packages with │~M │ │A-mark (auto │ │ │installed │ │ │package) │ │ ├──────────────┼──────────────────────────────────────────────────┤ │match │ │ │installed │ │ │packages │ │ │without A-mark│~i!~M │ │(administrator│ │ │selected │ │ │package) │ │ ├──────────────┼──────────────────────────────────────────────────┤ │match │ │ │installed and │~U │ │upgradable │ │ │packages │ │ ├──────────────┼──────────────────────────────────────────────────┤ │match removed │ │ │but not purged│~c │ │packages │ │ ├──────────────┼──────────────────────────────────────────────────┤ │match removed,│ │ │purged or │~g │ │can-be-removed│ │ │packages │ │ ├──────────────┼──────────────────────────────────────────────────┤ │match packages│ │ │with broken │~b │ │relation │ │ ├──────────────┼──────────────────────────────────────────────────┤ │match packages│ │ │with broken │ │ │depends/ │~B │ │predepends/ │ │ │conflict │ │ ├──────────────┼──────────────────────────────────────────────────┤ │match packages│ │ │from which │ │ │relation │~D[:] │ │ is │ │ │defined to │ │ │ package│ │ ├──────────────┼──────────────────────────────────────────────────┤ │match packages│ │ │from which │ │ │broken │ │ │relation │~DB[:] │ │ is │ │ │defined to │ │ │ package│ │ ├──────────────┼──────────────────────────────────────────────────┤ │match packages│ │ │to which the │ │ │ package│~R[:] │ │defines │ │ │relation │ │ │ │ │ ├──────────────┼──────────────────────────────────────────────────┤ │match packages│ │ │to which the │ │ │ package│~RB[:] │ │defines broken│ │ │relation │ │ │ │ │ ├──────────────┼──────────────────────────────────────────────────┤ │match packages│ │ │to which some │ │ │other │~R~i │ │installed │ │ │packages │ │ │depend on │ │ ├──────────────┼──────────────────────────────────────────────────┤ │match packages│ │ │to which no │ │ │other │!~R~i │ │installed │ │ │packages │ │ │depend on │ │ ├──────────────┼──────────────────────────────────────────────────┤ │match packages│ │ │to which some │ │ │other │ │ │installed │~R~i|~Rrecommends:~i │ │packages │ │ │depend or │ │ │recommend on │ │ ├──────────────┼──────────────────────────────────────────────────┤ │match │ │ │package with │~S filter │ │filtered │ │ │version │ │ ├──────────────┼──────────────────────────────────────────────────┤ │match all │ │ │packages │~T │ │(true) │ │ ├──────────────┼──────────────────────────────────────────────────┤ │match no │ │ │packages │~F │ │(false) │ │ └──────────────┴──────────────────────────────────────────────────┘ ● The regex part is the same ERE as the one used in typical Unix-like text tools using "^", ".*", "$" etc. as in egrep (1), awk(1) and perl(1). ● The relation is one of (depends, predepends, recommends, suggests, conflicts, replaces, provides). ● The default relation type is "depends". Tip When is a null string, place "~T" immediately after the command. Here are some short cuts. ● "~P" == "~Dprovides:" ● "~C" == "~Dconflicts:" ● "…~W term" == "(…|term)" Users familiar with mutt pick up quickly, as mutt was the inspiration for the expression syntax. See "SEARCHING, LIMITING, AND EXPRESSIONS" in the "User's Manual" "/usr/share/doc/aptitude /README". Note With the lenny version of aptitude(8), the new long form syntax such as "?broken" may be used for regex matching in place for its old short form equivalent "~b". Now space character " " is considered as one of the regex terminating character in addition to tilde character "~". See "User's Manual" for the new long form syntax. 2.2.7. Dependency resolution of aptitude The selection of a package in aptitude not only pulls in packages which are defined in its "Depends:" list but also defined in the "Recommends:" list if the menu "F10 → Options → Dependency handling" is set accordingly. These auto installed packages are removed automatically if they are no longer needed under aptitude. Note Before the lenny release, apt-get and other standard APT tools did not offer the autoremove functionality. 2.2.8. Package activity logs You can check package activity history in the log files. Table 2.12. The log files for package activities ┌─────────────────┬────────────────────────────────────────────┐ │file │content │ ├─────────────────┼────────────────────────────────────────────┤ │/var/log/dpkg.log│Log of dpkg level activity for all package │ │ │activities │ ├─────────────────┼────────────────────────────────────────────┤ │/var/log/apt/ │Log of generic APT activity │ │term.log │ │ ├─────────────────┼────────────────────────────────────────────┤ │/var/log/aptitude│Log of aptitude command activity │ └─────────────────┴────────────────────────────────────────────┘ In reality, it is not so easy to get meaningful understanding quickly out from these logs. See Section 9.2.9, “Recording changes in configuration files” for easier way. 2.2.9. Aptitude advantages Aptitude has advantages over other APT based packaging systems (apt-get, apt-cache, synaptic, …). ● aptitude removes unused auto installed packages automatically using its own extra layer of package state file (/var/lib/aptitude/pkgstates). (For new "lenny", other APT does the same.) ● aptitude makes it easy to resolve package conflicts and to add recommended packages. ● aptitude makes it easy to keep track of obsolete software by listing under "Obsolete and Locally Created Packages". ● aptitude gives a log of its history in "/var/log/aptitude". ● aptitude offers access to all versions of the package if available. ● aptitude includes a fairly powerful regex based system for searching particular packages and limiting the package display. ● aptitude in the full screen mode has su functionality embedded and can be run from normal user until you really need administrative privileges. For the old etch release version, synaptic also gives you the history log; apt-get did not but you can rely on the log of dpkg. Anyway, aptitude is nice for interactive console use. 2.3. Examples of aptitude operations Here are few examples of aptitude(8) operations. 2.3.1. Listing packages with regex matching on package names The following command lists packages with regex matching on package names. $ aptitude search '~n(pam|nss).*ldap' p libnss-ldap - NSS module for using LDAP as a naming service p libpam-ldap - Pluggable Authentication Module allowing LDAP interfaces This is quite handy for you to find the exact name of a package. 2.3.2. Browsing with the regex matching The regex "~dipv6" in the "New Flat Package List" view with "l" prompt, limits view to packages with the matching description and let you browse their information interactively. 2.3.3. Purging removed packages for good You can purge all remaining configuration files of removed packages. Check results of the following command. # aptitude search '~c' If you think listed packages are OK to be purged, execute the following command. # aptitude purge '~c' You may want to do the similar in the interactive mode for fine grained control. You provide the regex "~c" in the "New Flat Package List" view with "l" prompt. This limits the package view only to regex matched packages, i.e., "removed but not purged". All these regex matched packages can be shown by pressing "[" at top level headings. Then you press "_" at top level headings such as "Installed Packages". Only regex matched packages under the heading are marked to be purged by this. You can exclude some packages to be purged by pressing "=" interactively for each of them. This technique is quite handy and works for many other command keys. 2.3.4. Tidying auto/manual install status Here is how I tidy auto/manual install status for packages (after using non-aptitude package installer etc.). 1. Start aptitude in interactive mode as root. 2. Type "u", "U", "f" and "g" to update and upgrade package list and packages. 3. Type "l" to enter the package display limit as "~i(~R~i| ~Rrecommends:~i)" and type "M" over "Installed Packages" as auto installed. 4. Type "l" to enter the package display limit as "~prequired| ~pimportant|~pstandard|~E" and type "m" over "Installed Packages" as manual installed. 5. Type "l" to enter the package display limit as "~i!~M" and remove unused package by typing "-" over each of them after exposing them by typing "[" over "Installed Packages". 6. Type "l" to enter the package display limit as "~i" and type "m" over "Tasks" as manual installed. 7. Exit aptitude. 8. Start "apt-get -s autoremove|less" as root to check what are not used. 9. Restart aptitude in interactive mode and mark needed packages as "m". 10. Restart "apt-get -s autoremove|less" as root to recheck REMOVED contain only expected packages. 11. Start "apt-get autoremove|less" as root to autoremove unused packages. The "m" action over "Tasks" is an optional one to prevent mass package removal situation in future. 2.3.5. System wide upgrade with aptitude Note When moving to a new release etc, you should consider to perform a clean installation of new system even though Debian is upgradable as described below. This provides you a chance to remove garbages collected and exposes you to the best combination of latest packages. Of course, you should make a full backup of system to a safe place (see Section 10.1.6, “Backup and recovery”) before doing this. I recommend to make a dual boot configuration using different partition to have the smoothest transition. You can perform system wide upgrade to a newer release by changing contents of the "/etc/apt/sources.list" file pointing to a new release and running the "aptitude update; aptitude full-upgrade" command. To upgrade from stable to testing or unstable, you replace "lenny" in the "/etc/apt/sources.list" example of Section 2.1.4, “Debian archive basics” with "squeeze" or "sid". In reality, you may face some complications due to some package transition issues, mostly due to package dependencies. The larger the difference of the upgrade, the more likely you face larger troubles. For the transition from the old stable to the new stable after its release, you can read its new Release Notes and follow the exact procedure described in it to minimize troubles. When you decide to move from stable to testing before its formal release, there are no Release Notes to help you. The difference between stable and testing could have grown quite large after the previous stable release and makes upgrade situation complicated. You should make precautionary moves for the full upgrade while gathering latest information from mailing list and using common senses. 1. Read previous "Release Notes". 2. Backup entire system (especially data and configuration information). 3. Have bootable media handy for broken bootloader. 4. Inform users on the system well in advance. 5. Record upgrade activity with script(1). 6. Apply "unmarkauto" to required packages, e.g., "aptitude unmarkauto vim", to prevent removal. 7. Minimize installed packages to reduce chance of package conflicts, e.g., remove desktop task packages. 8. Remove the "/etc/apt/preferences" file (disable apt-pinning). 9. Try to upgrade step wise: oldstable → stable → testing → unstable. 10. Update the "/etc/apt/sources.list" file to point to new archive only and run "aptitude update". 11. Install, optionally, new core packages first, e.g., "aptitude install perl". 12. Run the "aptitude full-upgrade -s" command to assess impact. 13. Run the "aptitude full-upgrade" command at last. Caution It is not wise to skip major Debian release when upgrading between stable releases. Caution In previous "Release Notes", GCC, Linux Kernel, initrd-tools, Glibc, Perl, APT tool chain, etc. have required some special attention for system wide upgrade. For daily upgrade in unstable, see Section 2.4.3, “Safeguarding for package problems”. 2.4. Advanced package management operations 2.4.1. Advanced package management operations with commandline Here are list of other package management operations for which aptitude is too high-level or lacks required functionalities. Table 2.13. List of advanced package management operations ┌───────────────────────────┬──────────────────────────────────┐ │command │action │ ├───────────────────────────┼──────────────────────────────────┤ │COLUMNS=120 dpkg -l │list status of an installed │ │ │package for the bug report │ ├───────────────────────────┼──────────────────────────────────┤ │dpkg -L │list contents of an installed │ │ │package │ ├───────────────────────────┼──────────────────────────────────┤ │dpkg -L | │list manpages for an installed │ │egrep '/usr/share/man/man.*│package │ │/.+' │ │ ├───────────────────────────┼──────────────────────────────────┤ │dpkg -S │list installed packages which have│ │ │matching file name │ ├───────────────────────────┼──────────────────────────────────┤ │apt-file search │list packages in archive which │ │ │have matching file name │ ├───────────────────────────┼──────────────────────────────────┤ │apt-file list │list contents of matching packages│ │ │in archive │ ├───────────────────────────┼──────────────────────────────────┤ │dpkg-reconfigure │reconfigure the exact package │ │ │ │ ├───────────────────────────┼──────────────────────────────────┤ │dpkg-reconfigure -p=low │reconfigure the exact package with│ │ │the most detailed question │ ├───────────────────────────┼──────────────────────────────────┤ │configure-debian │reconfigure packages from the full│ │ │screen menu │ ├───────────────────────────┼──────────────────────────────────┤ │dpkg --audit │audit system for partially │ │ │installed packages │ ├───────────────────────────┼──────────────────────────────────┤ │dpkg --configure -a │configure all partially installed │ │ │packages │ ├───────────────────────────┼──────────────────────────────────┤ │apt-cache policy │show available version, priority, │ │ │and archive information of a │ │ │binary package │ ├───────────────────────────┼──────────────────────────────────┤ │apt-cache madison │show available version, archive │ │ │information of a package │ ├───────────────────────────┼──────────────────────────────────┤ │apt-cache showsrc │show source package information of│ │ │a binary package │ ├───────────────────────────┼──────────────────────────────────┤ │apt-get build-dep │install required packages to build│ │ │package │ ├───────────────────────────┼──────────────────────────────────┤ │apt-get source │download a source (from standard │ │ │archive) │ ├───────────────────────────┼──────────────────────────────────┤ │dget │download a source packages (from │ │ │other archive) │ ├───────────────────────────┼──────────────────────────────────┤ │dpkg-source -x │build a source tree from a set of │ │_- │source packages ("*.tar.gz" and │ │.dsc │"*.diff.gz") │ ├───────────────────────────┼──────────────────────────────────┤ │debuild binary │build package(s) from a local │ │ │source tree │ ├───────────────────────────┼──────────────────────────────────┤ │make-kpkg kernel_image │build a kernel package from a │ │ │kernel source tree │ ├───────────────────────────┼──────────────────────────────────┤ │make-kpkg --initrd │build a kernel package from a │ │kernel_image │kernel source tree with initramfs │ │ │enabled │ ├───────────────────────────┼──────────────────────────────────┤ │dpkg -i │install a local package to the │ │- │system │ │.deb │ │ ├───────────────────────────┼──────────────────────────────────┤ │debi │install local package(s) to the │ │- │system │ │.dsc │ │ ├───────────────────────────┼──────────────────────────────────┤ │dpkg --get-selection '*' > │save dpkg level package selection │ │selection.txt │state information │ ├───────────────────────────┼──────────────────────────────────┤ │dpkg --set-selection │set dpkg level package selection │ │" can be used search package names which contain files with the matching name installed by dpkg. But this overlooks files created by the maintainer scripts. If you need to make more elaborate search on the dpkg meta data, you need to run "grep -e regex_pattern *" command in the "/var/ lib/dpkg/info/" directory. This makes you search words mentioned in package scripts and installation query texts. If you wish to look up package dependency recursively, you should use apt-rdepends(8). 2.5. Debian package management internals Let's learn how the Debian package management system works internally. This should help you to create your own solution to some package problems. 2.5.1. Archive meta data Meta data files for each distribution are stored under "dist/ " on each Debian mirror sites, e.g., "http:// ftp.us.debian.org/debian/". Its archive structure can be browsed by the web browser. There are 6 types of key meta data. Table 2.14. The content of the Debian archive meta data ┌──────────────┬───────────────────────┬───────────────────────┐ │file │location │content │ ├──────────────┼───────────────────────┼───────────────────────┤ │Release │top of distribution │archive description and│ │ │ │integrity information │ ├──────────────┼───────────────────────┼───────────────────────┤ │ │ │signature file for the │ │Release.gpg │top of distribution │"Release" file signed │ │ │ │with the archive key │ ├──────────────┼───────────────────────┼───────────────────────┤ │Contents- │ │list of all files for │ ││top of distribution │all the packages in the│ │ │ │pertinent archive │ ├──────────────┼───────────────────────┼───────────────────────┤ │ │top of each │archive description │ │Release │distribution/component/│used for the rule of │ │ │architecture │apt_preferences(5) │ │ │combination │ │ ├──────────────┼───────────────────────┼───────────────────────┤ │ │top of each │concatenated debian/ │ │Packages │distribution/component/│control for binary │ │ │binary-architecture │packages │ │ │combination │ │ ├──────────────┼───────────────────────┼───────────────────────┤ │ │top of each │concatenated debian/ │ │Sources │distribution/component/│control for source │ │ │source combination │packages │ └──────────────┴───────────────────────┴───────────────────────┘ In the recent archive, these meta data are stored as the compressed and differential files to reduce network traffic. 2.5.2. Top level "Release" file and authenticity Tip The top level "Release" file is used for signing the archive under the secure APT system. Each suite of the Debian archive has a top level "Release" file, e.g., "http://ftp.us.debian.org/debian/dists/unstable/Release", as follows. Origin: Debian Label: Debian Suite: unstable Codename: sid Date: Sat, 26 Jan 2008 20:13:58 UTC Architectures: alpha amd64 arm hppa hurd-i386 i386 ia64 m68k mips mipsel powerpc s390 sparc Components: main contrib non-free Description: Debian x.y Unstable - Not Released MD5Sum: e9f11bc50b12af7927d6583de0a3bd06 22788722 main/binary-alpha/Packages 43524d07f7fa21b10f472c426db66168 6561398 main/binary-alpha/Packages.gz ... Note Here, you can find my rationale to use the "suite", "codeneme", and "components" in Section 2.1.4, “Debian archive basics”. The "distribution" is used when referring to both "suite" and "codeneme". The integrity of the top level "Release" file is verified by cryptographic infrastructure called the secure apt. ● The cryptographic signature file "Release.gpg" is created from the authentic top level "Release" file and the secret Debian archive key. ● The public Debian archive key can be seeded into "/etc/apt/ trusted.gpg"; ○ automatically by installing the keyring with the latest base-files package, or ○ manually by gpg or apt-key tool with the latest public archive key posted on the ftp-master.debian.org . ● The secure APT system verifies the integrity of the downloaded top level "Release" file cryptographically by this "Release.gpg" file and the public Debian archive key in "/etc/apt/trusted.gpg". The integrity of all the "Packages" and "Sources" files are verified by using MD5sum values in its top level "Release" file. The integrity of all package files are verified by using MD5sum values in the "Packages" and "Sources" files. See debsums(1) and Section 2.4.2, “Verification of installed package files”. Since the cryptographic signature verification is very CPU intensive process than the MD5sum value calculation, use of MD5sum value for each package while using cryptographic signature for the top level "Release" file provides the good security with the performance (see Section 10.3, “Data security infrastructure”). 2.5.3. Archive level "Release" files Tip The archive level "Release" files are used for the rule of apt_preferences(5). There are archive level "Release" files for all archive locations specified by "deb" line in "/etc/apt/sources.list", such as "http://ftp.us.debian.org/debian/dists/unstable/main/ binary-amd64/Release" or "http://ftp.us.debian.org/debian/dists/ sid/main/binary-amd64/Release" as follows. Archive: unstable Component: main Origin: Debian Label: Debian Architecture: amd64 Caution For "Archive:" stanza, suite names ("stable", "testing", "unstable", …) are used in the Debian archive while codenames ("dapper", "feisty", "gutsy", "hardy", "intrepid", …) are used in the Ubuntu archive. For some archives, such as experimental, volatile-sloppy, and lenny-backports, which contain packages which should not be installed automatically, there is an extra line, e.g., "http:// ftp.us.debian.org/debian/dists/experimental/main/binary-amd64/ Release" as follows. Archive: experimental Component: main Origin: Debian Label: Debian NotAutomatic: yes Architecture: amd64 Please note that for normal archives without "NotAutomatic: yes", the default Pin-Priority value is 500, while for special archives with "NotAutomatic: yes", the default Pin-Priority value is 1 (see apt_preferences(5) and Section 2.7.3, “Tweaking candidate version”). 2.5.4. Fetching of the meta data for the package When APT tools, such as aptitude, apt-get, synaptic, apt-file, auto-apt…, are used, we need to update the local copies of the meta data containing the Debian archive information. These local copies have following file names corresponding to the specified distribution, component, and architecture names in the "/etc/apt /sources.list" (see Section 2.1.4, “Debian archive basics”). ● "/var/lib/apt/lists/ftp.us.debian.org_debian_dists_ _Release" ● "/var/lib/apt/lists/ftp.us.debian.org_debian_dists_ _Release.gpg" ● "/var/lib/apt/lists/ftp.us.debian.org_debian_dists_ __binary-_Packages" ● "/var/lib/apt/lists/ftp.us.debian.org_debian_dists_ __source_Sources" ● "/var/cache/apt/apt-file/ftp.us.debian.org_debian_dists_ _Contents-.gz" (for apt-file) First 4 types of files are shared by all the pertinent APT commands and updated from command line by "apt-get update" and "aptitude update". The "Packages" meta data are updated if there is the "deb" line in "/etc/apt/sources.list". The "Sources" meta data are updated if there is the "deb-src" line in "/etc/apt/ sources.list". The "Packages" and "Sources" meta data contain "Filename:" stanza pointing to the file location of the binary and source packages. Currently, these packages are located under the "pool/ " directory tree for the improved transition over the releases. Local copies of "Packages" meta data can be interactively searched with the help of aptitude. The specialized search command grep-dctrl(1) can search local copies of "Packages" and "Sources" meta data. Local copy of "Contents-" meta data can be updated by "apt-file update" and its location is different from other 4 ones. See apt-file(1). (The auto-apt uses different location for local copy of "Contents-.gz" as default.) 2.5.5. The package state for APT In addition to the remotely fetched meta data, the APT tool after lenny stores its locally generated installation state information in the "/var/lib/apt/extended_states" which is used by all APT tools to track all auto installed packages. 2.5.6. The package state for aptitude In addition to the remotely fetched meta data, the aptitude command stores its locally generated installation state information in the "/var/lib/aptitude/pkgstates" which is used only by it. 2.5.7. Local copies of the fetched packages All the remotely fetched packages via APT mechanism are stored in the "/var/cache/apt/packages" until they are cleaned. 2.5.8. Debian package file names Debian package files have particular name structures. Table 2.15. The name structure of Debian packages ┌──────────────────────────┬───────────────────────────────────┐ │package type │name structure │ ├──────────────────────────┼───────────────────────────────────┤ │The binary package (a.k.a │_: │ │deb) │-│ │ │-.deb │ ├──────────────────────────┼───────────────────────────────────┤ │The binary package for the│_: │ │debian-installer (a.k.a │-│ │udeb) │-.udeb │ ├──────────────────────────┼───────────────────────────────────┤ │The source package │_: │ │(upstream source) │- │ │ │.tar.gz │ ├──────────────────────────┼───────────────────────────────────┤ │The source package (Debian│_: │ │changes) │- │ │ │.diff.gz │ ├──────────────────────────┼───────────────────────────────────┤ │The source package │_: │ │(description) │- │ │ │.dsc │ └──────────────────────────┴───────────────────────────────────┘ Table 2.16. The usable characters for each component in the Debian package names ┌──────────────────┬─────────────────────────┬─────────┐ │name component │usable characters (regex)│existence│ ├──────────────────┼─────────────────────────┼─────────┤ │ │[a-z,A-Z,0-9,.,,-] │required │ ├──────────────────┼─────────────────────────┼─────────┤ │: │[0-9]+: │optional │ ├──────────────────┼─────────────────────────┼─────────┤ ││[a-z,A-Z,0-9,.,,-,:] │required │ ├──────────────────┼─────────────────────────┼─────────┤ │ │[a-z,A-Z,0-9,.,,~] │optional │ └──────────────────┴─────────────────────────┴─────────┘ Note You can check package version order by dpkg(1), e.g., "dpkg --compare-versions 7.0 gt 7.~pre1 ; echo $?" . Note The debian-installer (d-i) uses udeb as the file extension for its binary package instead of normal deb. An udeb package is a stripped down deb package which removes few non-essential contents such as documentation to save space while relaxing the package policy requirements. Both deb and udeb packages share the same package structure. The "u" stands for micro. 2.5.9. The dpkg command dpkg(1) is the lowest level tool for the Debian package management. This is very powerful and needs to be used with care. While installing package called "", dpkg process it in the following order. 1. Unpack the deb file ("ar -x" equivalent) 2. Execute ".preinst" using debconf(1) 3. Install the package content to the system ("tar -x" equivalent) 4. Execute ".postinst" using debconf(1) The debconf system provides standardized user interaction with I18N and L10N (Chapter 8, I18N and L10N) supports. Table 2.17. The notable files created by dpkg ┌────────────────────────┬─────────────────────────────────────┐ │file │description of contents │ ├────────────────────────┼─────────────────────────────────────┤ │/var/lib/dpkg/info/ │list of configuration files. (user │ │.conffiles│modifiable) │ ├────────────────────────┼─────────────────────────────────────┤ │/var/lib/dpkg/info/ │list of files and directories │ │.list │installed by the package │ ├────────────────────────┼─────────────────────────────────────┤ │/var/lib/dpkg/info/ │list of MD5 hash values for files │ │.md5sums │installed by the package │ ├────────────────────────┼─────────────────────────────────────┤ │/var/lib/dpkg/info/ │package script run before the package│ │.preinst │installation │ ├────────────────────────┼─────────────────────────────────────┤ │/var/lib/dpkg/info/ │package script run after the package │ │.postinst │installation │ ├────────────────────────┼─────────────────────────────────────┤ │/var/lib/dpkg/info/ │package script run before the package│ │.prerm │removal │ ├────────────────────────┼─────────────────────────────────────┤ │/var/lib/dpkg/info/ │package script run after the package │ │.postrm │removal │ ├────────────────────────┼─────────────────────────────────────┤ │/var/lib/dpkg/info/ │package script for debconf system │ │.config │ │ ├────────────────────────┼─────────────────────────────────────┤ │/var/lib/dpkg/ │the alternative information used by │ │alternatives/ │the update-alternatives command │ │ │ │ ├────────────────────────┼─────────────────────────────────────┤ │/var/lib/dpkg/available │the availability information for all │ │ │the package │ ├────────────────────────┼─────────────────────────────────────┤ │/var/lib/dpkg/diversions│the diversions information used by │ │ │dpkg(1) and set by`dpkg-divert`(8) │ ├────────────────────────┼─────────────────────────────────────┤ │/var/lib/dpkg/ │the stat override information used by│ │statoverride │dpkg(1) and set by`dpkg-statoverride │ │ │`(8) │ ├────────────────────────┼─────────────────────────────────────┤ │/var/lib/dpkg/status │the status information for all the │ │ │packages │ ├────────────────────────┼─────────────────────────────────────┤ │/var/lib/dpkg/status-old│the first-generation backup of the │ │ │"var/lib/dpkg/status" file │ ├────────────────────────┼─────────────────────────────────────┤ │/var/backups/ │the second-generation backup and │ │dpkg.status* │older ones of the "var/lib/dpkg/ │ │ │status" file │ └────────────────────────┴─────────────────────────────────────┘ The "status" file is also used by the tools such as dpkg(1), "dselect update" and "apt-get -u dselect-upgrade". The specialized search command grep-dctrl(1) can search the local copies of "status" and "available" meta data. Tip In the debian-installer environment, the udpkg command is used to open udeb packages. The udpkg command is a stripped down version of the dpkg command. 2.5.10. The update-alternative command The Debian system has mechanism to install somewhat overlapping programs peacefully using update-alternatives(8). For example, you can make the vi command select to run vim while installing both vim and nvi packages. $ ls -l $(type -p vi) lrwxrwxrwx 1 root root 20 2007-03-24 19:05 /usr/bin/vi -> /etc/alternatives/vi $ sudo update-alternatives --display vi ... $ sudo update-alternatives --config vi Selection Command ---------------------------------------------- 1 /usr/bin/vim *+ 2 /usr/bin/nvi Enter to keep the default[*], or type selection number: 1 The Debian alternatives system keeps its selection as symlinks in "/etc/alternatives/". The selection process uses corresponding file in "/var/lib/dpkg/alternatives/". 2.5.11. The dpkg-statoverride command Stat overrides provided by the dpkg-statoverride(8) command are a way to tell dpkg(1) to use a different owner or mode for a file when a package is installed. If "--update" is specified and file exists, it is immediately set to the new owner and mode. Caution The direct alteration of owner or mode for a file owned by the package using chmod or chown commands by the system administrator is reset by the next upgrade of the package. Note I use the word file here, but in reality this can be any filesystem object that dpkg handles, including directories, devices, etc. 2.5.12. The dpkg-divert command File diversions provided by the dpkg-divert(8) command are a way of forcing dpkg(1) not to install a file into its default location, but to a diverted location. The use of dpkg-divert is meant for the package maintenance scripts. Its casual use by the system administrator is deprecated. 2.6. Recovery from a broken system When running unstable system, the administrator is expected to recover from broken package management situation. Caution Some methods described here are high risk actions. You have been warned! 2.6.1. Incompatibility with old user configuration If a desktop GUI program experienced instability after significant upstream version upgrade, you should suspect interferences with old local configuration files created by it. If it is stable under newly created user account, this hypothesis is confirmed. (This is a bug of packaging and usually avoided by the packager.) To recover stability, you should move corresponding local configuration files and restart the GUI program. You may need to read old configuration file contents to recover configuration information later. (Do not erase them too quickly.) 2.6.2. Different packages with overlapped files Archive level package management systems, such as aptitude(8) or apt-get(1), do not even try to install packages with overlapped files using package dependencies (see Section 2.1.5, “Package dependencies”). Errors by the package maintainer or deployment of inconsistently mixed source of archives (see Section 2.7.2, “Packages from mixed source of archives”) by the system administrator may create situation with incorrectly defined package dependencies. When you install a package with overlapped files using aptitude (8) or apt-get(1) under such situation, dpkg(1) which unpacks package ensures to return error to the calling program without overwriting existing files. Caution The use of third party packages introduces significant system risks via maintainer scripts which are run with root privilege and can do anything to your system. The dpkg(1) command only protects against overwriting by the unpacking. You can work around such broken installation by removing the old offending package, , first. $ sudo dpkg -P 2.6.3. Fixing broken package script When a command in the package script returns error for some reason and the script exits with error, the package management system aborts their action and ends up with partially installed packages. When a package contains bugs in its removal scripts, the package may become impossible to remove and quite nasty. For the package script problem of "", you should look into following package scripts. ● "/var/lib/dpkg/info/.preinst" ● "/var/lib/dpkg/info/.postinst" ● "/var/lib/dpkg/info/.prerm" ● "/var/lib/dpkg/info/.postrm" Edit the offending package script from the root using following techniques. ● disable the offending line by preceding "#" ● force to return success by appending the offending line with "|| true" Configure all partially installed packages with the following command. # dpkg --configure -a 2.6.4. Rescue with the dpkg command Since dpkg is very low level package tool, it can function under the very bad situation such as unbootable system without network connection. Let's assume foo package was broken and needs to be replaced. You may still find cached copies of older bug free version of foo package in the package cache directory: "/var/cache/apt/ archives/". (If not, you can download it from archive of http:// snapshot.debian.net/ or copy it from package cache of a functioning machine.) If you can boot the system, you may install it by the following command. # dpkg -i /path/to/foo__.deb Tip If system breakage is minor, you may alternatively downgrade the whole system as Section 2.7.7, “Emergency downgrading” using the higher level APT system. If your system is unbootable from hard disk, you should seek other ways to boot it. 1. Boot the system using the debian-installer CD in rescue mode. 2. Mount the unbootable system on the hard disk to "/target". 3. Install older version of foo package by the following. # dpkg --root /target -i /path/to/foo__.deb This example works even if the dpkg command on the hard disk is broken. Tip Any GNU/Linux system started by another system on hard disk, live GNU/Linux CD, bootable USB-key drive, or netboot can be used similarly to rescue broken system. If attempting to install a package this way fails due to some dependency violations and you really need to do this as the last resort, you can override dependency using dpkg's "--ignore-depends", "--force-depends" and other options. If you do this, you need to make serious effort to restore proper dependency later. See dpkg(8) for details. Note When your system is seriously broken, you should make a full backup of system to a safe place (see Section 10.1.6, “Backup and recovery”) and should perform a clean installation. This is less time consuming and produces better results in the end. 2.6.5. Recovering package selection data If "/var/lib/dpkg/status" becomes corrupt for any reason, the Debian system loses package selection data and suffers severely. Look for the old "/var/lib/dpkg/status" file at "/var/lib/dpkg/ status-old" or "/var/backups/dpkg.status.*". Keeping "/var/backups/" in a separate partition may be a good idea since this directory contains lots of important system data. For serious breakage, I recommend to make fresh re-install after making backup of the system. Even if everything in "/var/" is gone, you can still recover some information from directories in "/usr/share/doc/" to guide your new installation. Reinstall minimal (desktop) system. # mkdir -p /path/to/old/system Mount old system at "/path/to/old/system/". # cd /path/to/old/system/usr/share/doc # ls -1 >~/ls1.txt # cd /usr/share/doc # ls -1 >>~/ls1.txt # cd # sort ls1.txt | uniq | less Then you are presented with package names to install. (There may be some non-package names such as "texmf".) 2.7. Tips for the package management 2.7.1. How to pick Debian packages You can seek packages which satisfy your needs with aptitude from the package description or from the list under "Tasks". When you encounter more than 2 similar packages and wonder which one to install without "trial and error" efforts, you should use some common sense. I consider following points are good indications of preferred packages. ● Essential: yes > no ● Component: main > contrib > non-free ● Priority: required > important > standard > optional > extra ● Tasks: packages listed in tasks such as "Desktop environment" ● Packages selected by the dependency package (e.g., python2.4 by python) ● Popcon: higher in the vote and install number ● Changelog: regular updates by the maintainer ● BTS: No RC bugs (no critical, no grave, and no serious bugs) ● BTS: responsive maintainer to bug reports ● BTS: higher number of the recently fixed bugs ● BTS: lower number of remaining non-wishlist bugs Debian being a volunteer project with distributed development model, its archive contains many packages with different focus and quality. You must make your own decision what to do with them. 2.7.2. Packages from mixed source of archives Caution Installing packages from mixed source of archives is not supported by the official Debian distribution except for officially supported particular combinations of archives such as stable with security updates and volatile updates. Here is an example of operations to include specific newer upstream version packages found in unstable while tracking testing for single occasion. 1. Change the "/etc/apt/sources.list" file temporarily to single "unstable" entry. 2. Run "aptitude update". 3. Run "aptitude install ". 4. Recover the original "/etc/apt/sources.list" file for testing. 5. Run "aptitude update". You do not create the "/etc/apt/preferences" file nor need to worry about apt-pinning with this manual approach. But this is very cumbersome. Caution When using mixed source of archives, you must ensure compatibility of packages by yourself since the Debian does not guarantee it. If package incompatibility exists, you may break system. You must be able to judge these technical requirements. The use of mixed source of random archives is completely optional operation and its use is not something I encourage you to use. General rules for installing packages from different archives are followings. ● Non-binary packages ("Architecture: all") are safer to install. ○ documentation packages: no special requirements ○ interpreter program packages: compatible interpreter must be available ● Binary packages (non "Architecture: all") usually face many road blocks and unsafe to install. ○ library version compatibility (including "libc") ○ related utility program version compatibility ○ Kernel ABI compatibility ○ C++ ABI compatibility ○ … Note In order to make a package to be safer to install, some commercial non-free binary program packages may be provided with completely statically linked libraries. You should still check ABI compatibility issues etc. for them. Note Except to avoid broken package for a short term, installing binary packages from officially unsupported archives is generally bad idea. This is true even if you use apt-pinning (see Section 2.7.3, “Tweaking candidate version”). You should consider chroot or similar techniques (see Section 9.8, “Virtualized system”) to run programs from different archives. 2.7.3. Tweaking candidate version Warning In lenny, aptitude(8) has a bug for handling "/etc/apt/ preferences" file. (Bug#514930) Without the "/etc/apt/preferences" file, APT system choses the latest available version as the candidate version using the version string. This is the normal state and most recommended usage of APT system. All officially supported combinations of archives do not require the "/etc/apt/preferences" file since some archives which should not be used as the automatic source of upgrades are marked as NotAutomatic and dealt properly. Tip The version string comparison rule can be verified with, e.g., "dpkg --compare-versions ver1.1 gt ver1.1~1; echo $?" (see dpkg (1)). When you install packages from mixed source of archives (see Section 2.7.2, “Packages from mixed source of archives”) regularly, you can automate these complicated operations by creating the "/etc/apt/preferences" file with proper entries and tweaking the package selection rule for candidate version as described in apt_preferences(5). This is called apt-pinning. Warning Use of apt-pinning by a novice user is sure call for major troubles. You must avoid using apt-pinning except when you absolutely need it. Caution When using apt-pinning, you must ensure compatibility of packages by yourself since the Debian does not guarantee it. The apt-pinning is completely optional operation and its use is not something I encourage you to use. Caution Archive level Release files (see Section 2.5.3, “Archive level "Release" files”) are used for the rule of apt_preferences(5). Thus apt-pinning works only with "suite" name for normal Debian archives and security Debian archives. (This is different from Ubuntu archives). For example, you can do "Pin: release a= unstable" but can not do "Pin: release a=sid" in the "/etc/apt/ preferences" file. Caution When you use non-Debian archive as a part of apt-pinning, you should check what they are intended for and also check their credibility. For example, Ubuntu and Debian are not meant to be mixed. Note Even if you do not create the "/etc/apt/preferences" file, you can do fairly complex system operations (see Section 2.6.4, “Rescue with the dpkg command” and Section 2.7.2, “Packages from mixed source of archives”) without apt-pinning. Here is a simplified explanation of apt-pinning technique. APT system choses highest Pin-Priority upgrading package from available package sources defined in the "/etc/apt/sources.list" file as the candidate version package. If the Pin-Priority of the package is larger than 1000, this version restriction for upgrading is dropped to enable downgrading (see Section 2.7.7, “Emergency downgrading”). Pin-Priority value of each package is defined by "Pin-Priority" entries in the "/etc/apt/preferences" file or uses its default value. Table 2.18. List of the default Pin-Priority value for each package source type ┌────────────────────┬──────────────────────┐ │default Pin-Priority│package source type │ ├────────────────────┼──────────────────────┤ │990 │target release archive│ ├────────────────────┼──────────────────────┤ │500 │normal archive │ ├────────────────────┼──────────────────────┤ │100 │installed package │ ├────────────────────┼──────────────────────┤ │1 │NotAutomatic archive │ └────────────────────┴──────────────────────┘ The target release archive can be set by several methods. ● "/etc/apt/apt.conf" configuration file with "APT::Default-Release "stable";" line ● command line option, e.g., "apt-get install -t testing some-package" The NotAutomatic archive is set by archive server having its archive level Release file (see Section 2.5.3, “Archive level "Release" files”) containing "NotAutomatic: yes". The apt-pinning situation of from multiple archive sources is displayed by "apt-cache policy ". ● A line started with "Package pin:" lists the package version of pin if association just with is defined, e.g., "Package pin: 0.190". ● No line with "Package pin:" exists if no association just with is defined. ● The Pin-Priority value associated just with is listed right side of all version strings, e.g., "0.181 700". ● "0" is listed right side of all version strings if no association just with is defined, e.g., "0.181 0". ● The Pin-Priority values of archives (defined as "Package: *" in the "/etc/apt/preferences" file) are listed left side of all archive paths, e.g., "200 http://backports.org etch-backports/main Packages". Here is an example of apt-pinning technique to include specific newer upstream version packages found in unstable regularly upgraded while tracking testing. You list all required archives in the "/etc/apt/sources.list" file as the following. deb http://ftp.us.debian.org/debian/ testing main contrib non-free deb http://ftp.us.debian.org/debian/ unstable main contrib non-free deb http://security.debian.org/ testing/updates main contrib Set the "/etc/apt/preferences" file as as the following. Package: * Pin: release a=testing Pin-Priority: 500 Package: * Pin: release a=unstable Pin-Priority: 200 When you wish to install a package named "" with its dependencies from unstable archive under this configuration, you issue the following command which switches target release with "-t" option (Pin-Priority of unstable becomes 990.). $ sudo apt-get install -t unstable With this configuration, usual execution of "apt-get upgrade" and "apt-get dist-upgrade" (or "aptitude safe-upgrade" and "aptitude full-upgrade" for squeeze) upgrades packages which were installed from testing archive using current testing archive and packages which were installed from unstable archive using current unstable archive. Caution Be careful not to remove "testing" entry from the "/etc/apt/ sources.list" file. Without "testing" entry in it, APT system upgrades packages using newer unstable archive. Tip I usually edit the "/etc/apt/sources.list" file to comment out "unstable" archive entry right after above operation. This avoids slow update process of having too many entries in the "/ etc/apt/sources.list" file although this prevents upgrading packages which were installed from unstable archive using current unstable archive. Tip If "Pin-Priority: 20" is used instead of "Pin-Priority: 200" for the "/etc/apt/preferences" file, already installed packages having Pin-Priority value of 100 are not upgraded by unstable archive even if "testing" entry in the "/etc/apt/sources.list" file is removed. If you wish to track particular packages in unstable automatically without initial "-t unstable" installation, you must create the "/etc/apt/preferences" file and explicitly list all those packages at the top of it as the following. Package: Pin: release a=unstable Pin-Priority: 700 Package: Pin: release a=unstable Pin-Priority: 700 These set Pin-Priority value for each specific package. For example, in order to track the latest unstable version of this "Debian Reference" in English, you should have following entries in the "/etc/apt/preferences" file. Package: debian-reference-en Pin: release a=unstable Pin-Priority: 700 Package: debian-reference-common Pin: release a=unstable Pin-Priority: 700 Tip This apt-pinning technique is valid even when you are tracking stable archive. Documentation packages have been always safe to install from unstable archive in my experience, so far. Here is another example of apt-pinning technique to include specific newer upstream version packages found in experimental while tracking unstable. You list all required archives in the " /etc/apt/sources.list" file as the following. deb http://ftp.us.debian.org/debian/ unstable main contrib non-free deb http://ftp.us.debian.org/debian/ experimental main contrib non-free deb http://security.debian.org/ testing/updates main contrib The default Pin-Priority value for experimental archive is always 1 (<<100) since it is NotAutomatic archive (see Section 2.5.3, “Archive level "Release" files”). There is no need to set Pin-Priority value explicitly in the "/etc/apt/ preferences" file just to use experimental archive unless you wish to track particular packages in it automatically for next upgrading. 2.7.4. Volatile and Backports.org There are debian-volatile project and backports.org archives which provide updgrade packages for stable. Warning Do not use all packages available in the NotAutomatic archives such as lenny-backports and volatile-sloppy. Use only selected packages which fits your needs. Caution backports.org is a non-Debian archive, although its packages are signed by Debian developers. Caution Archive level Release files (see Section 2.5.3, “Archive level "Release" files”) are used for the rule of apt_preferences(5). Thus apt-pinning works only with "code" name for volatile Debian archives. This is different from other Debian archives. For example, you can do "Pin: release a=lenny" but can not do "Pin: release a=stable" in the "/etc/apt/preferences" file for volatile Debian archives. Here is an example of apt-pinning technique to include specific newer upstream version packages found in lenny-backports while tracking lenny and volatile. You list all required archives in the "/etc/apt/sources.list" file as the following. deb http://ftp.us.debian.org/debian/ lenny main contrib non-free deb http://security.debian.org/ lenny/updates main contrib deb http://volatile.debian.org/debian-volatile/ lenny/volatile main contrib non-free deb http://volatile.debian.org/debian-volatile/ lenny/volatile-sloppy main contrib non-free deb http://backports.org/debian/ lenny-backports main contrib non-free The default Pin-Priority value for backports.org and volatile-sloppy archives are always 1 (<<100) since they are NotAutomatic archive (see Section 2.5.3, “Archive level "Release" files”). There is no need to set Pin-Priority value explicitly in the "/etc/apt/preferences" file just to use for backports.org and volatile-sloppy archive unless you wish to track packages automatically for next upgrading. So whenever you wish to install a package named "" with its dependency from lenny-backports archive, you use following command while switching target release with "-t" option. $ sudo apt-get install -t lenny-backports If you wish to upgrade particular packages, you must create the "/etc/apt/preferences" file and explicitly lists all packages in it as the following. Package: Pin: release o=Backports.org archive Pin-Priority: 700 Package: Pin: release o=volatile.debian.org Pin-Priority: 700 Alternatively, with the "/etc/apt/preferences" file as the following. Package: * Pin: release a=stable , o=Debian Pin-Priority: 500 Package: * Pin: release a=lenny, o=volatile.debian.org Pin-Priority: 500 Package: * Pin: release a=lenny-backports, o=Backports.org archive Pin-Priority: 200 Package: * Pin: release a=lenny-sloppy, o=volatile.debian.org Pin-Priority: 200 Execution of "apt-get upgrade" and "apt-get dist-upgrade" (or "aptitude safe-upgrade" and "aptitude full-upgrade" for squeeze) upgrades packages which were installed from stable archive using current stable archive and packages which were installed from other archives using current corresponding archive for all archives in the "/etc/apt/sources.list" file. 2.7.5. Automatic download and upgrade of packages The apt package comes with its own cron script "/etc/cron.daily/ apt" to support the automatic download of packages. This script can be enhanced to perform the automatic upgrade of packages by installing the unattended-upgrades package. These can be customized by parameters in "/etc/apt/apt.conf.d/02backup" and " /etc/apt/apt.conf.d/50unattended-upgrades" as described in "/usr /share/doc/unattended-upgrades/README". The unattended-upgrades package is mainly intended for the security upgrade for the stable system. If the risk of breaking an existing stable system by the automatic upgrade is smaller than that of the system broken by the intruder using its security hole which has been closed by the security update, you should consider using this automatic upgrade with configuration parameters as the following. APT::Periodic::Update-Package-Lists "1"; APT::Periodic::Download-Upgradeable-Packages "1"; APT::Periodic::Unattended-Upgrade "1"; If you are running an unstable system, you do not want to use the automatic upgrade since it certainly breaks system some day. Even for such unstable case, you may still want to download packages in advance to save time for the interactive upgrade with configuration parameters as the following. APT::Periodic::Update-Package-Lists "1"; APT::Periodic::Download-Upgradeable-Packages "1"; APT::Periodic::Unattended-Upgrade "0"; 2.7.6. Limiting download bandwidth for APT If you want to limit the download bandwidth for APT to e.g. 800Kib/sec (=100kiB/sec), you should configure APT with its configuration parameter as the following. APT::Acquire::http::Dl-Limit "800"; 2.7.7. Emergency downgrading Caution Downgrading is not officially supported by the Debian by design. It should be done only as a part of emergency recovery process. Despite of this situation, it is known to work well in many incidents. For critical systems, You should backup all important data on the system after the recovery operation and re-install the new system from the scratch. You may be lucky to downgrade from newer archive to older archive to recover from broken system upgrade by manipulating candidate version (see Section 2.7.3, “Tweaking candidate version”). This is lazy alternative to tedious actions of many "dpkg -i _.deb" commands (see Section 2.6.4, “Rescue with the dpkg command”). Search lines in the "/etc/apt/sources.list" file tracking unstable as the following. deb http://ftp.us.debian.org/debian/ sid main contrib non-free Replace it with the following to track testing. deb http://ftp.us.debian.org/debian/ squeeze main contrib non-free Set the "/etc/apt/preferences" file as the following. Package: * Pin: release a=testing Pin-Priority: 1010 Run "apt-get dist-upgrade" to force downgrading of packages across the system. Remove this special "/etc/apt/preferences" file after this emergency downgrading. Tip It is good idea to remove (not purge!) as much packages to minimize dependency problems. You may need to manually remove and install some packages to get system downgraded. Linux kernel, bootloader, udev, PAM, APT, and networking related packages and their configuration files require special attention. 2.7.8. Who uploaded the package? Although the maintainer name listed in "/var/lib/dpkg/available" and "/usr/share/doc/package_name/changelog" provide some information on "who is behind the packaging activity", the actual uploader of the package is somewhat obscure. who-uploads (1) in the devscripts package identifies the actual uploader of Debian source packages. 2.7.9. The equivs package If you are to compile a program from source to replace the Debian package, it is best to make it into a real local debianized package (*.deb) and use private archive. If you chose to compile a program from source and to install them under "/usr/local" instead, you may need to use equivs as a last resort to satisfy the missing package dependency. Package: equivs Priority: extra Section: admin Description: Circumventing Debian package dependencies This is a dummy package which can be used to create Debian packages, which only contain dependency information. 2.7.10. Porting a package to the stable system For partial upgrades of the stable system, rebuilding a package within its environment using the source package is desirable. This avoids massive package upgrades due to their dependencies. Add the following entries to the "/etc/apt/sources.list" of a stable system. deb-src http://http.us.debian.org/debian unstable main contrib non-free Install required packages for the compilation and download the source package as the following. # apt-get update # apt-get dist-upgrade # apt-get install fakeroot devscripts build-essential $ apt-get build-dep foo $ apt-get source foo $ cd foo* Adjust installed packages if needed. Execute the following. $ dch -i Bump package version, e.g. one appended with "+bp1" in "debian/ changelog" Build packages and install them to the system as the following. $ debuild $ cd .. # debi foo*.changes 2.7.11. Proxy server for APT Since mirroring whole subsection of Debian archive wastes disk space and network bandwidth, deployment of a local proxy server for APT is desirable consideration when you administer many systems on LAN. APT can be configure to use generic web (http) proxy servers such as squid (see Section 6.10, “Other network application servers”) as described in apt.conf(5) and in "/usr/ share/doc/apt/examples/configure-index.gz". The "$http_proxy" environment variable can be used to override proxy server setting in the "/etc/apt/apt.conf" file. There are proxy tools specially for Debian archive. You should check BTS before using them. Table 2.19. List of the proxy tools specially for Debian archive ┌─────────────┬───────┬────┬───────────────────────────────────┐ │package │popcon │size│description │ ├─────────────┼───────┼────┼───────────────────────────────────┤ │ │V:0.2, │ │caching proxy server for Debian │ │approx * │I:0.3 │3868│archive files (compiled OCaml │ │ │ │ │program) │ ├─────────────┼───────┼────┼───────────────────────────────────┤ │apt-proxy * │V:0.4, │428 │Debian archive proxy and partial │ │ │I:0.5 │ │mirror builder (Python program) │ ├─────────────┼───────┼────┼───────────────────────────────────┤ │apt-cacher * │V:0.3, │244 │Caching proxy for Debian package │ │ │I:0.5 │ │and source files (Perl program) │ ├─────────────┼───────┼────┼───────────────────────────────────┤ │apt-cacher-ng│V:0.2, │ │Caching proxy for distribution of │ │* │I:0.2 │708 │software packages (compiled C++ │ │ │ │ │program) │ ├─────────────┼───────┼────┼───────────────────────────────────┤ │debtorrent * │V:0.15,│1173│Bittorrent proxy for downloading │ │ │I:0.2 │ │Debian packages (Python program) │ └─────────────┴───────┴────┴───────────────────────────────────┘ Caution When Debian reorganizes its archive structure, these specialized proxy tools tend to require code rewrites by the package maintainer and may not be functional for a while. On the other hand, generic web (http) proxy servers are more robust and easier to cope with such changes. 2.7.12. Small public package archive Here is an example for creating a small public package archive compatible with the modern secure APT system (see Section 2.5.2, “Top level "Release" file and authenticity”). Let's assume few things. ● Account name: "foo" ● Host name: "www.example.com" ● Required packages: apt-utils, gnupg, and other packages ● URL: "http://www.example.com/~foo/" ( → "/home/foo/ public_html/index.html") ● Architecture of packages: "amd64" Create an APT archive key of Foo on your server system as the following. $ ssh foo@www.example.com $ gpg --gen-key ... $ gpg -K ... sec 1024D/3A3CB5A6 2008-08-14 uid Foo (ARCHIVE KEY) ssb 2048g/6856F4A7 2008-08-14 $ gpg --export -a 3A3CB5A6 >foo.public.key Publish the archive key file "foo.public.key" with the key ID "3A3CB5A6" for Foo Create an archive tree called "Origin: Foo" as the following. $ umask 022 $ mkdir -p ~/public_html/debian/pool/main $ mkdir -p ~/public_html/debian/dists/unstable/main/binary-amd64 $ mkdir -p ~/public_html/debian/dists/unstable/main/source $ cd ~/public_html/debian $ cat > dists/unstable/main/binary-amd64/Release << EOF Archive: unstable Version: 4.0 Component: main Origin: Foo Label: Foo Architecture: amd64 EOF $ cat > dists/unstable/main/source/Release << EOF Archive: unstable Version: 4.0 Component: main Origin: Foo Label: Foo Architecture: source EOF $ cat >aptftp.conf <aptgenerate.conf < "www.example.com", method => "scpb", incoming => "/home/foo/public_html/debian/pool/main", # The dinstall on ftp-master sends emails itself dinstall_runs => 1, }; $cfg{'foo'}{postupload}{'changes'} = " echo 'cd public_html/debian ; apt-ftparchive generate -c=aptftp.conf aptgenerate.conf; apt-ftparchive release -c=aptftp.conf dists/unstable >dists/unstable/Release ; rm -f dists/unstable/Release.gpg ; gpg -u 3A3CB5A6 -bao dists/unstable/Release.gpg dists/unstable/Release'| ssh foo@www.example.com 2>/dev/null ; echo 'Package archive created!'"; The postupload hook script initiated by dupload(1) creates updated archive files for each upload. You can add this small public archive to the apt-line of your client system by the following. $ sudo bash # echo "deb http://www.example.com/~foo/debian/ unstable main" \ >> /etc/apt/sources.list # apt-key add foo.public.key Tip If the archive is located on the local filesystem, you can use "deb file:///home/foo/debian/ …" instead. 2.7.13. Recording and copying system configuration You can make a local copy of the package and debconf selection states by the following. # dpkg --get-selections '*' > selection.dpkg # debconf-get-selections > selection.debconf Here, "*" makes "selection.dpkg" to include package entries for "purge" too. You can transfer these 2 files to another computer, and install there with the following. # dselect update # debconf-set-selections < myselection.debconf # dpkg --set-selections < myselection.dpkg # apt-get -u dselect-upgrade # or dselect install If you are thinking about managing many servers in a cluster with practically the same configuration, you should consider to use specialized package such as fai to manage the whole system. 2.7.14. Converting or installing an alien binary package alien(1) enables the conversion of binary packages provided in Red Hat rpm, Stampede slp, Slackware tgz, and Solaris pkg file formats into a Debian deb package. If you want to use a package from another Linux distribution than the one you have installed on your system, you can use alien to convert it from your preferred package format and install it. alien also supports LSB packages. Warning alien(1) should not be used to replace essential system packages, such as sysvinit, libc6, libpam-modules, etc. Practically, alien(1) should only used for non-free binary-only packages which are LSB compliant or statically linked. For free softwares, you should use their source packages to make real Debian packages. 2.7.15. Extracting package without dpkg The current "*.deb" package contents can be extracted without using dpkg(1) on any Unix-like environment using standard ar(1) and tar(1). # ar x /path/to/dpkg__.deb # ls total 24 -rw-r--r-- 1 bozo bozo 1320 2007-05-07 00:11 control.tar.gz -rw-r--r-- 1 bozo bozo 12837 2007-05-07 00:11 data.tar.gz -rw-r--r-- 1 bozo bozo 4 2007-05-07 00:11 debian-binary # mkdir control # mkdir data # tar xvzf control.tar.gz -C control # tar xvzf data.tar.gz -C data You can also browse package content using the mc command. 2.7.16. More readings for the package management You can learn more on the package management from following documentations. ● Primary documentations on the package management: ○ aptitude(8), dpkg(1), tasksel(8), apt-get(8), apt-config (8), apt-key(8), sources.list(5), apt.conf(5), and apt_preferences(5); ○ "/usr/share/doc/apt-doc/guide.html/index.html" and "/usr /share/doc/apt-doc/offline.html/index.html" from the apt-doc package; and ○ "/usr/share/doc/aptitude/html/en/index.html" from the aptitude-doc-en package. ● Official and detailed documentations on the Debian archive: ○ "Debian Policy Manual Chapter 2 - The Debian Archive", ○ "Debian Developer's Reference, Chapter 4 - Resources for Debian Developers 4.6 The Debian archive", and ○ "The Debian GNU/Linux FAQ, Chapter 5 - The Debian FTP archives". ● Tutorial for building of a Debian package for Debian users: ○ "Debian New Maintainers' Guide". Chapter 3. The system initialization It is wise for you as the system administrator to know roughly how the Debian system is started and configured. Although the exact details are in the source files of the packages installed and their documentations, it is a bit overwhelming for most of us. I did my best to provide a quick overview of the key points of the Debian system and their configuration for your reference, based on the current and previous knowledge of mine and others. Since the Debian system is a moving target, the situation over the system may have been changed. Before making any changes to the system, you should refer to the latest documentation for each package. 3.1. An overview of the boot strap process The computer system undergoes several phases of boot strap processes from the power-on event until it offers the fully functional operating system (OS) to the user. For simplicity, I limit discussion to the typical PC platform with the default installation. The typical boot strap process is like a four-stage rocket. Each stage rocket hands over the system control to the next stage one. ● Section 3.2, “Stage 1: the BIOS” ● Section 3.3, “Stage 2: the boot loader” ● Section 3.4, “Stage 3: the mini-Debian system” ● Section 3.5, “Stage 4: the normal Debian system” Of course, these can be configured differently. For example, if you compiled your own kernel, you may be skipping the step with the mini-Debian system. So please do not assume this is the case for your system until you check it yourself. Note For non-legacy PC platform such as the SUN or the Macintosh system, the BIOS on ROM and the partition on the disk may be quite different (Section 9.3.1, “Disk partition configuration”). Please seek the platform specific documentations elsewhere for such a case. 3.2. Stage 1: the BIOS The BIOS is the 1st stage of the boot process which is started by the power-on event. The BIOS residing on the read only memory (ROM) is executed from the particular memory address to which the program counter of CPU is initialized by the power-on event. This BIOS performs the basic initialization of the hardware ( POST: power on self test) and hands the system control to the next step which you provide. The BIOS is usually provided with the hardware. The BIOS startup screen usually indicates what key(s) to press to enter the BIOS setup screen to configure the BIOS behavior. Popular keys used are F1, F2, F10, Esc, Ins, and Del. If your BIOS startup screen is hidden by a nice graphics screen, you may press some keys such as Esc to disable this. These keys are highly dependent on the hardware. The hardware location and the priority of the code started by the BIOS can be selected from the BIOS setup screen. Typically, the first few sectors of the first found selected device (hard disk, floppy disk, CD-ROM, …) are loaded to the memory and this initial code is executed. This initial code can be any one of the following. ● The boot loader code ● The kernel code of the stepping stone OS such as FreeDOS ● The kernel code of the target OS if it fits in this small space Typically, the system is booted from the specified partition of the primary hard disk partition. First 2 sectors of the hard disk on legacy PC contain the master boot record (MBR). The disk partition information including the boot selection is recorded at the end of this MBR. The first boot loader code executed from the BIOS occupies the rest of this MBR. 3.3. Stage 2: the boot loader The boot loader is the 2nd stage of the boot process which is started by the BIOS. It loads the system kernel image and the initrd image to the memory and hands control over to them. This initrd image is the root filesystem image and its support depends on the bootloader used. The Debian system normally uses the Linux kernel as the default system kernel. The initrd image for the current 2.6 Linux kernel is technically the initramfs (initial RAM filesystem) image. The initramfs image is a gzipped cpio archive of files in the root filesystem. The default install of the Debian system places first-stage GRUB boot loader code into the MBR for the PC platform. There are many boot loaders and configuration options available. Table 3.1. List of boot loaders ┌──────────┬──────────────┬───────┬────┬─────────┬─────────────┐ │bootloader│package │popcon │size│initrd │description │ ├──────────┼──────────────┼───────┼────┼─────────┼─────────────┤ │ │ │ │ │ │This is smart│ │ │ │ │ │ │enough to │ │ │ │ │ │ │understand │ │ │ │ │ │ │disk │ │GRUB │ │V:21, │ │ │partitions │ │Legacy │grub * │I:88 │168 │Supported│and │ │ │ │ │ │ │filesystems │ │ │ │ │ │ │such as vfat,│ │ │ │ │ │ │ext3, …. │ │ │ │ │ │ │(lenny │ │ │ │ │ │ │default) │ ├──────────┼──────────────┼───────┼────┼─────────┼─────────────┤ │ │ │ │ │ │This is smart│ │ │ │ │ │ │enough to │ │ │ │ │ │ │understand │ │ │ │V:2, │ │ │disk │ │GRUB 2 │grub-pc * │I:8 │1804│Supported│partitions │ │ │ │ │ │ │and │ │ │ │ │ │ │filesystems │ │ │ │ │ │ │such as vfat,│ │ │ │ │ │ │ext3, …. │ ├──────────┼──────────────┼───────┼────┼─────────┼─────────────┤ │ │ │ │ │ │This is GRUB │ │ │ │ │ │ │2 bootable │ │GRUB 2 │grub-rescue-pc│V:0.05,│2992│Supported│rescue images│ │ │* │I:0.5 │ │ │(CD and │ │ │ │ │ │ │floppy) (PC/ │ │ │ │ │ │ │BIOS version)│ ├──────────┼──────────────┼───────┼────┼─────────┼─────────────┤ │ │ │ │ │ │This relies │ │ │ │ │ │ │on the sector│ │Lilo │lilo * │V:0.7, │1124│Supported│locations of │ │ │ │I:3 │ │ │data on the │ │ │ │ │ │ │hard disk. │ │ │ │ │ │ │(Old) │ ├──────────┼──────────────┼───────┼────┼─────────┼─────────────┤ │ │ │ │ │ │This │ │ │ │ │ │ │understands │ │ │ │V:1.2, │ │ │the ISO9660 │ │Isolinux │syslinux * │I:7 │120 │Supported│filesystem. │ │ │ │ │ │ │This is used │ │ │ │ │ │ │by the boot │ │ │ │ │ │ │CD. │ ├──────────┼──────────────┼───────┼────┼─────────┼─────────────┤ │ │ │ │ │ │This │ │ │ │ │ │ │understands │ │ │ │ │ │ │the MSDOS │ │Syslinux │syslinux * │V:1.2, │120 │Supported│filesystem │ │ │ │I:7 │ │ │(FAT). This │ │ │ │ │ │ │is used by │ │ │ │ │ │ │the boot │ │ │ │ │ │ │floppy. │ ├──────────┼──────────────┼───────┼────┼─────────┼─────────────┤ │ │ │ │ │ │New system is│ │Loadlin │loadlin * │V:0.03,│144 │Supported│started from │ │ │ │I:0.2 │ │ │the FreeDOS/ │ │ │ │ │ │ │MSDOS system.│ ├──────────┼──────────────┼───────┼────┼─────────┼─────────────┤ │ │ │ │ │ │This is free │ │ │ │ │ │ │software │ │ │ │ │ │ │which │ │MBR by │ │V:1.0, │ │Not │substitutes │ │Neil │mbr * │I:6 │96 │supported│MSDOS MBR. │ │Turton │ │ │ │ │This only │ │ │ │ │ │ │understands │ │ │ │ │ │ │disk │ │ │ │ │ │ │partitions. │ └──────────┴──────────────┴───────┴────┴─────────┴─────────────┘ Warning Do not play with boot loaders without having bootable rescue media (CD or floppy) created from images in the grub-rescue-pc package. It makes you boot your system even without functioning bootloader on the hard disk. For GRUB Legacy, the menu configuration file is located at "/ boot/grub/menu.lst". For example, it has entries as the following. title Debian GNU/Linux root (hd0,2) kernel /vmlinuz root=/dev/hda3 ro initrd /initrd.img For GRUB 2, the menu configuration file is located at "/boot/ grub/grub.cfg". It is automatically generated by "/usr/sbin/ update-grub" using templates from "/etc/grub.d/*" and settings from "/etc/default/grub". For example, it has entries as the following. menuentry "Debian GNU/Linux" { set root=(hd0,3) linux /vmlinuz root=/dev/hda3 initrd /initrd.img } For these examples, these GRUB parameters mean the following. Table 3.2. The meaning of GRUB parameters ┌─────────┬────────────────────────────────────────────────────┐ │GRUB │meaning │ │parameter│ │ ├─────────┼────────────────────────────────────────────────────┤ │ │use 3rd partition on the primary disk by setting it │ │root │as "(hd0,2)" in GRUB legacy or as "(hd0,3)" in GRUB │ │ │2 │ ├─────────┼────────────────────────────────────────────────────┤ │kernel │use kernel located at "/vmlinuz" with kernel │ │ │parameter: "root=/dev/hda3 ro" │ ├─────────┼────────────────────────────────────────────────────┤ │initrd │use initrd/initramfs image located at "/initrd.img" │ └─────────┴────────────────────────────────────────────────────┘ Note The value of the partition number used by GRUB legacy program is one less than normal one used by Linux kernel and utility tools. GRUB 2 program fixes this problem. Tip UUID (see Section 9.3.2, “Accessing partition using UUID”) may be used to identify a block special device instead of its file name such as "/dev/hda3", e.g."root=UUID= 81b289d5-4341-4003-9602-e254a17ac232 ro". Tip You can start a boot loader from another boot loader using techniques called chain loading. See "info grub" and grub-install(8). 3.4. Stage 3: the mini-Debian system The mini-Debian system is the 3rd stage of the boot process which is started by the boot loader. It runs the system kernel with its root filesystem on the memory. This is an optional preparatory stage of the boot process. Note The term "the mini-Debian system" is coined by the author to describe this 3rd stage boot process for this document. This system is commonly referred as the initrd or initramfs system. Similar system on the memory is used by the Debian Installer. The "/init" script is executed as the first program in this root filesystem on the memory. It is a shell script program which initializes the kernel in user space and hands control over to the next stage. This mini-Debian system offers flexibility to the boot process such as adding kernel modules before the main boot process or mounting the root filesystem as an encrypted one. You can interrupt this part of the boot process to gain root shell by providing "break=init" etc. to the kernel boot parameter. See the "/init" script for more break conditions. This shell environment is sophisticated enough to make a good inspection of your machine's hardware. Commands available in this mini-Debian system are stripped down ones and mainly provided by a GNU tool called busybox(1). Caution You need to use "-n" option for mount command when you are on the readonly root filesystem. 3.5. Stage 4: the normal Debian system Note This section describes classical System V style boot system on lenny. Debian is moving to the event driven boot system. See The future of the boot system in Debian and Dependency based boot sequence. The normal Debian system is the 4th stage of the boot process which is started by the mini-Debian system. The system kernel for the mini-Debian system continues to run in this environment. The root filesystem is switched from the one on the memory to the one on the real hard disk filesystem. The "/sbin/init" program is executed as the first program and performs the main boot process. The Debian normally uses the traditional sysvinit scheme with the sysv-rc package. See init (8), inittab(5), and "/usr/share/doc/sysv-rc/ README.runlevels.gz" for the exact explanation. This main boot process essentially goes through the following. 1. The Debian system goes into runlevel N (none) to initialize the system by following the "/etc/inittab" description. 2. The Debian system goes into runlevel S to initialize the system under the single-user mode to complete hardware initialization etc. 3. The Debian system goes into one of the specified multi-user runlevels (2 to 5) to start the system services. The initial runlevel used for multi-user mode is specified with the "init=" kernel boot parameter or in the "initdefault" line of the "/etc/inittab". The Debian system as installed starts at the runlevel 2. All actual script files executed by the init system are located in the directory "/etc/init.d/". Tip For alternative boot mechanism to the sysv-rc package using a single configuration file "/etc/runlevel.conf", see the file-rc package. Both mechanisms are compatible through "/etc/init.d/ rc", "/etc/init.d/rcS", "/usr/sbin/update-rc.d", and "/usr/sbin/ invoke-rc.d" scripts. 3.5.1. The meaning of the runlevel Each runlevel uses a directory for its configuration and has specific meaning as the following. Table 3.3. List of runlevels and description of their usage ┌────────┬─────────┬───────────────────────────────────────────┐ │runlevel│directory│description of runlevel usage │ ├────────┼─────────┼───────────────────────────────────────────┤ │N │none │system bootup (NONE) level (no "/etc/rcN.d/│ │ │ │" directory) │ ├────────┼─────────┼───────────────────────────────────────────┤ │0 │/etc/ │halt the system │ │ │rc0.d/ │ │ ├────────┼─────────┼───────────────────────────────────────────┤ │S │/etc/ │single-user mode on boot (alias: "s") │ │ │rcS.d/ │ │ ├────────┼─────────┼───────────────────────────────────────────┤ │1 │/etc/ │single-user mode switched from multi-user │ │ │rc1.d/ │mode │ ├────────┼─────────┼───────────────────────────────────────────┤ │2 │/etc/ │multi-user mode │ │ │rc2.d/ │ │ ├────────┼─────────┼───────────────────────────────────────────┤ │3 │/etc/ │,, │ │ │rc3.d/ │ │ ├────────┼─────────┼───────────────────────────────────────────┤ │4 │/etc/ │,, │ │ │rc4.d/ │ │ ├────────┼─────────┼───────────────────────────────────────────┤ │5 │/etc/ │,, │ │ │rc5.d/ │ │ ├────────┼─────────┼───────────────────────────────────────────┤ │6 │/etc/ │reboot the system │ │ │rc6.d/ │ │ ├────────┼─────────┼───────────────────────────────────────────┤ │7 │/etc/ │valid multi-user mode but not normally used│ │ │rc7.d/ │ │ ├────────┼─────────┼───────────────────────────────────────────┤ │8 │/etc/ │,, │ │ │rc8.d/ │ │ ├────────┼─────────┼───────────────────────────────────────────┤ │9 │/etc/ │,, │ │ │rc9.d/ │ │ └────────┴─────────┴───────────────────────────────────────────┘ You can change the runlevel from the console to, e.g., 4 by the following. $ sudo telinit 4 Caution The Debian system does not pre-assign any special meaning differences among the runlevels between 2 and 5. The system administrator on the Debian system may change this. (I.e., Debian is not Red Hat Linux nor Solaris by Sun Microsystems nor HP-UX by Hewlett Packard nor AIX by IBM nor …) Caution The Debian system does not populate directories for the runlevels between 7 and 9 when the package is installed. Traditional Unix variants don’t use these runlevels. 3.5.2. The configuration of the runlevel The name of the symlink in each runlevel directory has the form "S<2-digit-number>" or "K<2-digit-number> ". The 2-digit-number is used to determine the order in which to run the scripts. "S" is for "Start" and "K" is for "Kill". When init(8) or telinit(8) commands goes into the runlevel to " ", it execute following scripts. 1. The script names starting with a "K" in "/etc/rc.d/" are executed in alphabetical order with the single argument "stop". (killing services) 2. The script names starting with an "S" in "/etc/rc.d/" are executed in alphabetical order with the single argument "start". (starting services) For example, if you had the links "S10sysklogd" and "S20exim4" in a runlevel directory, "S10sysklogd" which is symlinked to ".. /init.d/sysklogd" would run before "S20exim4" which is symlinked to "../init.d/exim4". Warning It is not advisable to make any changes to symlinks in "/etc/ rcS.d/" unless you know better than the maintainer. 3.5.3. The runlevel management example For example, let's set up runlevel system somewhat like Red Hat Linux as the following. ● init starts the system in runlevel=3 as the default. ● init does not start gdm(1) in runlevel=(0,1,2,6). ● init starts gdm(1) in runlevel=(3,4,5). This can be done by using editor on the "/etc/inittab" file to change starting runlevel and using user friendly runlevel management tools such as sysv-rc-conf or bum to edit the runlevel. If you are to use command line only instead, here is how you do it (after the default installation of the gdm package and selecting it to be the choice of display manager). # cd /etc/rc2.d ; mv S21gdm K21gdm # cd /etc ; perl -i -p -e 's/^id:.:/id:3:/' inittab Please note the "/etc/X11/default-display-manager" file is checked when starting the display manager daemons: xdm, gdm, kdm, and wdm. Note You can still start X from any console shell with the startx(1) command. 3.5.4. The default parameter for each init script The default parameter for each init script in "/etc/init.d/" is given by the corresponding file in "/etc/default/" which contains environment variable assignments only. This choice of directory name is specific to the Debian system. It is roughly the equivalent of the "/etc/sysconfig" directory found in Red Hat Linux and other distributions. For example, "/etc/default/ cron" can be used to control how "/etc/init.d/cron" works. The "/etc/default/rcS" file can be used to customize boot-time defaults for motd(5), sulogin(8), etc. If you cannot get the behavior you want by changing such variables then you may modify the init scripts themselves. These are configuration files editable by system administrators. 3.5.5. The hostname The kernel maintains the system hostname. The init script in runlevel S which is symlinked to "/etc/init.d/hostname.sh" sets the system hostname at boot time (using the hostname command) to the name stored in "/etc/hostname". This file should contain only the system hostname, not a fully qualified domain name. To print out the current hostname run hostname(1) without an argument. 3.5.6. The filesystem Although the root filesystem is mounted by the kernel when it is started, other filesystems are mounted in the runlevel S by the following init scripts. ● "`/etc/init.d/mountkernfs.sh" for kernel filesystems in "/ proc", "/sys", etc. ● "`/etc/init.d/mountdevsubfs.sh" for virtual filesystems in " /dev" ● "`/etc/init.d/mountall.sh" for normal filesystems using "/ etc/fstab" ● "`/etc/init.d/mountnfs.sh" for network filesystems using"/ etc/fstab" The mount options of the filesystem are set in "/etc/fstab". See Section 9.3.5, “Optimization of filesystem by mount options”. Note The actual mounting of network filesystems waits for the start of the network interface. Warning After mounting all the filesystems, temporary files in "/tmp", " /var/lock", and "/var/run" are cleaned for each boot up. 3.5.7. Network interface initialization Network interfaces are initialized in runlevel S by the init script symlinked to "/etc/init.d/ifupdown-clean" and "/etc/ init.d/ifupdown". See Chapter 5, Network setup for how to configure them. 3.5.8. Network service initialization Many network services (see Chapter 6, Network applications) are started under multi-user mode directly as daemon processes at boot time by the init script, e.g., "/etc/rc2.d/S20exim4" (for RUNLEVEL=2) which is a symlink to "/etc/init.d/exim4". Some network services can be started on demand using the super-server inetd (or its equivalents). The inetd is started at boot time by "/etc/rc2.d/S20inetd" (for RUNLEVEL=2) which is a symlink to "/etc/init.d/inetd". Essentially, inetd allows one running daemon to invoke several others, reducing load on the system. Whenever a request for service arrives at super-server inetd , its protocol and service are identified by looking them up in the databases in "/etc/protocols" and "/etc/services". inetd then looks up a normal Internet service in the "/etc/inetd.conf" database, or a Open Network Computing Remote Procedure Call (ONC RPC)/Sun RPC based service in "/etc/rpc.conf". Sometimes, inetd does not start the intended server directly but starts the TCP wrapper program, tcpd(8), with the intended server name as its argument in "/etc/inetd.conf". In this case, tcpd runs the appropriate server program after logging the request and doing some additional checks using "/etc/hosts.deny" and "/etc/hosts.allow". For system security, disable as much network service programs as possible. See Section 4.6.3, “Restricting access to some server services”. See inetd(8), inetd.conf(5), protocols(5), services(5), tcpd(8), hosts_access(5), hosts_options(5), rpcinfo(8), portmap(8), and " /usr/share/doc/portmap/portmapper.txt.gz". 3.5.9. The system message The system message can be customized by "/etc/default/syslogd" and "/etc/syslog.conf" for both the log file and on-screen display. See syslogd(8) and syslog.conf(5). See also Section 9.2.2, “Log analyzer”. 3.5.10. The kernel message The kernel message can be customized by "/etc/default/klogd" for both the log file and on-screen display. Set "KLOGD='-c 3'" in this file and run "/etc/init.d/klogd restart". See klogd(8). You may directly change the error message level by the following. # dmesg -n3 Table 3.4. List of kernel error levels ┌────────────────┬───────────────┬─────────────────────────────┐ │error level │error level │meaning │ │value │name │ │ ├────────────────┼───────────────┼─────────────────────────────┤ │0 │KERN_EMERG │system is unusable │ ├────────────────┼───────────────┼─────────────────────────────┤ │1 │KERN_ALERT │action must be taken │ │ │ │immediately │ ├────────────────┼───────────────┼─────────────────────────────┤ │2 │KERN_CRIT │critical conditions │ ├────────────────┼───────────────┼─────────────────────────────┤ │3 │KERN_ERR │error conditions │ ├────────────────┼───────────────┼─────────────────────────────┤ │4 │KERN_WARNING │warning conditions │ ├────────────────┼───────────────┼─────────────────────────────┤ │5 │KERN_NOTICE │normal but significant │ │ │ │condition │ ├────────────────┼───────────────┼─────────────────────────────┤ │6 │KERN_INFO │informational │ ├────────────────┼───────────────┼─────────────────────────────┤ │7 │KERN_DEBUG │debug-level messages │ └────────────────┴───────────────┴─────────────────────────────┘ 3.5.11. The udev system For Linux kernel 2.6, the udev system provides mechanism for the automatic hardware discovery and initialization (see udev(7)). Upon discovery of each device by the kernel, the udev system starts a user process which uses information from the sysfs filesystem (see Section 1.2.12, “procfs and sysfs”), loads required kernel modules supporting it using the modprobe(8) program (see Section 3.5.12, “The kernel module initialization” ), and creates corresponding device nodes. Tip If "/lib/modules//modules.dep" was not generated properly by depmod(8) for some reason, modules may not be loaded as expected by the udev system. Execute "depmod -a" to fix it. The name of device nodes can be configured by udev rule files in "/etc/udev/rules.d/". Current default rules tend to create dynamically generated names resulting non-static device names except for cd and network devices. By adding your custom rules similar to what cd and network devices do, you can generate static device names for other devices such as USB memory sticks, too. See "Writing udev rules" or "/usr/share/doc/udev/ writing_udev_rules/index.html". Since the udev system is somewhat a moving target, I leave details to other documentations and describe the minimum information here. Tip For mounting rules in "/etc/fstab", device nodes do not need to be static ones. You can use UUID to mount devices instead of device names such as "/dev/sda". See Section 9.3.2, “Accessing partition using UUID”. 3.5.12. The kernel module initialization The modprobe(8) program enables us to configure running Linux kernel from user process by adding and removing kernel modules. The udev system (see Section 3.5.11, “The udev system”) automates its invocation to help the kernel module initialization. There are non-hardware modules and special hardware driver modules as the following which need to be pre-loaded by listing them in the "/etc/modules" file (see modules(5)). ● TUN/TAP modules providing virtual Point-to-Point network device (TUN) and virtual Ethernet network device (TAP), ● netfilter modules providing netfilter firewall capabilities (iptables(8), Section 5.8, “Netfilter infrastructure”), and ● watchdog timer driver modules. The configuration files for the modprobe(8) program are located under the "/etc/modprobes.d/" directory as explained in modprobe.conf(5). (If you want to avoid some kernel modules to be auto-loaded, consider to blacklist them in the "/etc/ modprobes.d/blacklist" file.) The "/lib/modules//modules.dep" file generated by the depmod(8) program describes module dependencies used by the modprobe(8) program. Note If you experience module loading issues with boot time module loading or with modprobe(8), "depmod -a" may resolve these issues by reconstructing "modules.dep". The modinfo(8) program shows information about a Linux kernel module. The lsmod(8) program nicely formats the contents of the "/proc/ modules", showing what kernel modules are currently loaded. Tip You can identify exact hardware on your system. See Section 9.6.3, “Hardware identification”. Tip You may configure hardware at boot time to activate expected hardware features. See Section 9.6.4, “Hardware configuration”. Tip You can add support for your device by recompiling kernel. See Section 9.7, “The kernel”. Chapter 4. Authentication When a person (or a program) requests access to the system, authentication confirms the identity to be a trusted one. Warning Configuration errors of PAM may lock you out of your own system. You must have a rescue CD handy or setup an alternative boot partition. To recover, boot the system with them and correct things from there. 4.1. Normal Unix authentication Normal Unix authentication is provided by the pam_unix(8) module under the PAM (Pluggable Authentication Modules). Its 3 important configuration files, with ":" separated entries, are the following. Table 4.1. 3 important configuration files for pam_unix(8) ┌─────────┬──────────┬────┬──────┬─────────────────────────────┐ │file │permission│user│group │description │ ├─────────┼──────────┼────┼──────┼─────────────────────────────┤ │/etc/ │-rw-r--r--│root│root │(sanitized) user account │ │passwd │ │ │ │information │ ├─────────┼──────────┼────┼──────┼─────────────────────────────┤ │/etc/ │-rw-r-----│root│shadow│secure user account │ │shadow │ │ │ │information │ ├─────────┼──────────┼────┼──────┼─────────────────────────────┤ │/etc/ │-rw-r--r--│root│root │group information │ │group │ │ │ │ │ └─────────┴──────────┴────┴──────┴─────────────────────────────┘ "/etc/passwd" contains the following. ... user1:x:1000:1000:User1 Name,,,:/home/user1:/bin/bash user2:x:1001:1001:User2 Name,,,:/home/user2:/bin/bash ... As explained in passwd(5), each ":" separated entry of this file means the following. ● Login name ● Password specification entry ● Numerical user ID ● Numerical group ID ● User name or comment field ● User home directory ● Optional user command interpreter The second entry of "/etc/passwd" was used for the encrypted password entry. After the introduction of "/etc/shadow", this entry is used for the password specification entry. Table 4.2. The second entry content of "/etc/passwd" ┌───────┬──────────────────────────────────────────┐ │content│meaning │ ├───────┼──────────────────────────────────────────┤ │(empty)│passwordless account │ ├───────┼──────────────────────────────────────────┤ │x │the encrypted password is in "/etc/shadow"│ ├───────┼──────────────────────────────────────────┤ │* │no login for this account │ ├───────┼──────────────────────────────────────────┤ │! │no login for this account │ └───────┴──────────────────────────────────────────┘ "/etc/shadow" contains the following. ... user1:$1$Xop0FYH9$IfxyQwBe9b8tiyIkt2P4F/:13262:0:99999:7::: user2:$1$vXGZLVbS$ElyErNf/agUDsm1DehJMS/:13261:0:99999:7::: ... As explained in shadow(5), each ":" separated entry of this file means the following. ● Login name ● Encrypted password (The initial "$1$" indicates use of the MD5 encryption. The "*" indicates no login.) ● Days since Jan 1, 1970 that password was last changed ● Days before password may be changed ● Days after which password must be changed ● Days before password is to expire that user is warned ● … "/etc/group" contains the following. group1:x:20:user1,user2 As explained in shadow(5), each ":" separated entry of this file means the following. ● Group name ● Encrypted password (not really used) ● Numerical group ID ● "," separated list of user names Note "/etc/gshadow" provides the similar function as "/etc/shadow" for "/etc/group" but is not really used. Note The actual group membership of a user may be dynamically added if "auth optional pam_group.so" line is added to "/etc/pam.d/ common-auth" and set it in "/etc/security/group.conf". See pam_group(8). Note The base-passwd package contains an authoritative list of the user and the group: "/usr/share/doc/base-passwd/ users-and-groups.html". 4.2. Managing account and password information Here are few notable commands to manage account information. Table 4.3. List of commands to manage account information ┌────────────────────┬─────────────────────────────────────────┐ │command │function │ ├────────────────────┼─────────────────────────────────────────┤ │getent passwd │browse account information of " │ │" │ ├────────────────────┼─────────────────────────────────────────┤ │getent shadow │browse shadowed account information of " │ │" │ ├────────────────────┼─────────────────────────────────────────┤ │getent group │browse group information of "│ │ │" │ ├────────────────────┼─────────────────────────────────────────┤ │passwd │manage password for the account │ ├────────────────────┼─────────────────────────────────────────┤ │passwd -e │set one-time password for the account │ │ │activation │ ├────────────────────┼─────────────────────────────────────────┤ │chage │manage password aging information │ └────────────────────┴─────────────────────────────────────────┘ You may need to have the root privilege for some functions to work. See crypt(3) for the password and data encryption. Note On the system set up with PAM and NSS as the Debian alioth machine, the content of local "/etc/passwd", "/etc/group" and "/ etc/shadow" may not be actively used by the system. Above commands are valid even under such environment. 4.3. Good password When creating an account during your system installation or with the passwd(1) command, you should choose a good password which consists of 6 to 8 characters including one or more characters from each of the following sets according to passwd(1). ● Lower case alphabetics ● Digits 0 through 9 ● Punctuation marks Warning Do not chose guessable words for the password. 4.4. Creating encrypted password There are independent tools to generate encrypted password with salt. Table 4.4. List of tools to generate password ┌───────┬───────┬────┬─────────┬───────────────────────────────┐ │package│popcon │size│command │function │ ├───────┼───────┼────┼─────────┼───────────────────────────────┤ │whois *│V:12, │204 │mkpasswd │over-featured front end to the │ │ │I:88 │ │ │crypt(3) library │ ├───────┼───────┼────┼─────────┼───────────────────────────────┤ │openssl│V:43, │2352│openssl │compute password hashes │ │* │I:90 │ │passwd │(OpenSSL). passwd(1ssl) │ └───────┴───────┴────┴─────────┴───────────────────────────────┘ 4.5. PAM and NSS Modern Unix-like systems such as the Debian system provide PAM (Pluggable Authentication Modules) and NSS (Name Service Switch) mechanism to the local system administrator to configure his system. The role of these can be summarizes as the following. ● PAM offers a flexible authentication mechanism used by the application software thus involves password data exchange. ● NSS offers a flexible name service mechanism which is frequently used by the C standard library to obtain the user and group name for programs such as ls(1) and id(1). These PAM and NSS systems need to be configured consistently. The notable packages of PAM and NSS systems are the following. Table 4.5. List of notable PAM and NSS systems ┌───────────────────┬───────┬─────┬────────────────────────────┐ │package │popcon │size │description │ ├───────────────────┼───────┼─────┼────────────────────────────┤ │libpam-modules * │V:82, │1080 │Pluggable Authentication │ │ │I:99 │ │Modules (basic service) │ ├───────────────────┼───────┼─────┼────────────────────────────┤ │ │V:1.9, │ │Pluggable Authentication │ │libpam-ldap * │I:4 │384 │Module allowing LDAP │ │ │ │ │interfaces │ ├───────────────────┼───────┼─────┼────────────────────────────┤ │ │V:0.8, │ │Pluggable Authentication │ │libpam-cracklib * │I:1.6 │148 │Module to enable cracklib │ │ │ │ │support │ ├───────────────────┼───────┼─────┼────────────────────────────┤ │ │ │ │Pluggable Authentication │ │libpam-doc * │I:0.7 │1208 │Modules (documentation in │ │ │ │ │html and text) │ ├───────────────────┼───────┼─────┼────────────────────────────┤ │ │ │ │GNU C Library: Shared │ │libc6 * │V:96, │9908 │libraries which also │ │ │I:99 │ │provides "Name Service │ │ │ │ │Switch" service │ ├───────────────────┼───────┼─────┼────────────────────────────┤ │glibc-doc * │I:4 │1948 │GNU C Library: Manpages │ ├───────────────────┼───────┼─────┼────────────────────────────┤ │glibc-doc-reference│ │ │GNU C Library: Reference │ │* │I:1.5 │12020│manual in info, pdf and html│ │ │ │ │format (non-free) │ ├───────────────────┼───────┼─────┼────────────────────────────┤ │libnss-mdns * │I:53 │116 │NSS module for Multicast DNS│ │ │ │ │name resolution │ ├───────────────────┼───────┼─────┼────────────────────────────┤ │libnss-ldap * │I:4 │312 │NSS module for using LDAP as│ │ │ │ │a naming service │ ├───────────────────┼───────┼─────┼────────────────────────────┤ │ │V:0.13,│ │NSS module for using LDAP as│ │libnss-ldapd * │I:0.3 │136 │a naming service (new folk │ │ │ │ │of libnss-ldap) │ └───────────────────┴───────┴─────┴────────────────────────────┘ ● "The Linux-PAM System Administrators' Guide" in libpam-doc is essential for learning PAM configuration. ● "System Databases and Name Service Switch" section in glibc-doc-reference is essential for learning NSS configuration. Note You can see more extensive and current list by "aptitude search 'libpam-|libnss-'" command. The acronym NSS may also mean "Network Security Service" which is different from "Name Service Switch". Note PAM is the most basic way to initialize environment variables for each program with the system wide default value. 4.5.1. Configuration files accessed by the PAM and NSS Here are few notable configuration files accessed by the PAM. Table 4.6. List of configuration files accessed by the PAM ┌──────────────┬───────────────────────────────────────────────┐ │configuration │function │ │file │ │ ├──────────────┼───────────────────────────────────────────────┤ │/etc/pam.d/ │set up PAM configuration for the " │ │" program; see pam(7) and pam.d │ │ │(5) │ ├──────────────┼───────────────────────────────────────────────┤ │/etc/ │set up NSS configuration with the entry for │ │nsswitch.conf │each service. See nsswitch.conf(5) │ ├──────────────┼───────────────────────────────────────────────┤ │/etc/nologin │limit the user login by the pam_nologin(8) │ │ │module │ ├──────────────┼───────────────────────────────────────────────┤ │/etc/securetty│limit the tty for the root access by the │ │ │pam_securetty(8) module │ ├──────────────┼───────────────────────────────────────────────┤ │/etc/security/│set access limit by the pam_access(8) module │ │access.conf │ │ ├──────────────┼───────────────────────────────────────────────┤ │/etc/security/│set group based restraint by the pam_group(8) │ │group.conf │module │ ├──────────────┼───────────────────────────────────────────────┤ │/etc/security/│set environment variables by the pam_env(8) │ │pam_env.conf │module │ ├──────────────┼───────────────────────────────────────────────┤ │/etc/ │set additional environment variables by the │ │environment │pam_env(8) module with the "readenv=1" argument│ ├──────────────┼───────────────────────────────────────────────┤ │/etc/default/ │set locale by pam_env(8) module with the │ │locale │"readenv=1 envfile=/etc/default/locale" │ │ │argument. (Debian) │ ├──────────────┼───────────────────────────────────────────────┤ │/etc/security/│set resource restraint (ulimit, core, …) by the│ │limits.conf │pam_linits(8) module │ ├──────────────┼───────────────────────────────────────────────┤ │/etc/security/│set time restraint by the pam_time(8) module │ │time.conf │ │ └──────────────┴───────────────────────────────────────────────┘ The limitation of the password selection is implemented by the PAM modules, pam_unix(8) and pam_cracklib(8). They can be configured by their arguments. Tip PAM modules use suffix ".so" for their filenames. 4.5.2. The modern centralized system management The modern centralized system management can be deployed using the centralized Lightweight Directory Access Protocol (LDAP) server to administer many Unix-like and non-Unix-like systems on the network. The open source implementation of the Lightweight Directory Access Protocol is OpenLDAP Software. The LDAP server provides the account information through the use of PAM and NSS with libpam-ldap and libnss-ldap packages for the Debian system. Several actions are required to enable this (I have not used this setup and the following is purely secondary information. Please read this in this context.). ● You set up a centralized LDAP server by running program such as stand-alone LDAP daemon, slapd(8). ● You change the PAM configuration files in the "/etc/pam.d/" directory to use "pam_ldap.so" instead of the default "pam_unix.so". ○ Debian uses "/etc/pam_ldap.conf" as the configuration file for libpam-ldap and "/etc/pam_ldap.secret" as the file to store the password of the root. ● You change the NSS configuration in the "/etc/nsswitch.conf" file to use "ldap" instead of the default ("compat" or "file"). ○ Debian uses "/etc/libnss-ldap.conf" as the configuration file for libnss-ldap. ● You must make libpam-ldap to use SSL (or TLS) connection for the security of password. ● You may make libnss-ldap to use SSL (or TLS) connection to ensure integrity of data at the cost of the LDAP network overhead. ● You should run nscd(8) locally to cache any LDAP search results in order to reduce the LDAP network traffic. See documentations in pam_ldap.conf(5) and "/usr/share/doc/ libpam-doc/html/" offered by the libpam-doc package and "info libc 'Name Service Switch'" offered by the glibc-doc package. Similarly, you can set up alternative centralized systems with other methods. ● NIS (originally called YP) or NIS+ with older Unix-like systems ● Winbind with Windows NT and SAMBA 4.5.3. "Why GNU su does not support the wheel group" This is the famous phrase at the bottom of the old "info su" page by Richard M. Stallman. Not to worry: the current su command in Debian uses PAM, so that one can restrict the ability to use su to the root group by enabling the line with "pam_wheel.so" in "/etc/pam.d/su". 4.5.4. Stricter password rule Installing the libpam-cracklib package enables you to force stricter password rule, for example, by having active lines in " /etc/pam.d/common-password" as the following. For lenny: password required pam_cracklib.so retry=3 minlen=9 difok=3 password required pam_unix.so use_authtok nullok md5 For squeeze: password required pam_cracklib.so retry=3 minlen=9 difok=3 password [success=1 default=ignore] pam_unix.so use_authtok nullok md5 password requisite pam_deny.so password required pam_permit.so 4.6. Other access controls Note See Section 9.5.15, “Alt-SysRq key” for restricting the kernel secure attention key (SAK) feature. 4.6.1. sudo sudo(8) is a program designed to allow a sysadmin to give limited root privileges to users and log root activity. sudo requires only an ordinary user's password. Install sudo package and activate it by setting options in "/etc/sudoers". See configuration example at "/usr/share/doc/sudo/examples/sudoers". My usage of sudo for the single user system (see Section 1.1.12, “sudo configuration”) is aimed to protect myself from my own stupidity. Personally, I consider using sudo a better alternative to using the system from the root account all the time. For example, the following changes the owner of " " to "". $ sudo chown Of course if you know the root password (as self-installed Debian users do), any command can be run under root from any user's account using "su -c". 4.6.2. SELinux Security-Enhanced Linux (SELinux) is a framework to tighten privilege model tighter than the ordinary Unix-like security model with the mandatory access control (MAC) policies. The root power may be restricted under some conditions. 4.6.3. Restricting access to some server services For system security, It is a good idea to disable as much server programs as possible. This becomes critical for network servers. Having unused servers, activated either directly as daemon or via super-server program, are considered security risks. Many programs, such as sshd(8), use PAM based access control. There are many ways to restrict access to some server services. ● configuration files: "/etc/default/" ● runlevel configuration for daemon ● PAM (Pluggable Authentication Modules) ● "/etc/inetd.conf" for super-server ● "/etc/hosts.deny" and "/etc/hosts.allow" for TCP wrapper, tcpd(8) ● "/etc/rpc.conf" for Sun RPC ● "/etc/at.allow" and "/etc/at.deny" for atd(8) ● "/etc/cron.allow" and "/etc/cron.deny" for crontab(1) ● Network firewall of netfilter infrastructure See Section 3.5.3, “The runlevel management example”, Section 3.5.4, “The default parameter for each init script”, Section 4.5.1, “Configuration files accessed by the PAM and NSS” , Section 3.5.8, “Network service initialization”, and Section 5.8, “Netfilter infrastructure”. Tip Sun RPC services need to be active for NFS and other RPC based programs. Tip If you have problems with remote access in a recent Debian system, comment out offending configuration such as "ALL: PARANOID" in "/etc/hosts.deny" if it exists. (But you must be careful on security risks involved with this kind of action.) 4.7. Security of authentication The information here may not be sufficient for your security needs but it should be a good start. 4.7.1. Secure password over the Internet Many popular transportation layer services communicate messages including password authentication in the plain text. It is very bad idea to transmit password in the plain text over the wild Internet where it can be intercepted. You can run these services over "Transport Layer Security" (TLS) or its predecessor, "Secure Sockets Layer" (SSL) to secure entire communication including password by the encryption. Table 4.7. List of insecure and secure services and ports ┌─────────────────────┬────┬───────────────────┬────┐ │insecure service name│port│secure service name│port│ ├─────────────────────┼────┼───────────────────┼────┤ │www (http) │80 │https │443 │ ├─────────────────────┼────┼───────────────────┼────┤ │smtp (mail) │25 │ssmtp (smtps) │465 │ ├─────────────────────┼────┼───────────────────┼────┤ │ftp-data │20 │ftps-data │989 │ ├─────────────────────┼────┼───────────────────┼────┤ │ftp │21 │ftps │990 │ ├─────────────────────┼────┼───────────────────┼────┤ │telnet │23 │telnets │992 │ ├─────────────────────┼────┼───────────────────┼────┤ │imap2 │143 │imaps │993 │ ├─────────────────────┼────┼───────────────────┼────┤ │pop3 │110 │pop3s │995 │ ├─────────────────────┼────┼───────────────────┼────┤ │ldap │389 │ldaps │636 │ └─────────────────────┴────┴───────────────────┴────┘ The encryption costs CPU time. As a CPU friendly alternative, you can keep communication in plain text while securing just password with the secure authentication protocol such as "Authenticated Post Office Protocol" (APOP) for POP and "Challenge-Response Authentication Mechanism MD5" (CRAM-MD5) for SMTP and IMAP. (For sending mail messages over the Internet to your mail server from your mail client, it is recently popular to use new message submission port 587 instead of traditional SMTP port 25 to avoid port 25 blocking by the network provider while authenticating yourself with CRAM-MD5.) 4.7.2. Secure Shell The Secure Shell (SSH) program provides secure encrypted communications between two untrusted hosts over an insecure network with the secure authentication. It consists of the OpenSSH client, ssh(1), and the OpenSSH daemon, sshd(8). This SSH can be used to tunnel the insecure protocol communication such as POP and X securely over the Internet with the port forwarding feature. The client tries to authenticate itself using host-based authentication, public key authentication, challenge-response authentication, or password authentication. The use of public key authentication enables the remote password-less login. See Section 6.9, “The remote access server and utility (SSH)”. 4.7.3. Extra security measures for the Internet Even when you run secure services such as Secure Shell (SSH) and Point-to-point tunneling protocol (PPTP) servers, there are still chances for the break-ins using brute force password guessing attack etc. from the Internet. Use of the firewall policy (see Section 5.8, “Netfilter infrastructure”) together with the following secure tools may improve the security situation. Table 4.8. List of tools to provide extra security measures ┌─────────────┬─────────┬────┬─────────────────────────────────┐ │package │popcon │size│description │ ├─────────────┼─────────┼────┼─────────────────────────────────┤ │knockd * │V:0.14, │164 │small port-knock daemon knockd(1)│ │ │I:0.3 │ │and client konck(1) │ ├─────────────┼─────────┼────┼─────────────────────────────────┤ │denyhosts * │V:1.7, │432 │utility to help sysadmins thwart │ │ │I:2 │ │ssh hackers │ ├─────────────┼─────────┼────┼─────────────────────────────────┤ │fail2ban * │V:3, I:4 │660 │ban IPs that cause multiple │ │ │ │ │authentication errors │ ├─────────────┼─────────┼────┼─────────────────────────────────┤ │libpam-shield│V:0.01, │172 │lock out remote attackers trying │ │* │I:0.06 │ │password guessing │ └─────────────┴─────────┴────┴─────────────────────────────────┘ 4.7.4. Securing the root password To prevent people to access your machine with root privilege, you need to make following actions. ● Prevent physical access to the hard disk ● Lock BIOS and prevent booting from the removable media ● Set password for GRUB interactive session ● Lock GRUB menu from editing With physical access to hard disk, resetting the password is relatively easy with following steps. 1. Move the hard disk to a PC with CD bootable BIOS. 2. Boot system with a rescue media (Debian boot disk, Knopix CD, GRUB CD, …). 3. Mount root partition with read/write access. 4. Edit "/etc/passwd" in the root partition and make the second entry for the root account empty. If you have the edit access to the GRUB menu entry (see Section 3.3, “Stage 2: the boot loader”) for grub-rescue-pc at the boot time, it is even easier with following steps. 1. Boot system with the kernel parameter changed to something like "root=/dev/hda6 rw init=/bin/sh". 2. Edit "/etc/passwd" and make the second entry for the root account empty. 3. Reboot system. The root shell of the system is now accessible without password. Note Once one has root shell access, he can access everything on the system and reset any passwords on the system. Further more, he may compromise password for all user accounts using brute force password cracking tools such as john and crack packages (see Section 9.6.11, “System security and integrity check”). This cracked password may lead to compromise other systems. The only reasonable software solution to avoid all these concerns is to use software encrypted root partition (or "/etc" partition) using dm-crypt and initramfs (see Section 9.4, “Data encryption tips”). You always need password to boot the system, though. Chapter 5. Network setup Tip For general guide to the GNU/Linux networking, read the Linux Network Administrators Guide. The traditional TCP/IP network setup on Debian system uses ifupdown package as a high level tool. There are 2 typical cases. ● For dynamic IP system such as mobile PCs, you should setup TCP/IP network with the resolvconf package and enable you to switch your network configuration easily (see Section 5.3.4, “The network interface served by the DHCP”). ● For static IP system such as servers, you should setup TCP/ IP network without the resolvconf package and keep your system simple (see Section 5.3.5, “The network interface with the static IP”). We describe these traditional cases in detail here. We also touch on some alternative high level tools such as network-manager and wicd which ease configuration of wireless networks (see Section 5.5.2, “Automatic network configuration”). 5.1. The basic network infrastructure Let's review the basic network infrastructure on the modern Debian system. 5.1.1. The domain name The naming for the domain name is a tricky one for the normal PC workstation users. The PC workstation may be mobile one hopping around the network or located behind the NAT firewall inaccessible from the Internet. For such case, you may not want the domain name to be a valid domain name to avoid name collision. According to rfc2606, "invalid" seems to be a choice for the top level domain (TLD) to construct domain names that are sure to be invalid from the Internet. The mDNS network discovery protocol (Apple Bonjour / Apple Rendezvous, Avahi on Debian) uses "local" as the pseudo-top-level domain. Microsoft also seems to promote "local" for the TLD of local area network. Other popular choices for the invalid TLD seem to be "localdomain", "lan", "localnet", or "home" according to my incoming mail analysis. 5.1.2. The hostname resolution The hostname resolution is currently supported by the NSS (Name Service Switch) mechanism too. The flow of this resolution is the following. 1. The "/etc/nsswitch.conf" file with stanza like "hosts: files dns" dictates the hostname resolution order. (This replaces the old functionality of the "order" stanza in "/etc/ host.conf".) 2. The files method is invoked first. If the hostname is found in the "/etc/hosts" file, it returns all valid addresses for it and exits. (The "/etc/host.conf" file contains "multi on".) 3. The dns method is invoked. If the hostname is found by the query to the Internet Domain Name System (DNS) identified by the "/etc/resolv.conf" file, it returns all valid addresses for it and exits. The "/etc/hosts" file associates IP addresses with hostnames contains the following. 127.0.0.1 localhost 127.0.1.1 . # The following lines are desirable for IPv6 capable hosts ::1 ip6-localhost ip6-loopback fe00::0 ip6-localnet ff00::0 ip6-mcastprefix ff02::1 ip6-allnodes ff02::2 ip6-allrouters ff02::3 ip6-allhosts Here the in this matches the own hostname defined in the "/etc/hostname". The in this is the fully qualified domain name (FQDN) of this host. Tip For the mobile PC without real FQDN, you may pick a TLD such as bogus "invalid" or mDNS "local" as the TLD part of in this configuration. The "/etc/resolv.conf" is a static file if the resolvconf package is not installed. If installed, it is a symbolic link. Either way, it contains information that initialize the resolver routines. If the DNS is found at IP="192.168.11.1", it contains the following. nameserver 192.168.11.1 The resolvconf package makes this "/etc/resolv.conf" into a symbolic link and manages its contents by the hook scripts automatically. The hostname resolution via Multicast DNS (using Zeroconf, aka Apple Bonjour / Apple Rendezvous) which effectively allows name resolution by common Unix/Linux programs in the ad-hoc mDNS domain "local", can be provided by installing the libnss-mdns package. The "/etc/nsswitch.conf" file should have stanza like "hosts: files mdns4_minimal [NOTFOUND=return] dns mdns4" to enable this functionality. 5.1.3. The network interface name The network interface name, e.g. eth0, is assigned to each hardware in the Linux kernel through the user space configuration mechanism, udev (see Section 3.5.11, “The udev system”), as it is found. The network interface name is referred as physical interface in ifup(8) and interfaces(5). In order to ensure each network interface to be named persistently for each reboot using MAC address etc., there is a record file "/etc/udev/rules.d/70-persistent-net.rules". This file is automatically generated by the "/lib/udev/ write_net_rules" program, probably run by the "persistent-net-generator.rules" rules file. You can modify it to change naming rule. Caution When editing the "/etc/udev/rules.d/70-persistent-net.rules" rules file, you must keep each rule on a single line and the MAC address in lowercase. For example, if you find "Firewire device" and "PCI device" in this file, you probably want to name "PCI device" as eth0 and configure it as the primary network interface. 5.1.4. The network address range for the LAN Let us be reminded of the IPv4 32 bit address ranges in each class reserved for use on the local area networks (LANs) by rfc1918. These addresses are guaranteed not to conflict with any addresses on the Internet proper. Table 5.1. List of network address ranges ┌─────┬────────────────────┬─────────────┬───────────┬─────────┐ │Class│network addresses │net mask │net mask / │# of │ │ │ │ │bits │subnets │ ├─────┼────────────────────┼─────────────┼───────────┼─────────┤ │A │10.x.x.x │255.0.0.0 │/8 │1 │ ├─────┼────────────────────┼─────────────┼───────────┼─────────┤ │B │172.16.x.x — │255.255.0.0 │/16 │16 │ │ │172.31.x.x │ │ │ │ ├─────┼────────────────────┼─────────────┼───────────┼─────────┤ │C │192.168.0.x — │255.255.255.0│/24 │256 │ │ │192.168.255.x │ │ │ │ └─────┴────────────────────┴─────────────┴───────────┴─────────┘ Note If one of these addresses is assigned to a host, then that host must not access the Internet directly but must access it through a gateway that acts as a proxy for individual services or else does Network Address Translation(NAT). The broadband router usually performs NAT for the consumer LAN environment. 5.1.5. The network configuration infrastructure There are 2 types of low level networking programs for Linux networking system (see Section 5.6.1, “Iproute2 commands”). ● Old net-tools programs (ifconfig(8), …) are from the Linux NET-3 networking system. Most of these are obsolete now. ● New Linux iproute2 programs (ip(8), …) are the current Linux networking system. Although these low level networking programs are powerful, they are cumbersome to use. So high level network configuration systems have been created. The ifupdown package is the de facto standard for such high level network configuration system on Debian. It enables you to bring up network simply by doing , e.g., "ifup eth0". Its configuration file is the "/etc/network/interfaces" file and its typical contents are the following. auto lo iface lo inet loopback auto eth0 iface eth0 inet dhcp The resolvconf package was created to supplement ifupdown system to support smooth reconfiguration of network address resolution by automating rewrite of resolver configuration file "/etc/ resolv.conf". Now, most Debian network configuration packages are modified to use resolvconf package (see "/usr/share/doc/ resolvconf/README.Debian"). Helper scripts to the ifupdown package such as ifplugd, guessnet, ifscheme, etc. are created to automate dynamic configuration of network environment such as one for mobile PC on wired LAN. These are relatively difficult to use but play well with existing ifupdown system. Alternative high level network configuration systems, independent of ifupdown system, such as network-manager, wicd, etc. are created to ease configuration of network environment even for mobile PC on wireless network. Since these are relatively new system and their integration to Debian system is in progress, you may still need to disable the corresponding network interface configuration manually in "/etc/network/ interfaces" to avoid conflicts between these and ifupdown (see Section 5.5.2, “Automatic network configuration”). Table 5.2. List of network configuration tools ┌─────────────────────┬───────┬─────┬─────────────────┬───────────────┐ │packages │popcon │size │type │description │ ├─────────────────────┼───────┼─────┼─────────────────┼───────────────┤ │ │ │ │ │standardized │ │ │V:61, │ │ │tool to bring │ │ifupdown * │I:99 │148 │config::ifupdown │up and down the│ │ │ │ │ │network (Debian│ │ │ │ │ │specific) │ ├─────────────────────┼───────┼─────┼─────────────────┼───────────────┤ │ │V:0.5, │ │ │manage the │ │ifplugd * │I:0.9 │332 │, , │wired network │ │ │ │ │ │automatically │ ├─────────────────────┼───────┼─────┼─────────────────┼───────────────┤ │ │ │ │ │network testing│ │ │V:0.05,│ │ │script to │ │ifupdown-extra * │I:0.2 │124 │, , │enhance │ │ │ │ │ │"ifupdown" │ │ │ │ │ │package │ ├─────────────────────┼───────┼─────┼─────────────────┼───────────────┤ │ │ │ │ │set routing │ │ifmetric * │V:0.02,│100 │, , │metrics for a │ │ │I:0.08 │ │ │network │ │ │ │ │ │interface │ ├─────────────────────┼───────┼─────┼─────────────────┼───────────────┤ │ │ │ │ │mapping script │ │ │ │ │ │to enhance │ │ │V:0.09,│ │ │"ifupdown" │ │guessnet * │I:0.4 │496 │, , │package via "/ │ │ │ │ │ │etc/network/ │ │ │ │ │ │interfaces" │ │ │ │ │ │file │ ├─────────────────────┼───────┼─────┼─────────────────┼───────────────┤ │ │ │ │ │mapping scripts│ │ifscheme * │V:0.03,│132 │, , │to enhance │ │ │I:0.10 │ │ │"ifupdown" │ │ │ │ │ │package │ ├─────────────────────┼───────┼─────┼─────────────────┼───────────────┤ │ │ │ │ │Zugschlus' │ │ifupdown-scripts-zg2 │V:0.00,│ │ │interface │ │* │I:0.05 │220 │, , │scripts for │ │ │ │ │ │ifupdown's │ │ │ │ │ │manual method │ ├─────────────────────┼───────┼─────┼─────────────────┼───────────────┤ │ │ │ │ │NetworkManager │ │ │V:27, │ │ │(daemon): │ │network-manager * │I:35 │2176 │config::NM │manage the │ │ │ │ │ │network │ │ │ │ │ │automatically │ ├─────────────────────┼───────┼─────┼─────────────────┼───────────────┤ │network-manager-gnome│V:19, │ │ │NetworkManager │ │* │I:31 │3372 │, , │(GNOME │ │ │ │ │ │frontend) │ ├─────────────────────┼───────┼─────┼─────────────────┼───────────────┤ │network-manager-kde *│V:2, │3088 │, , │NetworkManager │ │ │I:4 │ │ │(KDE frontend) │ ├─────────────────────┼───────┼─────┼─────────────────┼───────────────┤ │ │V:1.4, │ │ │wired and │ │wicd * │I:1.8 │2524 │config::wicd │wireless │ │ │ │ │ │network manager│ ├─────────────────────┼───────┼─────┼─────────────────┼───────────────┤ │ │ │ │ │administration │ │ │V:26, │ │ │tools for │ │iptables * │I:99 │1384 │config::Netfilter│packet │ │ │ │ │ │filtering and │ │ │ │ │ │NAT (Netfilter)│ ├─────────────────────┼───────┼─────┼─────────────────┼───────────────┤ │ │ │ │ │iproute2, IPv6 │ │ │ │ │ │and other │ │ │V:37, │ │ │advanced │ │iproute * │I:79 │1000 │config::iproute2 │network │ │ │ │ │ │configuration: │ │ │ │ │ │ip(8), tc(8), │ │ │ │ │ │etc │ ├─────────────────────┼───────┼─────┼─────────────────┼───────────────┤ │ │ │ │ │rename network │ │ │ │ │ │interfaces │ │ifrename * │V:0.2, │108 │, , │based on │ │ │I:0.7 │ │ │various static │ │ │ │ │ │criteria: │ │ │ │ │ │ifrename(8) │ ├─────────────────────┼───────┼─────┼─────────────────┼───────────────┤ │ │V:3, │ │ │display or │ │ethtool * │I:12 │200 │, , │change Ethernet│ │ │ │ │ │device settings│ ├─────────────────────┼───────┼─────┼─────────────────┼───────────────┤ │ │ │ │ │test network │ │ │ │ │ │reachability of│ │iputils-ping * │V:37, │140 │test::iproute2 │a remote host │ │ │I:99 │ │ │by hostname or │ │ │ │ │ │IP address ( │ │ │ │ │ │iproute2) │ ├─────────────────────┼───────┼─────┼─────────────────┼───────────────┤ │ │ │ │ │test network │ │ │V:0.9, │ │ │reachability of│ │iputils-arping * │I:13 │84 │, , │a remote host │ │ │ │ │ │specified by │ │ │ │ │ │the ARP address│ ├─────────────────────┼───────┼─────┼─────────────────┼───────────────┤ │ │V:0.4, │ │ │trace the │ │iputils-tracepath * │I:2 │120 │, , │network path to│ │ │ │ │ │a remote host │ ├─────────────────────┼───────┼─────┼─────────────────┼───────────────┤ │ │ │ │ │NET-3 │ │ │ │ │ │networking │ │ │ │ │ │toolkit ( │ │net-tools * │V:71, │1016 │config::net-tools│net-tools, IPv4│ │ │I:99 │ │ │network │ │ │ │ │ │configuration):│ │ │ │ │ │ifconfig(8) │ │ │ │ │ │etc. │ ├─────────────────────┼───────┼─────┼─────────────────┼───────────────┤ │ │ │ │ │test network │ │ │ │ │ │reachability of│ │inetutils-ping * │V:0.05,│268 │test::net-tools │a remote host │ │ │I:0.13 │ │ │by hostname or │ │ │ │ │ │IP address │ │ │ │ │ │(legacy, GNU) │ ├─────────────────────┼───────┼─────┼─────────────────┼───────────────┤ │ │ │ │ │test network │ │ │ │ │ │reachability of│ │arping * │V:0.6, │64 │, , │a remote host │ │ │I:2 │ │ │specified by │ │ │ │ │ │the ARP address│ │ │ │ │ │(legacy) │ ├─────────────────────┼───────┼─────┼─────────────────┼───────────────┤ │ │ │ │ │trace the │ │ │V:14, │ │ │network path to│ │traceroute * │I:98 │184 │, , │a remote host │ │ │ │ │ │(legacy, │ │ │ │ │ │console) │ ├─────────────────────┼───────┼─────┼─────────────────┼───────────────┤ │dhcp3-client * │V:48, │604 │config::low-level│DHCP client │ │ │I:93 │ │ │ │ ├─────────────────────┼───────┼─────┼─────────────────┼───────────────┤ │ │ │ │ │client support │ │wpasupplicant * │V:13, │960 │, , │for WPA and │ │ │I:42 │ │ │WPA2 (IEEE │ │ │ │ │ │802.11i) │ ├─────────────────────┼───────┼─────┼─────────────────┼───────────────┤ │ │ │ │ │tools for │ │wireless-tools * │V:8, │276 │, , │manipulating │ │ │I:26 │ │ │Linux Wireless │ │ │ │ │ │Extensions │ ├─────────────────────┼───────┼─────┼─────────────────┼───────────────┤ │ │V:7, │ │ │PPP/PPPoE │ │ppp * │I:26 │1100 │, , │connection with│ │ │ │ │ │chat │ ├─────────────────────┼───────┼─────┼─────────────────┼───────────────┤ │ │ │ │ │configuration │ │pppoeconf * │V:0.5, │200 │config::helper │helper for │ │ │I:4 │ │ │PPPoE │ │ │ │ │ │connection │ ├─────────────────────┼───────┼─────┼─────────────────┼───────────────┤ │ │ │ │ │configuration │ │pppconfig * │V:0.3, │900 │, , │helper for PPP │ │ │I:3 │ │ │connection with│ │ │ │ │ │chat │ ├─────────────────────┼───────┼─────┼─────────────────┼───────────────┤ │ │ │ │ │configuration │ │wvdial * │V:0.6, │420 │, , │helper for PPP │ │ │I:2 │ │ │connection with│ │ │ │ │ │wvdial and ppp │ ├─────────────────────┼───────┼─────┼─────────────────┼───────────────┤ │ │ │ │ │trace the │ │mtr-tiny * │V:4, │120 │test::low-level │network path to│ │ │I:45 │ │ │a remote host │ │ │ │ │ │(curses) │ ├─────────────────────┼───────┼─────┼─────────────────┼───────────────┤ │ │ │ │ │trace the │ │ │V:0.5, │ │ │network path to│ │mtr * │I:2 │176 │, , │a remote host │ │ │ │ │ │(curses and │ │ │ │ │ │GTK+) │ ├─────────────────────┼───────┼─────┼─────────────────┼───────────────┤ │ │ │ │ │tools for │ │ │V:4, │ │ │common network │ │gnome-nettool * │I:38 │2816 │, , │information │ │ │ │ │ │operations │ │ │ │ │ │(GNOME) │ ├─────────────────────┼───────┼─────┼─────────────────┼───────────────┤ │ │V:6, │ │ │network mapper │ │nmap * │I:31 │6220 │, , │/ port scanner │ │ │ │ │ │(Nmap, console)│ ├─────────────────────┼───────┼─────┼─────────────────┼───────────────┤ │ │V:0.2, │ │ │network mapper │ │zenmap * │I:1.2 │1784 │, , │/ port scanner │ │ │ │ │ │(GTK+) │ ├─────────────────────┼───────┼─────┼─────────────────┼───────────────┤ │ │V:0.16,│ │ │network mapper │ │knmap * │I:0.7 │1980 │, , │/ port scanner │ │ │ │ │ │(KDE) │ ├─────────────────────┼───────┼─────┼─────────────────┼───────────────┤ │ │ │ │ │network traffic│ │tcpdump * │V:3, │796 │, , │analyzer ( │ │ │I:22 │ │ │Tcpdump, │ │ │ │ │ │console) │ ├─────────────────────┼───────┼─────┼─────────────────┼───────────────┤ │ │ │ │ │network traffic│ │wireshark * │V:2, │1716 │, , │analyzer ( │ │ │I:10 │ │ │Wireshark, │ │ │ │ │ │GTK+) │ ├─────────────────────┼───────┼─────┼─────────────────┼───────────────┤ │ │V:0.6, │ │ │network traffic│ │tshark * │I:3 │308 │, , │analyzer │ │ │ │ │ │(console) │ ├─────────────────────┼───────┼─────┼─────────────────┼───────────────┤ │ │ │ │ │monitoring and │ │ │ │ │ │management │ │nagios3 * │V:0.8, │32 │, , │system for │ │ │I:1.3 │ │ │hosts, services│ │ │ │ │ │and networks ( │ │ │ │ │ │Nagios) │ ├─────────────────────┼───────┼─────┼─────────────────┼───────────────┤ │ │ │ │ │produce a │ │ │ │ │ │summarization │ │tcptrace * │V:0.08,│436 │, , │of the │ │ │I:0.4 │ │ │connections │ │ │ │ │ │from tcpdump │ │ │ │ │ │output │ ├─────────────────────┼───────┼─────┼─────────────────┼───────────────┤ │ │ │ │ │flexible │ │ │V:0.7, │ │ │network │ │snort * │I:0.9 │8384 │, , │intrusion │ │ │ │ │ │detection │ │ │ │ │ │system (Snort) │ ├─────────────────────┼───────┼─────┼─────────────────┼───────────────┤ │ │V:1.3, │ │ │display network│ │ntop * │I:2 │11288│, , │usage in web │ │ │ │ │ │browser │ ├─────────────────────┼───────┼─────┼─────────────────┼───────────────┤ │ │ │ │ │network clients│ │ │V:15, │ │ │provided with │ │dnsutils * │I:91 │392 │, , │BIND: nslookup │ │ │ │ │ │(8), nsupdate │ │ │ │ │ │(8), dig(8) │ ├─────────────────────┼───────┼─────┼─────────────────┼───────────────┤ │ │ │ │ │check DNS zone │ │ │V:0.5, │ │ │information │ │dlint * │I:7 │96 │, , │using │ │ │ │ │ │nameserver │ │ │ │ │ │lookups │ ├─────────────────────┼───────┼─────┼─────────────────┼───────────────┤ │ │V:0.10,│ │ │trace a chain │ │dnstracer * │I:0.5 │88 │, , │of DNS servers │ │ │ │ │ │to the source │ └─────────────────────┴───────┴─────┴─────────────────┴───────────────┘ 5.1.6. The network device support Although most hardware devices are supported by the Debian system, there are some network devices which require DSFG non-free external hardware drivers to support them. Please see Section 9.7.7, “Non-free hardware drivers”. 5.2. The network connection method Caution The connection test method described in this section are meant for testing purposes. It is not meant to be used directly for the daily network connection. You are advised to use them via the ifupdown package (see Section 5.3, “The basic network configuration with ifupdown”). The typical network connection method and connection path for a PC can be summarized as the following. Table 5.3. List of network connection methods and connection paths ┌───────────┬──────────┬───────────────────────────────────────┐ │PC │connection│connection path │ │ │method │ │ ├───────────┼──────────┼───────────────────────────────────────┤ │Serial port│PPP │⇔ modem ⇔ POTS ⇔ dial-up access point ⇔│ │(ppp0) │ │ISP │ ├───────────┼──────────┼───────────────────────────────────────┤ │Ethernet │PPPoE/DHCP│⇔ BB-modem ⇔ BB service ⇔ BB access │ │port (eth0)│/Static │point ⇔ ISP │ ├───────────┼──────────┼───────────────────────────────────────┤ │Ethernet │DHCP/ │⇔ LAN ⇔ BB-router with network address │ │port (eth0)│Static │translation (NAT) (⇔ BB-modem …) │ └───────────┴──────────┴───────────────────────────────────────┘ Here is the summary of configuration script for each connection method. Table 5.4. List of network connection configurations ┌──────────────┬────────────────────────────────┬──────────────┐ │connection │configuration │backend │ │method │ │package(s) │ ├──────────────┼────────────────────────────────┼──────────────┤ │PPP │pppconfig to create │pppconfig, ppp│ │ │deterministic chat │ │ ├──────────────┼────────────────────────────────┼──────────────┤ │PPP │wvdialconf to create heuristic │ppp, wvdial │ │(alternative) │chat │ │ ├──────────────┼────────────────────────────────┼──────────────┤ │PPPoE │pppoeconf to create │pppoeconf, ppp│ │ │deterministic chat │ │ ├──────────────┼────────────────────────────────┼──────────────┤ │DHCP │described in "/etc/dhcp3/ │dhcp3-client │ │ │dhclient.conf" │ │ ├──────────────┼────────────────────────────────┼──────────────┤ │static IP │described in "/etc/network/ │net-tools │ │(IPv4) │interfaces" │ │ ├──────────────┼────────────────────────────────┼──────────────┤ │static IP │described in "/etc/network/ │iproute │ │(IPv6) │interfaces" │ │ └──────────────┴────────────────────────────────┴──────────────┘ The network connection acronyms mean the following. Table 5.5. List of network connection acronyms ┌──────────┬───────────────────────────────────────────────────┐ │acronym │meaning │ ├──────────┼───────────────────────────────────────────────────┤ │POTS │plain old telephone service │ ├──────────┼───────────────────────────────────────────────────┤ │BB │broadband │ ├──────────┼───────────────────────────────────────────────────┤ │BB-service│e.g., the digital subscriber line (DSL), the cable │ │ │TV, or the fiber to the premises (FTTP) │ ├──────────┼───────────────────────────────────────────────────┤ │BB-modem │e.g., the DSL modem, the cable modem, or the │ │ │optical network terminal (ONT) │ ├──────────┼───────────────────────────────────────────────────┤ │LAN │local area network │ ├──────────┼───────────────────────────────────────────────────┤ │WAN │wide area network │ ├──────────┼───────────────────────────────────────────────────┤ │DHCP │dynamic host configuration protocol │ ├──────────┼───────────────────────────────────────────────────┤ │PPP │point-to-point protocol │ ├──────────┼───────────────────────────────────────────────────┤ │PPPoE │point-to-point protocol over Ethernet │ ├──────────┼───────────────────────────────────────────────────┤ │ISP │Internet service provider │ └──────────┴───────────────────────────────────────────────────┘ Note The WAN connection services via cable TV are generally served by DHCP or PPPoE. The ones by ADSL and FTTP are generally served by PPPoE. You have to consult your ISP for exact configuration requirements of the WAN connection. Note When BB-router is used to create home LAN environment, PCs on LAN are connected to the WAN via BB-router with network address translation (NAT). For such case, PC's network interfaces on the LAN are served by static IP or DHCP from the BB-router. BB-router must be configured to connect the WAN following the instruction by your ISP. 5.2.1. The DHCP connection with the Ethernet The typical modern home and small business network, i.e. LAN, are connected to the WAN(Internet) using some consumer grade broadband router. The LAN behind this router is usually served by the dynamic host configuration protocol (DHCP) server running on the router. Just install the dhcp3-client package for the Ethernet served by the dynamic host configuration protocol (DHCP). 5.2.2. The static IP connection with the Ethernet No special action is needed for the Ethernet served by the static IP. 5.2.3. The PPP connection with pppconfig The configuration script pppconfig configures the PPP connection interactively just by selecting the following. ● The telephone number ● The ISP user name ● The ISP password ● The port speed ● The modem communication port ● The authentication method Table 5.6. List of configuration files for the PPP connection with pppconfig ┌────────────────┬─────────────────────────────────────────────┐ │file │function │ ├────────────────┼─────────────────────────────────────────────┤ │/etc/ppp/peers/ │The pppconfig generated configuration file │ │ │for pppd specific to │ ├────────────────┼─────────────────────────────────────────────┤ │/etc/chatscripts│The pppconfig generated configuration file │ │/ │for chat specific to │ ├────────────────┼─────────────────────────────────────────────┤ │/etc/ppp/options│The general execution parameter for pppd │ ├────────────────┼─────────────────────────────────────────────┤ │/etc/ppp/ │Authentication data for the PAP (security │ │pap-secret │risk) │ ├────────────────┼─────────────────────────────────────────────┤ │/etc/ppp/ │Authentication data for the CHAP (more │ │chap-secret │secure) │ └────────────────┴─────────────────────────────────────────────┘ Caution The "" value of "provider" is assumed if pon and poff commands are invoked without arguments. You can test configuration using low level network configuration tools as the following. $ sudo pon ... $ sudo poff See "/usr/share/doc/ppp/README.Debian.gz". 5.2.4. The alternative PPP connection with wvdialconf A different approach to using pppd(8) is to run it from wvdial (1) which comes in the wvdial package. Instead of pppd running chat(8) to dial in and negotiate the connection, wvdial does the dialing and initial negotiating and then starts pppd to do the rest. The configuration script wvdialconf configures the PPP connection interactively just by selecting the following. ● The telephone number ● The ISP user name ● The ISP password wvdial succeeds in making the connection in most cases and maintains authentication data list automatically. Table 5.7. List of configuration files for the PPP connection with wvdialconf ┌──────────────┬───────────────────────────────────────────────┐ │file │function │ ├──────────────┼───────────────────────────────────────────────┤ │/etc/ppp/peers│The wvdialconf generated configuration file for│ │/wvdial │pppd specific to wvdial │ ├──────────────┼───────────────────────────────────────────────┤ │/etc/ │The wvdialconf generated configuration file │ │wvdial.conf │ │ ├──────────────┼───────────────────────────────────────────────┤ │/etc/ppp/ │The general execution parameter for pppd │ │options │ │ ├──────────────┼───────────────────────────────────────────────┤ │/etc/ppp/ │Authentication data for the PAP (security risk)│ │pap-secret │ │ ├──────────────┼───────────────────────────────────────────────┤ │/etc/ppp/ │Authentication data for the CHAP (more secure) │ │chap-secret │ │ └──────────────┴───────────────────────────────────────────────┘ You can test configuration using low level network configuration tools as the following. $ sudo wvdial ... $ sudo killall wvdial See wvdial(1) and wvdial.conf(5). 5.2.5. The PPPoE connection with pppoeconf When your ISP serves you with PPPoE connection and you decide to connect your PC directly to the WAN, the network of your PC must be configured with the PPPoE. The PPPoE stand for PPP over Ethernet. The configuration script pppoeconf configures the PPPoE connection interactively. The configuration files are the following. Table 5.8. List of configuration files for the PPPoE connection with pppoeconf ┌─────────────────┬────────────────────────────────────────────┐ │file │function │ ├─────────────────┼────────────────────────────────────────────┤ │/etc/ppp/peers/ │The pppoeconf generated configuration file │ │dsl-provider │for pppd specific to pppoe │ ├─────────────────┼────────────────────────────────────────────┤ │/etc/ppp/options │The general execution parameter for pppd │ ├─────────────────┼────────────────────────────────────────────┤ │/etc/ppp/ │Authentication data for the PAP (security │ │pap-secret │risk) │ ├─────────────────┼────────────────────────────────────────────┤ │/etc/ppp/ │Authentication data for the CHAP (more │ │chap-secret │secure) │ └─────────────────┴────────────────────────────────────────────┘ You can test configuration using low level network configuration tools as the following. $ sudo /sbin/ifconfig eth0 up $ sudo pon dsl-provider ... $ sudo poff dsl-provider $ sudo /sbin/ifconfig eth0 down See "/usr/share/doc/pppoeconf/README.Debian". 5.3. The basic network configuration with ifupdown The ifupdown package provides the standardized framework for the high level network configuration in the Debian system. In this section, we learn the basic network configuration with ifupdown with simplified introduction and many typical examples. 5.3.1. The command syntax simplified The ifupdown package contains 2 commands: ifup(8) and ifdown(8). They offer high level network configuration dictated by the configuration file "/etc/network/interfaces". Table 5.9. List of basic network configuration commands with ifupdown ┌───────┬──────────────────────────────────────────────────────┐ │command│action │ ├───────┼──────────────────────────────────────────────────────┤ │ifup │bring up a network interface eth0 with the │ │eth0 │configuration eth0 if "iface eth0" stanza exists │ ├───────┼──────────────────────────────────────────────────────┤ │ifdown │bring down a network interface eth0 with the │ │eth0 │configuration eth0 if "iface eth0" stanza exists │ └───────┴──────────────────────────────────────────────────────┘ Warning Do not use low level configuration tools such as ifconfig(8) and ip(8) commands to configure an interface in up state. Note There is no command ifupdown. 5.3.2. The basic syntax of "/etc/network/interfaces" The key syntax of "/etc/network/interfaces" as explained in interfaces(5) can be summarized as the following. Table 5.10. List of stanzas in "/etc/network/interfaces" ┌───────────────────────┬──────────────────────────────────────┐ │stanza │meaning │ ├───────────────────────┼──────────────────────────────────────┤ │"auto "│start interface upon │ │ │start of the system │ ├───────────────────────┼──────────────────────────────────────┤ │"allow-auto │, , │ │" │ │ ├───────────────────────┼──────────────────────────────────────┤ │"allow-hotplug │start interface when │ │" │the kernel detects a hotplug event │ │ │from the interface │ ├───────────────────────┼──────────────────────────────────────┤ │Lines started with │define the network configuration │ │"iface …"│ │ ├───────────────────────┼──────────────────────────────────────┤ │Lines started with │define mapping value of │ │"mapping │for the matching │ │ "│ │ ├───────────────────────┼──────────────────────────────────────┤ │A line starting with a │ignore as comments (end-of-line │ │hash "#" │comments are not supported) │ ├───────────────────────┼──────────────────────────────────────┤ │A line ending with a │extend the configuration to the next │ │backslash "\" │line │ └───────────────────────┴──────────────────────────────────────┘ Lines started with iface stanza has the following syntax. iface ... For the basic configuration, the mapping stanza is not used and you use the network interface name as the network configuration name (See Section 5.4.5, “The mapping stanza”). Warning Do not define duplicates of the "iface" stanza for a network interface in "/etc/network/interfaces". 5.3.3. The loopback network interface The following configuration entry in the "/etc/network/ interfaces" file brings up the loopback network interface lo upon booting the system (via auto stanza). auto lo iface lo inet loopback This one always exists in the "/etc/network/interfaces" file. 5.3.4. The network interface served by the DHCP After prepairing the system by Section 5.2.1, “The DHCP connection with the Ethernet”, the network interface served by the DHCP is configured by creating the configuration entry in the "/etc/network/interfaces" file as the following. allow-hotplug eth0 iface eth0 inet dhcp hostname "mymachine" When the Linux kernel detects the physical interface eth0, the allow-hotplug stanza causes ifup to bring up the interface and the iface stanza causes ifup to use DHCP to configure the interface. 5.3.5. The network interface with the static IP The network interface served by the static IP is configured by creating the configuration entry in the "/etc/network/ interfaces" file as the following. allow-hotplug eth0 iface eth0 inet static address 192.168.11.100 netmask 255.255.255.0 broadcast 192.168.11.255 gateway 192.168.11.1 dns-domain lan dns-nameservers 192.168.11.1 When the Linux kernel detects the physical interface eth0, the allow-hotplug stanza causes ifup to bring up the interface and the iface stanza causes ifup to use the static IP to configure the interface. Here, I assumed the following. ● IP address range of the LAN network: 192.168.11.0 - 192.168.11.255 ● IP address of the gateway: 192.168.11.1 ● IP address of the PC: 192.168.11.100 ● The resolvconf package: installed ● The domain name: "lan" ● IP address of the DNS server: 192.168.11.1 When the resolvconf package is not installed, DNS related configuration needs to be done manually by editing the "/etc/ resolv.conf" as the following. nameserver 192.168.11.1 domain lan Caution The IP addresses used in the above example are not meant to be copied literally. You have to adjust IP numbers to your actual network configuration. 5.3.6. The basics of wireless LAN interface The wireless LAN (WLAN for short) provides the fast wireless connectivity through the spread-spectrum communication of unlicensed radio bands based on the set of standards called IEEE 802.11. The WLAN interfaces are almost like normal Ethernet interfaces but require some network ID and encryption key data to be provided when they are initialized. Their high level network tools are exactly the same as that of Ethernet interfaces except interface names are a bit different like eth1, wlan0, ath0, wifi0, … depending on the kernel drivers used. Tip The wmaster0 device is the master device which is an internal device used only by SoftMAC with new mac80211 API of Linux. Here are some keywords to remember for the WLAN. Table 5.11. List of acronyms for WLAN ┌───────┬─────────────┬────────────────────────────────────────┐ │acronym│full word │meaning │ ├───────┼─────────────┼────────────────────────────────────────┤ │NWID │Network ID │16 bit network ID used by pre-802.11 │ │ │ │WaveLAN network (very deprecated) │ ├───────┼─────────────┼────────────────────────────────────────┤ │ │(Extended) │network name of the Wireless Access │ │(E)SSID│Service Set │Points (APs) interconnected to form an │ │ │Identifier │integrated 802.11 wireless LAN, Domain │ │ │ │ID │ ├───────┼─────────────┼────────────────────────────────────────┤ │WEP, │Wired │1st generation 64-bit (128-bit) wireless│ │(WEP2) │Equivalent │encryption standard with 40-bit key │ │ │Privacy │(deprecated) │ ├───────┼─────────────┼────────────────────────────────────────┤ │ │Wi-Fi │2nd generation wireless encryption │ │WPA │Protected │standard (most of 802.11i), compatible │ │ │Access │with WEP │ ├───────┼─────────────┼────────────────────────────────────────┤ │ │Wi-Fi │3rd generation wireless encryption │ │WPA2 │Protected │standard (full 802.11i), non-compatible │ │ │Access 2 │with WEP │ └───────┴─────────────┴────────────────────────────────────────┘ The actual choice of protocol is usually limited by the wireless router you deploy. 5.3.7. The wireless LAN interface with WPA/WPA2 You need to install the wpasupplicant package to support the WLAN with the new WPA/WPA2. In case of the DHCP served IP on WLAN connection, the "/etc/ network/interfaces" file entry should be as the following. allow-hotplug ath0 iface ath0 inet dhcp wpa-ssid homezone # hexadecimal psk is encoded from a plaintext passphrase wpa-psk 000102030405060708090a0b0c0d0e0f101112131415161718191a1b1c1d1e1f See "/usr/share/doc/wpasupplicant/README.modes.gz". 5.3.8. The wireless LAN interface with WEP You need to install the wireless-tools package to support the WLAN with the old WEP. (Your consumer grade router may still be using this insecure infrastructure but this is better than nothing.) Caution Please note that your network traffic on WLAN with WEP may be sniffed by others. In case of the DHCP served IP on WLAN connection, the "/etc/ network/interfaces" file entry should be as the following. allow-hotplug eth0 iface eth0 inet dhcp wireless-essid Home wireless-key1 0123-4567-89ab-cdef wireless-key2 12345678 wireless-key3 s:password wireless-defaultkey 2 wireless-keymode open See "/usr/share/doc/wireless-tools/README.Debian". 5.3.9. The PPP connection You need to configure the PPP connection first as described before (see Section 5.2.3, “The PPP connection with pppconfig”). Then, add the "/etc/network/interfaces" file entry for the primary PPP device ppp0 as the following. iface ppp0 inet ppp provider 5.3.10. The alternative PPP connection You need to configure the alternative PPP connection with wvdial first as described before (see Section 5.2.4, “The alternative PPP connection with wvdialconf”). Then, add the "/etc/network/ interfaces" file entry for the primary PPP device ppp0 as the following. iface ppp0 inet wvdial 5.3.11. The PPPoE connection For PC connected directly to the WAN served by the PPPoE, you need to configure system with the PPPoE connection as described before (see Section 5.2.5, “The PPPoE connection with pppoeconf” ). Then, add the "/etc/network/interfaces" file entry for the primary PPPoE device eth0 as the following. allow-hotplug eth0 iface eth0 inet manual pre-up /sbin/ifconfig eth0 up up ifup ppp0=dsl down ifdown ppp0=dsl post-down /sbin/ifconfig eth0 down # The following is used internally only iface dsl inet ppp provider dsl-provider 5.3.12. The network configuration state of ifupdown The "/etc/network/run/ifstate" file stores the intended network configuration states for all the currently active network interfaces managed by the ifupdown package are listed. Unfortunately, even if the ifupdown system fails to bring up the interface as intended, the "/etc/network/run/ifstate" file lists it active. Unless the output of the ifconfig(8) command for an interface does not have a line like following example, it can not be used as a part of IPV4 network. inet addr:192.168.11.2 Bcast:192.168.11.255 Mask:255.255.255.0 Note For the Ethernet device connected to the PPPoE, the output of the ifconfig(8) command lacks a line which looks like above example. 5.3.13. The basic network reconfiguration When you try to reconfigure the interface, e.g. eth0, you must disable it first with the "sudo ifdown eth0" command. This removes the entry of eth0 from the "/etc/network/run/ifstate" file. (This may result in some error message if eth0 is not active or it is configured improperly previously. So far, it seems to be safe to do this for the simple single user work station at any time.) You are now free to rewrite the "/etc/network/interfaces" contents as needed to reconfigure the network interface, eth0. Then, you can reactivate eth0 with the "sudo ifup eth0" command. Tip You can (re)initialize the network interface simply by "sudo ifdown eth0;sudo ifup eth0". 5.3.14. The ifupdown-extra package The ifupdown-extra package provides easy network connection tests for use with the ifupdown package. ● The network-test(1) command can be used from the shell. ● The automatic scripts are run for each ifup command execution. The network-test command frees you from the execution of cumbersome low level commands to analyze the network problem. The automatic scripts are installed in "/etc/network/*/" and performs the following. ● Check the network cable connection ● Check duplicate use of IP address ● Setup system's static routes based on the "/etc/network/ routes" definition ● Check if network gateway is reachable ● Record results in the "/var/log/syslog" file This syslog record is quite useful for administration of the network problem on the remote system. Tip The automatic behavior of the ifupdown-extra package is configurable with the "/etc/default/network-test". Some of these automatic checks slow down the system boot-up a little bit since it takes some time to listen for ARP replies. 5.4. The advanced network configuration with ifupdown The functionality of the ifupdown package can be improved beyond what was described in Section 5.3, “The basic network configuration with ifupdown” with the advanced knowledge. The functionalities described here are completely optional. I, being lazy and minimalist, rarely bother to use these. Caution If you could not set up network connection by information in Section 5.3, “The basic network configuration with ifupdown”, you make situation worse by using information below. 5.4.1. The ifplugd package The ifplugd package is older automatic network configuration tool which can manage only Ethernet connections. This solves unplugged/replugged Ethernet cable issues for mobile PC etc. If you have NetworkManager or Wicd (see Section 5.5.2, “Automatic network configuration”) installed, you do not need this package. This package runs daemon and replaces auto or allow-hotplug functionalities (see Table 5.10, “List of stanzas in "/etc/ network/interfaces"”) and starts interfaces upon their connection to the network. Here is how to use the ifplugd package for the internal Ethernet port, e.g. eth0. 1. Remove stanza in "/etc/network/interfaces": "auto eth0" or "allow-hotplug eth0". 2. Keep stanza in "/etc/network/interfaces": "iface eth0 inet …" and "mapping …". 3. Install the ifplugd package. 4. Run "sudo dpkg-reconfigure ifplugd". 5. Put eth0 as the "static interfaces to be watched by ifplugd". Now, the network reconfiguration works as you desire. ● Upon power-on or upon hardware discovery, the interface is not brought up by itself. ○ Quick boot process without the long DHCP timeout. ○ No funny activated interface without proper IPv4 address (see Section 5.3.12, “The network configuration state of ifupdown”). ● Upon finding the Ethernet cable, the interface is brought up. ● Upon some time after unplugging the Ethernet cable, the interface is brought down automatically. ● Upon plugging in another Ethernet cable, the interface is brought up under the new network environment. Tip The arguments for the ifplugd(8) command can set its behaviors such as the delay for reconfiguring interfaces. 5.4.2. The ifmetric package The ifmeric package enables us to manipulate metrics of routes a posteriori even for DHCP. The following sets the eth0 interface to be preferred over the wlan0 interface. 1. Install the ifmetric package. 2. Add an option line with "metric 0" just below the "iface eth0 inet dhcp" line. 3. Add an option line with "metric 1" just below the "iface wlan0 inet dhcp" line. The metric 0 means the highest priority route and is the default one. The larger metric value means lower priority routes. The IP address of the active interface with the lowest metric value becomes the originating one. See ifmetric(8). 5.4.3. The virtual interface A single physical Ethernet interface can be configured as multiple virtual interfaces with different IP addresses. Usually the purpose is to connect an interface to several IP subnetworks. For example, IP address based virtual web hosting by a single network interface is one such application. For example, let's suppose the following. ● A single Ethernet interface on your host is connected to a Ethernet hub (not to the broadband router). ● The Ethernet hub is connected to both the Internet and LAN network. ● The LAN network uses subnet 192.168.0.x/24. ● Your host uses DHCP served IP address with the physical interface eth0 for the Internet. ● Your host uses 192.168.0.1 with the virtual interface eth0:0 for the LAN. The following stanzas in "/etc/network/interfaces" configure your network. iface eth0 inet dhcp metric 0 iface eth0:0 inet static address 192.168.0.1 netmask 255.255.255.0 network 192.168.0.0 broadcast 192.168.0.255 metric 1 Caution Although this configuration example with network address translation (NAT) using netfilter/iptables (see Section 5.8, “Netfilter infrastructure”) can provide cheap router for the LAN with only single interface, there is no real firewall capability with such set up. You should use 2 physical interfaces with NAT to secure the local network from the Internet. 5.4.4. The advanced command syntax The ifupdown package offers advanced network configuration using the network configuration name and the network interface name. I use slightly different terminology from one used in ifup(8) and interfaces(5). Table 5.12. List of terminology for network devices ┌───────────┬─────────────┬────────────────┬───────────────────┐ │manpage │my │examples in the │description │ │terminology│terminology │following text │ │ ├───────────┼─────────────┼────────────────┼───────────────────┤ │physical │network │lo, eth0, │name given by the │ │interface │interface ││Linux kernel (using│ │name │name │ │udev mechanism) │ ├───────────┼─────────────┼────────────────┼───────────────────┤ │logical │network │config1, │name token │ │interface │configuration│config2, │following iface in │ │name │name │ │the "/etc/network/ │ │ │ │ │interfaces" │ └───────────┴─────────────┴────────────────┴───────────────────┘ Basic network configuration commands in Section 5.3.1, “The command syntax simplified” require the network configuration name token of the iface stanza to match the network interface name in the "/etc/network/interfaces". Advanced network configuration commands enables separation of the network configuration name and the network interface name in the "/etc/network/interfaces" as the following. Table 5.13. List of advanced network configuration commands with ifupdown ┌───────────┬──────────────────────────────────────────────────┐ │command │action │ ├───────────┼──────────────────────────────────────────────────┤ │ifup eth0= │bring up a network interface eth0 with the │ │config1 │configuration config1 │ ├───────────┼──────────────────────────────────────────────────┤ │ifdown eth0│bring down a network interface eth0 with the │ │=config1 │configuration config1 │ ├───────────┼──────────────────────────────────────────────────┤ │ifup eth0 │bring up a network interface eth0 with the │ │ │configuration selected by mapping stanza │ ├───────────┼──────────────────────────────────────────────────┤ │ifdown eth0│bring down a network interface eth0 with the │ │ │configuration selected by mapping stanza │ └───────────┴──────────────────────────────────────────────────┘ 5.4.5. The mapping stanza We skipped explaining the mapping stanza in the "/etc/network/ interfaces" in Section 5.3.2, “The basic syntax of "/etc/network /interfaces"” to avoid complication. This stanza has the following syntax. mapping script map map map ... This provides advanced feature to the "/etc/network/interfaces" file by automating the choice of the configuration with the mapping script specified by . Let's follow the execution of the following. $ sudo ifup eth0 When the "" matches "eth0", this execution produces the execution of the following command to configure eth0 automatically. $ sudo ifup eth0=$(echo -e ' \n \n ...' | eth0) Here, script input lines with "map" are optional and can be repeated. Note The glob for mapping stanza works like shell filename glob (see Section 1.5.6, “Shell glob”). 5.4.6. The manually switchable network configuration Here is how to switch manually among several network configurations without rewriting the "/etc/network/interfaces" file as in Section 5.3.13, “The basic network reconfiguration” . For all the network configuration you need to access, you create a single "/etc/network/interfaces" file as the following. auto lo iface lo inet loopback iface config1 inet dhcp hostname "mymachine" iface config2 inet static address 192.168.11.100 netmask 255.255.255.0 broadcast 192.168.11.255 gateway 192.168.11.1 dns-domain lan dns-nameservers 192.168.11.1 iface pppoe inet manual pre-up /sbin/ifconfig eth0 up up ifup ppp0=dsl down ifdown ppp0=dsl post-down /sbin/ifconfig eth0 down # The following is used internally only iface dsl inet ppp provider dsl-provider iface pots inet ppp provider provider Please note the network configuration name which is the token after iface does not use the token for the network interface name. Also, there are no auto stanza nor allow-hotplug stanza to start the network interface eth0 automatically upon events. Now you are ready to switch the network configuration. Let's move your PC to a LAN served by the DHCP. You bring up the network interface (the physical interface) eth0 by assigning the network configuration name (the logical interface name) config1 to it by the following. $ sudo ifup eth0=config1 Password: ... The interface eth0 is up, configured by DHCP and connected to LAN. $ sudo ifdown eth0=config1 ... The interface eth0 is down and disconnected from LAN. Let's move your PC to a LAN served by the static IP. You bring up the network interface eth0 by assigning the network configuration name config2 to it by the following. $ sudo ifup eth0=config2 ... The interface eth0 is up, configured with static IP and connected to LAN. The additional parameters given as dns-* configures "/etc/resolv.conf" contents. This "/etc/resolv.conf" is better manged if the resolvconf package is installed. $ sudo ifdown eth0=config2 ... The interface eth0 is down and disconnected from LAN, again. Let's move your PC to a port on BB-modem connected to the PPPoE served service. You bring up the network interface eth0 by assigning the network configuration name pppoe to it by the following. $ sudo ifup eth0=pppoe ... The interface eth0 is up, configured with PPPoE connection directly to the ISP. $ sudo ifdown eth0=pppoe ... The interface eth0 is down and disconnected, again. Let's move your PC to a location without LAN or BB-modem but with POTS and modem. You bring up the network interface ppp0 by assigning the network configuration name pots to it by the following. $ sudo ifup ppp0=pots ... The interface ppp0 is up and connected to the Internet with PPP. $ sudo ifdown ppp0=pots ... The interface ppp0 is down and disconnected from the Internet. You should check the "/etc/network/run/ifstate" file for the current network configuration state of the ifupdown system. Warning You may need to adjust numbers at the end of eth*, ppp*, etc. if you have multiple network interfaces. 5.4.7. Scripting with the ifupdown system The ifupdown system automatically runs scripts installed in "/ etc/network/*/" while exporting environment variables to scripts. Table 5.14. List of environment variables passed by the ifupdown system ┌────────────┬─────────────────────────────────────────────────┐ │environment │value passed │ │variable │ │ ├────────────┼─────────────────────────────────────────────────┤ │"$IFACE" │physical name (interface name) of the interface │ │ │being processed │ ├────────────┼─────────────────────────────────────────────────┤ │"$LOGICAL" │logical name (configuration name) of the │ │ │interface being processed │ ├────────────┼─────────────────────────────────────────────────┤ │"$ADDRFAM" │ of the interface │ ├────────────┼─────────────────────────────────────────────────┤ │"$METHOD" │ of the interface. (e.g., "static") │ ├────────────┼─────────────────────────────────────────────────┤ │"$MODE" │"start" if run from ifup, "stop" if run from │ │ │ifdown │ ├────────────┼─────────────────────────────────────────────────┤ │ │as per "$MODE", but with finer granularity, │ │"$PHASE" │distinguishing the pre-up, post-up, pre-down and │ │ │post-down phases │ ├────────────┼─────────────────────────────────────────────────┤ │"$VERBOSITY"│indicates whether "--verbose" was used; set to 1 │ │ │if so, 0 if not │ ├────────────┼─────────────────────────────────────────────────┤ │"$PATH" │command search path: "/usr/local/sbin:/usr/local/│ │ │bin:/usr/sbin:/usr/bin:/sbin:/bin" │ ├────────────┼─────────────────────────────────────────────────┤ │"$IF_ │value for the corresponding option under the │ │