01 Lesson 01: Basic Tools / Installation and Management, psql
This page is extracted from the official PDF into native Markdown. Slide text, code and speaker notes are kept in reading order; the original PDF and HTML remain the layout references. HTML source: dev1_01_tools_overview.html Archive: DEV1-handouts-16-en.zip PDF pages: 20
Page 1: Basic Tools / Installation and Management, psql
Copyright
© Postgres Professional, 2017–2025 Authors: Egor Rogov, Pavel Luzanov, Ilya Bashtanov, Igor Gnatyuk Translated by: Liudmila Mantrova, Alexander Meleshko, Elena Sharafutdinova Photo by: Oleg Bartunov (Phu monastery, Bhrikuti summit, Nepal)
Use of Course Materials
Non-commercial use of course materials (presentations, demonstrations) is allowed without restrictions. Commercial use is possible only with the written permission of Postgres Professional. It is prohibited to make changes to the course materials.
Feedback
Please send your feedback, comments and suggestions to: edu@postgrespro.ru
Disclaimer
Postgres Professional assumes no responsibility for any damages and losses, including loss of income, caused by direct or indirect, intentional or accidental use of course materials. Postgres Professional company specifically disclaims any warranties on course materials. Course materials are provided “as is,” and Postgres Professional company has no obligations to provide maintenance, support, updates, enhancements, or modifications.
Page 2: Topics
- PostgreSQL Installation Types
- Server Management
- Server Message Log
- Configuration Parameters
- Using psql
Page 3: Database Cluster
{#source-figure-003 caption=“Source slide page 3: Database Cluster”}
- client
Speaker notes
Let’s start with the general concepts.
PostgreSQL is a program that belongs to the class of database management systems.
When this program is running, we call it a PostgreSQL server or a server instance. So far, the server seems to be a “black box” for us, but gradually we will get acquainted with how it works.
The data managed by PostgreSQL is stored in databases. One instance of PostgreSQL simultaneously manages several databases. This set of databases is called a database cluster. We will talk more about databases in the “Data organization. Logical structure” lesson.
The server interacts with clients. Clients are external applications that can connect to server databases and send queries for execution.
To summarize: a database cluster is the data in files; a server or a server instance is a program that manages the database cluster; a client is an application that “talks” to the server.
Page 4: Installation
Installation types
- pre-build packages (preferred)
- installation from source code
- without installation (cloud services)
- Extensions
- provide additional features
- installed separately
- shipped with the server as modules and programs (~50 extensions)
Speaker notes
The preferred option for installing PostgreSQL is via a package manager (such as apt or rpm) using pre-built packages. This gives you a comprehensive installation that is easy to support and upgrade. There are packages available for most operating systems.
Another option is to build PostgreSQL from source code. This may be necessary if you want to set up a non-standard configuration or deploy on an exotic platform.
Pre-built packages and source codes: http://www.postgresql.org/download/
Besides, you can work with cloud-based managed databases that do not require any installation at all. Such ready-to-use services are provided by all major cloud platforms (Amazon RDS, Google Cloud SQL, Microsoft Azure).
In this course, we are going to use a virtual machine with Xubuntu 24 OS; PostgreSQL 16 is installed from the package for this OS. In addition, the installation is set up to start and stop PostgreSQL when the OS starts and stops.
There are a lot of PostgreSQL extensions that add new database functionality “on the fly”, without modifying the system core. About 50 extensions are included into the PostgreSQL distribution itself.
https://postgrespro.com/docs/postgresql/16/contrib
https://postgrespro.com/docs/postgresql/16/contrib-prog
The list of available extensions and their installation status can be accessed using the pg_available_extensions view.
Page 5: Server Management
Management tools
- pg_ctlcluster
- pg_ctl
- Main operations
- start the server
- stop the server
- reload configuration parameters
Speaker notes
The main server management tasks are: initializing the database cluster, starting and stopping the server, reloading configuration parameters, and a few others. To perform these actions, use the pg_ctl utility, which comes together with PostgreSQL.
In the Ubuntu PostgreSQL package, pg_ctl is not run directly, but through a special wrapper pg_ctlcluster. The help information for pg_ctlcluster can be viewed using man:
$ man pg_ctlcluster
Information about existing clusters and their status can be obtained using the following commands:
$ pg_lsclusters
$ pg_ctlcluster status
For more details about server management for database administrators, see:
https://postgrespro.com/docs/postgresql/16/app-pg-ctl
https://postgrespro.com/docs/postgresql/16/runtime
Page 6: Installation and Management
PostgreSQL in the course VM is installed from packages. PostgreSQL installation directory:
The owner of the server software is the root user. The database cluster is automatically initialized when installed from a package and is located in the /var/lib/postgresql/16/main directory. This directory will be referred to as PGDATA in subsequent topics. The owner of the directory is postgres. Here is what the directory contains:
With the package installation, PostgreSQL is automatically added to the OS startup list, so there is no need to start it manually. Use the following commands to control the server as a privileged OS user with sudo: Stop the server:
Start the server:
Restart:
Reload configuration:
Get server information:
List of installed instances (works without sudo):
Page 7: Page 7
Page 8: Server Message Log
Server message log contains
- server messages
- user session messages
- application messages
- Log configuration
- log file location
- message format
- events to log
Speaker notes
Database operations are tracked in the server message log. The log keeps records about starting and stopping the server, as well as various signal messages, including messages about possible issues.
It can also store records of executed commands, their running time, locks that occur, etc. It can be used to trace user sessions.
Application developers can generate and log their own messages.
PostgreSQL allows you to flexibly configure which messages and in which format should be recorded into the log.
For example, the CSV and JSON output formats are convenient for automating log analysis.
https://postgrespro.com/docs/postgresql/16/runtime-config-logging
Page 9: Server Message Log
The server message log is located here:
Let’s look at the last messages:
Page 10: Configuration Parameters
For the entire instance
- the main configuration file is postgresql.conf
- For the current session
- View the current setting
Speaker notes
The PostgreSQL server configuration is governed by a variety of parameters that are used to manage resource consumption, tune system processes and user sessions, manage the server log, and handle many other tasks. It’s important to know how to check and update the parameter values.
Server configuration is usually stored in configuration files. The main configuration file is postgresql.conf, it has to be edited manually. The second configuration file is postgresql.auto.conf. Its contents are set using the ALTER SYSTEM command. The parameters set via ALTER SYSTEM take precedence over the parameters in postgresql.conf.
File and directory inclusion directives include and include_dir allow splitting large postgresql.conf into manageable parts. This may come in handy when managing multiple servers with similar configurations.
Most configuration parameters can be adjusted within a session with no server restart. You can define custom parameters and manage them in the same manner as with the system parameters.
Different ways of setting and updating parameters are described here:
https://postgrespro.com/docs/postgresql/16/config-setting
And the current parameter values are shown in the pg_settings view.
https://postgrespro.com/docs/postgresql/16/view-pg-settings
Page 11: Configuration Parameters
The main postgresql.conf configuration file is located in this directory:
Other configuration files are also stored here. Let’s check the value of the work_mem parameter:
The work_mem parameter specifies the amount of memory that will be used for internal sorting and hash table allocation operations before using temporary files on disk. 4MB is the default value and it is too small. Let’s say we want to increase it to 16MB for the entire instance. There are various ways to do this. First, you can uncomment and change the line in postgresql.conf where the parameter is defined:
Secondly, you can place the parameter definition in a file with the .conf suffix in the /etc/postgresql/16/main/conf.d directory or in a custom configuration file, the location of which should be set in the include directive of the main configuration file postgresql.conf. Thirdly, you can change the parameter value using an SQL command, and this is what we will do:
This change does not end up in postgresql.conf, but in postgresql.auto.conf, which is located in the PGDATA directory:
Reload the configuration files for the change to take effect. To do this, you can use pg_ctlcluster, or use the SQL function:
Verify that the new value is applied. Here is another way to check it:
To restore the default value of a parameter, you can use the RESET command instead of SET (and then reload the configuration, of course):
Page 12: Page 12
Most parameter values can be set for the current session live. For example, you can increase work_mem just for a single query that you expect to sort a large volume of data:
The new value will remain for the current session or even for the current transaction (with SET LOCAL). Yet another way to check a current value is to query the pg_settings view:
Lastly, there is the \dconfig command:
Page 13: The psql Client
Terminal client for working with PostgreSQL
- Comes with the DBMS
- Used by administrators and developers for interactive work and script execution
Speaker notes
There are other third-party tools available, but they are not considered in the scope of the course.
In the course, we will use the psql terminal client:
psql is the only client supplied with the DBMS.
The knowledge of psql will be useful to both developers and DB administrators, regardless of which tool they choose to work with at the end of the day.
For interactivity, psql provides built-in support for readline and pager programs (such as less and pspg), as well as the option to use external editing tools. Using psql, you can interact with the operating system, browse through the system catalog, and write scripts to automate routine tasks.
https://postgrespro.com/docs/postgresql/16/app-psql
Page 14: Connection
When starting psql, you need to specify the connection parameters. Required parameters include: database name, same as the user name by default user name (role), same as the OS user name by default host, local connection by default port, usually 5432 by default This is how you put them in:
The VM is set up to connect to PostgreSQL without specifying the parameters:
Check the connection:
The \connect command creates a new connection without leaving psql. You can use its short form: \c. From here on, the optional parts of commands will be shown in square brackets: \c[onnect].
Help and Documentation
You don’t need to open the documentation every time you need to check something. Instead, you can get the information from within the system directly. The command
displays the help information for psql. If PostgreSQL was installed with documentation, then you can get the reference manual with the command
psql can execute SQL commands and its own commands which start with a backslash, such as \conninfo. psql commands are always single-line, unlike SQL commands. Inside psql, you can get a list and a brief description of all psql commands: \? displays the list of psql commands \h[elp] provides a list of SQL commands that the server supports, as well as the syntax of an SQL command (if specified).
Formatting the Output
The psql client can return output in different formats: aligned format unaligned format extended format The aligned format is the default:
It sets each column’s width based on its contents. There’s also the header and the footer. psql commands to switch display modes: \a — mode switch: aligned/unaligned. \t — toggle the header and footer display. Let’s disable alignment, heading, and summary line:
Page 15: Page 15
This format is not the best for readability, but can be more suitable for automatic processing. The extended format is best for displaying multiple columns for one or several records. It is switched on for the current command with the \gx flag at the end in place of a semicolon:
Or you can toggle it on and off with the \x flag. The \pset command displays all formatting options.
Interacting with the OS and Executing Scripts
psql can run shell commands:
An SQL query can generate several other SQL queries and write them to a file using the \o[ut] command:
Nothing is printed to the screen (to the standard output). Check the file:
Revert to default output and formatting.
We can run the commands from the file using \i[nclude]:
There are other ways to execute commands, including from files. After executing the commands, the psql session will be terminated: psql < filename psql -f < filename psql -c ‘command’ (one command per call)
Page 16: psql Variables
Like shell, psql has its own variables. Set a variable:
To replace a variable with its value, use the colon in front of its name:
Unset a value:
Variables can be used, for example, to store the text of frequently used queries. Here is a query that returns a list of the five largest tables:
Now we can execute the query by just typing:
You can add the command to set the value of top5 to the .psqlrc starter file in the user’s home directory. Commands from .psqlrc are executed automatically every time psql starts. If used with no parameters, \set displays all currently set variables and their values, including the built-in ones. You can check the help for the built-in variables:
To exit psql, either use \q[uit], quit, or exit, or press Ctrl+D:
Page 17: Takeaways
Installing PostgreSQL from pre-built packages is the preferred installation type
- Pre-packaged distributions bring some OS specifics that you should be aware of:
- how to start and stop the server
- location of configuration files
- location of the server message log
- psql is a client application for working with PostgreSQL
Page 18: Practice
- Set the parameter work_mem to 8MB for all sessions. Reload the configuration and verify that the changes have taken effect. Restore the default value. 2. Put the command to create a table to the ddl.sql file. Put the commands to insert rows into this table to the populate.sql file. Run both scripts and verify that the table has been created and populated. 3. Find today’s records in the server message log.
Speaker notes
It is convenient to have separate databases for tasks that are related to different topics:
Use the ALTER SYSTEM command.
If the output does not fit into one terminal screen, psql forwards it to the tool called less for page-by-page output. To return to the prompt, press q.
Page 19: 1. Configuration Parameters
Connect to the database:
Set the parameters:
Reload the configuration:
Check the parameter value:
Reset to the default value:
Apply changes:
2. Executing Scripts in psql
Write a command into ’ddl.sql’ to create a table in PostgreSQL (you can use any text editor):
Write commands into ’populate.sql’ to populate the table:
Create a new database and connect to it:
Run the scripts and check the records in the table:
Page 20: Page 20
3. Viewing the Log
The log can be viewed in any text editor. Each log record starts with a date, contains the server process ID (with the packaged installation configuration), and may span several lines. The last records will be at the end of the file.
The database can now be disconnected and deleted (see the Data Organization module for more)