How To Manage and Use Journalctl Logging System on RHEL 7 or CentOS 7

Overview

The guides will help in understanding the RHEL / CentOS 7 logging system “journalctl”. Journald Daemon centralizes the management of logs regardless of where the messages are originated.

Applies To

RHEL 7, CentOS 7

Pre-requisites

None

journald.conf – Configurable Values Table

Configuration Attribute	Configurable Values
Storage	"volatile", "persistent", "auto" and "none"
Compress	"yes", "no"
Seal	"yes", "no"
SplitMode	"login", "uid" and "none"
SyncIntervalSec	User defined
RateLimitInterval	User defined
RateLimitBurst	User defined
SystemMaxUse	Customizable, applies when Storage set to persistent
SystemKeepFree	Customizable, applies when Storage set to persistent
SystemMaxFileSize	Customizable, applies when Storage set to persistent
RuntimeMaxUse	Customizable, applies when Storage set to persistent
RuntimeKeepFree	Customizable, applies when Storage set to persistent
RuntimeMaxFileSize	Customizable, applies when Storage set to persistent
MaxRetentionSec	User defined
MaxFileSec	User defined
ForwardToSyslog	"yes", "no"
ForwardToKMsg	"yes", "no"
ForwardToConsole	"yes", "no"
TTYPath	/dev/console
MaxLevelStore	"emerg", "alert", "crit", "err", "warning", "notice", "info", "debug" or integer value from 0 – 7
MaxLevelSyslog	"emerg", "alert", "crit", "err", "warning", "notice", "info", "debug"
MaxLevelKMsg	or integer value from 0 – 7
MaxLevelConsole	"emerg", "alert", "crit", "err", "warning", "notice", "info", "debug"

journald.conf – Attribute Purpose Table

Configuration Attribute	Purpose
Storage	Where to store journal data, for persistent data storage, create /run/log/journal/
Compress	Takes a boolean value. If enabled (the default)
Seal	Takes a boolean value. If enabled (the default)
SplitMode	Controls whether to split up journal files per user
SyncIntervalSec	Configures the rate limiting that is applied to all messages generated on the system
RateLimitInterval	Configures the rate limiting that is applied to all messages generated on the system. If, in the time interval defined by RateLimitInterval=, more messages than specified in RateLimitBurst= are logged by a service, all further messages within the interval are dropped until the interval is over
RateLimitBurst
SystemMaxUse	Enforce size limits on the journal files stored. The options prefixed with "System" apply to the journal files when stored on a persistent file system, more specifically /var/log/journal. The options prefixed with "Runtime" apply to the journal files when stored on a volatile in-memory file system, more specifically /run/log/journal. The former is used only when /var is mounted, writable, and the directory /var/log/journal exists. Otherwise, only the latter applies.
SystemKeepFree
SystemMaxFileSize
RuntimeMaxUse
RuntimeKeepFree
RuntimeMaxFileSize
MaxRetentionSec	The maximum time to store journal entries. This controls whether journal files containing entries older then the specified time span are deleted.
MaxFileSec	The maximum time to store entries in a single journal file before rotating to the next one.
ForwardToSyslog	Control whether log messages received by the journal daemon shall be forwarded to a traditional syslog daemon, to the kernel log buffer (kmsg), to the system console, or sent as wall messages to all logged-in users.
ForwardToKMsg
ForwardToConsole
ForwardToWall
TTYPath	Change the console TTY to use if ForwardToConsole=yes
MaxLevelStore	Controls the maximum log level of messages that are stored on disk, forwarded to syslog, kmsg, the console or wall (if that is enabled)
MaxLevelSyslog
MaxLevelKMsg
MaxLevelConsole
MaxLevelWall

 Default Setting – Configuration File – journald.conf

Logs are controlled by file “journald.conf” which is located in the folder “/etc/systemd/”.

[Journal]

Storage=

…….

The below snippet is default setting configuration file “journald.conf” for journalctl daemon.

journalctl – List Entries – Old Top

List all journal entry that is in the system will be displayed within a pager. The oldest entries will be displayed at top.

journalctl

journalctl – List Entries – New Top

List all journal entry that is in the system will be displayed within a pager. The newest entries will be displayed at top.

journalctl --reverse 

journalctl – List Entries – Tail

List all journal entry that is in the system will be displayed. Last 10 lines would be displayed, similar to running tail /var/log/messages.

journalctl -n

journalctl – List Entries – Tail N Lines

List all journal entry that is in the system will be displayed. Last 15 lines would be displayed, similar to running tail /var/log/messages -n 15.

journalctl -n 15

journalctl – List Entries – Tail Real-time

List all journal entry that is in the system will be displayed in real-time. Last 15 lines would be displayed, similar to running tail /var/log/messages -f.

journalctl -f

journalctl – List Entries – From a Time

List all journal entries that is in the system starting from specific time onwards.

journalctl --since 02:50

journalctl – List Entries – Current Boot

List all journal entries that is in the system starting from current boot.

journalctl -b

journalctl – List Entries – Kernel Logs

List all journal entries pertaining to that of kernel, that is in the system will be displayed.

journalctl -k

journalctl – List Entries – Between Timeframes

List all journal entries that is in the system starting from specific time onwards and until a specific time.

journalctl --since 02:00 --until 02:50

journalctl – List Entries – From Today

List all journal entries that is in the system for today only.

journalctl --since=today

journalctl – List Entries – From Yesterday

List all journal entries that is in the system for yesterday onwards and till now.

journalctl --since=yesterday

Filter Message – By UID (User ID)

List all journal entries that is in the system for a User ID.

journalctl _UID=1000

Filter Message – By GID (Group ID)

List all journal entries that is in the system for a Group ID.

journalctl _GID=1000

Filter Message – By PID (Process ID)

List all journal entries that is in the system for Process ID.

journalctl _PID=1

Filter Message – By Unit (service)

List all journal entries that is in the system for a unit name (service).

journalctl -u httpd.service

Filter Message – By Unit (service) - Verbose

List all journal entries that is in the system for a unit name (service) in verbose mode.

journalctl -f -o verbose UNIT=httpd.service

Filter Message – By Unit (service) - Debugging

List all journal entries that is in the system for a unit name (service) for debugging.

journalctl -f -u httpd.service -l

Filter Message – By Unit from today

Alternatively, you can filter messages by today, yesterday, since and until as well.

journalctl -u httpd.service --since=today

Filter Message – By Hours ago

You can filter messages by hours, minutes and seconds elapsed as well.

journalctl -u httpd.service --since "20 hours ago"

Filter Message – By Minutes ago

You can filter messages by minutes, hours and seconds elapsed as well.

journalctl -u httpd.service --since "1460 min ago"

Filter Message – By Seconds ago

you can filter messages by seconds, hour and minutes elapsed as well.

journalctl -u httpd.service --since "40 sec ago"

Filter Message – By DateTime

you can also filter messages by, from datetime to from datetime using the datetime format "yyyy-mm-dd hh:mm:ss" elapsed as well.

journalctl -u httpd --since "2015-11-16 23:15:00" --until "2015-11-17 23:20:00"

Filter Message – By Syslog Priority

List all journal entries that is in the system by syslog priority.

Value	Severity
0	Emergency
1	Alert
2	Critical
3	Error
4	Warning
5	Notice
6	Informational
7	Debug

Filter Message – By Priority Name

journalctl -p err

or

Filter Message – By Priority Number

journalctl -p 3

Filter Message – By Executable

journalctl /usr/bin/dbus-daemon

journalctl /usr/lib/systemd/systemd

Journalctl – Service Management

Start system-journald

systemctl start systemd-journald

Stop system-journald

systemctl stop systemd-journald

Note: You will not be able to stop journalctl service, because of dependency with systemd

Restart system-journald

systemctl restart systemd-journald

Status system-journald

systemctl status systemd-journald

Verify Corruption

Check the journal file for internal consistency.

journalctl --verify

Disk Space Consumed

To check disk space consumed by all the journals files.

journalctl --disk-usage

Slideshare Information

Guide to manage and use Journalctl is uploaded with screenshots.