Graylog search tips

Log types

Environments will automatically write several log types which can be searched with the log_type keyword in Graylog, as long as the project includes the cwp-core module.

• SilverStripe_log: standard log output of the Framework, will capture all events occurring after successful Framework bootstrap. This includes uncaught exceptions and any Injector::inst()->get(LoggerInterface::class)->... events.
• SilverStripe_audit: audit trail of security-related events provided by the silverstripe/auditor module.
• apache: apache access logs
• apache-errors: errors reported by Apache, which could include mod_php segmentation faults
• php: PHP errors logged by the PHP binary directly, such as command-line PHP execution
• postfix/error: Email error logs
• postfix/bounce: Email bounce logs

Searching

Each search requires a time period to search and query strings (the default is 5 minutes). The query string uses Graylog's query syntax which you can find out more about by reading Graylog documentation.

To get you started with Graylog there are some predefined searches you can select from the top right corner. 

Following are a couple of examples on search queries:

Find logs of a certain type

log_type:apache

Find web requests for the url "/about-us/"

http_url:"/about-us/"

Find web requests that begins with "/about-us/"

http_url:\/about-us\/*

Note that the following characters needs to be escaped with a backslash:
&& || : \ / + - ! ( ) { } [ ] ^ " ~ * ?

Find web requests that resulted in a 5xx error response

http_response:>=500

Find web requests that resulted in a 4xx error response

http_response:(>=400 AND <500)

Long loading pages (requests longer than 10 seconds)

log_type:apache AND http_resp_usec:>10000000

Large assets / pages (requests larger than 1mb)

log_type:apache AND http_bytes:>1000000

Identifying 404s (missing pages / assets)

log_type:apache AND http_response:404

Filter requests made by IP

log_type:apache AND http_clientip:"x.x.x.x"

Filter requests made by User agent

log_type:apache AND http_agent:"Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"

Authenticated user activity

log_type:SilverStripe_audit

Analysis of logs

Graylog provides several tools to analyse your search results. To analyse a field from your search results, expand the field in the search sidebar and click on the button of the analysis you want to perform.search analysis

Field statistics

You can compute different statistics on your fields, to help you better summarise and understand the data in them.

The statistical information consists of: total, mean, minimum, maximum, standard deviation, variance, sum, and cardinality. On non-numeric fields, you can only see the total amount of messages containing that field, and the cardinality of the field, i.e. the number of unique values it has.

Quick values

You can use quick values to help you find out the distribution of values for a field. Alongside a graphic representation of the common values contained in a field, Graylog will display a table with all different values, allowing you to see the number of times they appear. You can include any value in your search query by clicking on the magnifying glass icon located in the value row.quick values

Field graphs

You can create field graphs for any numeric field, by clicking on the Generate chart button in the search sidebar. Using the options in the Customise menu on top of the field graph, you can change the statistical function used in the graph, the kind of graph to use to represent the values, the graph interpolation, as well as the time resolution.stable