2.25.4. Analysis of logs with console commands

You can analyze logs using console commands on a hosting or on a local PC:

Server response codes in descending order of their number:

awk '{print $9}' access.log | sort | uniq -c | sort -r

25 most active IPs:

cat access.log | awk '{ print $1 }' | sort | uniq -c | sort -rn | head -n 25

The number of requests from each IP in descending order:

cat access.log | awk '{print "requests from " $1}' | sort | uniq -c | sort -r

10 Most Popular Referer:

cat access.log | awk -F \" ' { print $4 } ' | grep -v '-' | sort | uniq -c | sort -rn | head -n 10

Top 10 User-Agents:

cat access.log | awk -F \" ' { print $6 } ' | sort | uniq -c | sort -rn | head -n 10

Total number of requests per day:

awk '{print $4}' access.log | cut -d: -f1 | uniq -c

Hourly number of requests per day:

• If the log contains information for only one day:

  cat access.log | cut -d [ -f2 | cut -d] -f1 | awk -F: '{print $2":00"}' | sort -n | uniq -c

• If the log contains information for several days (instead of DD/Mon substitute the desired day of the month and the first three letters of the month name in English):

  grep "DD/Mon" access.log | cut -d [ -f2 | cut -d] -f1 | awk -F: '{print $2":00"}' | sort -n | uniq -c

Per-minute number of requests for the specified hour of the specified day (instead of DD/Mon/YEAR:HH substitute the desired day of the month, the first three letters of the name of the month in English, year and hour):

grep "DD/Mon/YEAR:HH" access.log | cut -d [ -f2 | cut -d] -f1 | awk -F: '{print $2":"$3}' | sort -nk1 -nk2 | uniq -c | awk '{ if ($1 > 10) print $0}'

Number of unique visitors:

cat access.log | awk '{print $1}' | sort | uniq -c | wc -l

25 most popular URI:

cat access.log | awk '{ print $7 }' | sort | uniq -c | sort -rn | head -n 25

List of unique IPs:

cat access.log | awk '{print $1}' | sort | uniq

List of unique IPs with date and time for each request from them:

cat access.log | awk '{print $1 " " $4}' | sort | uniq

List of unique IPs with date, time and method for each request from them:

cat access.log | awk '{print $1 " " $4 " " $6}' | sort | uniq

List of unique IPs with date, time and URI for each request from them:

cat access.log | awk '{print $1 " " $4 " " $7}' | sort | uniq