Spis Treści

Konfiguracja collectd
collectweb

Konfiguracja collectd

Wersja 5

Różnice w stosunku do wersji 4. Przede wszystkim korzystamy z "natywnego" panelu CGI zamiast "obcego". Jeśli chodzi o plik konfiguracyjny, to zmiany analogiczne do wersji 4.

Extra patch dla FreeBSD 10.1+

Zapisany jako /usr/ports/net-mgmt/collectd5/files/patch-src__utils_dns.c:

--- src/utils_dns.c.orig        2015-02-26 11:43:48.496180000 +0100
+++ src/utils_dns.c     2015-05-17 21:32:23.000000000 +0200
@@ -772,6 +772,11 @@
 # if __NAMESER >= 19991006
            case ns_t_tsig:     return ("TSIG");
 # endif
+# if __NAMESER >= 20090302
+           case ns_t_sshfp:    return ("SSHFP");
+           case ns_t_dnskey:   return ("DNSKEY");
+           case ns_t_spf:      return ("SPF");
+# endif
            case ns_t_ixfr:     return ("IXFR");
            case ns_t_axfr:     return ("AXFR");
            case ns_t_mailb:    return ("MAILB");

uruchomienie rrdcached

Ustawienie zmiennych w rc.conf

rrdcached_enable="YES"
rrdcached_flags="-s www -l unix:/var/run/rrdcached.sock -p /var/run/rrdcached.pid -j /var/db/rrdcached/journal -w 60"

A potem wystartowanie samego demona

mkdir -p /var/db/rrdcached/journal
service rrdcached start

collectd.conf

W samym collectd.conf trzeba ustawić następujące rzeczy:

<Plugin syslog>
        LogLevel warning
</Plugin>

<Plugin df>
        FSType "ufs"
        FSType "zfs"
        IgnoreSelected false
        ReportByDevice false
        ReportReserved false
        ReportInodes false
</Plugin>

# dyski ADA, a nie domyślnie ([as]d[0-9])
<Plugin disk>
        Disk "/^ada[0-9]?$/"
        IgnoreSelected false
</Plugin>

<Plugin ping>
        Host __ADRES_IP_DO_SPRAWDZANIA__
</Plugin>

# musi by/c przynajmniej komentarz - nie może być "pusto"
<Plugin processes>
#       Process "name"
</Plugin>

<Plugin tcpconns>
        ListeningPorts false
        LocalPort "22"
        LocalPort "80"
        LocalPort "443"
        LocalPort "25"
        LocalPort "110"
        LocalPort "143"
        LocalPort "993"
        LocalPort "995"
        RemotePort "25"
        RemotePort "80"
        RemotePort "443"
</Plugin>

Konfiguracja pluginu DNS

Konfiguracja w celu monitoringu DNS:

<Plugin dns>
        Interface "bge0"
        #IgnoreSource "192.168.0.1"
        SelectNumericQueryTypes false
</Plugin>

plugin load

Po aktualizacji collectd do 5.9.0 zmianie uległa konfiguracja pluginu load.

collectd.conf.d/load.conf

# --- za/ladowanie samego pluginu -----------------------------------
#
LoadPlugin load

# --- konfiguracja pluginu ------------------------------------------
#

Synchronizacja baz rrd - pozostałości po relative

Należy pobrać skrypt w pythonie, który sklei starą bazę load-relative.rrd z nową bazą load.rrd:

fetch https://gist.githubusercontent.com/arantius/2166343/raw/34cfb0f77f381073206fd3c2c356c77b097739e6/simple-rrd-merge.py -o /root/simple-rrd-merge.py

Zatrzymanie demonów

service collectdmon stop
service rrdcached stop

Sklejenie baz

# na przykładzie serwera HQ; katalogi docelowe mogą się różnić
python2.7 /root/simple-rrd-merge.py /var/db/collectd/rrd/rtm.bsdserwis.com/load/load-relative.rrd /var/db/collectd/rrd/rtm.bsdserwis.com/load/load.rrd | rrdtool restore /dev/stdin /root/load.rrd

mv /root/load.rrd /var/db/collectd/rrd/rtm.bsdserwis.com/load/load.rrd

Ponowne uruchomienie demonów

service rrdcached start
service collectdmon start

Konfiguracja pluginu mySQL

Konfiguracja w bazie mySQL

CREATE USER 'collectd'@'localhost' IDENTIFIED BY 'haslo_dla_collectd';
GRANT USAGE ON *.* TO 'collectd'@'localhost';

/* Byc moze nie potrzebne */
/* UPDATE user SET Repl_client_priv='Y' WHERE user='collectd'; */

FLUSH PRIVILEGES;

Konfiguracja collectd.conf

<Plugin mysql>
        <Database test>
                Host "localhost"
                User "collectd"
                Password "uptUrt5wekdoyswo"
                Database "test"
                MasterStats false
        </Database>
</Plugin>

Plugin OpenLDAP

slapd.conf

Dopisanie załadowania modułu, aktywacja "bazy" oraz konfiguracja ACL:

moduleload      back_monitor.la

# [...]
access to dn.subtree="cn=Monitor"
        by dn="cn=lokietek,dc=gpharma,dc=local" write
        by * read

# [...]
database        monitor

database        mdb
# [...]

collectd.conf

LoadPlugin openldap

# [...]
<Plugin "openldap">
    <Instance "localhost">
        URL "ldap://localhost/"
    </Instance>
</Plugin>

Poprawki w collection.modified.cgi

W pliku do wyświetlania statystyk trzeba dopisać (chyba, że już jest zrobione) ok. linii 3350:

    $GraphDefs->{'current_connections'} = $GraphDefs->{'connections'};
    $GraphDefs->{'total_connections'}   = $GraphDefs->{'connections'};

Konfiguracja pluginu postgresql

Konfiguracja postgresql.conf

W pliku postgresql.conf należy zapewnić następującą konfigurację (ustawienia w sekcji jak pokazano):

#------------------------------------------------------------------------------
# RUNTIME STATISTICS
#------------------------------------------------------------------------------

# - Query/Index Statistics Collector -

track_activities = on
track_counts = on
track_io_timing = on
track_functions = none                  # none, pl, all

Na wszelki wypadek trzeba wykonać restart serwera:

service postgresql stop
service postgresql start

Utworzenie użytkownika w bazie danych

createuser -U pgsql -DESRP collectd

collectd.conf

LoadPlugin postgresql
# [...]
<Plugin postgresql>
  <Query custom_deadlocks>
      Statement "SELECT deadlocks as num_deadlocks \
          FROM pg_stat_database \
          WHERE datname = $1;"
      Param database
      <Result>
          Type "pg_xact"
          InstancePrefix "num_deadlocks"
          ValuesFrom "num_deadlocks"
      </Result>
  </Query>
  <Query database_size>
    Statement "SELECT datname AS name, \
               pg_catalog.pg_database_size(datname) AS size \
               FROM pg_catalog.pg_database;"
    <Result>
        Type pg_db_size
        InstancePrefix "dbsize"
        InstancesFrom "name"
        ValuesFrom "size"
    </Result>
  </Query>

  # ----------------------------------------------------------------------
  #
  #                   PAMI/ETAJ O PONI/ZSZYCH
  #
  # ----------------------------------------------------------------------
  #
  # 1. w bazie postgres nie ma danych z tabeli pg_stat_user_tables, gdy/z
  # baza postgres jest baz/a systemow/a. Zamiast tego trzeba u/zy/c bazy,
  # kt/or/a chce si/e monitorowa/c, np. dspam_db
  #
  # 2. U/zytkownik collectd musi mie/c nadane uprawnienia do wykonywania
  # SELECT-/ow, aby pokazywa/ly si/e jakiekolwiek statystyki. Najlepiej to
  # zrobi/c poprzez:
  # echo "GRANT SELECT ON ALL TABLES IN SCHEMA public TO collectd;" | psql -U pgsql <database>
  #
  # 3. Zapytania "domy/slne" (kt/orych definicji nie ma powy/zej) s/a
  # umieszczone w pliku /usr/local/share/collectd/postgresql_default.conf
  #

  <Database postgres>
    Host "localhost"
    User "collectd"
    Password "__haslo_nadane_createuser__"
    Query custom_deadlocks
    Query backends
    Query transactions
    Query queries
    Query queries_by_table
    Query query_plans
    Query table_states
    Query query_plans_by_table
    Query table_states_by_table
    Query disk_io
    Query disk_io_by_table
    Query disk_usage
  </Database>
</Plugin>

Plugin Redis

collectd.conf

LoadPlugin redis

# [...]
<Plugin "redis">
    <Node "localhost">
        Host "127.0.0.1"
        Port "6379"
    </Node>
</Plugin>

Poprawki w collection.modified.cgi

W pliku do wyświetlania statystyk trzeba dopisać (chyba, że już jest zrobione) ok. linii 3352:

    $GraphDefs->{'blocked_clients'}     = $GraphDefs->{'gauge'};
    $GraphDefs->{'evicted_keys'}        = $GraphDefs->{'derive'};
    $GraphDefs->{'expired_keys'}        = $GraphDefs->{'derive'};
    $GraphDefs->{'memory_lua'}          = $GraphDefs->{'gauge'};
    $GraphDefs->{'operations_per_second'}    = $GraphDefs->{'gauge'};
    $GraphDefs->{'volatile_changes'}    = $GraphDefs->{'gauge'};

    $MetaGraphDefs->{'pubsub'}          = \&meta_graph_memory;

Poprawki w katalogu z bazami rrd

Jedną z baz RRD, które powstają jest memory.rrd. Niemniej schemat generowania wykresów bazuje na plikach memory-xxxxxx.rrd. Aby wykresy były pokazywane prawidłowo należy w katalogu /redis wykonać link symboliczny:

cd /var/db/collectd/rdd/__hostname__/redis-local/
ln -s memory.rrd memory-redis.rrd

apache.conf

Konfiguracja dla Apache składa się z dwóch elementów: definicji katalogu¹ oraz ScriptAlias dla wybranej wirtualki:

# --- definicja katalog/ow ze stronkami -----------------------------
#
<Directory /usr/local/www/collectd>
    Options ExecCGI
    Order allow,deny
    Allow from all
</Directory>

ScriptAlias /stats/ "/usr/local/www/collectd/collection.cgi"

collection.cgi

Oryginalny skrypt jest nieco zmodyfikowanym skryptem z poprzedniej wersji (v4) collectd. Tak więc wymaga dużej liczby poprawek:

--- /usr/local/www/collectd/collection.cgi      2014-07-24 15:50:23.459797000 +0200
+++ /usr/local/www/collectd/collection.cgi.new  2014-07-24 18:16:55.532191754 +0200
@@ -957,9 +957,9 @@

   $GraphDefs =
   {
-    apache_bytes => ['DEF:min_raw={file}:count:MIN',
-    'DEF:avg_raw={file}:count:AVERAGE',
-    'DEF:max_raw={file}:count:MAX',
+    apache_bytes => ['DEF:min_raw={file}:value:MIN',
+    'DEF:avg_raw={file}:value:AVERAGE',
+    'DEF:max_raw={file}:value:MAX',
     'CDEF:min=min_raw,8,*',
     'CDEF:avg=avg_raw,8,*',
     'CDEF:max=max_raw,8,*',
@@ -976,9 +976,9 @@
     'GPRINT:avg:LAST:%5.1lf%s Last',
     'GPRINT:avg_sum:LAST:(ca. %5.1lf%sB Total)\l'
     ],
-   apache_connections => ['DEF:min={file}:count:MIN',
-    'DEF:avg={file}:count:AVERAGE',
-    'DEF:max={file}:count:MAX',
+   apache_connections => ['DEF:min={file}:value:MIN',
+    'DEF:avg={file}:value:AVERAGE',
+    'DEF:max={file}:value:MAX',
     "AREA:max#$HalfBlue",
     "AREA:min#$Canvas",
     "LINE1:avg#$FullBlue:Connections",
@@ -987,9 +987,9 @@
     'GPRINT:max:MAX:%6.2lf Max,',
     'GPRINT:avg:LAST:%6.2lf Last'
     ],
-    apache_idle_workers => ['DEF:min={file}:count:MIN',
-    'DEF:avg={file}:count:AVERAGE',
-    'DEF:max={file}:count:MAX',
+    apache_idle_workers => ['DEF:min={file}:value:MIN',
+    'DEF:avg={file}:value:AVERAGE',
+    'DEF:max={file}:value:MAX',
     "AREA:max#$HalfBlue",
     "AREA:min#$Canvas",
     "LINE1:avg#$FullBlue:Idle Workers",
@@ -998,9 +998,9 @@
     'GPRINT:max:MAX:%6.2lf Max,',
     'GPRINT:avg:LAST:%6.2lf Last'
     ],
-    apache_requests => ['DEF:min={file}:count:MIN',
-    'DEF:avg={file}:count:AVERAGE',
-    'DEF:max={file}:count:MAX',
+    apache_requests => ['DEF:min={file}:value:MIN',
+    'DEF:avg={file}:value:AVERAGE',
+    'DEF:max={file}:value:MAX',
     "AREA:max#$HalfBlue",
     "AREA:min#$Canvas",
     "LINE1:avg#$FullBlue:Requests/s",
@@ -1009,9 +1009,9 @@
     'GPRINT:max:MAX:%6.2lf Max,',
     'GPRINT:avg:LAST:%6.2lf Last'
     ],
-    apache_scoreboard => ['DEF:min={file}:count:MIN',
-    'DEF:avg={file}:count:AVERAGE',
-    'DEF:max={file}:count:MAX',
+    apache_scoreboard => ['DEF:min={file}:value:MIN',
+    'DEF:avg={file}:value:AVERAGE',
+    'DEF:max={file}:value:MAX',
     "AREA:max#$HalfBlue",
     "AREA:min#$Canvas",
     "LINE1:avg#$FullBlue:Processes",
@@ -1081,12 +1081,12 @@
     'GPRINT:avg:LAST:%5.1lf%sA Last\l'
     ],
     df => ['-v', 'Percent', '-l', '0',
-    'DEF:free_avg={file}:free:AVERAGE',
-    'DEF:free_min={file}:free:MIN',
-    'DEF:free_max={file}:free:MAX',
-    'DEF:used_avg={file}:used:AVERAGE',
-    'DEF:used_min={file}:used:MIN',
-    'DEF:used_max={file}:used:MAX',
+    'DEF:free_avg={file}:value:AVERAGE',
+    'DEF:free_min={file}:value:MIN',
+    'DEF:free_max={file}:value:MAX',
+    'DEF:used_avg={file}:value:AVERAGE',
+    'DEF:used_min={file}:value:MIN',
+    'DEF:used_max={file}:value:MAX',
     'CDEF:total=free_avg,used_avg,+',
     'CDEF:free_pct=100,free_avg,*,total,/',
     'CDEF:used_pct=100,used_avg,*,total,/',
@@ -2525,9 +2525,9 @@
     'GPRINT:multimeter_avg:LAST:%4.1lf Last\l'
     ],
     users => ['-v', 'Users',
-    'DEF:users_avg={file}:users:AVERAGE',
-    'DEF:users_min={file}:users:MIN',
-    'DEF:users_max={file}:users:MAX',
+    'DEF:users_avg={file}:value:AVERAGE',
+    'DEF:users_min={file}:value:MIN',
+    'DEF:users_max={file}:value:MAX',
     "AREA:users_max#$HalfBlue",
     "AREA:users_min#$Canvas",
     "LINE1:users_avg#$FullBlue:Users",
@@ -2666,7 +2666,117 @@
     'GPRINT:max:MAX:%4.1lf Max,',
     'GPRINT:avg:LAST:%4.1lf Last\l'
     ],
+    cache_size => [
+    'DEF:min={file}:value:MIN',
+    'DEF:avg={file}:value:AVERAGE',
+    'DEF:max={file}:value:MAX',
+    "AREA:max#$HalfBlue",
+    "AREA:min#$Canvas",
+    "LINE1:avg#$FullBlue:ARC cache size (Bytes)",
+    'GPRINT:min:MIN:%6.2lf%s Min,',
+    'GPRINT:avg:AVERAGE:%6.2lf%s Avg,',
+    'GPRINT:max:MAX:%6.2lf%s Max,',
+    'GPRINT:avg:LAST:%6.2lf%s Last'
+    ],
+    cache_result => [
+    'DEF:min={file}:value:MIN',   'DEF:avg={file}:value:AVERAGE',
+    'DEF:max={file}:value:MAX',   "AREA:max#$HalfBlue",
+    "AREA:min#$Canvas",           "LINE1:avg#$FullBlue:Requests/s",
+    'GPRINT:min:MIN:%6.2lf Min,', 'GPRINT:avg:AVERAGE:%6.2lf Avg,',
+    'GPRINT:max:MAX:%6.2lf Max,', 'GPRINT:avg:LAST:%6.2lf Last'
+    ],
+    io_octets => [
+    'DEF:out_min={file}:tx:MIN',
+    'DEF:out_avg={file}:tx:AVERAGE',
+    'DEF:out_max={file}:tx:MAX',
+    'DEF:inc_min={file}:rx:MIN',
+    'DEF:inc_avg={file}:rx:AVERAGE',
+    'DEF:inc_max={file}:rx:MAX',
+    'CDEF:overlap=out_avg,inc_avg,GT,inc_avg,out_avg,IF',
+    'CDEF:mytime=out_avg,TIME,TIME,IF',
+    'CDEF:sample_len_raw=mytime,PREV(mytime),-',
+    'CDEF:sample_len=sample_len_raw,UN,0,sample_len_raw,IF',
+    'CDEF:out_avg_sample=out_avg,UN,0,out_avg,IF,sample_len,*',
+    'CDEF:out_avg_sum=PREV,UN,0,PREV,IF,out_avg_sample,+',
+    'CDEF:inc_avg_sample=inc_avg,UN,0,inc_avg,IF,sample_len,*',
+    'CDEF:inc_avg_sum=PREV,UN,0,PREV,IF,inc_avg_sample,+',
+    "AREA:out_avg#$HalfGreen",
+    "AREA:inc_avg#$HalfBlue",
+    "AREA:overlap#$HalfBlueGreen",
+    "LINE1:out_avg#$FullGreen:Written",
+    'GPRINT:out_avg:AVERAGE:%5.1lf%s Avg,',
+    'GPRINT:out_max:MAX:%5.1lf%s Max,',
+    'GPRINT:out_avg:LAST:%5.1lf%s Last',
+    'GPRINT:out_avg_sum:LAST:(ca. %5.1lf%sB Total)\l',
+    "LINE1:inc_avg#$FullBlue:Read   ",
+    'GPRINT:inc_avg:AVERAGE:%5.1lf%s Avg,',
+    'GPRINT:inc_max:MAX:%5.1lf%s Max,',
+    'GPRINT:inc_avg:LAST:%5.1lf%s Last',
+    'GPRINT:inc_avg_sum:LAST:(ca. %5.1lf%sB Total)\l'
+    ],
+    counter => [
+    '-v',
+    'Count',
+    'DEF:avg={file}:value:AVERAGE',
+    'DEF:min={file}:value:MIN',
+    'DEF:max={file}:value:MAX',
+    "AREA:max#$HalfRed",
+    "AREA:min#$Canvas",
+    "LINE1:max#$FullRed:Count ",
+    'GPRINT:min:MIN:%4.1lf Min,',
+    'GPRINT:avg:AVERAGE:%4.1lf Avg,',
+    'GPRINT:max:MAX:%4.1lf Max,',
+    'GPRINT:avg:LAST:%4.1lf Last\l'
+    ],
+    derive => [
+    '-v',
+    'Derive',
+    'DEF:min={file}:value:MIN',
+    'DEF:avg={file}:value:AVERAGE',
+    'DEF:max={file}:value:MAX',
+    'CDEF:mytime=avg,TIME,TIME,IF',
+    'CDEF:sample_len_raw=mytime,PREV(mytime),-',
+    'CDEF:sample_len=sample_len_raw,UN,0,sample_len_raw,IF',
+    'CDEF:avg_sample=avg,UN,0,avg,IF,sample_len,*',
+    'CDEF:avg_sum=PREV,UN,0,PREV,IF,avg_sample,+',
+    "AREA:max#$HalfRed",
+    "AREA:min#$Canvas",
+    "LINE1:avg#$FullRed:Derive ",
+    'GPRINT:avg:AVERAGE:%5.1lf%s Avg,',
+    'GPRINT:max:MAX:%5.1lf%s Max,',
+    'GPRINT:avg:LAST:%5.1lf%s Last',
+    'GPRINT:avg_sum:LAST:(ca. %5.1lf%s Total)\l'
+    ],
+    uptime => [
+    '-v',
+    'Days',
+    'DEF:raw={file}:value:MAX',
+    'CDEF:out_val=raw,86400,/',
+    'CDEF:out_days=out_val,FLOOR',
+    'CDEF:out_hours=out_val,out_days,-,24,*,FLOOR',
+    'CDEF:out_mins=out_val,out_days,-,24,*,out_hours,-,60,*,FLOOR',
+    'CDEF:out_secs=out_val,out_days,-,24,*,out_hours,-,60,*,out_mins,-,60,*',
+    "AREA:out_val#$HalfBlue",
+    "LINE1:out_val#$FullBlue:Uptime",
+    'GPRINT:out_days:LAST:%6.0lf day(s),',
+    'GPRINT:out_hours:LAST:%6.0lf hour(s),',
+    'GPRINT:out_mins:LAST:%6.0lf min(s),',
+    'GPRINT:out_secs:LAST:%6.0lf sec(s)\l'
+    ],
+    bytes => ['-v', 'Bytes',
+    'DEF:avg={file}:value:AVERAGE',
+    'DEF:min={file}:value:MIN',
+    'DEF:max={file}:value:MAX',
+    "AREA:max#$HalfGreen",
+    "AREA:min#$Canvas",
+    "LINE1:avg#$FullGreen:Bytes",
+    'GPRINT:min:MIN:%4.1lf Min,',
+    'GPRINT:avg:AVERAGE:%4.1lf Avg,',
+    'GPRINT:max:MAX:%4.1lf Max,',
+    'GPRINT:avg:LAST:%4.1lf Last\l'
+    ],
   };
+
   $GraphDefs->{'if_multicast'} = $GraphDefs->{'ipt_packets'};
   $GraphDefs->{'if_tx_errors'} = $GraphDefs->{'if_rx_errors'};
   $GraphDefs->{'dns_qtype'} = $GraphDefs->{'dns_opcode'};
@@ -2674,6 +2784,13 @@
   $GraphDefs->{'vmpage_io-memory'} = $GraphDefs->{'vmpage_io'};
   $GraphDefs->{'vmpage_io-swap'} = $GraphDefs->{'vmpage_io'};
   $GraphDefs->{'virt_cpu_total'} = $GraphDefs->{'virt_cpu_total'};
+  $GraphDefs->{'cache_ratio'} = $GraphDefs->{'cache_result'};
+  $GraphDefs->{'cache_eviction'} = $GraphDefs->{'cache_result'};
+  $GraphDefs->{'cache_operation'} = $GraphDefs->{'cache_result'};
+  $GraphDefs->{'mutex_operations'} = $GraphDefs->{'cache_result'};
+  $GraphDefs->{'hash_collisions'} = $GraphDefs->{'cache_result'};
+  $GraphDefs->{'operations'} = $GraphDefs->{'derive'};
+  $GraphDefs->{'queue_length'} = $GraphDefs->{'gauge'};

   $MetaGraphDefs->{'cpu'} = \&meta_graph_cpu;
   $MetaGraphDefs->{'dns_qtype'} = \&meta_graph_dns;
@@ -2684,6 +2801,7 @@
   $MetaGraphDefs->{'nfs_procedure'} = \&meta_graph_nfs_procedure;
   $MetaGraphDefs->{'ps_state'} = \&meta_graph_ps_state;
   $MetaGraphDefs->{'swap'} = \&meta_graph_swap;
+  $MetaGraphDefs->{'df_complex'} = \&meta_graph_swap;
   $MetaGraphDefs->{'mysql_commands'} = \&meta_graph_mysql_commands;
   $MetaGraphDefs->{'mysql_handler'} = \&meta_graph_mysql_commands;
   $MetaGraphDefs->{'tcp_connections'} = \&meta_graph_tcp_connections;

Dodanie użytkownika oraz grupy na potrzeby skryptów - plugin exec

pw add group collectd
pw add user collectd -g collectd -d /var/www/htdocs/virtuals/collectd -s /sbin/nologin

# albo 
pw add user collectd -g collectd -d /usr/local/www/collectd-web -s /sbin/nologin

Sprawdzenie konfiguracji collectd

collectd -t

collectweb

Jest to zmodyfikowany skrypt CGI do prezentowania statystyk collectd.

Źródła do pobrania

Strona domowa projektu, to:

wget --no-check-certificate https://github.com/httpdss/collectd-web/archive/0.4.1.tar.gz

Przygotowania w systemie

Z wykorzystaniem własnego repozytorium pakietów instalujemy:

pkg install converters/p5-JSON p5-HTML-Parser p5-URI www/p5-CGI

Dodatkowo partycja, gdzie są skrypty CGI-BIN nie może mieć wyłączonej opcji exec (np. zfs). Inaczej w logach będzie komunikat typu:

(13)Permission denied: exec of '/var/www/htdocs/virtuals/collectd/data/cgi-bin/collection.modified.cgi' failed,

Łatki, łatki do skryptów CGI

Poniżej lista poprawek, które trzeba nałożyć na oryginalne pliki w celu poprawnego działania. Chodzi głównie o ścieżki (zmiany w PERLu) jak również rozwój samego Collect Demona.

Łatanie collection.modified.cgi

$/!\$ Stan na dzień 07.12.2019 (dodanie pg_db_size):

--- collection.modified.cgi.orig        2012-12-31 01:51:09.000000000 +0100
+++ collection.modified.cgi     2019-12-07 11:40:28.656518000 +0100
@@ -1,4 +1,4 @@
-#!/usr/bin/env perl
+#!/usr/local/bin/perl
 # Collectd-web - collection.modified.cgi
 # Copyright (C) 2006-2010  Florian "octo" Forster
 # Copyright (C) 2009-2010  Kenneth Belitzky
@@ -27,7 +27,7 @@ use URI::Escape    ('uri_escape');
 use RRDs           ();
 use Data::Dumper   ();
 use JSON ('to_json');
-our $Config   = "/etc/collectd/collection.conf";
+our $Config   = "/usr/local/etc/collectd-web.conf";
 our @DataDirs = ();
 our $LibDir;
 our $make_transparent = 1;
@@ -39,6 +39,9 @@ our $ValidTimespan    = {
     year  => 366 * 86400
 };

+# --- wy/l/aczenie logowania b/l/ed/ow w logacha serwera www. U/zywa/c z rozs/adkiem - s/lu/zy do poprawek
+#
+##$CGI::LIST_CONTEXT_WARN = 0;

 our @RRDDefaultArgs = (
 '--rigid', '-w', '500', '-h', '160', '--alt-autoscale-max', '--alt-y-grid',
@@ -191,7 +194,9 @@ sub _get_menu_buttons {
 sub _get_param_host {
     my %all_hosts = map { $_ => 1 } ( _find_hosts() );
     my @selected_hosts = ();
-    for ( param('host') ) {
+    my @loc_hosts = ();
+    push @loc_hosts, scalar param('host');
+    for ( @loc_hosts ) {
         if ( defined( $all_hosts{$_} ) ) {
             push( @selected_hosts, "$_" );
         }
@@ -619,7 +624,7 @@ sub action_show_plugin_json {
         $plugins_per_host->{ $hosts[$i] } = _find_files_for_host( $hosts[$i] );
         _files_union( $all_plugins, $plugins_per_host->{ $hosts[$i] } );
     }
-    for ( param('plugin') ) {
+    for ( scalar param('plugin') ) {
         if ( defined( $all_plugins->{$_} ) ) {
             $selected_plugins->{$_} = 1;
         }
@@ -1059,6 +1064,35 @@ sub load_graph_definitions {
             'GPRINT:max:MAX:%6.2lf Max,',
             'GPRINT:avg:LAST:%6.2lf Last'
         ],
+        io_octets => [
+        'DEF:out_min={file}:tx:MIN',
+        'DEF:out_avg={file}:tx:AVERAGE',
+        'DEF:out_max={file}:tx:MAX',
+        'DEF:inc_min={file}:rx:MIN',
+        'DEF:inc_avg={file}:rx:AVERAGE',
+        'DEF:inc_max={file}:rx:MAX',
+        'CDEF:overlap=out_avg,inc_avg,GT,inc_avg,out_avg,IF',
+        'CDEF:mytime=out_avg,TIME,TIME,IF',
+        'CDEF:sample_len_raw=mytime,PREV(mytime),-',
+        'CDEF:sample_len=sample_len_raw,UN,0,sample_len_raw,IF',
+        'CDEF:out_avg_sample=out_avg,UN,0,out_avg,IF,sample_len,*',
+        'CDEF:out_avg_sum=PREV,UN,0,PREV,IF,out_avg_sample,+',
+        'CDEF:inc_avg_sample=inc_avg,UN,0,inc_avg,IF,sample_len,*',
+        'CDEF:inc_avg_sum=PREV,UN,0,PREV,IF,inc_avg_sample,+',
+        "AREA:out_avg#$HalfGreen",
+        "AREA:inc_avg#$HalfBlue",
+        "AREA:overlap#$HalfBlueGreen",
+        "LINE1:out_avg#$FullGreen:Written",
+        'GPRINT:out_avg:AVERAGE:%5.1lf%s Avg,',
+        'GPRINT:out_max:MAX:%5.1lf%s Max,',
+        'GPRINT:out_avg:LAST:%5.1lf%s Last',
+        'GPRINT:out_avg_sum:LAST:(ca. %5.1lf%sB Total)\l',
+        "LINE1:inc_avg#$FullBlue:Read   ",
+        'GPRINT:inc_avg:AVERAGE:%5.1lf%s Avg,',
+        'GPRINT:inc_max:MAX:%5.1lf%s Max,',
+        'GPRINT:inc_avg:LAST:%5.1lf%s Last',
+        'GPRINT:inc_avg_sum:LAST:(ca. %5.1lf%sB Total)\l'
+        ],
         cache_result => [
             'DEF:min={file}:value:MIN',
             'DEF:avg={file}:value:AVERAGE',
@@ -1071,6 +1105,18 @@ sub load_graph_definitions {
             'GPRINT:max:MAX:%6.2lf Max,',
             'GPRINT:avg:LAST:%6.2lf Last'
         ],
+        cache_size => [
+        'DEF:min={file}:value:MIN',
+        'DEF:avg={file}:value:AVERAGE',
+        'DEF:max={file}:value:MAX',
+        "AREA:max#$HalfBlue",
+        "AREA:min#$Canvas",
+        "LINE1:avg#$FullBlue:ARC cache size (Bytes)",
+        'GPRINT:min:MIN:%6.2lf%s Min,',
+        'GPRINT:avg:AVERAGE:%6.2lf%s Avg,',
+        'GPRINT:max:MAX:%6.2lf%s Max,',
+        'GPRINT:avg:LAST:%6.2lf%s Last'
+        ],
         apache_scoreboard => [
             'DEF:min={file}:value:MIN',
             'DEF:avg={file}:value:AVERAGE',
@@ -1356,6 +1402,34 @@ sub load_graph_definitions {
             'GPRINT:inc_max:MAX:%6.2lf Max,',
             'GPRINT:inc_avg:LAST:%6.2lf Last\l'
         ],
+        disk_io_time => [
+            '-v',
+            'Seconds/s',
+            'DEF:out_min_raw={file}:io_time:MIN',
+            'DEF:out_avg_raw={file}:io_time:AVERAGE',
+            'DEF:out_max_raw={file}:io_time:MAX',
+            'DEF:inc_min_raw={file}:weighted_io_time:MIN',
+            'DEF:inc_avg_raw={file}:weighted_io_time:AVERAGE',
+            'DEF:inc_max_raw={file}:weighted_io_time:MAX',
+            'CDEF:out_min=out_min_raw,1000,/',
+            'CDEF:out_avg=out_avg_raw,1000,/',
+            'CDEF:out_max=out_max_raw,1000,/',
+            'CDEF:inc_min=inc_min_raw,1000,/',
+            'CDEF:inc_avg=inc_avg_raw,1000,/',
+            'CDEF:inc_max=inc_max_raw,1000,/',
+            'CDEF:overlap=out_avg,inc_avg,GT,inc_avg,out_avg,IF',
+            "AREA:out_avg#$HalfGreen",
+            "AREA:inc_avg#$HalfBlue",
+            "AREA:overlap#$HalfBlueGreen",
+            "LINE1:out_avg#$FullGreen:IO time",
+            'GPRINT:out_avg:AVERAGE:%5.1lf%ss Avg,',
+            'GPRINT:out_max:MAX:%5.1lf%ss Max,',
+            'GPRINT:out_avg:LAST:%5.1lf%ss Last\l',
+            "LINE1:inc_avg#$FullBlue:Weighted IO time   ",
+            'GPRINT:inc_avg:AVERAGE:%5.1lf%ss Avg,',
+            'GPRINT:inc_max:MAX:%5.1lf%ss Max,',
+            'GPRINT:inc_avg:LAST:%5.1lf%ss Last\l'
+        ],
         disk_time => [
             '-v',
             'Seconds/s',
@@ -3176,6 +3250,18 @@ sub load_graph_definitions {
             'GPRINT:max:MAX:%6.2lf Max,',
             'GPRINT:avg:LAST:%6.2lf Last'
         ],
+        mysql_slow_queries => [
+            'DEF:min={file}:value:MIN',
+            'DEF:avg={file}:value:AVERAGE',
+            'DEF:max={file}:value:MAX',
+            "AREA:max#$HalfBlue",
+            "AREA:min#$Canvas",
+            "LINE1:avg#$FullBlue:Queries/s",
+            'GPRINT:min:MIN:%6.2lf Min,',
+            'GPRINT:avg:AVERAGE:%6.2lf Avg,',
+            'GPRINT:max:MAX:%6.2lf Max,',
+            'GPRINT:avg:LAST:%6.2lf Last'
+        ],
 # jaf-19aug11 END
 # jaf-18aug11 additional Varnish graphs
         threads => [
@@ -3215,13 +3301,30 @@ sub load_graph_definitions {
             'GPRINT:avg:LAST:%6.2lf Last'
         ],
 # jaf-18aug11 END
-
-
+        bytes => ['-v', 'Bytes',
+        'DEF:avg={file}:value:AVERAGE',
+        'DEF:min={file}:value:MIN',
+        'DEF:max={file}:value:MAX',
+        "AREA:max#$HalfGreen",
+        "AREA:min#$Canvas",
+        "LINE1:avg#$FullGreen:Bytes",
+        'GPRINT:min:MIN:%4.1lf Min,',
+        'GPRINT:avg:AVERAGE:%4.1lf Avg,',
+        'GPRINT:max:MAX:%4.1lf Max,',
+        'GPRINT:avg:LAST:%4.1lf Last\l'
+        ],
     };
+    $GraphDefs->{'ps_code'}             = $GraphDefs->{'ps_state'};
+    $GraphDefs->{'ps_data'}             = $GraphDefs->{'ps_state'};
+    $GraphDefs->{'ps_stacksize'}        = $GraphDefs->{'ps_state'};
+    $GraphDefs->{'ps_vm'}               = $GraphDefs->{'ps_state'};
     $GraphDefs->{'if_multicast'}        = $GraphDefs->{'ipt_packets'};
     $GraphDefs->{'if_tx_errors'}        = $GraphDefs->{'if_rx_errors'};
     $GraphDefs->{'dns_qtype'}           = $GraphDefs->{'dns_opcode'};
     $GraphDefs->{'dns_rcode'}           = $GraphDefs->{'dns_opcode'};
+    $GraphDefs->{'full_join'}        = $GraphDefs->{'mysql_qcache'};
+    $GraphDefs->{'mysql_sort_merge_passes'} = $GraphDefs->{'mysql_slow_queries'};
+    $GraphDefs->{'mysql_sort_rows'} = $GraphDefs->{'mysql_slow_queries'};
 # jaf-18aug11 additional Varnish graphs
     $GraphDefs->{'memcached_command'}       = $GraphDefs->{'total_operations'};
     $GraphDefs->{'memcached_connections'}   = $GraphDefs->{'apache_connections'};
@@ -3234,6 +3337,26 @@ sub load_graph_definitions {
 # jaf-18aug11 END
     $GraphDefs->{'vmpage_io-memory'}    = $GraphDefs->{'vmpage_io'};
     $GraphDefs->{'vmpage_io-swap'}      = $GraphDefs->{'vmpage_io'};
+    $GraphDefs->{'cache_ratio'}         = $GraphDefs->{'cache_result'};
+    $GraphDefs->{'cache_eviction'}      = $GraphDefs->{'cache_result'};
+    $GraphDefs->{'cache_operation'}     = $GraphDefs->{'cache_result'};
+    $GraphDefs->{'mutex_operations'}    = $GraphDefs->{'cache_result'};
+    $GraphDefs->{'memory_throttle_count'} = $GraphDefs->{'cache_result'};
+    $GraphDefs->{'hash_collisions'}     = $GraphDefs->{'cache_result'};
+    $GraphDefs->{'operations'}         = $GraphDefs->{'derive'};
+    $GraphDefs->{'queue_length'}       = $GraphDefs->{'gauge'};
+    $GraphDefs->{'records'}            = $GraphDefs->{'gauge'};
+    $GraphDefs->{'total_values'}       = $GraphDefs->{'gauge'};
+    $GraphDefs->{'total_events'}       = $GraphDefs->{'gauge'};
+    $GraphDefs->{'current_connections'} = $GraphDefs->{'connections'};
+    $GraphDefs->{'total_connections'}   = $GraphDefs->{'connections'};
+    $GraphDefs->{'blocked_clients'}     = $GraphDefs->{'gauge'};
+    $GraphDefs->{'evicted_keys'}        = $GraphDefs->{'derive'};
+    $GraphDefs->{'expired_keys'}        = $GraphDefs->{'derive'};
+    $GraphDefs->{'memory_lua'}          = $GraphDefs->{'gauge'};
+    $GraphDefs->{'operations_per_second'}    = $GraphDefs->{'gauge'};
+    $GraphDefs->{'volatile_changes'}    = $GraphDefs->{'gauge'};
+    $GraphDefs->{'pending_operations'} = $GraphDefs->{'derive'};
     $MetaGraphDefs->{'cpu'}             = \&meta_graph_cpu;
     $MetaGraphDefs->{'df_complex'}      = \&meta_graph_df_complex;
     $MetaGraphDefs->{'dns_qtype'}       = \&meta_graph_dns;
@@ -3241,11 +3364,15 @@ sub load_graph_definitions {
     $MetaGraphDefs->{'if_rx_errors'}    = \&meta_graph_if_rx_errors;
     $MetaGraphDefs->{'if_tx_errors'}    = \&meta_graph_if_rx_errors;
     $MetaGraphDefs->{'memory'}          = \&meta_graph_memory;
+    $MetaGraphDefs->{'pubsub'}          = \&meta_graph_memory;
     $MetaGraphDefs->{'nfs_procedure'}   = \&meta_graph_nfs_procedure;
+    $MetaGraphDefs->{'pg_db_size'}      = \&meta_graph_pg_db_size;
     $MetaGraphDefs->{'ps_state'}        = \&meta_graph_ps_state;
     $MetaGraphDefs->{'swap'}            = \&meta_graph_swap;
     $MetaGraphDefs->{'mysql_commands'}  = \&meta_graph_mysql_commands;
     $MetaGraphDefs->{'mysql_handler'}   = \&meta_graph_mysql_commands;
+    $MetaGraphDefs->{'mysql_select'}    = \&meta_graph_mysql_commands;
+    $MetaGraphDefs->{'mysql_sort'}      = \&meta_graph_mysql_commands;
     $MetaGraphDefs->{'tcp_connections'} = \&meta_graph_tcp_connections;
     $MetaGraphDefs->{'vmpage_number'}   = \&meta_graph_vmpage_number;
     $MetaGraphDefs->{'vmpage_action'}   = \&meta_graph_vmpage_action;
@@ -3468,7 +3595,7 @@ sub meta_graph_dns

   @$type_instances = sort @$type_instances;

-  $opts->{'colors'} = _get_n_colors ($type_instances);
+  ###$opts->{'colors'} = _get_n_colors ($type_instances);

   for (@$type_instances)
   {
@@ -3657,6 +3784,45 @@ sub meta_graph_nfs_procedure {
     }    # for (@$type_instances)
     return ( meta_graph_generic_stack( $opts, $sources ) );
 }    # meta_graph_nfs_procedure
+
+sub meta_graph_pg_db_size {
+    confess("Wrong number of arguments") if ( @_ != 5 );
+    my $host            = shift;
+    my $plugin          = shift;
+    my $plugin_instance = shift;
+    my $type            = shift;
+    my $type_instances  = shift;
+    my $opts            = {};
+    my $sources         = [];
+    $opts->{'title'} =
+        "$host/$plugin"
+      . ( defined($plugin_instance) ? "-$plugin_instance" : '' )
+      . "/$type";
+    $opts->{'number_format'} = '%5.1lf%s';
+    $opts->{'rrd_opts'} = [ '-b', '1024', '-v', 'Bytes' ];
+    my @files = ();
+
+    for ( sort @$type_instances ) {
+        my $inst  = $_;
+        my $file  = '';
+        my $title = $opts->{'title'};
+        for (@DataDirs) {
+            if ( -e "$_/$title-$inst.rrd" ) {
+                $file = "$_/$title-$inst.rrd";
+                last;
+            }
+        }
+        confess("No file found for $title") if ( $file eq '' );
+        push(
+            @$sources,
+            {
+                name => $inst,
+                file => $file
+            }
+        );
+    }    # for (@$type_instances)
+    return ( meta_graph_generic_stack( $opts, $sources ) );
+}    # meta_graph_pg_db_size

 sub meta_graph_ps_state {
     confess("Wrong number of arguments") if ( @_ != 5 );

Łatanie pliku JS

Zgodnie z poprawką na GitHub-ie trzeba dodać poniższy patch (albo skasować linie jak w/w poprawce):

--- base.js.orig      2012-12-31 01:51:09.000000000 +0100
+++ base.js  2014-09-12 01:17:17.915656805 +0200
@@ -143,11 +143,12 @@
     $.extend(params, new_params);
     var url = original_url.split('?')[0] + '?';

+/**
     // The backend returns absolute URLs
     if (url.charAt(0) == '/') {
         url = url.substring(1, url.length);
     }
-
+**/
     for (key in params) {
         if (params[key] !== '' || params[key] === 0) {
         url += key + '=' + params[key] + ';';

Łatanie pliku graphdefs.cgi

Jest to plik pozwalający na wczytanie definicji RRD. Ale po pierwsze trzeba mu nadać uprawnienia wykonywalne:

chmod +x /var/www/htdocs/virtuals/collectdweb/data/cgi-bin/graphdefs.cgi

a po drugie nieco go "poprawić":

--- graphdefs.cgi.orig  2012-12-31 01:51:09.000000000 +0100
+++ graphdefs.cgi       2016-02-10 14:36:42.236505826 +0100
@@ -1,4 +1,4 @@
-#!/usr/bin/env perl
+#!/usr/local/bin/perl

 # Collectd-web - graphdefs.cgi
 # Copyright (C) 2009-2010  Kenneth Belitzky
@@ -19,7 +19,7 @@
 use strict;
 use warnings;

-use JSON ('objToJson');
+use JSON ('to_json');
 use CGI (':cgi');

 our $GraphDefs;
@@ -27,7 +27,7 @@ load_graph_definitions();

 my $q = CGI->new;

-my $response = objToJson({'graph_defs' => $GraphDefs});
+my $response = to_json({'graph_defs' => $GraphDefs});
 print $q->header('application/json');
 print $response;

Łatanie pliku time.cgi

--- time.cgi.orig       2012-12-31 01:51:09.000000000 +0100
+++ time.cgi    2016-02-10 14:36:42.236505826 +0100
@@ -1,4 +1,4 @@
-#!/usr/bin/env perl
+#!/usr/local/bin/perl

 # Collectd-web - time.cgi
 # Copyright (C) 2009-2010  Kenneth Belitzky

Dodanie konfiguracji dla skryptów webowych do collectd

# spróbuję to dodać do unimaster-a
#
echo "datadir: \"/var/db/collectd/rrd/\"" > /usr/local/etc/collectd-web.conf

Konfiguracja apache

Konfiguracja dla Apache składa się z dwóch elementów: definicji katalogu¹ jak również Alias oraz ScriptAlias dla wybranej wirtualki:

          Alias /stats/ "/var/www/htdocs/virtuals/collectdweb/data/"
          ScriptAlias /cgi-bin/ "/var/www/htdocs/virtuals/collectdweb/data/cgi-bin/"

Definicja jest już wykonana w repozytorium CVS/SVN w pliku 010-collectd.conf (1 2)