d-fens GmbH

Graylog2 is a powerful and scalable open source log management and data analytics system on top of elasticsearch. Since our logging module is based on log4net, the integration of Graylog2 as log destination is very easy thanks to the available gelf4net adapter.

The gelf4net project provides a log4net adapter that formats logs to the GELF specification and makes it easy to send them over UDP or AMQP (but we did not test AMQP yet). Currently there are no TCP or HTTP adapters available.

Graylog2 Input Configuration

The first step is to create a GELF UDP input listener on a desired port. Otherwise the system will not accept the forwarded messages from the client. Note down the port as you will use it later in you log4net configuration. Here you can also choose whether you want to have the input accepting messages on all nodes (“global”) or you on a single node.

Client configuration
The easiest way to install gelf4net is using the nuget command line tool, which also downloads the log4net package.

The latest version of the nuget.exe command-line tool is always available from http://nuget.org/nuget.exe.

PS C:\Software> C:\Software\nuget.exe install gelf4net
Attempting to resolve dependency 'log4net (≥ 2.0.3)'.
Installing 'log4net 2.0.3'.
Successfully installed 'log4net 2.0.3'.
Installing 'Gelf4Net 2.0.3.1'.
Successfully installed 'Gelf4Net 2.0.3.1'.

There where some issues with the way log4net resolves appender assemblies. However it works when gelf4net.dll is located in the same directory as the log 4net assembly:

Copy-Item "C:\Software\Gelf4Net.2.0.3.1\lib\net40\gelf4net.dll" "C:\Software\log4net.2.0.3\lib\net40-full\gelf4net.dll"

You now have to add an appender setting in your existing log4net configuration, where you specify IP address and UDP port number that you configured on the server side.
Here is an example static xml file for the log4net configuration. The configuration logs to the Console and to the GelfUdp Input Listener. Keep in mind that currently Graylog2 does not support neither authentication nor encryption on its inputs/listeners! In this example we log Gelf messages to 192.168.60.130 at UDP port 12201:

<appender name="GelfUdpAppender" type="gelf4net.Appender.GelfUdpAppender, gelf4net">
  <remoteAddress value="192.168.60.130"/>
  <remotePort value="12201" />
  <layout type="Gelf4net.Layout.GelfLayout, Gelf4net">
  <param name="AdditionalFields" value="app:RandomSentence,version:1.0,Level:%level" />
  <param name="Facility" value="RandomPhrases" />
  <param name="IncludeLocationInformation" value="true"/>
  <!-- Sets the full_message and short_message to the specified pattern-->
  <!--<param name="ConversionPattern" value="[%t] %c{1} - %m" />-->
  </layout>
</appender>

Note how you can also specify additional fields (e.g. “AdditionalField”) in the configuration. Later in Graylog2 these fields appear prefixed with an underscore: “_AdditionaFields”. It is recommended that you -if you have the chance- do the proper formatting before you insert the data in to Graylog2, as you then do not have to do any extraction/rules. When using the d-fens logging module you can split the message into separate fields already so you do not have to do the splitting based on default pipe character (“|”) we are using.
One more note about the GELF format: In the specification you can read that specifying the “Facility” is deprecated. You should have this as an additional field as well (in contrast to the above example).

In your root/logger configuration section you then specify GelfUdp as an active appender (along with the log level you would like):

<root>
  <level value="ALL" />
  <appender-ref ref="GelfUdpAppender" />
</root>

Loading log4net and Gelf4net
To be able to actually use log4net and gelf4net you have to load the assemblies via “Add-Type” first:

You can then load the log4net and gelf4net assemblies and start logging:
Add-Type -Path "C:\PSModules\biz.dfch.PS.System.Logging\log4net.dll";
Add-Type -Path "C:\PSModules\biz.dfch.PS.System.Logging\gelf4net.dll";
[log4net.Config.XmlConfigurator]::ConfigureAndWatch("C:\data\Log4netConfigurationFile.xml");
$mvar.Logger = [log4net.LogManager]::GetLogger("root");

Logging messages
You can then send log message either via using the “Info/Warn/Error” methods or via the corresponding “Format”- messages where you can have Log4net format your string according to .NET string formatting conventions (“{0}” etc):

$mvar.Logger.Info("my message");
$mvar.Logger.Info("{0} and my formatted message.", $someOtherVariable");

Hint: though this is not gelf4net specific but applies to lgo4net in general, you can use “global properties” in log4net that you can reference in your appender settings. You can set these properties as follows …

[log4net.GlobalContext]::Properties["myProperty"]

… and references them for example in your conversion pattern or fields alike this:

<conversionPattern value="%%property{myProperty}|%message%newline" />

In case you would like to test your Gelf input manually you can easily do so from PowerShell directly via ‘Invoke-RestMethod’ (and adapting the curl example from the official documentation):

# curl -XPOST http://graylog2.example.com:12201/gelf -p0 -d '{"short_message"="Hello there"; "host"="example.com"; "facility"="test"; "_foo"="bar"}'
$Body = @{"short_message"="Hello there"; "host"="example.org"; "facility"="test"; "_foo"="bar"} | ConvertTo-Json
Invoke-RestMethod POST 'http://192.168.174.134:12202/gelf' -Body $Body

As soon as your log messages are in Graylog2 you can do whatever you have to with these messages just as usual. So in case your applications are using log4net anyway you don’t have to use any specific log forwarders but just adjusting your existing log4net configuration – and the best thing is, that all your scripts can now log centrally as well (e.g. when they use the ‘biz.dfch.PS.System.Logging’ module).

As a side note: you can also use this to log messages to logstash as they are accepting Gelf as an input as well.

Have fun and log!