Gravwell is designed to work with your data, in your infrastructure, and within your constraints. Whether you have petabytes of packet capture, data-at-rest sensitivity requirements, or are simply integrating existing infrastructure, Gravwell is built to enable a workflow that meets your needs. Today we’ll look at an example integration with multiple Google Stenographer installations, our new Gravwell Packet Fleet ingester, and a powerful new feature in Gravwell Big Bang - Actionables.
Gravwell provides native PCAP capture and query capabilities, but sometimes data volume, retention, or sensitivity requirements means you have to work with dedicated packet capture tools. Google Stenographer is one example, providing a write-optimized packet capture datastore with a simple, BPF-inspired query system.
Let’s illustrate how we can create a full Gravwell workflow to interrogate access logs, and only ingest PCAP from several Stenographer instances on demand. We’ll do this by leveraging Gravwell’s new Packet Fleet ingester and Actionables. If you are new to Gravwell or do not have an active Gravwell installation, request a Free Trial and then view the Quick Start Guide to get the ball rolling once you have received your license.
To begin, let’s sketch out our architecture. We’ll keep it simple - two Stenographer instances, capturing different segments of a network, and a single node installation of Gravwell. Connecting everything is the Gravwell Packet Fleet ingester, which has query access to each Stenographer installation.
Stenographer image copyright NBC Universal Media
The Packet Fleet ingester is easy to configure, just requiring the client certificates from Stenographer for authentication, and a clear route to Stenographer’s query service. Our test config is below:
[Global]
Ingester-UUID="c5eb1f22-20d0-456b-8da7-308451a931b1"
Ingest-Secret = IngestSecrets
Connection-Timeout = 0
Insecure-Skip-TLS-Verify=false
Cleartext-Backend-Target=127.0.0.1:4023
Log-Level=ERROR
Log-File=/opt/gravwell/log/stenographer_ingester.log
# listen address for the ingester web interface
Listen-Address=":9002"
Server-Cert="server.cert"
Server-Key="server.key"
[Stenographer "Region 1"]
URL="https://region-1-stenographer.mycompany.com”
CA-Cert=“region-1-ca_cert.pem"
Client-Cert=“region-1-client_cert.pem"
Client-Key=“region-1-client_key.pem"
Tag-Name=steno
[Stenographer "Region 1"]
URL="https://region-2-stenographer.mycompany.com:1234”
CA-Cert=“region-2-ca_cert.pem"
Client-Cert=“region-2-client_cert.pem"
Client-Key=“region-2-client_key.pem"
Tag-Name=steno
There are two key components to our config - first, we create a Listen-Address for the ingester, which we’ll use later to send queries from Gravwell. Second, we create multiple Stenographer connections, each with client certificates, URL, tags, etc., to use when querying and ingesting. We can have as many Stenographer connections as we need, and still provide a unified interface for Gravwell to query.
At this point no data is being automatically ingested into Gravwell. Data captured by Stenographer stays within Stenographer’s datastore, and is only ingested when we query the Packet Fleet ingester. Let’s take a closer look at that next.
If we point a browser at the Listen-Address host:port we defined in our config above, we’ll be presented with a simple landing page:
The Packet Fleet ingester allows providing a query in Stenographer’s BPF-like format, provides details on current in-flight queries (Packet Fleet queries are asynchronous, so it’s fine to spam the ingester with lots of queries), and a few other options. First is the source override, which allows you to provide a numeric identifier to your query for easy lookup in Gravwell. We’ll see more of that later. Second is a checkbox for both Region 1 and Region 2, as described in our config above. The Packet Fleet allows multiplexing queries across any or all connected Stenographer instances, enabling you to submit a single query across your infrastructure.
New in Gravwell Big Bang, Actionables provide a way to key on any text rendered in a query and take one or more actions when selected. Often this is used to pivot to another query based on some matched text in a current query, but actionables can also be leveraged to access external resources, such as our Packet Fleet ingester. Let’s use Actionables to highlight all IPv4 addresses we see in our Zeek logs, and use any match as a reference we can use to query our Stenographer instances.
We start by browsing to the Actionables menu and creating a new Actionable. We can create multiple actions available to the user based on the matched data. We’ll create a single action, called “Stenographer”, and set the action type to “Open a URL”. The target URL will be our ingester landing page, and we’ll even pass the contents of the matched regular expression (an IP address), to the ingester by using the _ENTRY_ token. More details on using actions can be found in the documentation.
We’ll set the “Open in a modal” option to render the URL inside our Gravwell user interface, save, and populate our Triggers with a regular expression that will match on IPv4 addresses.
From here we’ll save and start searching!
Actionables appear as context menus on any match in rendered output. Let’s look through our Zeek connection logs and see if anything interesting shows up.
Upon inspection I see something fishy - a connection initiated from 8.8.8.8, Google’s public DNS resolver. That should never happen. Let’s dig in by clicking on the IP, which provides a context menu, including our “Stenographer” target. Selecting that brings up our Stenographer query interface, complete with a pre-populated query. We’ll add a source override of “100” (any integer will do here) to simplify finding any returned data and submit the query to both Stenographer regions.
From here we’ll create a new query, referencing our source override, and investigate as we would with any PCAP source.
In this article we’ve highlighted Gravwell’s flexibility in integrating with external tools and introduced the Packet Fleet ingester and Actionables. For more information on the exciting upcoming Big Bang release and doing on-demand packet capture at scale, click the red button. We'd be happy to chat.
Thanks!