Sometimes, you just need to get data into Gravwell without setting up any ingesters--maybe you want to analyze a collection of log files somebody emailed you, or maybe you've got a pcap file from Wireshark. We've had command-line tools for this for years, but with Gravwell 4.1.0 we're pleased to announce a new feature: a flexible and easy-to-use interface for ingesting data inside the web interface! This UI lets you drag-and-drop line-delimited logs, packet capture files, or entries downloaded from a Gravwell query; Gravwell will figure out what you gave it and parse it appropriately.
You'll find the data ingester UI in the 'Tools and Resources' section of the Gravwell main menu (Don't have an instance of Gravwell yet? Grab a free trial). If you don't see the Data Ingester, check with your administrator. Admin users can always ingest data; to let regular users ingest, add them to a group (named e.g. "ingest") and put Webserver-Ingest-Groups=ingest in the Global section of gravwell.conf on the webserver.
Uploading Files
After opening the data ingester page, you'll see a "file queue" taking up most of the page. You can drag-and-drop files into it, or you can click to bring up a file section dialog.
You can queue up multiple files at a time, then ingest them all at once. For each file, you can pick a different tag. In the image below, I've queued a line-delimited log file and a packet capture, sending the former to the "web-warn" tag and the latter to "v9pcap".
Clicking "Ingest Data" uploads and ingests the files. Searching for the "v9pcap" tag brings up the entries, but note that I've had to search far back in time, because the packet capture is old:
If I re-upload the same pcap file, but check the 'Ingest entries with the current timestamp' box in the upload UI, Gravwell will ignore the timestamps in the packet capture and just apply the current time to every packet.
Manual Entry
You can also enter data by hand rather than uploading a file. This is useful if you just want a few entries for experimenting with the query language and don't feel like creating a real file. Just click "Manual Entry", then type some entries:
Each line you type will become one entry. Once you click "Queue Data", those lines will be enqueued for ingestion just like a file. You can add multiple manual data sets, or mix and match with files from the disk.
Conclusion
When you want to get some data into Gravwell fast, the Data Ingest page is the easiest way to accomplish that task. You can line up multiple files comprising multiple data types, set them each to their own tag, and then ingest the whole batch at once.
The web UI based ingester is just one of the features of our new 4.1.0 release, along with compound query support, a new "enrich" module, temporal mode in the "dump" module, and many other improvements. Schedule a demo with a Gravwell Guide and see these powerful features in action.
