Works
01 · Python

Splunkyard

A Splunk cluster planned, built and handed over from one screen. I wrote it so the job is done the same way every time, and so every step leaves a record.

  1. 01GatherWhat the design needs first: access, environment, profiles, sizing.
  2. 02DesignDraw the cluster. Machines, network, disks, hosts.
  3. 03ExecuteReview, prepare, install, configure, check.
  4. 04MaintainRead the running cluster back. Health, files, history.
  5. 05DocumentHand over files, or the whole project as one archive.

Splunk collects and searches the logs of a company. At any real size it is not one server. It is a group of machines, each with its own job, and they must be set up to agree with each other.

Splunkyard keeps that whole job in one place. You draw the group, it checks the drawing, builds the machines on VMware, installs Splunk and connects the machines to each other.

Before a step runs, it shows which machines the step will touch and which it leaves out. A step can be repeated. Nothing runs by surprise.

Every change goes into a journal: who, when, what, and the result. Passwords and keys are hidden in the journal, in the logs and in the exports.

It starts with what the design needs. Who may log in where, which environment, which machine and disk types, how big.

Gather screen, Access tab: a list of credentials with their purpose, and a form to add one. The password field shows dots only.
Gather. Passwords and keys live in one list. Values are never shown on screen.

Design is a drawing of the cluster. Nine machines here, grouped by job: management, search and indexing. Each colour is one job.

Design screen, Architecture view: nine machines in three groups, management, search and indexing, each as a coloured card.
Architecture. Nine machines in three groups. A new project: nothing is built yet.

Switch the view and the same machines answer another question. Network shows who talks to whom.

Design screen, Network view: machines joined by lines for incoming data, search traffic and management traffic.
Network. Solid lines bring data in. Dash-dot lines are search. Short dashes are management.

Storage shows where every disk lives.

Design screen, Storage view: each machine with its disks, all linked to a datastore.
Storage. Each machine, its disks, and the datastore they need.

Placement shows which VMware host runs which machine.

Design screen, Placement view: nine machines inside one VMware host, linked to a datastore.
Placement. Machines on a host. Automatic until you pin one.

Execute is five steps: review, prepare, install, configure, check. Review comes first. It lists exactly what a job would use.

Execute screen, Review step: a table of nine machines with role, processors, memory and disks.
Review. One line per machine: role, processors, memory, disks.

Prepare is the installation disc the machines start from.

Execute screen, Prepare step: installation profile and build environment, with a note that nothing has been built yet.
Prepare. The disc and the machine that builds it. Nothing is built in this project, and the page says so.

Install creates the machines and puts an operating system on each one.

Execute screen, Install step: a list of steps from checking the build host to verifying the operating system, each with a preview button.
Install. From building the disc to checking the system. Each step can be previewed first.

Configure sets up Splunk itself, in the right order. The manager first, then the indexers, then the search heads.

Execute screen, Configure step: package install, certificates, configuration files, cluster manager, peer nodes and search heads as ordered steps.
Configure. Package, certificates, files, manager, indexers, search heads. In that order.

After the build, Maintain reads the cluster back.

Maintain screen, Health page: it states that nothing has been collected yet, so nothing is known.
Health. Until something is read back, it says nothing is known. It does not guess.

Document is the way out.

Document screen, Project files: a handoff download and a project archive download, with an optional passphrase.
Files to hand over. With a passphrase the archive carries its secrets, encrypted. Without one it carries none.
Status

Version 1.2, alpha

It works end to end for the cluster shapes it supports. Source stays private.

The screens above show a new, empty project, so some pages say plainly that nothing has been collected yet.

Splunk is a trademark of Splunk Inc. Splunkyard is independent of Splunk Inc.