SPLK-3003 Core Certified Consultant Practice Questions

The maxTotalDataSizeMB attribute in an index stanza defines the maximum size the index can reach on a single peer/indexer. Once this size is exceeded, Splunk will start freezing the oldest (coldest) buckets to make space, enforcing the retention policy based on size.

Splunk's procedure for decommissioning a site starts from a complete cluster whose manager is not on that site, with site replication and search factors that keep at least one copy and one searchable copy of every bucket on other sites. Move search heads and indexer-discovery forwarders to a remaining site, then run splunk enable maintenance-mode on the manager to prevent unnecessary bucket fixing. Update available_sites, site_replication_factor, site_search_factor and site_mappings (which maps the retired site to a remaining one), and restart the manager. Run splunk disable maintenance-mode to start fix-up on the remaining sites, then splunk stop each peer on the retired site and confirm that the replication and search factors are met. Maintenance mode is always cluster-wide on the manager, never per peer. Buckets that existed on those peers before they joined the cluster are lost.

The search capability "lets a user run a search", which is the one capability this role needs. Which indexes the analysts can search is set separately with the role's allowed indexes (srchIndexesAllowed), not with a capability. admin_all_objects (access to all objects), edit_user (manage users) and rest_properties_set (edit the services/properties endpoint) are administrative. accelerate_datamodel only lets a user enable or disable data model acceleration and is not needed to run searches.

On the instance that parses data (an indexer or a heavy forwarder), the parsing function consists of three pipelines: parsing, merging and typing. Data from inputs enters the parsingQueue; the parsing pipeline does UTF-8 decoding, line breaking (LINE_BREAKER) and header handling. The aggQueue (aggregation queue) feeds the merging pipeline, whose aggregator processor merges lines into multi-line events and extracts timestamps. props.conf, for example, describes DATETIME_CONFIG = CURRENT as the time the event "passed through the aggregator processor". Regex transforms (TRANSFORMS-, SEDCMD, routing) run later, in the typing pipeline after the typingQueue. REPORT- field extractions are search-time and never run in these pipelines. Both queues exist wherever data is parsed, not on separate tiers.

replication_factor = 2 keeps two copies of every bucket on different peers, so losing one peer loses no data. search_factor = 2 makes both copies searchable, so when a peer fails the manager can immediately make the surviving searchable copy primary instead of first building index files for a non-searchable copy, which is what happens with search_factor = 1. RF=3 with SF=2 or SF=3 also meets the objective but stores more copies than necessary, so RF=2 with SF=2 is the minimum. After an unexpected failure the manager still waits for the heartbeat timeout (60 seconds by default) before it reassigns primary copies.

On a three-site cluster, site_replication_factor = origin:1, total:2 keeps one copy on the origin site and sends the remaining copy to a site that has no copy yet, so the two copies of every bucket sit on two different sites. site_search_factor = origin:1, total:2 makes both copies searchable. When LON fails, every bucket therefore still has a searchable copy in NYC or TOK. The manager detects the failure after the heartbeat timeout (60 seconds by default); until it has reassigned primacy to the surviving searchable copies, searches return partial results. Then the cluster is valid again and the search returns complete results. No copies need to be converted, because the surviving copies are already searchable.

useACK = true enables indexer acknowledgment, which is disabled by default. The forwarder keeps a copy of each data block in its in-memory wait queue until the indexer confirms it has written the data to disk. If no acknowledgment arrives (the indexer goes down, its disk is full, or the network drops), the forwarder resends the block, to the next indexer when load balancing is used. This protects data in flight, though a resend after a lost ACK can create duplicates. During a forwarder shutdown it waits up to ackTimeoutOnShutdown (30 s) for outstanding ACKs. compressed, autoLBFrequency and sendCookedData = false do not protect against data loss.

In a search head cluster every KV store write is delegated to the KV store captain, while reads stay local. Write-heavy apps therefore load one KV store that the whole cluster shares, and adding members does not spread that write load. Moving the apps with heavy KV store use to their own, smaller search head cluster gives them a separate KV store and removes their contention from the primary cluster. Moving high-write lookups to CSV goes against Splunk guidance: CSV lookups suit files that are small or rarely modified and need a full rewrite for every edit, while KV store lookups suit large or frequently updated tables. splunk clean kvstore deletes KV store data and is meant for resynchronization or restore, and search concurrency settings do not address KV store contention.

Splunk recommends stats (or transaction) over join and append in most cases. The usual rewrite is one search that retrieves every sourcetype (for example sourcetype=a OR sourcetype=b) followed by stats ... by , so the events are fetched once and correlated in a single pass. join must run a separate subsearch, and by default only 50,000 right-side rows can be joined, within 60 seconds ([join] subsearch_maxout and subsearch_maxtime in limits.conf), so large correlations are truncated. stats is a transforming command, not a streaming one. Reading accelerated summaries is what tstats does, and stats outputs a results table, not _raw events.

The correct way to route events to different indexes based on their content is at parse time on a Heavy Forwarder or on the Indexers. This is achieved by defining a stanza in props.conf that applies a TRANSFORMS- class. This class then points to a stanza in transforms.conf which uses a regular expression (REGEX) to match event patterns and sets the DEST_KEY to _MetaData:Index with the FORMAT set to the target index name (e.g., pci_index). This ensures events are written to the correct index upon arrival.