Case Study — Detection Engineering
Custom Detection Engineering and Incident Analysis Using Wazuh SIEM
This project focused on creating and validating a custom Wazuh detection rule for SSH brute-force activity. The goal was not just to trigger an alert, but to understand how attack behavior appears in logs, how detection logic works, and how alerts can support incident analysis.
Context and Objective
Brute-force attacks are a common way attackers attempt to gain unauthorized access to systems. In this project, I simulated that behavior in a controlled lab environment using Kali Linux and Hydra against a Linux server running SSH.
The objective was to move beyond basic log collection and create a custom detection rule that could identify repeated authentication failures from a single source. This helped show how multiple lower level events can be correlated into a higher confidence security alert.
The project was completed in a VirtualBox lab using Wazuh, Kali Linux, Hydra, and a Linux target server. The final result was a working detection that generated high-severity Level 12 alerts for repeated SSH authentication failures.
Attack Simulation
To generate realistic activity, I used Hydra from the Kali Linux machine to perform repeated SSH login attempts against the Linux server. This created authentication failures that appeared in the Linux logs and were forwarded into Wazuh.
The attack was intentionally performed in a controlled internal lab environment. The purpose was not offensive activity for its own sake, but to understand what brute-force behavior looks like from the defender’s side.
This helped connect the full chain: attacker activity, system logs, SIEM ingestion, rule processing, alert generation, and incident analysis.
Detection Logic
Wazuh already detected individual authentication failures, but the goal was to create a stronger rule that identified repeated failures as a pattern. This is important because one failed login attempt may not be meaningful, but many failures from the same source can indicate brute-force behavior.
The custom rule was designed to correlate multiple failed authentication events within a defined timeframe. When the pattern matched, Wazuh elevated the activity into a higher severity alert.
This showed how detection engineering turns raw event data into a more useful security signal. Instead of only seeing separate log entries, the SIEM could identify behavior that required attention.
Troubleshooting and Refinement
The detection rule did not work correctly on the first attempt. That became one of the most valuable parts of the project because it forced me to troubleshoot the rule structure, syntax, and matching behavior.
I ran into issues with incorrect rule syntax, including the wrong use of matching conditions and configuration problems involving frequency and timeframe logic. At one point, Wazuh failed to restart because of rule file errors, which required reviewing logs and correcting the configuration.
I also learned that the original detection logic was too narrow because SSH authentication failures could be associated with multiple rule IDs. The rule had to be refined so it could better capture the relevant events.
This troubleshooting process made the project stronger. It showed that detection work is not just writing a rule once. It requires testing, validation, debugging, and refinement until the detection behaves the way it should.
Validation and Results
After correcting the rule configuration, I ran the attack simulation again and confirmed that Wazuh successfully triggered the custom detection. The alert reached Level 12 severity and included useful investigation context such as the source IP, target system, and affected user account.
This validation step was important because it proved the detection was not just theoretical. The rule was tested against real lab activity and produced alerts that matched the intended behavior.
The final result demonstrated the value of custom detection logic: default rules provide baseline visibility, but custom rules can improve detection quality by identifying behavior patterns that matter in a real investigation.
Skills and Concepts Demonstrated
This project demonstrates hands-on detection engineering, SIEM troubleshooting, attack simulation, log analysis, validation, and incident thinking. It also connects directly to MITRE ATT&CK technique T1110 for brute-force activity.
Why This Project Matters
This project matters because it shows practical detection work from start to finish. I did not only rely on default alerts — I created custom logic, tested it, broke it, fixed it, and validated the final result.
It also shows how security work often happens in real life. Detection is not always clean on the first attempt. Rules need to be tested, logs need to be understood, and configuration errors need to be solved carefully.
For me, this project was a strong step toward thinking like a SOC analyst or detection engineer: understand the behavior, identify the evidence, build a detection, validate it, and explain what happened.
Future Improvements
One improvement would be expanding the detection logic to include additional SSH-related rule IDs and authentication patterns. This would make the detection more flexible and reduce the chance of missing related brute-force behavior.
Another improvement would be adding automated response actions, such as temporarily blocking the source IP or triggering an alert workflow for review. I would also like to expand the project with more MITRE ATT&CK mapping and additional endpoint monitoring.
Future versions could also include more attack scenarios so the same lab can support broader detection engineering practice beyond SSH brute force.
Full Report and Supporting Documentation
The full report includes the objective, scenario, lab environment, Hydra attack simulation, Wazuh detection logic, troubleshooting notes, validation screenshots, MITRE mapping, and evidence collected during the project.