The Way It Was
The concept of purple teaming is not new. We’ve been doing it for years; it was just disjointed and we didn’t call it “purple” or “red and blue”. Instead, the red team “broke in” (usually walked in), perused the aisles, took what they wanted, and then wrote up a scathing report with lots of items for the blue team to “fix”. The best penetration testing reports provided executive summaries with thoughtful root causes and enough details for the blue team to reproduce the observations. This included screen captures (yes, we accessed that) and sometimes video playbacks (yes, it was that easy). The end result was the red team left with a “mission accomplished” feeling and the blue team was left feeling dejected or confused or angry and then tasked with digging through log sources to find out what went wrong with all the defensive controls they painstakingly put in place but still “didn’t see anything”.
As some of the best and brightest on the red team decided to give the blue side a shot (how hard could this defender thing be?), they realized not only was the deck stacked against them, but winning wasn’t an option. Instead of winning, words like “containment” “eradication” and “response” were the best options. But why? The playbook was there; but no one was sharing it with the blue team. They found:
- Lack of coverage: Event sources lacked coverage over portions of the network and applications that attackers may be hitting the hardest. The red team didn’t have to be stealthy if no one was watching.
- Signals lost in the noise: The information was there and alerts were firing, but way too often to make them meaningful and separate out the false positives.
- Engineering required: Relying on research, blog posts and a base set of rules from toolsets that just didn’t match up wasn’t enough. Lots of engineering time needed to be spent to start reliably detecting the basics like password guessing against VPN endpoints and lateral movement across the network.
The New Approach
Collaboration. Let’s do this thing side-by-side. We’ve got smart red teamers and hyper capable blue teamers and we’re all taking an open-book exam together. Combined, we can start detecting, blocking and responding to early indicators of compromise (IOC’s), indicators of attack, and threat actors in the network. At Security Risk Advisors, we’ve seen our share of successful and unsuccessful purple teaming, here’s what we’ve learned to make it add value and efficiency to existing processes and toolsets:
Set the ground rules
- This is not the Red Team Show: We’re not going to extract passwords using only the most stealthy technique, we’re going to use lots of techniques from Mimikatz to procdump and we’re not going to establish a command and control channel using one method, we’re going to try encrypted and unencrypted channels over multiple ports. The red team is putting their cards on the table and executing test cases they assume get caught to show that the basic defensive countermeasures are in place.
- Give up the goods: The red team source IP’s and targets should be known to all parties during the exercise. The blue team needs to give the red team some accounts (basic users and administrators) to make things easier to track and minimize production disruption.
- Make sure the replay booth has all the angles: Good, consistent documentation is crucial to making a purple team exercise successful. Both red and blue should help record the outcomes and recap next actions.
Plan your campaigns
- Chart your course: The campaigns are at the heart of the purple team engagement and describe exactly what the red team will execute and the blue team will try to defend against. The example below is called the domain controller assault which has a main purpose of extracting passwords from the DC three different ways:
Fig 1: Example campaign for testing the defensive controls surrounding your Domain Controller in VECTR™
- Create custom campaigns tailored to your business: For example, we often run a campaign called “EMR (Electronic Medical Record) Data Exfiltration” with our healthcare clients or very specific fraud campaigns in the financial services sector.
Use a tool to track progress
- Don’t go at it alone: When we (quickly) realized Excel wasn’t going to cut it; we developed a tool to help. We call it VECTR™. It provides a centralized dashboard for tracking all things Purple. A quick feature list:
- Real-time task tracking during Purple Team exercises
- Measure progress across phases, test cases completed, and outcomes
- Maintains a view of red team arsenal and blue team garrison and their effectiveness
- Ability to add custom test cases and target assets
- Produce summary and detailed reporting for Purple Team outcomes
- Provide historical trending of Purple Team exercises
Fig 2: Example Detailed Test Case within a Campaign in VECTR™
Fig 3: Example VECTR™ Dashboard
Do it live
- Come Together: Get in a room together to conduct the majority of testing. This makes things more interactive and it’s a learning experience for all parties involved. The blue team gets to see the hacking magic and the red team gets to see what it looks like on the other side.
- Roll Three Deep: Display the toolset tracking your progress on one screen, the red team machine on a second and the blue team tools on a third. Alternatively, conduct the testing from the SOC.
- Use an MC: Assign someone the roll of broadcast announcer that can keep explaining what’s going on. Frequently stop and make sure everyone is on the same page and the room understands and explains:
- What Red is executing
- Why Red is executing it
- What the risk is if this is successful
- Where you’re at in the kill chain
- What tools Blue is using and why they’re using them
- The perceived stealthiest methods of the current test
- When things didn’t work; what happened or at least what we think happened
- Why we’re skipping certain tests if we are
Recap the results
- Daily Updates: At the end of each day, make sure to recap the toolset tracking progress:
- What tests were performed
- % completion overall
- Next steps and action items for the team
- What we’re going to do the next day
- Have a Q&A session
- Weekly Status: Have a discussion on how things went overall at the end of the week with:
- Discuss root causes and summarize next steps
- Have a Q&A session
Get better and do it again
- Blue Team Improvements: Analyze the results and make a game plan to improve detection capabilities using existing toolsets and make the case to potentially look towards implementing new tools and processes. Don’t try to tackle everything at once. Rather, look at each campaign and make decisions based on real-world pain points. If you’re dealing with ransomware every week or spear phishing is rampant, focus on the results of those campaigns, then move to the more difficult areas such as detecting lateral movement in the environment.
- Measurable Results: Once you feel that engineering and tuning efforts are ready to be tested, conduct the purple team exercise again to get measurable improvement. A full purple team exercise doesn’t need to be completed each round, but we suggest doing a full one annually or twice a year. The VECTR™ application will retain historical performance, making it easy to track improvement.
Toolset POC’s: Purple team exercises are a great way to score and test effectiveness of new software/appliances during product selection bake-offs. For example, EDR (Endpoint Detection and Response) and UBA (User Behavior Analytics) continue to be the most implemented new toolset in our client environments assuming they already have a SIEM and DLP. Continuous tuning and engineering of new toolsets is always required and conducting a purple team assessment specifically aimed at what the toolset is supposed to detect or block can help inform a product bakeoff in a demonstrable & repeatable way.
Purple team exercises highlight the strengths and gaps in defensive tools and processes and is a collaborative approach to getting better at defending against real attackers. Traditional penetration and red teaming should still be executed regularly throughout the year to get a true sense of the defensive posture of your organization; however purple is the key to measurable, effective improvements.
Security Risk Advisors conducts collaborative Purple Team exercises for many industry leading clients using the VECTR™ tool. Contact us and visit the VECTR™ website at vectr.io to learn more about this innovative approach.