I decided to write a short blog post describing the technical solutions I used during my BlackHat US training. All BlackHat trainings happened remotely this year, and this doesn’t come without challenges. The first one is of course attendees engagement, and the second is the labs, since they rely on real ICS hardware. I publish this blog post hopping this can help/inspire other trainers facing the same challenges.
This was not my first remote training, but I was surprised to see that this time, attendees were not turning the webcam on, neither were they participating vocally. Most of the questions / comments happened in Zoom’s chat, and that seemed a bit strange to me. “Reading the room” is important when training to make sure people are understanding what you’re trying to explain; it’s usually quite easy to tell visually 🙂
To make sure attendees were getting the core messages, I used a platform called Beekast. Beekast allows you to create polls, quizzes, and all kind of user interactions.
Below are a few examples of quizzes I used at the end of each module:
During the Capture the Flag event, it is also difficult to know who is following the instructions. I set up a CTFd server, not for the competition, but to have a way of knowing who’s able to complete the tasks, and set the pace accordingly.
I’m also using a shared Google doc, in which it’s easier to share command lines :
Technical setup for the CTF
I usually come to conferences for training with my ICS setup that fits in a luggage, and use a small WiFi access point to allow attendees to connect, using the virtual machines I provide. Remotely, it’s unfortunately not so easy.
I decided to try something new for BlackHat, by hosting the virtual machines for attendees in the cloud.
I used Guacamole to allow attendees to access the VM directly from a standard web browser. I was worried about the graphical performance, but using a very standard server with no graphic cards worked perfectly.
I rented a dedicated server from OVH, hosted in Canada (since most of the attendees would be from US region I thought). I installed ESXi from VMware to manage the VMs.
I created 2 template VMs for the attendees : one Windows 10 and one Kali Linux. I then used this script to deploy 20 copies of each one.
I used a 24-core server, with 256Gb of RAM, which was more than enough to run the ~45 VMs (CPU usage varied from 40% most of the time to 70/80).
Most of the exercises are performed directly on the virtual machines, but for the CTF we also need to access the specific hardware, that is temporarily installed in my home. So I set up a site-to-site VPN between the Cloud server in Canada and my local ESXi server, which is itself connected to PLCs and an additional laptop to host virtual machines.
The whole setup architecture looks like this :
I also set a Twitch stream that allowed attendees to see in “real time” the impact of their attack on the hardware setup. I used OBS with 3 webcam to capture the setup and the screens of the SCADA VMs. It looked like this:
Hope this will be useful to some!