Some IT providers talk about ransomware but they’ve never been to war in a real case.
In this interview with Tom, he shares an example of a real situation a client was under, what happened during that stressful time and how we were able to climb out of it.
A Real Security Breach Story
Derek: Tom, today, we are going to talk about an incident that you've been through and the lessons learned. Would you mind diving into this particular incident?
Tom: Let me tell you the story of one client, it's a large client with offices in different places like Connecticut, Westchester, New York. About a month and a half ago, we were doing some after-hours work on the server, updates and patching, and we noticed that there were files encrypted and in real time, as we were on the server, we noticed more files where getting encrypted. This is obviously a sign of something, like a virus, going on. Luckily for us, we were in there doing the work and we saw what was going on. We contacted the customer and we began the process of trying to contain this virus, identify it and eradicate.
What was different in this case is that the tools we had in place for this client weren't adequate because every time we got rid of the virus, it came right back, we couldn't do anything with it. Luckily we partner with a company called Continuum and they have a Security Operation Center (SOC) that we were already signed up at an advanced level endpoint protection called Sentinel One that we utilize and we called them up, we turned on our a SOC, we pushed out this tool for this specific client and allowed us to contain the virus and to start eradicating it.
There was a tremendous amount of damage already done. This happened on a Thursday night so Friday during the day it was when we got it under control and literally working around the clock, because the client needed to be back up and running on Monday. We were able to get the virus contained, eradicated, and start the restore process.
They had adequate backups in place, so they were able to fail over, operate on their backup and disaster recovery appliance and we were able to bring them back live by Monday morning. It was a tremendous amount of work and a lot of pain that that we shouldn't have had to go through.
What we should have had in place, and what we have in place now for all our clients, is this advanced endpoint protection. It's not a matter of a “nice to have” this Security Operation Center, you really need it. What we do today is we deploy this for all our clients.
What is a Security Operations Center (SOC)?
Derek: What exactly is a SOC?
Tom: It's a Security Operation Center, it's a fancy term for a bunch of very smart trained, certified security experts. Imagine NASA launching in the space shuttle, that's kind of what you can imagine sitting around looking at screens and utilizing advanced tools that we deploy for the client, which is only part of it. You need to know how to tune these tools and you need to know what you're looking at. They also backed by some Artificial Intelligence that's constantly taking all this data and information and tuning it across all these environments to pull out false positives and to see these threats.
Really what the key is it's real time detection, they see things happening and they're able to respond in minutes rather than hours or days, which is critical when something's happening in your system.
Derek: So we learned the hard lesson here, I guess, through this experience...
Tom: What we learned is that tools and things that we put in place six months ago, which were basically a state-of-the-art are no longer. It's a new world we live in, it's not a matter of if, it's a matter of when. We feel these tools and base level support and protection are not adequate enough for the clients that we support. This is the new normal in what we require for all clients going forward.
Derek: Hopefully we won't have to deal with this too many more times, but it's great that we're able to move forward and have better systems in place in case the worst case scenario comes to fruition.
Tom: The last piece of advice is that if you're running a business or you own a business, make sure to bring this up with whoever's doing your IT support. Ask them about options for advanced level endpoint protection, and what you should do to get 24/7 security professionals monitoring detection and response. It's not as expensive as you would think, and I would say it pays for itself in very short order.
