synkr0nized
01-16-2013, 08:12 PM
Putting this in General, 'cause it's not like I'm looking for some deep debate. I kind of want to slow-clap for the guy despite the potential security breach he enabled. I suppose we could argue about the ethics of this, too.
Short version: guy wants to spend his time at work looking at lolcats and browsing Reddit or whatever; outsources his own job to Chinese programmers and pays them a fraction of his own salary; VPN hits from China raise enough flags to eventually have him investigated despite continued high marks/performance in reviews
Verizon Business Security Article (http://securityblog.verizonbusiness.com/2013/01/14/case-study-pro-active-log-review-might-be-a-good-idea/) (where many of the other articles I found on this seem to be sourcing their data)
With the New Year having arrived, it’s difficult not to reflect back on last year’s caseload. While the large-scale data breaches make the headlines and are widely discussed among security professionals, often the small and unknown cases are the ones that are remembered as being the most interesting from the investigators point of view. Every now and again a case comes along that, albeit small, still involves some unique attack vector – some clever and creative way that an attacker victimized an organization. It’s the unique one-offs, the ones that are different that often become the most memorable and most talked about amongst the investigators.
Such a case came about in 2012. The scenario was as follows. We received a request from a US-based company asking for our help in understanding some anomalous activity that they were witnessing in their VPN logs. This organization had been slowly moving toward a more telecommuting oriented workforce, and they had therefore started to allow their developers to work from home on certain days. In order to accomplish this, they’d set up a fairly standard VPN concentrator approximately two years prior to our receiving their call. In early May 2012, after reading the 2012 DBIR, their IT security department decided that they should start actively monitoring logs being generated at the VPN concentrator. (As illustrated within our DBIR statistics, continual and pro-active log review happens basically never – only about 8% of breaches in 2011 were discovered by internal log review). So, they began scrutinizing daily VPN connections into their environment. What they found startled and surprised them: an open and active VPN connection from Shenyang, China! As in, this connection was LIVE when they discovered it.
Besides the obvious, this discovery greatly unnerved security personnel for three main reasons:
They’re a U.S. critical infrastructure company, and it was an unauthorized VPN connection from CHINA. The implications were severe and could not be overstated.
The company implemented two-factor authentication for these VPN connection. The second factor being a rotating token RSA key fob. If this security mechanism had been negotiated by an attacker, again, the implications were alarming.
The developer whose credentials were being used was sitting at his desk in the office.
Plainly stated, the VPN logs showed him logged in from China, yet the employee is right there, sitting at his desk, staring into his monitor. Shortly after making this discovery, they contacted our group for assistance. Based on what information they had obtained, the company initially suspected some kind of unknown malware that was able route traffic from a trusted internal connection to China, and then back. This was the only way they could intellectually resolve the authentication issue. What other explanation could there be?
Our investigators spent the initial hours with the victim working to facilitate a thorough understanding of their network topology, segmentation, authentication, log collection and correlation and so on. One red flag that was immediately apparent to investigators was that this odd VPN connection from Shenyang was not new by any means. Unfortunately, available VPN logs only went back 6 months, but they showed almost daily connections from Shenyang, and occasionally these connections spanned the entire workday. In other words, not only were the intruders in the company’s environment on a frequent basis, but such had been the case for some time.
Central to the investigation was the employee himself, the person whose credentials had been used to initiate and maintain a VPN connection from China.
Employee profile –mid-40’s software developer versed in C, C++, perl, java, Ruby, php, python, etc. Relatively long tenure with the company, family man, inoffensive and quiet. Someone you wouldn’t look at twice in an elevator. For the sake of case study, let’s call him “Bob.”
The company’s IT personnel were sure that the issue had to do with some kind of zero day malware that was able to initiate VPN connections from Bob’s desktop workstation via external proxy and then route that VPN traffic to China, only to be routed back to their concentrator. Yes, it is a bit of a convoluted theory, and like most convoluted theories, an incorrect one.
As just a very basic investigative measure, once investigators acquired a forensic image of Bob’s desktop workstation, we worked to carve as many recoverable files out of unallocated disk space as possible. This would help to identify whether there had been malicious software on the system that may have been deleted. It would also serve to illustrate Bob’s work habits and potentially reveal anything he inadvertently downloaded onto his system. What we found surprised us – hundreds of .pdf invoices from a third party contractor/developer in (you guessed it) Shenyang, China.
As it turns out, Bob had simply outsourced his own job to a Chinese consulting firm. Bob spent less that one fifth of his six-figure salary for a Chinese firm to do his job for him. Authentication was no problem, he physically FedExed his RSA token to China so that the third-party contractor could log-in under his credentials during the workday. It would appear that he was working an average 9 to 5 work day. Investigators checked his web browsing history, and that told the whole story.
A typical ‘work day’ for Bob looked like this:
9:00 a.m. – Arrive and surf Reddit for a couple of hours. Watch cat videos
11:30 a.m. – Take lunch
1:00 p.m. – Ebay time.
2:00 – ish p.m Facebook updates – LinkedIn
4:30 p.m. – End of day update e-mail to management.
5:00 p.m. – Go home
Evidence even suggested he had the same scam going across multiple companies in the area. All told, it looked like he earned several hundred thousand dollars a year, and only had to pay the Chinese consulting firm about fifty grand annually. The best part? Investigators had the opportunity to read through his performance reviews while working alongside HR. For the last several years in a row he received excellent remarks. His code was clean, well written, and submitted in a timely fashion. Quarter after quarter, his performance review noted him as the best developer in the building.
Short version: guy wants to spend his time at work looking at lolcats and browsing Reddit or whatever; outsources his own job to Chinese programmers and pays them a fraction of his own salary; VPN hits from China raise enough flags to eventually have him investigated despite continued high marks/performance in reviews
Verizon Business Security Article (http://securityblog.verizonbusiness.com/2013/01/14/case-study-pro-active-log-review-might-be-a-good-idea/) (where many of the other articles I found on this seem to be sourcing their data)
With the New Year having arrived, it’s difficult not to reflect back on last year’s caseload. While the large-scale data breaches make the headlines and are widely discussed among security professionals, often the small and unknown cases are the ones that are remembered as being the most interesting from the investigators point of view. Every now and again a case comes along that, albeit small, still involves some unique attack vector – some clever and creative way that an attacker victimized an organization. It’s the unique one-offs, the ones that are different that often become the most memorable and most talked about amongst the investigators.
Such a case came about in 2012. The scenario was as follows. We received a request from a US-based company asking for our help in understanding some anomalous activity that they were witnessing in their VPN logs. This organization had been slowly moving toward a more telecommuting oriented workforce, and they had therefore started to allow their developers to work from home on certain days. In order to accomplish this, they’d set up a fairly standard VPN concentrator approximately two years prior to our receiving their call. In early May 2012, after reading the 2012 DBIR, their IT security department decided that they should start actively monitoring logs being generated at the VPN concentrator. (As illustrated within our DBIR statistics, continual and pro-active log review happens basically never – only about 8% of breaches in 2011 were discovered by internal log review). So, they began scrutinizing daily VPN connections into their environment. What they found startled and surprised them: an open and active VPN connection from Shenyang, China! As in, this connection was LIVE when they discovered it.
Besides the obvious, this discovery greatly unnerved security personnel for three main reasons:
They’re a U.S. critical infrastructure company, and it was an unauthorized VPN connection from CHINA. The implications were severe and could not be overstated.
The company implemented two-factor authentication for these VPN connection. The second factor being a rotating token RSA key fob. If this security mechanism had been negotiated by an attacker, again, the implications were alarming.
The developer whose credentials were being used was sitting at his desk in the office.
Plainly stated, the VPN logs showed him logged in from China, yet the employee is right there, sitting at his desk, staring into his monitor. Shortly after making this discovery, they contacted our group for assistance. Based on what information they had obtained, the company initially suspected some kind of unknown malware that was able route traffic from a trusted internal connection to China, and then back. This was the only way they could intellectually resolve the authentication issue. What other explanation could there be?
Our investigators spent the initial hours with the victim working to facilitate a thorough understanding of their network topology, segmentation, authentication, log collection and correlation and so on. One red flag that was immediately apparent to investigators was that this odd VPN connection from Shenyang was not new by any means. Unfortunately, available VPN logs only went back 6 months, but they showed almost daily connections from Shenyang, and occasionally these connections spanned the entire workday. In other words, not only were the intruders in the company’s environment on a frequent basis, but such had been the case for some time.
Central to the investigation was the employee himself, the person whose credentials had been used to initiate and maintain a VPN connection from China.
Employee profile –mid-40’s software developer versed in C, C++, perl, java, Ruby, php, python, etc. Relatively long tenure with the company, family man, inoffensive and quiet. Someone you wouldn’t look at twice in an elevator. For the sake of case study, let’s call him “Bob.”
The company’s IT personnel were sure that the issue had to do with some kind of zero day malware that was able to initiate VPN connections from Bob’s desktop workstation via external proxy and then route that VPN traffic to China, only to be routed back to their concentrator. Yes, it is a bit of a convoluted theory, and like most convoluted theories, an incorrect one.
As just a very basic investigative measure, once investigators acquired a forensic image of Bob’s desktop workstation, we worked to carve as many recoverable files out of unallocated disk space as possible. This would help to identify whether there had been malicious software on the system that may have been deleted. It would also serve to illustrate Bob’s work habits and potentially reveal anything he inadvertently downloaded onto his system. What we found surprised us – hundreds of .pdf invoices from a third party contractor/developer in (you guessed it) Shenyang, China.
As it turns out, Bob had simply outsourced his own job to a Chinese consulting firm. Bob spent less that one fifth of his six-figure salary for a Chinese firm to do his job for him. Authentication was no problem, he physically FedExed his RSA token to China so that the third-party contractor could log-in under his credentials during the workday. It would appear that he was working an average 9 to 5 work day. Investigators checked his web browsing history, and that told the whole story.
A typical ‘work day’ for Bob looked like this:
9:00 a.m. – Arrive and surf Reddit for a couple of hours. Watch cat videos
11:30 a.m. – Take lunch
1:00 p.m. – Ebay time.
2:00 – ish p.m Facebook updates – LinkedIn
4:30 p.m. – End of day update e-mail to management.
5:00 p.m. – Go home
Evidence even suggested he had the same scam going across multiple companies in the area. All told, it looked like he earned several hundred thousand dollars a year, and only had to pay the Chinese consulting firm about fifty grand annually. The best part? Investigators had the opportunity to read through his performance reviews while working alongside HR. For the last several years in a row he received excellent remarks. His code was clean, well written, and submitted in a timely fashion. Quarter after quarter, his performance review noted him as the best developer in the building.