
SANS Stormcast Wednesday, March 25th, 2026: IP KVM Usage; TeampPCP, Trivy, liteLLM and More
Loading summary
A
Hello and welcome to the Wednesday, March 25, 2026 edition of the sans Internet StormCenter's Stormcast. My name is Johannes Ulrich, recording today from Jacksonville, Florida, and this episode is brought to you by the Sans Edu Bachelor's degree program in Applied Cybersecurity. Filling diaries today, just one quick diary about detecting IP KVMs. I've spoken about, spoken and written about these IP KVMs a couple times in the past. This was a little bit inspired by some news coverage that North Koreans that are getting hired here in the US as IT help are often using IP KVMs to then connect the laptops that these companies are sending to US addresses. So they're using it basically as a remote access tool. And one of the reasons they're using IP KVM's is that they don't have to install any software, which of course makes them a bit more difficult to detect. So I looked at some of the detection options with usb. In particular, the Sipeed nano KVM that I tested has a USB device that's outright called Sipeed nano kvm. For the pikvm, that's the other device I tested. It's tiny, little bit more difficult in the sense that the USB devices are a little bit more generic in their strings. However, there's also an HDMI interface that emulates a monitor and monitors are sending extended display identification data and that lists as identifier and model name PIKVM for the pikvm. Now, of course attackers can adjust it, but there's certainly something that, that you may want to look for for the pikvm. Actually, I believe there's a simple configuration file where you can adjust some of these strings. Next, we do have a little bit of a lengthy story, a supply chain story. In part, it's a long story because I've been neglecting some of these supply chain stories lately. Well, got a little bit bored of them to be honest. You know, that sort of one npm, PyPi, GitHub, repo that gets compromised after the other. But lately there is one particular group that sort of has made quite the impact and that's Team pcp. Now, Flare is one of the companies that sort of has been looking at this particular group for quite a while. I think back in December or so they started tracking them. But late February they made news because a bot called Hacker Bot Clause that's sort of hunting for CI CD workflows that it configured was able to identify exposed credentials and hit paydirt when it ran into a privileged Personal access Token used by AquaSecurity. The bot used this credential to push malicious artifacts into a Visual Studio code extension from Aqua Security. And well, that then led to the compromise of that Visual Studio code extension for Aqua Security product called Trivi. What makes this more interesting is that Aqua Security deploys supply chain security solutions and Trivi is a free vulnerability scanner that Aqua Security created. The scanner integrates with various development environments including Visual Studio code, and that's sort of where the extension comes from. Now, a week or so later, maybe a couple days later, the timeline isn't that terribly clear here. Aqua Security realized that they had a problem. They published an initial advisory, fixed the extension and rotated credentials. Well, you would think all good now. Well, that's what they thought initially, but it turns out they must have forgotten a spot. The attacker came back on March 19 and updated the Trivi binary release. They did not release a new version as you have so often supply chain attacks, but instead they replaced existing version. This meant that organizations using Trivi that were now using the malicious version if they were basically just pinning the version that hack and not a GitHub commit based on the GitHub or the Git hashtag. The code added to Trivi was a standard info stealer and exfiltrated SH keys, cloud tokens and other secrets from anybody running Trivi. And Trivi has quite a good following now. Luckily Aqua Security actually noticed this issue quickly and remediated it in a couple of hours. Again they published the details on March 20th and 22nd. So that would have been this weekend, essentially a day later, which was yesterday. Akma Security stated that they believe the attacker still has some access to environment and they have since released additional updates basically sort of detailing what they're doing to mitigate this entire breach. Now today Light LLM announced that they were affected by the Trivi compromise and that now their repository was infected as well. This turned the Trevi supply chain attack into sort of a multi level supply chain attack. And well, because now anybody running lightllm also again had the same problem. Now what is Lightlm? Lightlm is at its nature sort of a proxy that is able to forward prompts to different LLMs. It's sort of used in two ways. First of all, organizations can use it to centralize LLM requests. Users connect to Litempl and then LightLM forwards the request to the model the user. Selected Organizations deployed Lightlm in this form so they don't have to reveal the actual credentials to users. They can do some metering Rate limiting and such and basically control costs in using these LLMs. LightLM is also included in other projects as a simplified sort of API to connect to different LLMs. So you really just have to connect to Lightlm and it sort of does some of the translations and such for you. But really what it comes down to is Lightlm has access to the credentials to access all these different lms, which well again is kind of what this entire sort of compromise was after. So now the info stealer has access to even more credentials in this particular case of course credentials to different lms. Now Team pcp, the group identified as responsible for the Trivi compromise, also has apparently compromised check marks using malware similar that was used against Trivi. They used a similar tactic of replacing existing version of Checkmark's open source projects like KICS and again Visual Studio code extensions that were published by by Checkmarks. KICS is short for keeping infrastructure as code secure. So another security tool compromised by Team PCP and again a tool that often has access to credentials. Sysdig, the company that assisted Trivi with the incident response suggests that checkmarks may have been compromised using credentials stolen via Trivi. So they are a Trivi user. But at this point we no real confirmation for what exactly happened here and how these two events are linked. But that's not really all. We also have some new development coming from Team pcp. So so far they were very specifically going after weak cloud configuration, weak repositories and then stealing credentials. Stealing credentials was their thing. Aikido Security suggested that a new Kubernetes wiper they identified this weekend for was also created by Team pcp. This would be again a very different payload than what they did now. Until now Team PCP exclusively deployed credential stealers. The Kubernetes wiper is also interesting as it focuses specifically on systems in Iran. Now the way it identifies them is by looking at the time zone and the locale information. So not just by IP address or such. It may also get basically systems configured for Farsi or configured for the Iranian time zone in other countries. The reason Aikido attributes the malware to Team PCP is that one of the sort of specific techniques that Team PCP is using is ICP or short for Internet Computer Canister for command and control. So these ICP canisters are used by Team pcp. The canister by the same identifier is used for the Iran attack. So that's why they think that they are related. Also in this case, once you're affected by it, it will not just basically wipe out your kubernetes infrastructure, it will also attempt to spread the ssh and that's a very common technique where it looks at hey, what secret keys are in the system, what past connections do we have to that system? Can we somehow deduct some trust relationships or look at authorized key files and such to figure out, you know, where we can connect with these keys to other systems? So this is an event that's still very much under development at this particular point. From a mitigation point of view, Lightlm and checkmarks, they're probably just the tip of the iceberg. It's very likely that Team PCP had access to number of other repositories given how popular Privy is. And as a result you should be aware that yes, this malware may show up in other repositories. So definitely pay attention here if you want to do something about it. Well then definitely pinning by git hashes versus by versions tags is sort of one way how you could prevent some of this happening here at least spreading to you overall. Secret management of course is sort of one of the things I keep lately hitting more and more on. That's definitely sort of the unseen star of the show here because haven't really seen of any of the victims already using sort of solid secrets management here. I'm going to add links to multiple blogs I mentioned because this is a little bit lengthy story here and in some ways it's actually multiple stories. So some of these blog posts also include additional indicators of compromise and mitigation steps. With all these indicators of compromise, I would always be a little bit careful, you know how ephemeral they are. But well, this is it for today, so don't forget that I'll be teaching Defending Web Apps in Orlando next week and San Diego in May. Thanks for listening, liking and subscribing and talk to you again tomorrow. Hopefully we'll have some time tomorrow for more than just sort of one big story.
Host: Johannes B. Ullrich
Theme: Recent High-Impact Supply Chain Attacks – From IP KVM Detection to the TeamPCP-Trivi-LightLLM Multi-Level Compromise
This episode delivers a detailed look at recent cybersecurity threats, focusing on two main topics:
[00:30 – 02:05]
[02:06 – 08:55]
Trivy, a free vulnerability scanner from Aqua Security, was the primary victim.
Malicious Trivy binary acted as an info-stealer, targeting SSH keys, cloud tokens, and other secrets.
Quote:
“The code added to Trivy was a standard info stealer and exfiltrated SSH keys, cloud tokens and other secrets from anybody running Trivy.” – Johannes [04:35]
On March 25, LightLLM announced their repository was also infected due to the Trivy compromise, escalating the incident into a multi-level supply chain attack:
Quote:
“Really what it comes down to is, LightLLM has access to the credentials to access all these different LLMs, which, well, again is kind of what this entire compromise was after.” – Johannes [06:18]
New evidence links TeamPCP to creation of a Kubernetes-specific wiper malware, notably targeting environments configured for Iran (via locale and timezone, not just IP).
Quote:
“The Kubernetes wiper is also interesting as it focuses specifically on systems in Iran. … The reason Aikido attributes the malware to TeamPCP is that one of the sort of specific techniques that TeamPCP is using is ICP or short for Internet Computer Canister for command and control.” – Johannes [08:05]
[08:56 – 10:04]
Immediate advice:
Quote:
“Secret management, of course, is sort of one of the things I keep lately hitting more and more on. That’s definitely sort of the unseen star of the show here because haven’t really seen any of the victims already using solid secrets management…” – Johannes [09:35]
Johannes will share links to detailed blog analyses, including IoCs and mitigation details.
He stresses this might only be the visible part of a widescale attack:
“LightLLM and Checkmarx, they’re probably just the tip of the iceberg. It’s very likely that TeamPCP had access to a number of other repositories given how popular Trivy is.” [09:09]
For links to referenced blog posts, indicators of compromise, and further mitigation steps, see the show notes on the SANS ISC website.