How do I monitor agent health?

How do I monitor Agent Health?
Created by Dennis Underwood , last modified by Mark Weideman on Sep 27, 2023 
 
 
 
 
 Offline/Online Agent Monitoring 
 Agents Page 
 On the Agents page, the date and time of the last check-in per activity is tracked. Generically speaking, if a software agent is functioning and connected to the Internet, each behavior should have a date of less than a couple hours. Some activities are triggered by activities on the endpoint, while others are set per timers, but with a “jitter” offset of up to 30 seconds, so that customers don’t have surges in network activity. 
 
 
 
 
 
 
 
 Offline Notification Management 
 To setup notification of offline agents, use the Security Notifications page. 
 These notifications are intended for the following situations: 
 
 
 
 
 
 
 There is a network issue between an agent, and the Cyber Crucible servers. That can be either due to a firewall (hacker inspired, or just IT operations), or the agent is running but offline. Protection is maintained during offline periods, but security team notifications will not occur until the agent is back online. 
 
 
 There is an issue with the Cyber Crucible software. This is very, very rare. Please open a support ticket with Cyber Crucible immediately, if this occurs. 
 
 
 The machine is powered off. This is the most common scenario. Discussion of best practices is next. 
 
 
 
 
 
 
 Offline Notification best practices 
 An important consideration for offline notification, is that a machine which has been powered off, will appear offline to Cyber Crucible’s servers. Workstations, prone to being turned on and off during normal business operations, would produce offline notifications during lunch breaks, travel between locations, or during holidays. 
 Best practices for groups already split servers from workstations. Servers are typically best served with shorter offline notification periods, in case of an issue. Workstation focused groups are best for longer periods of appearing offline. 
 Many Cyber Crucible customers find it best to only create automated notifications for server groups, and periodically filter one of the date fields on the Manage Agents page by a date, such as “Show me agents who have not had their Machine Data Update field updated in the past week”. 
 
 
 
 
 
 
 
 Agents That Are Not Fully Updated 
 Cyber Crucible agents require a reboot to update. Greater understanding of the update algorithms can be found here . When an agent reports that a reboot will result in an update, the Agents page lists which machines require an update, and are ready to update upon reboot. Sometimes machines, if they are not rebooted often (typically servers), will stage multiple updates before actually upgrading. We see an example of that in the screenshot. The last reboot time is also available. 
 The weekly Cyber Crucible executive report identifies that number of agents which require an upgrade, and how many of those are ready to upgrade as soon as they are rebooted. The two most common reasons a machine requires an update, but are not ready/staged, are: 
 
 
 
 
 
 
 The machine has been offline, such as an employee on vacation. 
 
 
 The machine had a hardware or major software failure/refresh, and Cyber Crucible is actually no longer on that machine. 
 
 
 
 
 
 
 Other than those two instances, updates normally happen very quickly, and without administrator involvement outside of normal patching and rebooting activities. 
 
 
 
 
 
 
 
   
  

Testing if Cyber Crucible Software is "Working"
Created by Dennis Underwood on May 14, 2025
 
 
 Introduction Cyber Crucible is designed to run without interrupting end users or employees, while quietly notifying chosen Dashboard users (typically chosen IT and Security team members) of automated responses. A lot of the simulators for ransomware or malware do not accurately capture a hacker’s behavior. Even though that causes issues for tools like Cyber Crucible that (correctly) identify the simulation as a false positive, that isn’t necessarily a bad thing. Some vendors will actually build their detection engines around these simulators for some of their detection functionality, to ensure that any customer tests pass with flying colors. Real malware actions like ransomware encryption run the risk of unrecoverably encrypting or corrupting data. At Cyber Crucible, we’ve even seen ransomware secretly activates other malware on the network, or do other actions that could harm the company trying to test Cyber Crucible. So, you don’t want to use malware, and you don’t want to use a simulator so accurate that you might accidentally corrupt your data. Testing Cyber Crucible There is an easy way to test Cyber Crucible without using malware or a simulation so realistic you put your or your company’s data at risk. Cyber Crucible assesses digital identity accesses at every millisecond of the day. A lot of those digital identities are stored in the browsers - that is why attackers put web browsers to the top of the list for their automated targeting. Cyber Crucible identity access behavioral analysis will result in one of three responses: Do Nothing Hide the data known legitimate application that passes all memory and kernel checks, but does not need access to that particular identity data - example: unexploited Windows Explorer Suspend the process known legitimate application that should have access to the identity data but is hacked, or a strange application Here are two locations that are common for Windows users: In this case, the Windows username is “denni”, and these are the location where session data is stored. You may find the variable %LOCALAPPDATA% works for you if you don’t want to figure out the username. C:\Users\denni\AppData\Local\Google\Chrome\User Data\Default\Sessions %LOCALAPPDATA%\Google\Chrome\User Data\Default\Sessions C:\Users\denni\AppData\Local\Microsoft\Edge\User Data\Default\Sessions %LOCALAPPDATA%\Microsoft\Edge\User Data\Default\Sessions The View Without Cyber Crucible Installed Let’s first see what one of these folders looks like with Explorer.exe, without Cyber Crucible Installed. Here, we see multiple Chrome Sessions under the “default” Google Chrome profile. This information typically contains sensitive information that an attacker would want to hijack sessions with important programs like IT administration portal, banking, email, or storage tools like Google Drive or Dropbox. The View With Cyber Crucible Installed After installing Cyber Crucible, a quick test may be conducted, without using malware or any type of criminal techniques or misuse of IT equipment. In this case, the same browser session storage location is used: C:\Users\denni\AppData\Local\Google\Chrome\User Data\Default\Sessions %LOCALAPPDATA%\Google\Chrome\User Data\Default\Sessions C:\Users\denni\AppData\Local\Microsoft\Edge\User Data\Default\Sessions %LOCALAPPDATA%\Microsoft\Edge\User Data\Default\Sessions Cyber Crucible immediately after installation is already blocking access to the data in that folder. Please notice that this is the exact same Explorer process that was already running upon installation, so the program running suddenly has had access revoked at the “hard drive” or kernel layer. Even programs, benign, exploited, or malware that are already running are correctly assessed by Cyber Crucible. Seeing your test in the Cyber Crucible Dashboard There are a large number of identity data accesses in a running system, at all times. Software as a Service and cloud-based programs have resulted in many websites and applications that have limited on the website itself, while the browser (or embedded browser for applications) constantly communicates with data servers to populate spreadsheets, tables, graphs, and social media content. Identity Access page Identity accesses that are deemed appropriate for Cyber Crucible to suspend the offending application are listed in the Identity Access page. These are situations where the program accessing the identity data should be stopped, not just have data hidden. In this case, Explorer was not stopped after the multitude of checks Cyber Crucible conducted. Instead, data was just hidden. Therefore, there are no identity access violations listed here. Identity Access Hide page Cyber Crucible optionally has the ability to show Dashboard customers situations in which identity data was hidden, but which there the accessing program was not stopped. Explorer was not exploited or otherwise malicious in this test, it was just accessing very privileged identity data for the Chrome and Edge browsers. Most users do not have access to this information given the additional load on the customers' machines and networks transmitting that data to Cyber Crucible servers. Please ask if you would like access, but please know the activity you are seeing is not malicious activity, but identity data privacy behaviors Cyber Crucible conducts on a GRC, or policy enforcement, perspective. Identity Hide Explorer.mp4 Here, we see in this short video, that Explorer was blocked from viewing the browser session data, even though Explorer itself was not interrupted.