8 Must-Ask Questions for AI Security Vendors 

I attended the Black Hat conference last week, and to no surprise, artificial intelligence (AI) was the talk of the town. Nearly every conversation at Swimlane partners booths or with prospects of ours, touched on AI in one way or another. As an AI product leader, I enjoy these conversations, but one thing that struck me is how vague and shallow AI messaging tends to be. 

After spending the better part of my tenure at Swimlane talking with data scientists, industry analysts, customers and reading about AI trends, I can say I’ve heard it all. Keep reading this blog for my take on the top questions that you should ask your cybersecurity vendor about AI.

Introducing Hero AI

Before we dive into the must-ask questions, I should provide the context that Hero AI is Swimlane’s collection of AI-enhanced innovations available within the Turbine security automation platform. Its features include case summarization, text-to-code ChatBots, HelpDocs ChatBots, schema inference and the newly announced recommended actions and AI-augmented reporting. 

As we discuss the top questions to ask your AI vendors below, I will also share information about how we approach these areas at Swimlane. For more in-depth information about the value, use cases and architecture of Hero AI, I encourage you to check out this technical report from TAG Cyber: Using AI for SecOps Automation. 

Common Questions to Ask AI Vendors About AI Products in the SOC

1. What large language model(s) does your AI model use? 

Understanding the large language model (LLM) that is behind your AI vendors capabilities is a good place to start. A healthy degree of skepticism is needed when it comes to considering sharing cybersecurity information like alerts, cases, incidents, or automation pipelines with LLMs. 

Swimlane Hero AI features like case summarization, recommended actions, AI-augmented reporting and schema inference, interact with sensitive data and therefore use the Swimlane LLM rather than a public model. ChatBot features leverage OpenAI to help generate Python code or to help summarize Swimlane Knowledge Center information directly within Swimlane Turbine. 

2. What data sources does your AI system rely on, and how do you ensure data privacy and security?

AI systems often require access to a vast amount of data for training and analysis. It’s crucial to know where this data comes from and how it’s handled to ensure compliance with privacy regulations and to prevent unauthorized access.

Hero AI’s core features leverage a privately hosted LLM. The Swimlane LLM ensures that no sensitive data is ever stored centrally. The only centrally stored data is metadata related to model usage and performance. Customer data is never and will never be used to train shared models. All data processed by Hero AI is securely stored in dedicated database instances for each customer so that complete data segregation is ensured. 

3. Can you provide insights into the transparency and explainability of your AI algorithms?

Transparency and explainability are paramount when it comes to AI-powered security solutions. Security teams need to understand how decisions are made by the AI system and be able to interpret and trust its outputs.

Honesty and integrity in “all the things’ ‘ is a core value of Swimlane’s, and this value extends to our customers. While we want all Swimlane customers to experience the value and simplicity that comes from Hero AI, it’s important that they adopt AI on their own terms. To prioritize transparency, Hero AI is turned off by default for all customers. AI functionality is authenticated once customers have reviewed and agreed to the terms and conditions specified in the Hero AI addendum. 

4. How does your AI solution adapt to new and emerging threats?

The threat landscape is constantly evolving, and AI systems must be able to adapt accordingly. AI Vendors should have mechanisms in place to continuously update and improve their algorithms to stay ahead of emerging threats.

For data protection and security reasons, we do not use customer data to train our model. Our approach is different better. Rather than focusing on training data sets, which use months-old data, we constantly re-evaluate our model architecture to ensure that we are always using state-of-the-art technologies to support Hero AI innovations. These models are then trained on publicly available data like cybersecurity frameworks and public documentation. Customers own data and knowledge base best practices will be used within the confines of their Hero AI instance and thus train their own private model. 

5. What measures are in place to prevent adversarial attacks or exploitation of AI vulnerabilities?

Adversarial attacks aimed at exploiting weaknesses in AI systems are a growing concern. AI Vendors should have robust security measures in place to protect against such attacks and ensure the integrity of their AI algorithms.

By design, Hero AI follows the same robust security principles as the rest of the Swimlane Turbine platform. Only authorized users can access Hero AI, where they can communicate exclusively with the data they already have permissions to automate with. The data used for training and validating Hero AI features is thoroughly checked, sanitized and siloed, ensuring that customer data is never used to train shared models. 

6. How does your AI solution integrate with existing security infrastructure?

Seamless integration with existing security infrastructure is essential for maximizing the effectiveness of AI solutions. AI Vendors should provide clear guidance on how their solution can be integrated into the organization’s existing security stack.

By its very nature, Swimlane Turbine integrates with anything. It can connect with any API, and uses remote agents and webhooks to gain visibility into hard-to-reach telemetry. Hero AI is fully integrated into the platform, and thus, it can interact with any data visible to Turbine. 

7. What level of support and expertise do you offer to assist with implementation and ongoing maintenance?

Implementing and maintaining an AI-powered security solution can be complex. AI Vendors should offer comprehensive support and expertise to help security teams navigate the implementation process and address any challenges that arise.

Swimlane professional services implements Hero AI for all new customers as part of the SOC Solutions Implementation service. Technical Account Management (TAM) services are available for customers who desire on-going assistance or customization. All Turbine platform packages come with support at no added cost. 

8. What is your approach to compliance with relevant regulations and standards?

Compliance with regulations such as GDPR, HIPAA, and others is non-negotiable, especially when it comes to handling sensitive data. AI Vendors should have robust compliance programs in place to ensure adherence to relevant regulations and standards.

At Swimlane, we take complaints seriously. We constantly review and monitor the latest regulations for data protection and AI, as well as research on cybersecurity risks in the field of AI, and implement all the necessary steps and techniques to comply with them. For more information about Swimlane’s security and compliance controls, visit the Trust Center.

Learn more about AI-Enhanced Security Automation 

It is easy to be some combination of excited, overwhelmed, curious and confused about when and how to incorporate AI into your SecOps workflows. There is no single right answer or solution for all organizations, but by being knowledgeable and informed about your vendor’s offering you will be able to make the best decision for your SOC team. 
If you’re interested in learning more about AI-enhanced security automation, I’d love to connect.