Key Parameters When Evaluating a Web Application Firewall (WAF)
A Web Application Firewall (WAF) operates at layer 7 of the OSI network model (the application layer), inspecting data sent and received by web applications. By monitoring and filtering this traffic, a WAF protects against a range of attacks that traditional firewalls might miss.
With numerous solutions to choose from, identifying the best option for your business requires careful WAF performance evaluation. This involves understanding the various factors that determine WAF capabilities and performance, including deployment model, integrations, policy customization and management, reporting capabilities, and more.
However, it also involves specific WAF comparison metrics and parameters that provide a quantifiable measure of each platform’s security capabilities.
WAFs Explained
Deployed at the network edge, a WAF filters HTTP/S traffic between the internet and web applications to identify malicious packets or signs of an attack. Unlike traditional firewalls, which operate primarily at the network or transport layer, WAFs provide application-layer security controls.
At their core, a WAF analyzes traffic patterns, inspects payloads, and enforces rule sets to identify various threats. These internally developed rule sets or policies aim to block malicious traffic and protect potential vulnerabilities in the application logic. For example, a WAF might enforce a negative security model with a blocklist for known attacks or a positive security model that only admits pre-approved traffic using an allowlist.
A WAF is not designed to protect against all enterprise attacks and must be deployed in conjunction with other security tools. Typical attacks they secure applications against include SQL injection, cross-site scripting (XSS), and Distributed Denial of Service (DDoS) attacks.
For a more detailed understanding of potential threats, the Open Worldwide Application Security Project (OWASP) compiles a list of the top 10 web application security risks. The most recent list was published in 2021, with an updated top 10 expected in 2025.
WAFs have become a critical component of modern cybersecurity strategies, protecting against attacks targeted at the application layer. Typical capabilities of WAF solutions include:
- Real-Time Protection: Immediate alerts and response capabilities to minimize the impact of malicious traffic
- Customizable Policies: The ability to tailor policies based on the organization’s preferences and risk appetite
- Reporting and Logging Tools: Data gathering features and deep visibility into WAF traffic to track, analyze, and gain new insights
- Threat Intelligence Integration: Utilizing the latest information on new threats and attack signatures to improve detection methods
- Anomaly Detection: Using AI and machine learning tools to identify suspicious activity, outside of normal requests, and spot new or sophisticated attacks
- Application Programming Interface (API) Protections: Extending protections to APIs through Web Application and API protection (WAAP) services
WAF Deployment Types
There are three main ways of implementing WAFs, each of which offers different benefits to organizations:
- Appliance WAF: Usually hardware-based and installed locally, appliance WAFs offer reduced latency but have a higher upfront investment
- Host-Based WAF: Integrated into the application code, host-based WAFs offer a lower-cost alternative to appliance-based WAFs, as well as greater customization. However, they are more complex to implement and require local resources and maintenance
- Cloud WAF: A flexible, scalable, and cost-effective option that is easy to set up. A Cloud WAF only requires a DNS change or proxy configuration to deploy. Often delivered via an as-a-service model, Cloud WAFs offer a range of functionality that the vendor can easily update to respond to new threats
Comparing WAF Solutions
There are many factors to consider when comparing WAF solutions, beyond deployment type. With each vendor emphasizing different strengths, you must understand the most relevant factors for your organization. Core factors to consider include:
- Security Effectiveness: How well the WAF blocks real threats, including new and unknown attack types
- Customization: Ability to create and fine-tune policies for your unique application requirements
- Monitoring & Logging: Quality of visibility, analysis, and integration with Security Information and Event Management (SIEM) tools
- Ease of Management: Simplicity of rule creation, policy updates, and integration with DevOps pipelines
- Performance: Low latency and high throughput for security with minimal impact on user experience
- Integration & Compatibility: How seamlessly it connects with existing infrastructure and security tools
- Compliance: Alignment with regulations such as PCI DSS, HIPAA, or GDPR, as well as reporting features to help with audits
- Total Cost of Ownership (TCO): Balances licensing, operational overhead, and support costs.
- Vendor Reputation: Their track record, update frequency, and market leadership
WAF Comparison Metrics
These considerations provide a high-level overview of a WAF solution’s capabilities and how it could perform if implemented at your organization. But to compare WAF detection accuracy directly, you need quantifiable metrics such as true positive rate, false positive rate, and balanced accuracy.
These WAF comparison metrics reduce subjectivity and allow decision-makers to identify solutions that maximize security while minimizing unnecessary disruptions.
True Positive Rate
Also referred to as security quality, True Positive Rate (TPR) assesses a WAF’s ability to accurately identify and respond to malicious requests. In this context, a true positive refers to the WAF correctly responding to an attack. This includes both known attacks and zero-day attacks that are not yet well understood and lack specific signatures to monitor for.
The way TPR is calculated is as follows: It is the number of times the WAF is triggered divided by the number of attacks, or the number of true positives divided by the total number of true positives (caught attacks) and false negatives (missed attacks):
TPR = (True Positives) ÷ (True Positives + False Negatives)
Think of it as measuring the probability that the WAF responds when the web application is attacked. This is a direct indicator of WAF detection accuracy.
A WAF with a high true positive rate blocks more real threats and provides better protection. In contrast, a poor true positive rate indicates a WAF that is more likely to miss critical exploits, undermining its core purpose.
To maintain a high WAF detection accuracy, you must regularly update rule sets and signatures, enable anomaly detection to catch zero-day attacks, and test WAFs against industry benchmarks.
False Positive Rate
A closely related WAF comparison metric, false positive rate, measures how often legitimate requests are incorrectly flagged as malicious. Sometimes called detection quality, False Positive Rate (FPR) is calculated as the number of false positives (incorrectly flagged malicious traffic) divided by the total number of false positives and true negatives (correctly unflagged traffic).
FPR = (False Positives) ÷ (False Positives + True Negatives)
Effectively, it is the number of incorrectly flagged data packets divided by the total number of safe data packets inspected by the WAF. While high detection is beneficial, it must be achieved without generating excessive false positives that harm business operations. An extreme example would be a WAF that flagged every data packet as malicious. The WAF would catch every threat, but it would also be useless, as it blocks any legitimate use of the web application.
High false positive rates lead to alert fatigue for security teams as they waste time and resources investigating harmless activity. Best practices for minimizing false positive rates include fine-tuning WAF rules with application-specific context, utilizing testing environments to evaluate policy performance, and leveraging machine learning models to enhance WAF detection accuracy.
Balanced Accuracy
Balanced accuracy combines both of these WAF comparison metrics into a single number, accounting for successful attack detection as well as the correct handling of safe traffic. It is calculated as the average of the true positive rate and the true negative rate (TNR), the inverse of the false positive rate (1 – FPR):
Balanced Accuracy = (TPR + TNR) ÷ 2
Unlike taking the true positive rate or false positive rate in isolation, balanced accuracy WAF evaluation ensures a fair assessment of both parameters. It avoids the trap of focusing solely on blocking attacks while ignoring the impact of false alarms. A WAF with high balanced accuracy not only detects threats effectively but also minimizes business disruption. This makes it one of the most critical WAF comparison metrics for organizations choosing between vendors.
Balanced accuracy best practices to help improve WAF performance include:
- Regularly reviewing traffic logs to identify patterns causing misclassifications
- Incorporating continuous testing, including simulated attack traffic and real user activity
- Benchmarking multiple WAF solutions using the same dataset for fair comparison
Industry-Leading Balanced Accuracy WAF with CloudGuard
Tests of the leading vendors reveal CloudGuard WAF from Check Point offers the highest balanced accuracy results on the market. By testing numerous WAF solutions against two large data sets, one legitimate and one malicious, CloudGuard in two configurations (Default and Critical) achieved balanced accuracy of around 99%, significantly higher than the next best solution.
This data is corroborated by CloudGuard WAF being recognized as a leader in the industry for the second consecutive year in GigaOM’s 2024 Radar report, which examines Application and API Security (AAS) solutions.
CloudGuard WAF detection accuracy is powered by its contextual AI analytics, which use various data points to identify true positives from safe requests. Learn more about CloudGuard and how it can transform WAF performance for your business by booking a demo.
