This article looks at how YY implements a reliable WAF from the perspective of WAF product development, inspired by ModSecurity, etc. Thanks to open source.
This article includes three topics
- WAF implementation
What components does WAF include, and how do these components interact to implement WAF defense functions
- wAF rule (policy) maintenance
How to maintain rules (strategies), including acquisition channels, rule testing methods, and online effect evaluation (3)
- WAF support
What information database support is needed for the improvement of WAF products
WAF implementation
WAF one sentence description is to parse HTTP requests (protocol analysis module), rule detection (rule module), do different defense actions (action module), and record the defense process (log module). Regardless of hardware models, software models, cloud models, the core is this, and then the realization of YY WAF around this sentence. The implementation of WAF consists of five modules (configuration module, protocol analysis module, rule module, action module, error handling module)
Configuration module
Set the detection granularity of WAF and enable it on demand, as shown in the figure
Protocol analysis module (emphasis)
The output of the protocol analysis is the operation object of the next module rule detection, and the granularity of the analysis directly affects the WAF defense effect. For the cloud WAF model that parasitizes the WAF module on the web server, it generally depends on the parsing ability of the web server.
Rules module (emphasis)
Here comes the point. This is the core of WAF. I will subdivide this into three sub-modules.
Rule configuration module
IP black and white list configuration, URL black and white list configuration, and selection of appropriate rule packages.
Rule parsing module
The main function is to parse specific rule files. It is best to use a unified rule description language for rules to facilitate the provision of custom rules to third parties. ModSecurity has done a very good job in this regard.
The rule file consists of four parts, which are divided into variable part, operator part, transaction function part and action part.
Rule detection module
In the previous step, we set up various variables, and the next step is to do addition, subtraction, multiplication, and addition according to certain logic.
Action module (emphasis)
Through the rule detection module, we have identified the likes and dislikes of the request, and the next step is to respond, sentencing, and not just interception.
Log module (emphasis)
Log processing is very important and very hot. The content is so rich that it can be completely independent from WAF to form a separate security product (eg log treasure) and provide interfaces to support WAF. For cloud WAF with huge data volume, there will be a separate big data team to support the architecture, including data storage (eg hdfs), data transmission (kafka), offline data analysis (hadoop/spark), and real-time data analysis (storm) , Data association analysis (elasticsearch), etc., a separate explanation will be opened later.
Error handling module
Exception handling when the above modules run incorrectly
WAF rule (policy) maintenance
WAF needs to practice a picture to cover it
WAF Support Information Base
WAF needs to practice a picture to cover it
Almost all security personnel in the above support library are doing it repeatedly. The reason why resources are not shared is that they cannot be said internally; the other is that they cannot be combined without a unified description language. Alas, the Tower of Babel for security practitioners.
Supplementary knowledge (including articles and codes)
WAF related
WAF defense capability evaluation and tools
ModSecurity related articles (I am a diehard fan of ModSecurity)
[Science] Install Apache2+ModSecurity and customize WAF rules on ubuntu
ModSecurity SecRule cheatsheets
ModSecurity CRS notes, WAF defense checklist, and some thoughts on WAF architecture
ModSecurity promotion-how to call lua script for defense quick start
ModSecurity whitelist settings
Fingerprint recognition
Web application fingerprint recognition
IP related
Use a free local IP geography library to locate IP geography-GeoIP lookup
Obtaining IP Geolocation Information IP Geolocation and IP Location Visualization
IP geographic information offline acquisition script
IP geographic information online acquisition script
Identify search engine scripts
Determine which CDN script to use
CDN architecture
Website load balancing technology reading notes and some thoughts on webmaster products
Regular optimization
NFA engine regular optimization TIPS, Perl regular skills and regular performance evaluation methods
HTTP package tool
HTTP.pl-understand the HTTP protocol through the HTTP package tool
reference
I read the eighth, ninth, tenth, and elevenths over and over again, every time I have new inspirations. Chapters 14 and 15 are used as Xinhua dictionaries to avoid forgetting.
" Web Application Defenders Cookbook Battling Hackers and Protecting Users " (Red Book, still reading)