Exercise RFP SAST
1. Introduction
In this exercise, we will work on a fictive assignment: You are in charge of conducting an RFP (Request For Proposals) for purchasing a new static analysis software in a medium sized university in Switzerland.
This SAST tool should be used in the university for all publically available applications, and should be integrated in the university’s SDLC at the deployment stage, i.e. each in-house-developed application for Internet access should pass the SAST test before deployment.
Your normal job in the university is that of a senior developer, with a dotted reporting line to the CISO of the university. The dotted line is the reason why you were tasked with this RFP.
In discussion with your sponsor (CISO), you agree to take the best candidates out of a market survey and build a long list of candidates for further evaluation in the RFP (i.e. each of the candidates shoudl get a list of questions for the RFP subsequently (no longer in scope of the exercise)).
Costs as a factor will not be considered in this exercise.
Ressources:
- Gartner Magic Quadrant for Application Security Testing[1]
2. SAST Canditates
- Checkmarx (SAST product)
- Github
- Microfocus (Fortify)
- Snyk (new SAST)
- WhiteHat Security (SAST)
- …
3. SWOT Analysis
Checkmarx (CxSAST)
| Strength | Weakness |
|---|---|
| Supports over 22 programming and scripting languages | scan duration |
| Good integration with IDEs and local developer environments | *possibly high false positive rate? |
| Detection for all OWASP Top 10 and SANS 25 vulnerabilities | very cost intense! |
Github
| Strength | Weakness |
|---|---|
| Tight integration with GitHub Actions and the GitHub source code repository | limited language support** |
| Analysis is performed automatically as code is committed or pull requests are generated | Support for IDE is limited to Visual Studio Code |
| Developer enablement is good | no dynamic and interactive scanning available |
| Don’t support mobile application testing |
** C variants, Java, JavaScript, TypeScript, Python and Go
Microfocus (Fortify)
| Strength | Weakness |
|---|---|
| Realtime security checker inside IDE | |
| Artificial intelligence (AI) predictions on issues | |
| Reduce false positives | complexity and volume of unfiltered results can be a challenge |
Snyk
| Strength | Weakness |
|---|---|
| Free edition, unlimited use for opensource projects | |
| Gitlab and JIRA integration | limited support for mobile application testing (IOS) |
| AI based semantic analysis | |
Whitehat Security
| Strength | Weakness |
|---|---|
| Dev SecOp integraton | does not offer an IAST solution. |
| DAST-as-a-service provider, supports native API testing and cloud environments | |
| Transparent pricing model |
4. Conclusion
Based on a OWASP article[2] we have the following selection critieras for a DAST tool:
- Requirement: Must support your programming language, but not usually a key factor once it does.
- Types of vulnerabilities it can detect (out of the OWASP Top Ten (plus more?))
- How accurate is it? False Positive/False Negative rates? – Does the tool have an OWASP Benchmark score?
- Does it understand the libraries/frameworks you use?
- Does it require a fully buildable set of source?
- Can it run against binaries instead of source?
- Can it be integrated into the developer’s IDE?
- How hard is it to setup/use?
- Can it be run continuously and automatically?
- License cost for the tool. (Some are sold per user, per organization, per application, per line of code analyzed. Consulting licenses are frequently different than end user licenses.)
On my opinion there is no "all-in-one product suitable for every case". It depends on your project which SAST product satisfy your needs best. As university we work a lot with opensource projects and for many projects github is allready widely used. I’d recommend to give the SAST funcion of github a try and also use Snyk which provides unlimited use for opensource projects. I’d prefer to use tools which are free first or provides free capabilities for opensource projects. [3]
But free tools have often limited functionalities. Supporting different IDEs is often not given and the set of supported programming languages is reduced. A combination of opensource tools and a commercial one can be the best, but it depends from project to project which is the best. A good SAST tool should also not produce to much false positives otherwise it’s not a gain.
[1] https://www.gartner.com/en/documents/4001946-magic-quadrant-for-application-security-testing
[2] https://owasp.org/www-community/Source_Code_Analysis_Tools
[3] https://eldadfux.medium.com/this-is-how-we-use-snyk-to-protect-our-open-source-projects-from-evil-dependencies-6ee258ca5815