One of the most frequently asked questions is if we have static code analysis and a well defined DevOps process, why would we need run time code analysis? In this article, let’s explore the differences between the two and why you might want to have runtime code analysis (and IZ Runtime Analyzer) even if you have static code analysis in an organization.

In this article, we will take MuleSoft as an example and talk about Static Code Analysis and Runtime Analysis based on them. The concepts will remain the same no matter the technology we choose – but it helps to understand the tool better with concrete examples. 

We won’t cover why static source code analysis is important for MuleSoft – you can read it from our other article at

Let’s take a look at key challenges not addressed by static code analysis and why run time code analysis is imperative to work together with static code analysis to provide a comprehensive solution for organization for comprehensive analysis.

To understand the concepts better, lets assume ACME organization for our discussions. ACME uses Mulesoft’s Anypoint Platform and has multiple business groups and environments. They have a mixture of internal and external APIs, some of which handle sensitive data and hence security vulnerabilities are a top priority. ACME’s development team follows best practices and uses static code analysis before deployment, checking for potential vulnerabilities, bugs, and code smells. However, they are concerned about maintaining consistency across environments and runtime issues that static analysis might miss.

So, what are the biggest challenges that still exist after having static code analysis in place?

Integrity of deployments

As company policy in ACME, there are strict coding guidelines available to analyze writing of quality code. They even use a product like IZ Analyzer which has Anypoint Studio plugin to analyze the code write from the first second code is being written and validated against the enterprise set standards. When the code is committed to code repository, automatic build pipelines are run as part of Dev Sec Ops life cycle – which use a product like IZ Analyzer to analyze code to make sure that the code is in line with enterprise standards.

But what happens after the code is deployed to Test environment? Traditionally, if everything is as per the process, the code is not touched after the code is deployed to Test environment – but instead promoted from Test to Stage environment and Stage to Production environment. But what about hot fixes/bugs? In a large organization with multiple DevOps teams and access to platform shared with multiple users/applications, how can ACME be sure that the process is being followed correctly in the organization and there are no gaps/breaks in the process?

Runtime code Analyzer tool like IZ Runtime Analyzer fills this gap by continuously scanning any changes to all the environments (based on configuration of course!) and can intelligently determine when new code or API has been deployed to any of the servers. It can identify whether the same code in test environment has been promoted to Staging and Production environments or whether its a modified/new code deployed to the environments. It analyzes the code deployed if new and alerts based on any new vulnerabilities introduced with the deployment – so that even if the static code analysis didn’t capture the issues or if the process is by passed, it will capture and provides uptodate state of the organization clearly. This helps make sure that the organization can have a reliable way to check the runtime state of the estate – rather than just assuming that process works or to miss checking for any unauthorized deployments or changes at all!

In summary, the IZ Runtime Analyzer continuously monitors all environments, verifying that the deployed applications are identical. This gives organizations assurance that the code in promoted environments are consistent and the Dev Sec Ops process is working as expected without any exceptions.

Continuous Runtime Monitoring

ACME has one of their APIs deployed and been running successfully for a long time. Suddenly they find that one of their APIs is behaving unusually. Since they follow the Dev Sec Ops process through an established process, they know that there should be no new update/deployments since the original deployment. They have health check monitoring in place for the application which shows that the API is up and running without issues. What could be wrong?

Runtime analysis tool like IZ Runtime Analyzer keeps monitoring all the environments for any changes to application/APIs in the organization. It can find any changes made to the system (either intentional or rogue) and provide an accurate alerting to notify changes – in a proactive manner. It can check and confirm if an already analyzed piece of code is being promoted from one environment to another or a new code is getting deployed and exposed in the organization. It will auto analyze any exposed vulnerabilities or hotspots in the code – and provide ways of mitigating or fixing the issues that have been introduced. Most crucial aspect of this threat is the ability of someone replacing an existing application with a compromised application – keeping the health check and other functionality similar – and then replace the original application back without anyone ever figuring out that changes to the application has occurred – because no one reported it! But with a continuous monitoring with a tool like IZ Runtime Analyzer, any small changes will be reported as well – so that you can be absolutely sure that no unauthorized deployments/changes to the code has occurred in production (or pre-production) system!

In summary, IZ Runtime Analyzer provides continuous monitoring capabilities to make sure that all changes are strictly analyzed and reported – without any manual intervention. You could even automate the preventive measures to stop the application from continuing or resetting the application state if required to original version if it doesn’t meet enterprise guidelines!

Keep Compliance Current (KCC)

ACME had deployed hundreds of APIs on their platform which utilized various java packages or modules since their applications used lots of in built connectors, custom connectors and libraries. They encountered two main types changes to their security profile:

1. Critical vulnerabilities like Log4j vulnerability are discovered very frequently – and as part of company policy they want to make sure that such vulnerabilities are quickly identified and fixed both in standard components and custom components
2. If a company guidelines change (for example – allowed authentication methods or connectivity methods to a specific API/system changed from earlier policy to a new one or API specification guidelines are changed) – they would like to identify the technical debt easily 

Whenever guidelines change like this and they had hundreds of applications and APIs, the usual way of scanning each and every application/API is a huge laborious process. How do we get around this issue?

With a tool like IZ Runtime Analyzer, whenever company adds or modifies their policy guideline, this will trigger a complete re-analysis of the entire affected infrastructure applications/apis – so that organization readily obtains updated report on the compliance of the organization. Any application which was compliant under the old guideline or rule set, could be either compliant, partially compliant or non-compliant according to the new set of guidelines. This reduces the huge overhead on the Dev Sec Ops team to re-trigger analysis of the entire application code base to analyze things – as well as capture rogue/bypassed applications according to the new guidelines to always keep the compliance current.

In summary, IZ Runtime Analyzer provides ability to rescan the entire application infrastructure so that compliance of the organization remains current no matter the frequency or type of guideline change.

Coverage Analysis

ACME has hundreds of applications and APIs built over the years as part of many main initiatives / projects and deployed across various business groups/environments. Most of the static code analysis reports are usually generated at specific application or API level – which are great at identifying issues at component level. 

As part of compliance view of the estate, they would like to know:

• What’s the compliance status of the Group 1/Production environment and Group 2/Production environment?
• What’s the compliance status of APIs of Group 1 and APIs of Group 2?
• What’s the overall compliance status of the organization as a whole – across the entire estate?
• As part of ACME, there are two chief projects/initiatives. One of the project uses the following components:
  • App1 from Group 1 Production, App 1, App 2 from Group 2 Production environment, API 1 from Group 1 and API 2 from Group 2
  • What’s the compliance per initiative or project?
• ACME uses different vendors to build/maintain the applications. Can we find out what’s the compliance status per vendor?

In IZ Runtime Analyzer, not only component level code/specification is scanned with issues and metrics captured, it provides a capability of linking the assets across the entire estate. This enables easy way of analyzing the compliance data in one shot – whether you would like to understand it at component level, summary level or based on custom tag level! This enables to truly have a compliance dashboard specific to whatever level of data that you desire – and analyze the risk associated with issues discovered by the tool!

In summary, IZ Analyzer provides a broad auto analysis capability – which helps in making sense out of fine grained data – so that you can really get to the heart of the issues and spend time and effort in solving the issues rather than identifying them manually!

Feedback and Compliance Loop

ACME has good Dev Sec Ops process defined – which helps in identifying issues generated during the build and deployment part of their product life cycle. What they would like to achieve would be:

• Identify the bug frequency/issue frequency which are identified by run time analysis which is not identified as part of static code analysis
• Identify and optimize the Dev Sec Ops and life cycle management process and keep it updated
• Provide reports on maintaining the compliance as per the current guidelines

Through continuous monitoring and providing feedback to the process and development team, IZ runtime Analyzer helps make sure that entire feedback and compliance process comes back to full circle – instead of just leaving the process responsible for just deployment part of the life cycle. Runtime analysis tool also identifies the technical debt of current and old applications – helping organization understand and plan for tackling similar situations in the future more optimally.

In summary, IZ Runtime Analyzer helps bridge the gap between deployment and completing the feedback & compliance loop within the organization.