Elevate Your MuleSoft Game: 7 Tips for Superior Repository Management

Version control is crucial part of Dev Sec Ops strategy and helps in minimizing the errors, build reliable applications and APIs. But the most common issue we notice with many of the implementations is that they are not able to get the most out of version control — since the standard way of setting up a mule repository is to copy what works for other languages (like java).

Is this really the right way?
No. Tailoring the strategy to uniquely suit the mulesoft implementation leads to more efficient way for developers and operations team to work as part of Dev Sec Ops process.

Why proper repository structure and guidelines matter?

• A well organized repository strategy can significantly enhance the efficiency of your development process
• Repository strategy should scale with huge number of micro services/APIs to be built on Mulesoft platform
• Configuration changes should be treated in an effective manner to avoid too many rebuilds
• Effective repository strategy should align developer and operation visions — rather than create additional obstacles/efforts

We’ll explore 7 effective repository strategies in the following article that we can use to increase productivity in mulesoft implementations.

1. Component based repository

“When I wrote this code, only God and I understood what I did. Now only God knows” (Pic Designed By Freepik)

Mulesoft implementation usually includes a variety of components like API definitions (like OAS, RAML), Mule implementations (or mule applications), Connectors, Shared libraries/utilities (like enterprise exception strategy, logging connectors). The best way to organize and utilize these different components is to create a separate repository for each of the component and utilize one from another as required — rather than create a mono repository which everyone is working on at the same time. This gets rid of dependency such as development team 1 works on folder X while second development team works on folder Y.

Common repository structure followed for mulesoft implementations are:

• API-Specific Repositories: Create separate repositories for each API. This isolation allows teams to work independently on different APIs without causing conflicts.
• Implementation Specific Repositories: Create separate repositories for each API or interface implementations. For example — if you have a system API, process API and experience API being built, you should use separate repositories for each of them. In future, if you have changes to any one type of API, you wouldn’t need to change the repository containing code of other APIs.
• Connectors: Create separate repository for each of connector projects — along with relevant test cases and detailed documentation for its usage
• Shared Libraries and Utilities: Have a dedicated repository for shared libraries and utilities that can be used across multiple projects. This promotes reusability and consistency.

Five common benefits of component based organization are:

• Reusability — Components plug into a variety of applications without the need for modification or special accommodations.
• Extensibility — Components combine with other components to create new behaviors.
• Replaceability — Components with similar functionality can be swapped.
• Encapsulation — Components are self-contained and expose functionality through interfaces while hiding the details of internal processes.
• Independence — Components have minimal dependencies on other components and can operate in different environments and contexts.

Note: In traditional repository set up (like Java or legacy application development), you might have one or a handful of repositories which hold all your organization implementations. In API led and micro services implementations, you will have hundreds of APIs to be built and it isn’t advisable to create a mono repository like the traditional approach— unless there is specific organizational restrictions for it.

2. Follow appropriate branching strategy

Using a consistent branching strategy helps manage the development process effectively — especially with multiple developers working on the same component, versioning of the components and with providing hot fixes to production application.

Here is a common branching strategy provided as an example:

Git flow Flow Example (Source)

GitFlow

• Master: The stable branch containing the production-ready code.
• Develop: The branch where the latest development changes are integrated. It serves as an integration branch for features.
• Feature branches: Created from develop for working on new features. Once a feature is complete, it is merged back into develop.
• Release branches: Created from develop when preparing for a new production release. Bug fixes are made here before merging into master and develop.
• Hotfix branches: Created from master to fix critical issues in production. Changes are merged back into both master and develop.

Note: Even though gitflow is considered legacy compared to trunk based development, for most use cases of mulesoft implementation, gitflow (or some alternative customized to organization requirement) is sufficient. We rarely will have huge number of developers working parallelly on the same component requiring access to main branch (like in task based development) — although the best way to decide on the branching strategy will be to identify the organization standards and requirements and adopt the right strategy to suit the needs.

3. Standard directory structure within repositories

“Any fool can write code that a computer can understand. Good programmers write code that humans can understand.” (Pic Designed By Freepik)

Its extremely helpful to know what the structure of projects within the repository is and how the developers/designers can use them.

Below is a sample directory structure for a mule project

• src/main/mule: Store your common / global mule flows, configurations, and sub flows here.
• src/main/mule/<apiName>/<version>: Store your api implementation flows and sub flows here
• src/main/resources: Place your data weave files and other resources needed by your Mule application here — preferably with a standard sub structure (for example src/main/resources/dw/).
• src/test: Keep your test cases and test data in this directory.
• docs: Include documentation related to the project, such as API specifications, design documents, and user guides.

Special guidelines related to documentation will be really helpful in the long run — for maintainability of the repository. Some of the example of guidelines that is useful to set at a repository level include:

• README: Include a detailed README file in each repository explaining the purpose of the project, setup instructions, and usage guidelines.
• Mule Code Documentation: For mule code repositories, specify clear guidelines on the level of documentation — at project, at configuration file level, flows/sub flow level.
• API Documentation: For API documentation repositories, specify clear guidelines on the level of documentation that needs to accompany the RAML or OAS Specifications.
• Code Commit Comments: Encourage developers to write meaningful comments in the code to explain complex logic and important decisions.

Note: You can automate code commit messages to push the commits to project planning/management tools — and in turn increase developer productivity. For example, any commit to bitbucket repository can have a #<Jira Issue ID> as a prefix to the comment — which can automatically add the comment to jira task and change the task status. Approval of any pull request or merge can automatically move the related jira issue further along its life cycle.

4. Configuration / Property Storage

”A configuration item is an asset in its context.” (Pic Designed By Freepik)

Mulesoft already provides a way to define properties outside of the implementation — so that you can secure it and make it modular — but by default they also sit along the application code in the same repository.

Properties keep changing all the time (for example, you have a key rotation strategy in organization, then every 6 months — all keys will change) — if they are stored along with the repository code, then even if there is no change to source code, you need to recompile, re-package and deploy the application for every property change— since usually properties are bundled with the application jar file that is created to be deployed.

Why store configuration different location to code?

• Isolation: Development changes can be tested with their own configuration before merging into staging or production.
• Flexibility: Teams can quickly switch configurations without affecting the main codebase.

There are lots of different approaches to utilize configurations effectively ranging from static property configuration to runtime/dynamic property lookup. But one of the simplest way you can implement this with your dev sec ops will be to adopt a simple configuration management strategy.

Common ways to implement storage of configuration are:

a. Use an external configuration vault:

• Use a separate enterprise grade/common configuration vault similar to Hashi corp Vault for storing configurations
• Common disadvantage will be to have an additional overhead of who can access/edit/update the properties and dependency on another system for Dev Sec Ops
  

b. Create a configuration repository:

• Set up a separate repository dedicated to storing configuration files and properties.
• Ensure it has restricted access and is only available to authorized personnel.
• Common disadvantage will be to have an additional overhead of who can access/edit/update the properties

c. Organize within the repository as a separate branch:

• Structure the repository to have different branch for configuration like ‘config’
  

How to correctly use the configuration for mule projects?

When build pipeline runs for an application, it takes the source code from the source code (or pre-built code jar if there are no code changes — usually for staging and production environments) and combines it with the properties (whether from external repository, external tool or from a separate branch in the code repository) and deploys the application with dynamic properties values. MuleSoft provides multiple ways of achieving this (more details in 6. Integrate with CI/CD using the right tools section below)

Advantage with this approach is:

• We can always track whether there have been any changes to code or only configuration
• Re-use pre-tested and validated code jar as much as possible — increasing re-usability and reliability of mule applications
• When there are changes to configuration only, you can confidently deploy the application using the new configuration even if the code is being re-built

5. Include runtime configurations as code

”Code never lies, comments sometimes do.” (Pic Designed By Freepik)

Mulesoft applications when they are being designed, implemented and deployed, there are lots of non-implementation information that will be at play like:

• What business group the implementation is going to be deployed to
• What business group the implementation is going to be deployed to
• vCore and worker information (like how this application will be deployed)
• API Policies to be applied (like what policies to be configured if managed in API manager)
• Organizational information (like which project this implementation falls under, which initiative, what is the configuration ID or catalog ID for the project, etc)
  

Why this will be helpful?

• Takes the guess work out of how the component will be used and managed
• Allows Dev Sec Ops process to automate non-coding related Dev Sec Ops activities as well (like updating a service now configuration item for a component, deploying API policies along with mule application deployment or updating metadata on Any point Exchange after deploying an API, etc)
• Enforce additional governance (see section Enforce best practices and governance for more details)

6. Integrate with CI/CD pipelines using the right tools.

“A good tool improves the way you work. A great tool improves the way you think.” — Jeff Duntemann (Pic Designed By Freepik)

Storing components and configuration in a repository helps provide modularity, versioning and security. But the most important benefit of storing the code in repository is to automate the rest of the Dev Sec Ops process. Ideally, you would automate the deployment or validation of component using CI/CD tools whenever changes are committed to the repository.

Common tools used for CI/CD pipelines are Jenkins, GitLab CI/CD, or Azure DevOps — and depends mainly on the repository that you would have chosen to store your code.

But for Mulesoft, what would be the best tool to do deployment of components?

Mulesoft provides two major ways of integrating with pipelines:

a. Mule maven plugin

Mule maven plugin provides a java way of integration — if the organization is already used to java implementations. It provides many features like:

• Packaging mule applications
• Running MUnit test cases
• Deploying to mulesoft environments

Common disadvantages of using the mule maven plugin (please refer to the documentation for any updates which might correct the disadvantages below):

• Newer features are added slower than Anypoint CLI
• Depedence on Java knowledge for debug and configuration
• Lacks some of the features to work with Anypoint platform (like API manager)

b. Anypoint CLI

Anypoint CLI provides a command line utility to work with Anypoint Platform — which mainly uses the platform APIs for its working. It provides many features like:

• Working with runtime components much more easily (like API manager)
• Faster release cycles — kept up to date with platform API changes
• Easily customizable (since the CLI is nodejs based) — in case of customization/hot fixes

Note: You would still need to package the components using the mule maven plugin above — since its supported only with maven at the moment. Most organizations utilize mix and match of the above components to deliver the desired functionality

7. Enforce best practices and Governance

Gone are the days where you had time to “Do It” and then “Do It Right” and then “Do It Better”. The new mantra of the day is “Make it easy to do it right the first time!”

Cost of Defects found later in the life cycle vs earlier (Source)

Why to enforce best practices?

• Cost of fixing software issues found later in the SDLC cost exponentially more than finding and fixing issues earlier in the life cycle
• Missing code quality checks will lead to faulty or insecure code — leading to loss of reputation or financial loss to organizations
• Automated quality checks improves the quality, reliability and maintainability of applications exponentially

Common ways of enforcing code quality or governance of assets are:

a. Use a Static Code Analysis Tool (SAST)

Customers use a tool like Falcon Suite (https://integralzone.com/falcon-suite/) to do code analysis and governance of components (whether API or Mule code) being checked in. These tools will help not only during the design time (they have plugins to IDE like Anypoint Studio and Anypoint Code Builder to aid developers), but also plug into the build pipeline seamlessly — to sniff and check the checked in code for compliance. Organizations can customize the rules to suit the organizations needs and prove compliance.

Benefits of automated review:

• Validate component checked in at every level is compliant with organizational standards
• Custom rules and custom quality gates can be defined as per organization requirement
• Provide developers, operational team and management with a single pane view of things going through the pipeline

b. Peer review or manual code review

Many organizations earlier in the maturity cycle of Dev Sec Ops have process in place for manual code review of the component being checked into the repositories.

Common issues of manual code review:

• Costly and highly skilled resources needed for performing optimal code review
• Manual reviews are error prone and time consuming
• Usually performed too late in the life cycle — which introduces additional financial and timeline implications to the project

Conclusion

Having a well defined guideline and vision for Dev Sec Ops goes a long way in achieving consistency, reliability and security of the applications — and helps with being compliant with organizational policies as well. Well thought out and set guidelines helps achieve optimized process — benefitting developers, operational team and management team as well.

Frequently Asked Questions

Why should I use version control in Mulesoft DevSecOps projects?

Version control enhances collaboration, improves security, ensures code quality, and streamlines the development and deployment processes.
Some of the key benefits:

• Distributed Development
• Audit Trails and Access Controls
• Automated testing and seamless deployment
• Enforced standards
• Reliable backups
• Regulatory compliance

Why would I need to enforce best practices and standards as part of Mulesoft implementation?

While enforcing the best practices, one shouldn’t just look at the mule code part or API design bit — but rather look holistically across all aspects of the Dev Sec Ops process. In Mulesoft ecosystem, you would want to enforce best practices to make sure there are no security violations or bugs introduced which can cause financial and reputation implications. Having a clear process to identify issues, validate any commits into the repositories and provide clear guidelines for fixing them aids everyone to work more efficiently.

You can use governance to validate things in:

• Mulesoft Code — to make sure that code implemented doesn’t have any vulnerabilities or issues
• API Design — to make sure that the design of the API is following enterprise best practices and provides a clear and consistent design across all the APIs in the organization
• Runtime configuration — to make sure that you have a clear articulated process for runtime configuration changes
• Configuration management — to make sure that configurations are secured and managed appropriately throughout the life cycle
• API Policy configurations — to make sure that API policy configurations are designed and implemented properly — rather than as an after thought.

Note: You can use a tool like Falcon Suite (https://integralzone.com/falcon-suite/) to enforce governance like described above.